How clinical trials compare treatments.

Start with the question and comparator

Before reading a result, name the population, intervention, comparator and outcome. A placebo asks whether outcomes differ from a designed comparison without the active component. Usual care asks a different question: whether adding or replacing treatment improves outcomes in the setting in which care is normally delivered. An active comparator asks how the intervention compares with another specified treatment, which may be standard care or another active option.

A no-treatment or historical comparison may be more exposed to changes in background risk, diagnosis, supportive care and participant selection. A placebo group is not necessarily untreated: in an add-on trial, both groups may receive the same background care while one receives the experimental treatment and the other a placebo. The result then addresses the added treatment in that care setting.

Whether withholding an effective treatment is ethical depends on the condition, available care and trial design. The comparator determines what “better” means. A treatment can outperform placebo while remaining uncertain against current standard care.

A non-inferiority comparison asks whether a new intervention is no worse than an active comparator by more than a limit set in advance, called the margin. A result that is not statistically significant does not, by itself, establish non-inferiority. The comparator’s dose, timing, adherence and rescue treatment belong in the interpretation.134

A non-inferiority result can still mean a difference

Imagine a trial where higher scores are better. The new treatment scores five points lower on average than standard treatment, with a 95% confidence interval from eight points lower to two points lower.

Suppose the study set a non-inferiority margin of ten points lower in advance. If the other requirements are met, the entire interval is above that margin, so the result can meet the study’s non-inferiority rule. But the interval is also below zero: it does not show equal effects and is compatible with a smaller average benefit from the new treatment.

Whether a loss of up to ten points is acceptable depends on the outcome, evidence for standard treatment, any advantages of the new option and the clinical context. A margin cannot be chosen after the result simply because it is convenient.49

What randomisation contributes

In a randomised trial, a planned chance process assigns people to groups. If the sequence is properly generated and unpredictable, it helps balance known and unknown factors that could affect outcomes, on average. It does not guarantee identical groups in every trial.

Allocation concealment is a separate safeguard. It stops the team enrolling participants from knowing the next assignment and selecting people accordingly. Check how many people were assigned, their starting characteristics, the allocation ratio and how the sequence was generated and concealed.1912

Allocation concealment and blinding

These safeguards address different stages. Allocation concealment protects assignment before a person enters a group. Blinding reduces the influence of knowing the assignment afterwards. Participants, clinicians, outcome assessors and analysts may all be blinded, or only some may be.

When blinding is impossible, objective outcomes and independent assessment can reduce some bias. Expectations, other care, reporting and treatment decisions may still differ. FDA guidance explains that randomisation helps limit systematic starting differences, while blinding helps reduce differences in care, assessment and expectations.57

Read the outcome before the headline

The main outcome and its measurement time should be set in advance. Outcomes important to patients describe how people feel, function or survive. A laboratory or scan result can be useful, but may be a substitute, or surrogate. A clear change in a surrogate does not automatically prove a benefit people will notice.

Check what was measured, how, when and which direction represents improvement. Keep harms beside benefits. Ask how adverse events were defined and collected, how many people contributed data, whether severity and treatment stopping were reported, and whether follow-up was long enough to find delayed effects. Randomisation helps compare groups, but a small trial may still miss uncommon serious harms.628

Follow participants and analyse the groups assigned

A flow diagram should show how many people were assessed, randomised, treated, followed and included in each outcome analysis. The intention-to-treat principle associates participants with the groups to which they were assigned, even if they did not adhere perfectly; this helps preserve the original randomised comparison. It does not, by itself, specify how every missing outcome or later event is handled.

Ask what happens in the analysis if someone stops treatment or starts rescue medication. A treatment-policy question can use outcomes regardless of those events; a hypothetical question asks what outcomes might have been under a specified scenario in which an event did not occur. These are different effects, often requiring different assumptions. Per-protocol or as-treated analyses can answer other questions, but excluding participants after randomisation can reintroduce differences between groups. Look for the prespecified question, missing-data assumptions and sensitivity analyses rather than treating an “intention-to-treat” label as a complete description.5101112

Generalisability is part of the result

Randomisation helps support a cause-and-effect comparison within the people enrolled. It does not guarantee the result applies to people excluded because of age, other conditions or medicines, disease severity, language, location or ability to attend follow-up.

Compare who was eligible, their starting risk, treatment, monitoring and usual care with the setting that matters to you. A carefully controlled trial may answer a narrower question than routine care. A pragmatic trial may better reflect usual practice but allow more variation. Either way, inspect the actual people, plan and outcomes.17

A reader’s comparison checklist

What question did the study ask? What was the new treatment compared with: placebo, usual care or another treatment? How were people assigned, and were assignment and assessment protected from foreknowledge? What benefits and harms were measured, for how long and with what uncertainty?

How many people were lost or excluded, how were missing results handled, and do the participants resemble the people you care about? If a report does not answer these questions, the size, duration, safety and relevance of a difference may remain uncertain.711