The evidence pyramid: why not all studies are equal
"A study found" tells you almost nothing until you know what kind of study it was. A quick tour from anecdote to meta-analysis.
Not all evidence is created equal, and the single most useful habit in reading health news is to ask one question: what kind of study was this? The answer usually settles how seriously to take it before you read a word of the findings.
From the bottom up
Anecdote and expert opinion. "It worked for me," or "in my experience." Real, human, and almost worthless as proof, because for any remedy, some people improve anyway, and we remember the hits.
Case reports and case series. Careful descriptions of one or a handful of patients. Excellent for raising a question; unable to answer it.
Observational studies, cohort and case-control. These follow groups of people and look for patterns: do coffee drinkers live longer, do night-shift workers get sick more? They can find associations across thousands of people, which is powerful. But they cannot, on their own, prove that one thing caused the other. The people who differ in the habit usually differ in a hundred other ways too.
Randomised controlled trials (RCTs). Here the researchers, not life, decide who gets the intervention, by chance. That single move is what lets a trial speak about cause and effect, because the coin flip, on average, balances out all those hidden differences.
Systematic reviews and meta-analyses. At the top sits not a bigger study but a disciplined synthesis of all the credible studies on a question, appraised together and, where possible, pooled into one estimate. This is the level a meta-analysis occupies, and the reason its summary figure carries more weight than any single trial beneath it.
The catch
The pyramid is a guide, not a gavel. A sloppy trial can be worse than a superb cohort study; a meta-analysis that pools bad studies inherits their flaws, garbage in, garbage out. Level tells you where to start, not where to stop.
But it explains why we so often meet a breathless headline with a shrug. "Scientists discover" almost always sits near the base of the pyramid: a single study, frequently observational, sometimes in mice. Interesting. Rarely settled.
This is an evidence summary for the general reader — not a diagnosis or personalised medical advice. For any health decision, talk to a qualified professional.
More stories
How we weigh the evidence
Every claim we assess carries the same two marks: how strong the evidence is, and which way it points. This is the method behind them, and the reason we sometimes say "we don't know."
Relative vs absolute risk: the trick behind scary headlines
"Doubles your risk" and "raises it from 1 in 10,000 to 2 in 10,000" can describe the exact same finding. One sells papers; the other tells you what to do.
Does vitamin C stop you catching colds?
Decades of Cochrane trials find no meaningful drop in cold risk for ordinary adults, only a modest shortening of colds once they start.