What Does a 20-Person Study Actually Tell Us?
A personal essay, not an evidence assessment. Our graded assessments of individual ingredients — written to a published standard, from full texts — are in the evidence section. Nothing here is a statement about what any product does.
Twenty people can teach us something.
They cannot tell us everything.
That distinction sounds obvious until a study involving twenty participants produces an exciting percentage.
31% improvement.
47% reduction.
2.4x increase.
Suddenly the number twenty disappears.
The percentage becomes the headline.
I do not dismiss small studies
That would be a mistake.
Early research has to begin somewhere.
Pilot studies can determine whether an idea deserves further investigation.
Small crossover trials can sometimes be highly informative.
Carefully controlled laboratory experiments can detect acute effects with relatively modest samples.
The important question is not:
“Is twenty too small?”
It is:
“Too small for what conclusion?”
Precision becomes fragile
With fewer participants, estimates tend to be less precise.
One unusual responder can influence the average substantially.
A few dropouts can change the composition of the group.
Subgroup analysis becomes particularly unstable.
Rare adverse events are almost impossible to characterise.
Percentages become visually dangerous
Suppose two people in one small group report something and four people in another group report it.
The relative difference can sound dramatic.
The raw difference is two people.
Both descriptions can be mathematically correct.
Only one gives the reader an intuitive sense of what actually happened.
This is why I like to see:
n = 20 almost as prominently as:
+40%.
Context should not be typography's smallest element.
Small studies are especially vulnerable to exceptional
participants
Human response varies.
One person sleeps unusually badly.
Another metabolises caffeine slowly.
Another arrives after an exceptionally stressful week.
In a group of several thousand, individual peculiarities become diluted.
In a group of ten per arm, they can become the story.
Good experimental design can reduce some of this.
It cannot make the sample larger than it is.
Replication changes everything
One small study interests me.
Three independent studies pointing in the same direction interest me much more.
A large preregistered replication interests me more again.
Evidence accumulates.
That is the important point.
I do not need the first study to answer every question.
I need everyone - including us - to stop pretending that it did.
Sometimes twenty participants are enough to falsify an
assumption
Small studies are not only about finding benefits.
A carefully designed experiment can expose a problem with a mechanism.
It can show that an expected physiological change does not happen.
It can reveal tolerability problems.
It can show that a supposedly obvious effect is not obvious at all.
The value of a study is not proportional to how commercially useful its conclusion is.
My rule is simple
When I see:
n = 20 I do not think:
bad science.
I think:
interesting evidence with a limited capacity for certainty.
Then I ask what came next.
Was it replicated?
Did a larger trial agree?
Did the effect remain similar?
Did confidence narrow?
Did safety data accumulate?
A twenty-person trial can begin an important scientific story.
What it should not do is end the conversation.
Methodological sources
Reporting and appraisal standards referred to in this essay. They are not the evidence behind any product claim.