Recurring pattern. Somebody posts a careful summary of the evidence. Somebody replies "my mate did it and his knee is fine now". The second one wins the room.
Why is that so reliable, and is there anything to do about it?
5 replies·
Recurring pattern. Somebody posts a careful summary of the evidence. Somebody replies "my mate did it and his knee is fine now". The second one wins the room.
Why is that so reliable, and is there anything to do about it?
Because a meta-analysis is a claim about a distribution and an anecdote is a story about a person, and human reasoning was not built for the first thing.
A story has an actor, a sequence, and a consequence. It is memorable, it is transmissible, it is easy to picture. A pooled effect size with a confidence interval has none of those properties and cannot acquire them.
The uncomfortable part is that the anecdote is not worthless. It is a genuine observation. It is just one draw from a distribution, presented with all the vividness of a conclusion about the distribution.
And there is a selection effect that makes it worse than a random draw.
You only ever hear the anecdotes that are worth telling. Nobody says "I took it for eight weeks and honestly I could not tell". So the anecdotes reaching you are pre-filtered for being surprising, which means they are pre-filtered for being unrepresentative.
The distribution you are sampling from in a pub is not the distribution the meta-analysis measured. It is the tail of it.
That is the cleanest statement of it I have seen. The anecdote is not a bad sample, it is a good sample of a different population — "things people bothered to say".
Sign in or create an account to reply.