The publication bias

This post will present some interesting info on biases in the pharmacological research. The financial stakes are high so drug companies are keen to publish research reports putting their products in a favourable light. The research is carried out by independent labs, usually associated with universities. In the course of testing a particular drug the lab has to ensure that certain known biases do not taint the results – selection bias, observers bias, confirmation bias. This is largely a domain of statistics so there are known ways to deal with them. But there is one bias which even the most rigorous and honest research project can fall victim to. It is known as publication bias.

Let’s assume that a certain medical condition has a 50% rate of spontaneous recovery. Let’s further assume that a 60% recovery rate in a sample group of sufferers who have been administered a certain drug is considered statistically significant. This means that if an independent research indicates recovery rates below 60% the drug is a dud. So far so good. But let’s now assume that the company promoting the drug had commissioned not one but ten research projects at different universities. Even if their drug is a coated piece of chalk it is likely that one or two sample groups of patients will have at least 60% recovery rate (and, probably, one or two will fall below 40%). This is to be expected because of the statistical spread of results in a complex system like human body. There is nothing stopping the drug company chucking nine research reports in the bin and publishing only the most favourable one. This constitutes a publication bias.

There are ways of dealing with the issue – knowing that ten reports have been commissioned the pass bar has to go up and perhaps at least two reports showing the recovery rate over 70% should be considered statistically significant. The problem is that a GP reading The Lancet does not know how many reports had been commissioned and whether a recovery rate of 65% in the published report means anything or not. To combat a possible publication bias reputable medical mags (like The Lancet) have adopted an interesting editorial policy. They will only publish results from trials pre-registered with them. This way if a drug company reports a 65% recovery rate from one trial The Lancet editors can ask what the cure rate was in the other nine trials pre-registered with them.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: