Saturday April 14, 2007
[oh, I love the subs, but there was a slightly bonkers headline in the paper today on this column, as sometimes happens… this is why I don’t mock people for what’s written by someone else in their headlines…]
So here’s an interesting question. Lots of us wander around quite happily with a “dolphins good, drug companies bad” morality in our heads; and this is entirely reasonable, they are quite bad. But how easy is it to show that drug companies kludge their results, and to explain what they’ve done to a lay audience?
On an individual level, it is sometimes quite hard to show that one trial has been deliberately rigged to give the right answer for the sponsors. Overall, however, the picture emerges more clearly. The issue has been studied so frequently that in 2003 a systematic review found 30 separate studies looking at whether funding affected the findings, and overall, studies funded by a pharmaceutical company were four times more likely to give results that were favourable to the company than independent studies.
But one classic review of bias tells a genuine Alice in Wonderland story. They found 56 different trials looking at NSAID painkillers: drugs like ibuprofen, diclofenac and so on. People often invent new versions of these drugs in the hope that they might have fewer side effects, or be stronger.
Now, these were trials in which one painkiller was compared against another, rather than against a placebo. Because if you think about it, people in pain tend to look at you a bit impatiently when you offer them a placebo. And in every single trial, the sponsoring manufacturer’s drug came out as better than, or equal to, the others in the trial. On no single occasion did the manufacturer’s drug come out worse.
Philosophers and mathematicians talk about a phenomenon known as “transitivity“. If A is better than B, and B is better than C, then C cannot be better than A. To put it bluntly, the results of this review of 56 trials exposed a singular absurdity: these drugs, surely, cannot all be better than each other?
But people then began to study the methodological flaws in big pharma studies, and to their astonishment found that industry-funded trials turn out to have no worse research methods, or better research methods, on average, than independent trials.
The most they could pin the drug companies down on were some fairly trivial howlers: using inadequate doses of the competitor’s drug as a control, for example, or making claims that somewhat lyrically exaggerated a positive finding in the results. Both heinous and cheeky, of course, but in both cases, at least, these were transparent flaws. You only have to read a trial to see that they have given a miserly dose of a painkiller, and you should read the “results” section to decide what the “results” of a trial are, because the discussion and conclusions section, at the end of the paper, is like the comment pages in a newspaper. It’s not where you get your news from.
The real action in how drug company data seems to be biased overall seems to come largely from the burying of bad results, known as “publication bias“, and as I’ve geekily argued many times before, having a clinical trials database, where people are forced to register their trial before they start, knocks this on the head.
But there is a bigger political backstory to all this. We don’t like big pharma, because it makes money out of what we think should be a “helping people” industry: a wider, unspoken dislike of brutal capitalism, perhaps. But more than that, we don’t like big pharma because we have disempowered ourselves and our states.
Everybody has some kind of interest in their results; and everyone makes mistakes, perhaps – we’ll say unconsciously – more often in their own favour. So science relies on independent replication; but drug trials are so expensive, and state funding of research so miserly, that pharmaceutical research is rarely independently funded. By which, of course, we mean it’s rarely state funded.
I’m totally up for the stuff about the dolphins being good, and big pharma bad. But if only 10% of pharmaceutical research is funded outside the pharmaceutical industry, I’m not convinced that’s entirely the industry’s fault.
Please send your bad science to email@example.com
In the text as links above, but for formality’s sake…
A study of manufacturer-supported trials of nonsteroidal anti-inflammatory drugs in the treatment of arthritis. Rochon PA, Gurwitz JH, Simms RW, Fortin PR, Felson DT, et al. (1994) Arch Intern Med 154:157â€“163.
Pharmaceutical industry sponsorship and research outcome and quality: systematic review. Lexchin J, Bero LA, Djulbegovic B, Clark O (2003) BMJ 326:1167-1170