Ben Goldacre
The Guardian,
Saturday January 5 2008
It is rare that a bad pharma story is left untouched by the British media, but this one unfolded while everyone was drunk in December, and perhaps it was just too geeky. Luckily, you share my taste for details. Ezetimibe is a best-selling cholesterol drug with sales of more than £2bn last year. It can modify cholesterol levels but no one knows whether it cuts the incidence of real outcomes such as heart attacks, or, you know, death. Is that the bad thing?
No. There was a trial, called ENHANCE, which looked at this more closely, running for years and ending two years ago. Doctors still give out the drug and they could do with this medical evidence to help with prescribing: but it’s nowhere to be found.
Is that the bad thing? Kind of. Let’s imagine we’re playing snakes and ladders. I roll the die three times in a row then pick the best score as my actual roll, as if the other two were just practice. I invite you, winningly, to ignore those other rolls. You would rightly kick the board over, declaring I was a cheat. I waited until after I knew the results and then I chose the score that suited me best.
Similarly, in a trial, you might measure many things but you have to say which is the “primary outcome” before you start: you can’t change your mind about what you’re counting as your main outcome after you’ve finished and the results are in. It’s not just dodgy, it also messes with the statistics.
You cannot find your starting hypothesis in your final results, unless you are a time lord. What if the results were completely random, like our die, and you just selected the one six, making it look as if you were a winner?
But the people running the ENHANCE trial altered their chosen endpoint when the trial was over. They say they did so before they knew the results. That may be so, but it doesn’t look good, and they’ve now had a very serious letter from a US congressional committee demanding to know why it was done. (I seriously recommend you read it, the letter is a hoot, and it pulls no punches). Then they announced they were changing the endpoints back, in a press release FAQ dated the same day this committee’s letter was sent.
You cannot change the rules after the game has started. You cannot even be seen to do that.
To stop firms doing this kind of thing we have registers of clinical trials. Before your trial starts you put it on the register and publish detailed information about what you’re going to do – so you can’t change your mind once the game has begun or bury unfavourable data. That’s the point of a clinical trials register.
Was the Enhance trial registered? Yes. Oh no, hang on, yes, but registered on October 31 2007, more than 18 months after it finished. And looking at some original papers, this register entry in fact had the cheeky altered outcomes. So is that the really bad thing?
No. Here is the very bad thing. It also transpires there are more unpublished studies on Ezetimibe, from five years ago, and some regard safety. A Schering executive, Robert J Spiegel, says the firms had not considered those studies scientifically important enough to publish. Some, he suggests, could be eventually placed in the public domain.
And that is the final straw: because all of this breaks the most fundamental and sacred contract in clinical research, the moral contract between patient and researcher. Patients are in the hands of researchers on the understanding they are taking modest risks to benefit mankind as a whole, to improve prescribing. Ezetimibe may well be safe. But nonetheless the data belongs to patients – and to the people whose bodies are used.
le canard noir said,
January 5, 2008 at 1:01 am
It is interesting that alternative medicine critics of conventional medicine like to point to the faults of pharmaceutical companies and then claim evidence-based medicine is rotten. The reality is that the intellectual journey of creating healthcare systems based on science is driving the cultural and legislative environment that forces ‘Big Pharma’ to behave in appropriate ways.
That journey is not complete and has different problems in different countries; the US being quite different from the UK in many ways.
What is without doubt is that the homeopaths, nutritionists and other dubious health professions could be criticised of having similar faults. However, they do not benefit from their own internal criticism forcing their own standards higher.
Ben Goldacre said,
January 5, 2008 at 1:24 am
re: “You cannot find your starting hypothesis in your final results”
just been reminded by john connell of this truly excellent feynman quote on the same subject:
“You know, the most amazing thing happened to me tonight. I was coming here, on the way to the lecture, and I came in through the parking lot. And you won’t believe what happened. I saw a car with the license plate ARW 357. Can you imagine? Of all the millions of license plates in the state, what was the chance that I would see that particular one tonight? Amazing…”
Munin said,
January 5, 2008 at 11:24 am
Any excuse for a Feynman story:
“The gravitational force is weak,” he said at one conference, introducing his work on quantizing gravity. “In fact, it’s *damned* weak.” At that instant a loudspeaker demonically broke loose from the ceiling and crashed to the floor. Feynman barely hesitated: “Weak — but not negligible.”
neilcam2001 said,
January 5, 2008 at 12:29 pm
Patients are also in the hands of doctors. Does Ben also blame Big Pharma for doctors who misprescribe medication? Why are members of the medical profession always so mysteriously blame-free in the world that Ben inhabits?
jodyaberdein said,
January 5, 2008 at 12:37 pm
re 4:
I thought that doctors were quite open about their responsibility regarding patient safety.
Certainly I’m obliged (sure out of medico legal defence, but more importantly in order to provide best care) to record decisions, results etc, and my patients have a right to see that data.
In addition I’m obliged to be open and honest with my patients about what is happening to them and what we are thinking. That includes being open and honest about mistakes.
In addition I’m obliged to subscribe to the general medical council, who regulate my practice.
J
bootboy said,
January 5, 2008 at 12:41 pm
“I can also recommend the Feynman flow chart:
www.wellingtongrey.net/miscellanea/archive/2006-12-25-what-would-richard-feynman-do.html
J
At last, a life-roadmap in handy flow-chart form.
evidencebasedeating said,
January 5, 2008 at 5:54 pm
heh
this seems to be the ‘Equazen’ (TM) approach to research. Guess we can expect a publication from Madeleine Portwood and Durham LEA ooh, say, Jan 2009?
There’s much more fishiness in the opaque supplement pushers market than Big Pharma.
manigen said,
January 5, 2008 at 9:40 pm
neilcam2001:
Which misprescription where you thinking of? Doctors can hardly be blamed for prescribing Ezetimibe incorrectly if the reason for that mistake is that information was hidden from them.
I’m certain that there are many instances of a doctor giving a patient the wrong drug (they are human and therefore fallible), but did you have any particular example in mind?
Dr Aust said,
January 6, 2008 at 1:47 am
There are many reasons for prescribing errors. Being fed inaccurate information about the pills is certainly one, but of course poor doctoring is another. In this context it is worth pointing out that doctors and pharmacists can get into plenty of trouble for such errors, certainly including being suspended and being sued for incompetence / negligence if it is clearly their mistake, or based on substandard skills and knowledge.
One also comes back to the fact that the professionals handing out the drugs are actively trying to do it the best they can and to improve where necessary – e.g. the recent drive to beef up training of junior doctors in how to prescribe drugs has come largely from the junior doctors themselves. I am, erm, not entirely sure the same ethos – “trying hard to be scrupulously honest” can be assumed of those writing the glossy PharmaCo brochures trying to persuade the docs to hand out MyDrug rather than TheirDrug. Call me cynical…
manigen said,
January 6, 2008 at 11:21 am
Shockingly cynical, Dr Aust.
There is a fascinating discussion to be had about the many reasons that a doctor might prescribe the wrong drug. I wonder if that’s the conversation that neilcam2001 wants to have. Probably not, judging by his other comment at www.badscience.net/?p=602#comment-19265
ayupmeduck said,
January 6, 2008 at 7:32 pm
If I understand this correctly I guess that there are two ways of looking at this right? The one that Merck are now effectively presenting is that it’s just an innocent mistake – after all they are still blind to the results. To believe this, you have to accept that Merck somehow forgot about the importance of the endpoint even though this is core to their research procedures. It’s sort of difficult to believe this isn’t it?
But if you don’t believe that Merck just forgot such an important detail, then the only other way of looking at it would be that Merck were trying to pull very nasty deception. Just out of interest what would be the potential penalty for such a deception?
neilcam2001 said,
January 7, 2008 at 10:59 am
As a survivor of misprescribing by a psychiatrist 8 years ago I just keep wondering why Ben never seems to tackle the issue this issue, especially with regard to anti-psychotic drugs as I understand he is an expert in neuroscience. I recently asked why he ignored a Panorama documentary which touched on the issue with regard to an elderly person with dementia and merely received from one of his adulators some arrogant statement previously made by Ben regarding a similar request. Basically I am coming to the conclusion that this website should be renamed either ‘bad journalism’ or ‘ben goldacre (self)adulation’ rather than ‘bad science’, which is a pity because I think bad science is a better subject for debate.
superburger said,
January 7, 2008 at 1:59 pm
hey neilcam2001,
start a blog, especially as you seem so passionate about the topic.
post the link here, or on the badscience forums and you’ll get plenty of hits and comments.
jodyaberdein said,
January 7, 2008 at 2:42 pm
Re: prescribing errors,
Is the point that errors are common and serious (certainly true),or that the data relating to errors is unreliable in a way that is suggestive of bias?
I wish in no way to dismiss the importance of prescribing error, but I was under the impression that this post was on the subject of how scientific data about a drug effect is presented (or not)…
emilypk said,
January 7, 2008 at 3:02 pm
Indeed. There is a horrendous lack of apples in this thread about orange.
manigen said,
January 7, 2008 at 6:28 pm
Yes, you’re right, we’ve wandered off topic. So I’ll just second superburger’s great suggestion that neilcam2001 start a blog.
Delster said,
January 8, 2008 at 12:35 pm
Neilcam2001
An incorrect drug prescription is not bad science.
It is a human error caused either by the person prescribing being mistaken in diagnosis or possibly being wrong re. the name of the drug.
These things are not down to badscience but possibly bad doctors.
If, how ever, a company produces a study on the effects of it’s new wonder drug and misrepresents it’s effects to the doctors who do the prescribing then it is bad science.
The doctor who then prescribs it does so in good faith that the company making the stuff hadn’t told him a bunch of lies hidden by cherry picked data and poor study design.
jodyaberdein said,
January 9, 2008 at 6:16 am
This has reminded me of a relatively new company to enter the
scene:
www.bmj.com/cgi/content/full/327/7429/1442?maxtoshow=&HITS=10&hits=10&RESULTFORMAT=1&author1=sackett&title=oldest+professions+harlot&andorexacttitle=and&andorexacttitleabs=and&andorexactfulltext=and&searchid=1&FIRSTINDEX=0&sortspec=relevance&resourcetype=HWCIT
J
danbeck said,
January 9, 2008 at 10:33 pm
It is an unfortunate reality that when multiple-billion dollars are at stake, there follow attempts to conceal and mislead.
We have seen the cases of the COX-2 inhibitors, where manufacturers concealed the risks to cardiovascular health and with Ezetimibe this seems but another chapter.
As a junior doctor I have often witnessed the marketing spin of the pharmaceutical industry first hand (though I now take pains to avoid it). The only defence we have is knowledge- that is why Ben again deserves praise for bringing this to a mainstream audience.
Many of my peers lamentably fail to utilise their ‘critical appraisal’ skills when faced with a drug-rep sponsored meal or presentation. Every time this happens we are in a small way failing our patients. As this article demonstrates, the desire to market a product that ‘works’ goes all the way to the core.
The damage we have already done to our profession is for all to see; gone are the days of ‘doctor knows best’- and rightly so. If the medical profession are to maintain the trust of the public (still the most trusted profession in contemporary polls), the new breed of doctor must be alert to attempts to deceive and conceal in research. This requires more space in medical student training and a more general acknowledgement of the dangers on the part of medical establishments. Doctors too must shoulder the responsibility and utilise systematic review resources such as Cochrane, Bandolier and BMJ Clinical Evidence.
The sad reality remains that the most a junior doctor may hear about the ENHANCE trial will be the small-print references on the back of the Ezetimibe adverts.
[Competing Interests: For my part I have helped with the inception of Pharmaware, a UK based group that aims to create new health-care professionals that interact responsibly and ethically with the pharmaceutical industry.]