Ben Goldacre, The Guardian, Saturday 17 July 2010
This week the FDA voted not to ban GlaxoSmithKline’s diabetes drug rosiglitazone (brand name Avandia). Their vote has been reported as a victory for the company. I don’t think so: this saga tells an ugly story about our collective medical incompetence.
Rosiglitazone was first marketed in 1999. From the outset it was a magnet for disappointing behaviour: in that first year Dr John Buse discussed an increased risk of heart problems at a pair of academic meetings. He was silenced. GSK made direct contact, then moved on to his head of department. Buse felt pressured to sign various legal documents and, to cut a long story short, after wading through documents for several months, in 2007 the US Senate committee on finance released a report describing the treatment of Dr Buse as “intimidation”.
In 2003, the Uppsala Drug Monitoring Group of the World Health Organization contacted GSK about an unusually large number of spontaneous reports associating rosiglitazone with heart problems. GSK conducted 2 internal meta-analyses of their data on this, in 2005 and 2006. These showed that the risk was real, but although both GSK and the FDA had these results, neither made any public statement about them, and they were not published until 2008.
Why then? In 2004 GSK were caught – famously – hiding data showing side effects of the antidepressant paroxetine in children: a court settlement required them to post all clinical trial results voluntarily on a public website. Using this data source, cardiologist Professor Steve Nissen and colleagues published a landmark meta-analysis in 2007 showing a 43% increase in the risk of heart attack on rosiglitazone. People with diabetes are already at increased risk of heart problems.
The FDA found a similar risk in their own calculations, but voted in 2007 to keep the drug on the market. This is not insane: diabetes is tricky, 300m people have it worldwide, a great many people die from it, and rosiglitazone is unusually good at controlling blood sugar. Lots of dangerous drugs are kept on the market and then used less frequently, in extreme circumstances. A consensus algorithm for treating diabetes from the American Diabetes Association and the European Association for the Study of Diabetes, meanwhile, unanimously recommended against rosiglitazone. Although annual sales for rosiglitazone fell, they still remained over $1 billion.
Concerns continued to mount. So did the bad behaviour. In 2007, for example, by a stroke of luck Nissen caught GSK out discussing a copy of his unpublished paper, which they had obtained improperly.
Then on 28 June this year Nissen published an updated meta-analysis of 56 trials in over 35,000 patients. Again it found an increased risk of heart problems. GSK’s response to all this has been – and there is no harsher way to say it – like the responses you get from homeopaths. There are 7 trials since 2007, they said, showing no excess risk: fine, except there are 56 which collectively do show an excess risk. There is this other meta-analysis, they said, which looked at 164 trials: fine, except it’s published in a fairly obscure journal, and it looked at trials lasting more than 4 weeks, when the others set the bar at trials over 24 weeks, because a heart risk takes time to develop, and including 4 week trials will obviously dilute this effect, if it’s real. In any case, this other meta-analysis is not brilliant for GSK’s case, since it points out that the company denied access data from 6 trials which we know to have taken place [table 3 appendix].
There is no excuse for companies witholding data from academics and doctors. But most revealing, as ever, are the deeprooted flaws this story exposes in our rather ad hoc systems for gathering, analysing, and disseminating evidence on risks and benefits of treatments.
This drug has been on the market since 1999, and it has seen billions of dollars of sales every year. There has been plenty of real patient experience of this treatment, but we have failed to capture it for analysis. Most of the trials included in these meta-analyses weren’t specifically designed to look at heart problems, and so the data on these, collected incidentally, is unpredictably inaccurate.
In an ideal world, for every patient, wherever possible, we could be gathering anonymised outcome data and comparing this against medication history, making exceptions only for those who put their anxieties about privacy above the lives of others (I will have this argument with you any time). In an ideal world, wherever a patient is given any treatment, and there is genuine uncertainty about which treatment is best, they would be simply and efficiently randomised to one treatment, and their progress monitored. In an ideal world, these notions would be so routinely embedded in our notion of what healthcare looks like that no patient would be bothered by it.
This isn’t fanciful, or difficult, or disproportionately expensive. Instead we have unsatisfactory hotchpotch of incomplete monitoring systems and unforgivable secrecy. Medical successes and failures are one of the biggest factors impacting on whether people live or die. Living and dying matter a lot. We have a failure of vision.
CoralBloom said,
July 17, 2010 at 12:48 am
‘We have a failure of vision.’
And a failure of ethics too.
We can do better, but money and greed block out the light.
cenderis said,
July 17, 2010 at 12:48 am
According to Kaiser Fung, things are much more clear cut about whether to take Avandia: junkcharts.typepad.com/numbersruleyourworld/2010/07/trust-no-one-when-it-comes-to-the-numbers.html
danielrendall said,
July 17, 2010 at 1:14 am
Hang on, Ben, aren’t you supposed to be in the pocket of big pharma – ‘an ass who makes money from pharmaceutical giants’ as of our more prominent intellectuals described you recently? I don’t think they’ll be paying you for this!
Sili said,
July 17, 2010 at 1:23 am
I’d be interested in hearing more about how we’d ensure proper randomisation. Also for the case of sentencing as you’ve suggested before.
How do we ensure that GPs and judges don’t – wilfully or inadvertently – show bias in what patients/criminals they assign to the trial? I mean, it’s all fine and dandy that there are two equally valid treatments/sentences, but what’s to stop them from systematically assigning a third treatment/sentence to a non-random section of the population?
orpheus66 said,
July 17, 2010 at 3:42 am
My father-in-law was one of those whose heart attack can easily be attributable to the drug. Sadly, my mother-in-law wants to “leave well enough alone” and has never pursued legal recourse. But the fact remains that my wife lost her father because this drug has been left on the market.
Mark said,
July 17, 2010 at 7:07 am
“…showing a 43% increase in the risk of heart attack on rosiglitazone…”
I thought relative risk ratios were just about the worst way of providing information on risks (whilst providing the largest numbers).
TwentyMuleTeam said,
July 17, 2010 at 12:51 pm
Mannucci, et al: “Conversely, treatment with rosiglitazone is associated with a relevant increase in the risk of heart failure, particularly in insulin-treated patients.”
Nissen, et al: “*** the totality of randomized clinical trials continue to demonstrate increased risk for MI although not for CV or all-cause mortality *** current findings suggest an unfavorable benefit to risk ratio for rosiglitazone.”
As the public, I need to know more about what “associated with” and “unfavorable benefit to risk ratio” might mean. Were I Big Pharma, I’d already know: just shut up and don’t get in road between me and my bank.
lasker said,
July 17, 2010 at 2:13 pm
I am not surprised that GSK are currently trying to find evidence against Rosiglitazone’s harmful effects as they must be worried about defending a class action. Obviously clinicians should not be so biased and use of this medication, as with any other, should be guided by a balance of probabilities.
Your ideal world seems problematic. It would require patients to routinely accept a greater degree of uncertainty in their medications and from their doctors than they are currently accustomed to. Many would not be sufficiently sophisticated to accept this uncertainty as preferable to a spurious certainty. Not quickly, easily or cheaply at any rate. If an enlightened country were to introduce it then drug companies might delay the introduction of new, useful medications in that country so as to make easy money elsewhere.
hardindr said,
July 17, 2010 at 4:31 pm
Ben:
I thought you might be interested in this blog post, which is critical of you d-squareddigest.blogspot.com/2010/07/one-for-philosophers-me-from-couple-of.html . I generally like the output from this blogger, but I think he is quite wrong in this instance and that you might like to engage him in a dialogue/debate. Comments can be left here www.blogger.com/comment.g?blogID=3699020&postID=2358563306751101775 .
I was disappointed that you didn’t make it to this year TAM! in Las Vegas. It was a lot of fun and I would have loved it if you could have autographed my copy of Bad Science. Maybe next year.
Cheers!
hardindr said,
July 17, 2010 at 4:34 pm
Ben:
Also this blast from the past, particularly the comments section:
unspeak.net/evidence-based-medicine/
muscleman said,
July 17, 2010 at 4:57 pm
@TwentyMuleTeam
‘Associated with’ means the study found a high risk in insulin dependant people who also took rosiglitazone. But no more than that, the study design being insufficient to prove causation. Proving causation would require a double blind case controlled study at the very least.
‘Unfavourable risk ratio’ means they calculated that the risk of heart attacks outweighed the benefits of taking rosiglitazone. These things are a balancing act, all drugs have risk and all have a risk-benefit analysis. For eg the maximum daily dose of paracetamol is just below the level where it becomes toxic to the liver. But paracetamol taken below this level is a very useful drug. Ibuprofen like most NSAIDs should not be taken on an empty stomach, but it is much less harsh on the stomach than aspirin.
JQH said,
July 17, 2010 at 6:05 pm
Tut tut. Your “pharmaceutical giants” paymasters will be most displeased with this.
Must remember to link to this next time some homeopath or nutritionista starts banging on about their critics being in the pocket of Big Pharma.
chinaphil said,
July 18, 2010 at 12:44 pm
Apropos of this, I’d love to know what Ben thinks of sites like Patientslikeme.com – I watched a TED talk about it, and thought it sounded really interesting, but I don’t know how much the obvious self-selection would affect statistics. Is self reporting the next big thing?
geridoc said,
July 18, 2010 at 6:26 pm
The rosaglitazone affair definitely illustrates the importance of using electronic data to measure the real world outcomes of medicines after they are approved.
This case does make one wonder if the FDA applies the right standard in deciding whether a drug should stay on the market. In the case of rosiglitazone, it is not clear there are any patients in which the benefits of the drug clearly exceed the harms. Even in patients with difficult to control blood sugars, is it really reasonable to believe the benefits of better blood sugar control outweigh the potential cardiovascular harms? Especially when insulin is a reasonable alternative?
Given what we know now, would rosiglitazone be approved? My guess is no.
PhDChem said,
July 18, 2010 at 11:54 pm
One thing I’m very interested in, and haven’t seen anything about is whether or not the reviewer of the paper who forwarded the work to GSK was censured or not. Breaking the confidentiality of the review process should result in that reviewer being named and shamed to the academic community at large and they should no longer be allowed to be a reviewer for quite some time. Sadly, it seems that this person has gotten away with it.
Michael Gray said,
July 19, 2010 at 6:28 am
Ben, why don’t you ever beat up on BIG PHARMA?
Once again your bias against
bogusalternative therapies is showing.Hang on… 😉
AdamJacobs said,
July 19, 2010 at 11:44 am
I agree that there is no excuse for drug companies (or anyone else for that matter) withholding data. If you’ve done a clinical trial, the data should be in the public domain once the trial is over.
However, not sure about the idea of randomising patients whenever there is any uncertainty. You’d end up with a whole bunch of very heterogeneous randomised trials. It’s a nice idea in principle, but I don’t see how it could work in practice. Who would pay for it? How would you decide when to analyse the data? Would the trials be blinded, and if so, how long would patients be on a treatment before they could be unblinded? What if patients are not responding and need to switch to a different drug?
I think there are too many practical problems for it to work.
T.J. Crowder said,
July 19, 2010 at 2:08 pm
@Mark (#6): I flagged that up too, was surprised to see Ben using that kind of stat standalone (e.g., without indicating the risk went from 1:1000 to 1.43:1000 or whatever [those are not real numbers] something like that).
@Ben: Despite the above, this is your best post in months. The devil as ever would be in the detail, but as a vision it’s first class.
T.J. Crowder said,
July 19, 2010 at 2:41 pm
“…, making exceptions only for those who put their anxieties about privacy above the lives of others (I will have this argument with you any time)…”
I’d be very interested in seeing a post — heck, a book — solely on this topic from you, I think it would be challenging and useful reading (and probably challenging and useful writing as well). I’d like to see the medical argument, but also how you would then relate that same principle outside the medical arena. If anxiety about privacy shouldn’t be put above the lives of others in medicine, what about counter-terrorism? Community policing? Motoring? Spousal and child abuse?
A key aspect of such a system would be anonymizing the data without losing valuable information. For instance, if I’m diabetic and am enrolled in data collection about a diabetes drug, and it turns out I’m also alcoholic but my doctor doesn’t pick up on that (because like any addict I’m lying to him about my units) and it’s not until I see another doctor much, much later that it’s revealed, how do we ensure that the data is updated to flag patient X as an alcoholic? (What with alcohol having a fairly significant effect on blood sugar stability.)
Capturing that data without compromising anonymity, or more to the point, without creating a massive database subject to failures and abuse, would be extraordinarily difficult in at least four areas I can think of: Procedurally, technically, politically, and socially.
I’m not saying it can’t be done, and I’m certainly not saying that it wouldn’t be fantastic if possible to track effects of health inputs (including drugs) on outcomes; such a thing could be a huge boon to all of us. I’d just really like to see a well-thought-out discussion of how it can be done, touching on at least those four areas, and of how it can and can’t, should and shouldn’t, be generalized beyond medical outcomes.
Guy said,
July 20, 2010 at 9:27 am
Although it would have been better if Ben had expressed it as the Number needed to harm (NNH) rather than percentage increase in heart attacks, you have to remember that most older diabetics will die from heart attack. So a 43% increase in the most common cause of death in this group is a major impact, however expressed.
Gullshore said,
January 10, 2011 at 11:01 pm
Leaving aside GSK’s supposed ethics for the moment, I’m an ex-Rosiglitazone patient. With any drug, it’s a matter of assessing the benefits against the unwanted side effects. I accept that the latter have triumphed in the case of Rosiglitazone, but for the record, when I was on it I felt excellent. Since I’m now relying on insulin and metformin, managing blood sugar is achieveable but much more difficult. Its a pity that someone can’t come up with something with Rosiglitazone’s benefits but without its side-effects…Ho Hum
arouet760 said,
October 4, 2011 at 5:06 am
The design of clinical trials working in the doctors office is being planned in the USA. When doctors type in a prescription, some medications will trigger an invitation to randomize the patient to one of the available therapies. As one drug shows benefit over another, the randomization coin gets “bent” to aim more patients to that therapy.