My book Bad Pharma documents serious ongoing problems in the pharmaceutical industry. In particular, I show how vitally important information from clinical trials is still being withheld from doctors and patients, right now, today; and that patients experience avoidable suffering and death as a consequence. There is no nicer way to express that reality, and this ongoing problem of missing data is meticulously documented in the book, because it is widely discussed in the medical literature.
I am very sorry to say that the ABPI, the UK pharmaceutical industry PR group, has now responded, and their only response has been to flatly deny what is plainly and provably true. Here is their press release, titled “ABPI statement on Ben Goldacre’s book Bad Pharma”. Most of it explains that drug companies are great. Only one paragraph addresses the serious concerns I have raised:
“The issues presented in Ben Goldacre’s book Bad Pharma are important. They impact on society’s ability to deliver innovative medicines to solve areas of unmet clinical need and improve the quality of lives for countless patients and their carers. We must remember though that the examples he refers to are long documented and historical, and the companies concerned have long addressed these issues.”
This is simply not true. The examples I refer to – Tamiflu, the failure of clinicaltrials.gov, and more – are contemporary, and very much ongoing. They are set out in plain English in the very first chapter of the book. It is extraordinary and bizarre that anyone should claim that these problems are historical.
The ongoing story of Roche withholding vitally important information on the risks and benefits of Tamiflu is just one illustration, but it is meticulously documented, and a good account can be found in this open access journal article. Here is the BMJ’s ongoing Open Data campaign page on Tamiflu. There is absolutely no sense in which this ongoing scandal is historical or resolved: Roche continues, today, to withhold this information from doctors and patients. For this, Roche should be publicly criticised by the ABPI. At the very least, one might expect the ABPI to remain silent. Instead, the ABPI claims that these problems are historical and have been fixed. This is an extraordinary and demonstrably false claim.
Tamiflu is just one example. The most current systematic review on “publication bias” – the phenomenon whereby trials go missing in action – can be found here. It shows that overall, around half of all the trials that are conducted and completed go unpublished, and that trials with “positive” or flattering results are about twice as likely to be published as trials with negative results.
This is an ongoing problem, and as a result, we cannot know the true effects of the treatments we use in medicine. Various things have been done to try and address this issue. All these solutions have been announced with fanfare, and they are always lauded as if they have been successful. In reality, all of these interventions have failed.
In 2004, we saw a charade, as the academic journal editors claimed that they would hold the line, by forcing all companies and researchers to post their trials on a trials registry: in reality, as we discovered in 2009, the journals had repeatedly broken their promise. The problem persisted.
In 2007, the US government passed a law stating that all trial results must be posted on clinicaltrials.gov within one year of completion. Everybody announced that the problem was fixed, once again. But in 2012, an audit was finally published, which showed that only one in five trials had met this reporting requirement. Despite this astonishingly bad performance, no fine has ever been levied against any company or researcher for failing to post results. Even if a fine had been levied, it is only $10,000 a day, or $3.65m a year: a parking ticket to a company with annual revenue in the tens of billions.
Finally, even if these initiatives suddenly became successful tomorrow, that would still not solve the problem. We do not practice medicine using only the results of trials that were completed after October 2012: we use drugs from 2009, 2003, and 1998, from an era when we know the entire evidence base for medicine was hopelessly distorted, by companies withholding vitally important trial data. We need all of this trial data to be released, urgently, today, to prevent ongoing harm to patients.
There is absolutely no doubt whatsoever that unflattering trial data continues to be withheld from doctors and the public, and there is absolutely no doubt that patients suffer as a result. Even more tragically, as I show in the book, doctors, academics, regulators, journals, patient associations, and universities have all failed to adequately address these problems, behind closed doors, over the course of many years.
I believe that if patients and the public ever come to realise what has happened, then they will demand change. I believe that the Royal Colleges, patient associations, universities and ethics committees must do more to protect the interest of patients, by demanding that all trial results are published in full. That is why I am working to give the public the clearest possible explanation of what has gone wrong in medicine, and to set out several simple steps that would start to put these problems right.
I have taken the liberty of drafting what I believe the ABPI should have said, in response to my book, if they were sincere about engaging on this vitally important issue. I hope that they will reconsider their position, and engage more constructively on what is undoubtedly the most important ethical problem facing medicine today.
The ABPI’s outright denial, of ongoing problems that are so well documented, is worrying, because it hints at a wider malaise: that the pharmaceutical industry has learned there are no consequences for claiming black is white. As long as we live in a world where the ABPI can feel confident about claiming, quite falsely, in the face of all the published evidence, that these problems are in the past, then patients will continue to suffer unnecessary and avoidable harm. I truly wish this wasn’t so.
What I believe the ABPI should have said:
“We are pleased to see that Ben Goldacre’s book recognises the valuable contribution made to drug development by the pharmaceutical industry, and that our members are not motivated by evil intentions. We also recognise, though, that much harm has been done by unethical activity in our industry, however unintentional; and that the evidence clearly shows these problems are ongoing.
“Despite some international effort over the past two decades, many trial results continue to be left unpublished. This undoubtedly misleads doctors and harms patients, and this issue must be addressed. Members of the ABPI must continue to work harder to strengthen international legislation, and we call on the industry to ensure that all information about trials is shared with clinicians, academics, and patients. In particular we condemn Roche for continuing to withhold information on Tamiflu from Cochrane, though this is not an isolated incident.
“Lastly, we recognise that the harm done to patients by withheld trial results cannot be adequately addressed by ensuring only that future trials are fully published. Patients continue to be put at risk by the withholding of trial results over the past decades. Fixing this will not be easy. But we are committed to encouraging industry to cooperate fully with an amnesty on all previously withheld trial results.
“This will not be easy, and of course the industry is not all bad: this can be seen in the lifesaving products produced across all disease areas. But we strive to improve.”
I am sorry to say that tomorrow I will be sharing another, much more troubling response to the book, from a different organisation.
AdamJacobs said,
October 11, 2012 at 8:57 pm
I’m puzzled about why you claim with such certainty that the problems of publication bias have not been fixed.
The “most current systematic review on publication bias” that you cite was published in 2010, but reviews studies that were mostly done several years earlier than that.
Where is the *recent* evidence on publication bias?
Surely the honest response to this question is “we don’t know for sure whether the problem has been fixed or not”, rather than asserting that it definitely hasn’t been fixed based on data that come from a period when everyone agrees publication bias was a massive problem.
BTW, if you want to find the Tamiflu data, it’s here: tamiflu.clinit.net/ Not sure why you think it’s still being withheld.
rjmunro said,
October 11, 2012 at 10:36 pm
Perhaps you could sue them for Libel. You’ve probably only got a few months left before the libel reform laws come in.
Ghostsigns said,
October 12, 2012 at 2:26 am
Looks like a cover up to me. They’ve been rumbled and, just like News International and phone hacking, they are trying to dismiss it and move on. Hopefully this response will initiate a similarly detailed interrogation of the issues, vindicate you, and find this organisation severely wanting in terms of their credibility and ability to represent this industry.
pfsc12 said,
October 12, 2012 at 10:49 am
I have not read BP yet, but plan to do so as I enjoyed BS so much but understand the gist of what is being said here. I am not clear if there is any interest or commitment from our public servants and professional bodies to consider this issue.
It is easy to be sucked into finding and railing against bogeymen but I am interested in knowing what is stopping or preventing our public servants from doing the right thing to bring serious health matters into the open.
We vote for them and fund them, so perhaps Jeremy Hunt has a personal opinion about this topic too that he will shall with Ben Goldacre?
BTW – can we replace all future use of the Americanism “ongoing” by the English word “continues” or “continuing” please as my annoyance on this abuse of our language continues.
pfsc12 said,
October 12, 2012 at 10:51 am
whoops – ‘share with Ben Goldacre’ not ‘shall with’
Ben Goldacre said,
October 13, 2012 at 3:26 pm
AdamJacobs.
The most current systematic review is clear: publication bias is a huge problem in medicine. You assert, as someone working in industry, that things have changed since 2010. It is for you to present good quality systematic review evidence showing that this is the case. There are no theoretical grounds to believe that things have improved. We know from Prayle et al in 2012 that the rules for posting results on clinicaltrials.gov within a year have been ignored by four out of five trials. And more recent studies of publication bias than the 2010 review show that trials continue to go unpublished.
Furthermore, as I have said on many occasions, fixing the problem from now is not enough, we need all the withheld results, from all the trials that have ever been conducted, for all drugs in current use.
Roche continue to withhold information on Tamiflu and it is untenable to claim otherwise. The gaps in the limited information they have shared online are documented in the book, where you will also have read descriptions of Roche’s appalling behaviour, delaying the release of even that unsatisfactory information, and deploying bizarre inappropriate excuses. Here is the BMJ’s ongoing campaign to get Roche to finally release *all* the relevant information.
www.bmj.com/tamiflu
Trial results continue to be withheld. This inflicts avoidable suffering and harm on patients. The problem will not be fixed for as long as people in industry feel safe in denying what is so plainly documented as true. This is very, very sad.
David Hallett said,
October 13, 2012 at 5:32 pm
The BMJ review you cite as showing that only 20% of trials have results posted at clinicaltrials.gov actually shows that pharma does far better in this regard than independent researchers. 40% of industry-sponsored trials had results posted, and phase III and IV trials (which actually affect patient care) were much more likely to have results than phase II trials (which don’t). We can deduce that at least half of pharma phase III/IV trials have results posted, and probably rather more, as some were probably just delayed rather than withheld.
So while the situation is far from perfect, your implication that 80% of trials don’t have results posted because pharma companies are trying to bury unwanted data is clearly misleading. Far from showing that “the problem hasn’t gone away”, it suggests the companies are actually making efforts to comply, though I agree they could do better.
Did you not read the paper, or were you just trying to (ahem) suppress unwanted data in order to increase sales of your product?
Dr_MC said,
October 14, 2012 at 1:26 pm
Hate to be a pest..but when are we seeing this more troubling response? Or have they retracted/replaced/threatened to sue?
P.S Just ordered my copy of Bad Pharma. I’m sure, like bad science, it will be made compulsory reading for my colleagues at the practice. There’s a healthy questioning of all the latest rubbish proposals that are set to us now. It’s a sceptic’s (in the scientific “questioning” sense) dream.
AdamJacobs said,
October 14, 2012 at 3:25 pm
Thanks for the reply, Ben.
“The most current systematic review is clear: publication bias is a huge problem in medicine.”
No, what that shows is that publication bias was a huge problem in medicine. The review was published in 2010, but most of the studies it reviews come from many years before that.
“You assert, as someone working in industry, that things have changed since 2010”
If you read my earlier comment again, you will see that I do not assert that. I merely assert that we do not know what has happened to the problem of publication bias in the last few years. If I were claiming publication bias had been fixed, then I agree it would be up to me to provide evidence to support it, but I’m not claiming that.
You, however, are claiming that the problem hasn’t been fixed. Where is the evidence? If you can provide recent evidence that publication bias is still a problem, then I shall believe you. If you can’t, then I shall continue to believe that we don’t know whether it has been fixed or not.
“Furthermore, as I have said on many occasions, fixing the problem from now is not enough, we need all the withheld results, from all the trials that have ever been conducted, for all drugs in current use.”
Now there I agree with you. The problem of publication bias for studies that were completed a while ago absolutely remains a problem, and I would love to see more efforts to fix it. Totally on your side on this one.
“Roche continue to withhold information on Tamiflu and it is untenable to claim otherwise”
Well, this is all a little puzzling. Roche have made available the full clinical study reports on a website: see the link I posted earlier. Perhaps what you mean is that they haven’t released raw, individual patient level data, which may well be a valid criticism for all I know.
A little more clarity on exactly why you are criticising Roche would be useful here. In your book (page 88), you say:
Now, to be fair, I’m not sure when Roche posted the full study reports on a website. Maybe they didn’t do it until after your book went into production. But they have done it now, as you can see quite clearly from the link I posted earlier. If your current criticism is that they haven’t released raw data, it would be helpful to make that clear and point out that your book is now out of date.
Laura Graham said,
October 15, 2012 at 12:34 am
I’m not surprised by ABPI’s response but I am disappointed. I think I would have preferred their silence rather than such a rigid defence. I think we know that pharma is no hurry to improve a “system” which has developed into something so profitable to them. I’ve yet to read Bad Pharma in full (and I am looking forward to reading it), but have read the introduction. I note that you say this is not about bad people (and is about the system) but I have found many of the “medical opinion leaders” to be bad (in their ethics, motives etc). Many are high profile medical professionals who are taking thousands of pounds in research grants, honoraria, and direct payment in order to create a “panic” around a condition, and promote a type of medication to manage that condition. They have centre stage on the conference circuit, have their pharma sponsored research published in pharma sponsored journals (which have become another pharma marketing tool), and as we have experienced recently in the addiction treatment field, are commissioned by government bodies to review how these medications are delivered because of their published research interests, and without ever being asked to declare their pharma interests. Hence, their financial interests in protecting pharma clouds their clarity of vision in said reviews which means nothing changes. Worse still, these individuals are able silence anyone raising concern about this by using the threat of current UK libel law.
There are others who are trying to get these secrets out:
www.testingtreatments.org/
www.jameslindlibrary.org/testing-treatments.html
It is sad that the consequences of this are felt by the patient and in the worse case scenario, their grieving families.
Every best wish with your work.
Ben Goldacre said,
October 15, 2012 at 11:29 pm
Hi Adam.
I’m glad you agree companies need to release all trial results, current and previous.
I think your stance on the current evidence on publication bias is puzzling: there is no reason to believe anything has changed, the Prayle 2012 paper shows the clinicaltrials.gov rules have been ignored, and the individual studies I’ve seen since the 2010 systematic review show that trials continue to go unpublished at the normal, tragic rate.
The Tamiflu story is explained very clearly in the book. Roche are still withholding vitally important information about their clinical trials. This is not about individual patient recordsL that is an important but different issue, as explained clearly in the book. The book explains very clearly why the material Roche have posted online, at the website you link to, is insufficient (page 81 onwards). A good open access resource where others can read about this is here
www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.1001201
The paper above is also referenced and summarised in the book. I don’t really understand why you are ignoring the content of this paper and the book, to claim that the Roche website contains all the information that is needed by Cochrane. It doesn’t.
Meanwhile the BMJ Open Data campaign continues, with its focus on Roche and Tamiflu.
www.bmj.com/tamiflu
Furry Fran said,
October 17, 2012 at 12:09 pm
-The sky is blue.
-I can’t see how you can be so sure.
-I see it every time the clouds clear. I saw it yesterday.
-So your most recent observation was 24 hours ago. It was also based on scattered sunlight that takes several milliseconds to travel through the atmosphere, so your claim is old, based on even older data.
-There is no reason to think the colour of the sky has changed. The laws of physics are the same as they always were, human colour perception likewise. If you’re claiming there is any reason to doubt the colour of the sky, can you present your evidence?
-All you have shown is that the sky was blue slightly more than a day ago. I am not claiming the sky is any particular colour, just that we do not know the colour of the sky at the moment. If you cannot present evidence that the sky is blue now, I cannot see that your claim is supported.
-I’ve just looked outside again and there was a slight gap in the clouds. I could see the blue sky.
-You have not presented comprehensive evidence that the sky is blue right now.
AdamJacobs said,
October 17, 2012 at 2:42 pm
Thanks for the further reply, Ben. I appreciate your taking the time to do so. I do hope my posts aren’t coming across as too confrontational (I will happily admit that you identify some very genuine and important problems in your book, and would like to help fix them), but I do think it’s important to get our facts right.
Anyway…
Completeness of publications
I disagree when you say “there is no reason to believe anything has changed”. The fact is that in the last few years, most of the big pharma companies have developed publication policies that say they will publish all their trials. Here’s GSK’s as an example (pdf link), but other companies have similar ones.
Now, the question remains whether those policies have been effective. Just because the policies exist does not necessarily mean they’ve been followed. The fact is we simply don’t know whether they’ve been effective, because of a lack of recent evidence. But those policies just weren’t there a few years ago, so their creation does give us a reason to believe that things might have changed, and that relying on 10-year-old data isn’t going to be helpful. The Prayle paper doesn’t help here, as it’s not about publication (it was also rather flawed, but since it’s not about publication anyway, that’s not hugely relevant).
I’d be interested to see what references you have that are more recent than the systematic review.
BTW, I’m working on a little research project myself that may shed some light on recent trends in completeness of publication: watch this space.
Tamiflu
I continue to be puzzled. The data on Tamiflu have been made more available than the data on almost any other drug you could think of, so it seems an odd example to pick. You could make those criticisms about almost any other drug (I do agree that full study reports should be made publicly available), and you would be spot on. It strikes me as a bit like saying “The problem with politicians is that they’re all a bunch of posh boys: just look at that Dennis Skinner!”
I don’t really understand what you think is missing. According to the site I linked earlier, Roche have provided the main body of the CSRs (we’re not confusing it with the publicly accessible site where they only report the results summaries, are we? I agree that the results summaries are insufficient). This does seem to contradict the information given on page 88 of your book (which I quoted earlier). Do you stand by that statement on page 88? If so, why?
The PLoS Medicine article that you linked does not actually specify what information the Cochrane reviewers need that they can’t find in the study reports. I find it hard to imagine what information they might be missing. Clinical study reports are, as you rightly point out in your book, much more comprehensive than most published papers, which is all most meta-analysts have to work with. If a full clinical study report does not give enough detail for a meta-analysis, then it’s a wonder how any of them have been done in the past, as most simply have published papers to work with.
In your book on page 87, you say the study reports are missing vitally important information such as randomisation details, analysis plans, the protocol, and a list of deviations. I find that surprising. Most study reports would include that sort of information.
Now, I haven’t read the reports myself, which are on a password protected website and I don’t seem to be eligible to register to download them. Maybe they are just really bad reports that are lacking all those details that a study report would normally contain. If that’s true, then I think I understand your criticisms. Is it? Although if the study reports really were lacking those details, I find it astonishing that Tamiflu would ever have got licensed, as those are definitely the sorts of details that regulators would expect to see.
Tom Jefferson said,
October 17, 2012 at 8:36 pm
Hi Ben, I am troubled by some of the statements made by Mr Adam Jacobs.
In his first posting he gives the link to the Roche website. This was hastily put together after they received the BMJ enquiries on the ten Tamiflu Kaiser trials in December 2009.
Mr Jacobs comments your statement:
“Roche continue to withhold information on Tamiflu and it is untenable to claim otherwise”
by writing:
“Well, this is all a little puzzling. Roche have made available the full clinical study reports on a website: see the link I posted earlier. Perhaps what you mean is that they haven’t released raw, individual patient level data, which may well be a valid criticism for all I know.
A little more clarity on exactly why you are criticising Roche would be useful here. In your book (page 88), you say:” etc
The criticism goes a bit beyond failure to release raw data (whatever Mr Jacobs may mean by that term).
If you click on the Roche website link Mr Jacobs reported in the blog, you see the (pre log in) caption: “The information provided on this site includes Module I, the main body of the clinical study reports (CSRs), for ten Tamiflu (oseltamivir) studies (in total, there are seven stand-alone reports, two of which provide the results of multiple studies). Module I of the CSRs includes a detailed report and appendices with data tables and patient-level safety listings”.
Note the words “Module 1”.
Missing from the site are the remaining modules including protocol, certificates of analyses, SAP and other documents vital for the understanding of each trial. Individual listings are also missing as well as CRFs – all in all perhaps 95% of the available data for each of the 10 trials.
60% of Tamiflu treament trials remain unpublished. Of those which are visible through our (BMJ, Channel 4 and the Cochrane Collaboration) efforts perhaps only 5% of the available data are above the waterline.
This is after Roche’s promise to release “full study reports” in Dec 2009 in the BMJ.
Your blog and book are factually correct. Our campaign aims to hold Roche accountable for (so far) unfulfilled promises.
Best wishes,
Tom.
Colin Forster said,
October 18, 2012 at 3:07 pm
Like 8. Dr_MC, above, I am curious that we have still not seen this “much more troubling response”. What happened?
AdamJacobs said,
October 18, 2012 at 3:15 pm
I’m grateful to Mr Tom Jefferson for clearing up some of the confusion here. That’s very helpful.
So, we all agree that Roche have made the main body of the study reports available (ie considerably more than just the summaries), but the problem is that they have not made available various report appendices, such as protocols and SAPs. Is that correct?
I’m still a little puzzled, however, about what information might be missing that would be necessary for a meaningful meta-analysis. The main body of a study report would normally include all relevant information from the protocol and the statistical analysis plan.
Do these reports not include such information? If not, could Mr Jefferson give an example of what kind of information is missing?
I’m also intrigued by the importance of the missing certificates of analysis. What information contained in the certificates of analysis would be useful to a meta-analyst?
I note that Mr Jefferson is an experienced meta-analyst. I’d be interested to know his estimate of what proportion of meta-analysts have access to certificates of analysis from all the studies they include, and what the consequences are if the answer is “not many”.
I should point out that I’m not trying to defend Roche here. I totally agree that there is no good reason why they couldn’t just put the whole lot on a non-password protected website, and that they have indeed behaved badly. I just find it really puzzling why Mr Jefferson feels that the main body of study reports, which contain far more detail than most meta-analysts could ever dream of, are insufficient in this case.
Carpjaw said,
October 23, 2012 at 2:47 pm
Regarding the paper by Prayle et al in 2012, which Ben cites as evidence that pharma is not complying with the new legislation on mandatory disclosure of results, I am not sure we can make too much of it.
Sure, a low percentage of the companies did not make the 1-year reporting deadline, but this could be more of an organizational issue than anything. The postings on clinicaltrials.gov will usually be based on the clinical study reports, and if these are not available until very near the 1-year reporting deadline (to the health authorities), then I can imagine that delays would be inevitable. So it would be interesting to see what the compliance rate is at, say, one and a half years.
Finally, I would also be interested to know the delay between the end of a study and when a full publication in a peer-reviewed journal usually becomes available. Again, I could imagine that it would probably be more than a year.
Bob Fiddaman said,
October 28, 2012 at 6:18 pm
“I find it laughable that the CEO of the ABPI would defend the industry particularly when, last year, the HM Revenue & Customs [HMRC] were seeking information from pharmaceutical companies regarding payments made to doctors, psychiatrists, key opinion leaders etc.”
More here – fiddaman.blogspot.co.nz/2012/10/in-defence-of-glaxo-stephen-whitehead.html
twaza said,
November 1, 2012 at 10:37 pm
The ABPI’s response to BadPharma is a classic case of “Wilful Blindness”.
www.huffingtonpost.co.uk/margaret-heffernan-/jimmy-savile-epidemic-of-wilful-blindness_b_2044966.html
Margaret Hefferman’s book should be required reading for every politician, journalist, regulator, clinician, manager, BadPharma fan, BadPharma Skeptic, …
PaulH said,
November 5, 2012 at 12:23 pm
‘“The issues presented in Ben Goldacre’s book Bad Pharma are important. They impact on society’s ability to deliver innovative medicines to solve areas of unmet clinical need and improve the quality of lives for countless patients and their carers. We must remember though that the examples he refers to are long documented and historical, and the companies concerned have long addressed these issues.”
This is simply not true. ‘
Actually, it is true. Just misleading.
The examples are long documented. They are historical. The companies have been addressing the issues for a long time, just like the tobacco companies addressed the issue of lung cancer for a long time – they worked really hard to cover up and confuse the evidence.
overlylongusername said,
July 9, 2013 at 5:43 pm
It does seem that pharmaceutical companies may be a attracting disproportionate level of criticism for spinning their results. The manipulation of results for behavioural interventions which can be used to justify the medicalisation of the psychosocial aspects of people’s lives seems even more serious a problem.
This publicly funded trial is refusing to release outcome measures in the manner laid out in their published protocol, and one of the researchers involved has been trying to portray FOI requests as ‘harrassment’.
www.whatdotheyknow.com/request/pace_trial_recovery_rates_and_po
So long as it’s thought acceptable to mislead patients about the efficacy of treatments, we will go on encouraging quackery.