Why – and how – I wrote Bad Pharma

October 8th, 2013 by Ben Goldacre in bad science | 17 Comments »

This is my piece for Waterstones Book Club, where I was asked to write about why – and how – I wrote Bad Pharma. The full book club caboodle is here, and you can buy the book here.

Here it is…

++++++++++++++++++++++++++++++++

I wrote this book because we need to fix a set of problems that have been allowed to persist in my own profession – medicine – for far too long. Trial results can be withheld from doctors and patients, quite legally; trials are often poorly designed, or biased towards the sponsor’s product; doctors are misled about which treatments work best; and so on. These problems have a real impact on patient care, because we don’t have the information we need to choose the most effective treatments for patients. Often, we tolerate actively misleading information.

I’ve spent a long time, as a doctor, wondering why these problems have been able to persist for so long, especially since they’re all routinely documented in the academic literature, and they’re all perfectly fixable. Drug companies could easily turn a profit, without misleading doctors, or hiding unflattering data.

Complacency is clearly part of the explanation. Although we work with life and death, doctors, academics, pharmacists, patient groups, regulators and the rest of us in medicine are the same as any other set of workers. We can all easily fall into a rhythm of “getting by”, responding reflexly to proximate incentives: getting the next grant, getting the next academic publication, getting through the next clinic, processing the paperwork, and keeping our heads above the waterline.

Once you get into the detail, it’s easy to see how the problems described in Bad Pharma have persisted, because they exploit the small incentives in peoples’ everyday lives.

Doctors don’t want to pay their own money out, for example, to stay up to date with the new evidence on new treatments: not in their fifties, not when they feel like qualified doctors, and not if they don’t have to, with mortgages and their kids’ university fees to think about. A few education sessions from a drug company seem harmless enough, especially when all your friends do it, and you think you’re clever enough to spot a ruse.

And regulators have plenty to worry about with their own decisions, and the petty legal squabbles they fight every week with drug companies: they don’t want to open up a new front, and try to force companies to share their results with doctors and independent academics. Especially not when that information could be used to second guess, or criticise, the regulatory decisions made in the first place.

But more than that, these problems have persisted because there haven’t been enough people from outside medicine, peering in and asking us the embarrassing questions. Time and again, at public events and over email, people have asked me: why are people allowed to withhold trial results, and why didn’t I know about this before?

newbookThe best answer I can give you is that it takes a few dozen pages – a good hour of your life – to get a good understanding of why this problem even matters. Then it takes a few hours more reading to trust me when I say that the problem hasn’t been fixed, despite the protestations of industry and regulators to the contrary. In fact, I don’t think of myself as an angry person, but this is where my patience breaks down: it’s bad enough that these issues persist in medicine, passively; but when people then actively claim that they don’t exist, they are standing in the way of efforts to fix things and move forwards.

This, I think, is why people throw the book across the room so often. When they read, for example, in the last chapter, “Better Data”, about the “Ethical Standards in Health and Life Sciences Group” – the great and the good of British medicine and academia – who stand up and say outright that there is a robust regulatory framework around trial results, people are rightly appalled. I get emails on that one paragraph, spitting fire. People gasp when you put that organisation’s quotes up on slides at public talks. And yet, when I’ve tried to talk to the people who represent doctors and academics in that Group – to explain the harm that they’ve done, not just to medicine, but to our collective reputation – they seem to struggle.

So this, in a very roundabout way, is why I wrote the book: I think we in medicine need your help, the public, to see the wood for the trees. I think we need you to make us fix these broad, birds-eye-view issues, where we have failed.

But there’s a cheerier reason. I’ve always thought that explaining how science goes wrong is the best way to explain how science really works. There is a beauty in the clever ways that trials can be rigged by design, and it speaks to the reasons we do trials in the first place: because we want them to be fair test of which treatment works best.

And before you even get that far, to understand how evidence based medicine has been perverted, you have to understand how it’s supposed to work in the first place. I’m constantly staggered that this isn’t taught in schools, explained on telly, and understood by all normally educated people, as the parts of the body or the plays of Shakespeare are. Given so many patients’ endless preoccupation – quite undestandably – with side effects, it’s amazing to me that there aren’t clear and accessible explanations of exactly how we monitor and measure side effects flying off the shelves already. Secretly, that’s what this book is, too: a basic primer in epidemiology, the science we use to find out what’s good for us, or bad for us, in medicine. But a primer with bad guys, to keep you reading.

Lastly, it’s a book because it had to be. There are lots of things in science that you can explain in twenty minutes, and there’s nothing wrong with that. There are lots of books that are a string of twenty minute vignettes: fair enough, if you want to gorge on one writer’s vignettes. But the interesting thing to me, about the problems in Bad Pharma, is that they are all inter-related. That means they all reinforce each other, but it also poses some interesting challenges when you come to explain them. Computer programmers talk about “dependency issues”: you need to define A, B, and C, in a programme, before you can define G, H, and I. In medicine, and in Bad Pharma, you have the same problem, but you often also need to know a little bit about H and I to understand A and C. Things like this preoccupy me, and create rooms filled with post-it notes and string. It also explains why there’s a tiny bit of repetition, here and there, because it’s a book that had to be accessible to people who know nothing about the subject area, as well as people who think they know it backwards.

My publishers have asked me to talk about how I wrote the book, but as you can see, that will rapidly descend into talk of tools. What follows is more detail than anyone needs, and probably more than I should confess.

Broadly speaking, my life is spent hoovering up information, loving it, filing it, and using it. I read a lot through “Feedly”, which lets me subscribe to multiple journals, blogs, and other news feeds. I also pick things up from twitter, mailing lists, conferences, and conversations. When I stumble on anything I might want to use again – an academic paper, an insight, a thought, an explanatory framework, an author I want to read more from – I store it in a service called Evernote, which synchronises my notes across my phones, tablets, and laptops. I’m obsessed with devices and systems, and I’ll cheerfully spend four hours automating a task that could be done by hand in two minutes.

In Evernote, then, I have piles of things in queues, waiting to be pulled off and worked on: a bank of ideas that would work for single articles; a pile of “bad graphs” that I’ve not got a use for yet; piles of notes for books I’ll write one day; ideas for academic studies someone needs to do; and so on. When I sit down to work, I can go to those piles, and get going. Meanwhile, any academic papers also go into Zotero, an amazing free and open source reference manager, that lets you store papers in a hierarchy by subject (and builds bibliographies very neatly, when you’ve cited hundreds of them).

People sometimes ask how long it took to write Bad Pharma, and there’s no clear answer, because it can’t be disentangled from this ongoing game of populating the giant, delicious, monstrous, synchronising ecosystem of knowledge that lives and breathes across all these electronic devices and services. Sometimes I feel overwhelmed by the sheer scale of the information that’s available to us all now: it can make some people feel quite anxious (it makes me feel like I want to live forever). But working in this way has made me realise that putting a frame around disparate facts, and constructing an argument, is the one thing that humans will probably always be needed for.


++++++++++++++++++++++++++++++++++++++++++
If you like what I do, and you want me to do more, you can: buy my books Bad Science and Bad Pharma, give them to your friends, put them on your reading list, employ me to do a talk, or tweet this article to your friends. Thanks! ++++++++++++++++++++++++++++++++++++++++++

17 Responses



  1. logihoni said,

    October 8, 2013 at 8:54 pm

    So Ben are there any things which have been proven to be good for health ,ie., inulin quercetin, glutathione Vit D ,Nitric oxide ?

  2. muscleman said,

    October 9, 2013 at 5:36 pm

    @logihoni
    In biology it is all a matter of dose. VitD for eg is necessary for health, deficiencies are bad for you causing things like rickets. But take too much and it has bad effects. Hypervitaminosis D interferes with glucose metabolism as I discovered when I took too much via multivitamins (I must have naturally high levels). It caused me to ‘hit the wall’ on a run in about 10mn of running. It normally takes something between 2 1/2 and 3hours of running to do that.

    Water is absolutely necessary for life yet it has an LD50 in humans and marathon runners at the slower end of the field occasionally die from drinking too much of it.

    Oxygen is also absolutely necessary but again take too much and it will kill you as dead as not enough of it.

    So your question is not really answerable in general. If you are deficient in any necessary nutrient or element etc then obviously correcting that deficiency will improve health in the same way stopping smoking will because you have taken away a harm.

    It is all about balance. In physiology we talk about homeostasis and study how the body seeks to return to it after perturbations such as a build up of CO2 because I have started running.

  3. Ben Goldacre said,

    October 9, 2013 at 5:49 pm

    I pour out everything I can into the article above, on pharma, distortion of evidence based medicine, how professional bodies can be led astray, strategies for organising knowledge.. and you come in to discuss dreary quack remedies. FFS.

  4. wrincewind said,

    October 10, 2013 at 2:06 am

    Hey, since you talk about using Feedly and twitter, i thought you might be interested in this site – it turns twitter timelines into RSS feeds that you can plug into feedly. they only update once every hour or so but it’s a good way to keep up with everyone you follow.

    www.rssitfor.me/

    I hope it helps you in the future!

  5. beeflin said,

    October 10, 2013 at 8:25 am

    I, on the other hand, enjoyed this article because I love science. And I read it on an RSS reader which links to Feedly. Keep smiling, Ben :-)

  6. jf said,

    October 12, 2013 at 11:56 am

    Keep up the good work. I’m one of many researchers I know who read Bad Pharma thinking I knew what would be in there but was surprised/horrified by some of the examples.

    Are you going to make the extra chapter available for those of us that bought the original version like you did for bad science or is it only available through the new edition?

    P.S. Your books page needs updating as it still says:

    ““The Drug Pushers” (4th Estate) is my book about the misuse of evidence by the pharmaceutical industry, especially the way that negative trial data goes missing in action. It’s out in 2012.

    www.badscience.net/books/the-drug-pushers/

  7. davidrodgers said,

    October 15, 2013 at 9:46 pm

    Hi Ben, longtime lurker here, finally registered. Obviously you put the hours into archiving all the data you’ll need, I can’t quite get motivated when I always fear that formats will be withdrawn/change at short notice. Did you manage to archive your postings from posterous? I gave out some of the links there to patients fairly frequently and it seems a shame that it all disappeared on you (and us!)

  8. errorlady said,

    October 17, 2013 at 4:58 am

    I have been very impressed at the clarity and vividness of the statistical fallacies explained in Bad Pharma, in relation to such (normally) esoteric topics as cherry-picking, multiple modeling, hunting and searching for significance and a variety of other ‘selection effects” that alter error probabilities. I have been writing on these topics from the view of a philosophers of statistical science for many years, and I think Ben brings the gist into a more perspicacious focus than many of my own attempts! So I’m grateful for it. errorstatistics.com

  9. Ben Goldacre said,

    October 17, 2013 at 8:42 pm

    thanks davidrodgers, the posterous stuff will all be back up shortly along with a load of other stuff, after a long overdue site rebuild!

  10. Ben Goldacre said,

    October 17, 2013 at 8:47 pm

    Hi JF, I’ve made sure the extra chapter goes out free to anyone who bought it as an ebook on Amazon, Google Play etc. Most of what I do – writing, lobbying, speaking etc – I cheerfully do for free, books are one of the few things I charge for, so I’m a bit less able to give book content away. You can read about all the new developments at www.alltrials.net/ which is excellent and detailed. And thanks, I’ll fix that books page when I update the whole site soon, just need to find/pay someone to do it well!

  11. tombryce said,

    October 19, 2013 at 6:34 pm

    Hi,I have a ebook copy from amazon. How do I get the extra chapter?

  12. pfflyernc said,

    October 29, 2013 at 4:06 am

    Just finished BP–bravo. I left a glowing review over on Goodreads.

    Just wanted to let you know about a possible error in the text. On p. 248, you indicate that Celebrex was “later taken off the market because it was harming patients.” To the best of my knowledge, it was never actually removed, although there was certainly much discussion along that line (along with Vioxx and Bextra, of course).

    Thanks for your valuable public service. Keep up the good work!

  13. ajpatterson said,

    November 14, 2013 at 4:31 pm

    As tombryce asked, I bought the book on Google Books — how do I get the extra chapter? What is the extra chapter named, that would make it easier to tell when I *do* have access to it :)

    Thanks!

  14. Ben Goldacre said,

    November 21, 2013 at 3:22 pm

    thanks, that was a brainfart and changed as soon as i got an email about it (about a year ago!) sorry your edition must be v early!

  15. SSRIuser said,

    December 10, 2013 at 10:30 am

    I’m listening to the audio version of Bad Science. In the book, there are short comments on the lack of evidence for a positive effect of SSRI on depression. I don’t agree with some of these comments, so I read your 26 January 2008 article on the subject. To start with, it does not surprise me that Pharma cooks the books on trail results. It would certainly surprise me if they didn’t. I also know of people with depression that have used SSRI’s without positive effect. Even though I’m aware of the fact that my own experience with a drug does not prove that it works, I believe strongly that the SSRI sertralin cured my depression. I suffered from both anxiety and depression. This is often the case and SSRI’s are prescribed for both conditions. I believe that the SSRI relieved the anxiety and thereby cured the depression. I should also mention that many of my relatives suffer from different forms of anxiety. One of my parents was hospitalised for it. That parent required shock treatment before partial recovery and release from the hospital.

    Anxiety as an illness is not the same as everyday worries. It often leads to curious behaviours like incessant hand washing. I went back to check the lock three times every time I left the house. In extreme cases, suffers kill themselves. I think I would have done that if I hadn’t found effective help. This form of anxiety is very painful and therefore it results in depression. Physiatrists also admit that it is very difficult to diagnose psychological illness precisely because symptoms are often linked. This brings me to the problem of interpreting drug trial results. Some of the uncertainty of trial results may be due to the fact that the participants in fact suffer from different diseases with the same symptoms. There are probably patients with diseases that can be relieved by SSRI’s. If they choose not to use SSRI’s because of your comments, then you are doing something wrong. If they choose an alternative treatment you may be driving patients to the dodgy practitioner you want us to avoid. Therefore, I would encourage you to be more balanced in your comments on SSRI’s.

    There are many observations in Bad Science I agree with whole heartedly. One in particular stands out; we as a society are not willing to spend the money to do drug trials in the public domain. Please work to change this situation because, as long as we allow industry to do these trails, we will not get honest results.

  16. EKCarpenter said,

    January 3, 2014 at 3:07 pm

    SSRIuser, I understand your fears entirely, but unfortunately, we’re not in a position to be cautious about recommending for or against SSRIs – because we don’t know what course of action would be the cautious one.

    As you say, some people have experienced benefits from SSRIs, some have experienced nothing, and some have experience of SSRIs making them worse. Without trial data we don’t know which is more common, or what might cause an individual person to have one or the other experience.

    So, your answer to that is that we should avoid criticising SSRIs too harshly, because we might inadvertently lead people to not take them, to their detriment. Fine in principle, but the problem is that we don’t know, on the larger scale, if people not taking SSRIs would be to their detriment. So there are six possibilities, if you’ll forgive my lawyerly reductio ad ridiculum:

    1. We criticise SSRIs, fewer people take them, and in fifteen years we have a decent trial (hopefully) which shows that SSRIs are a good treatment for depression. So we’ve harmed people.

    2. We criticise SSRIs, fewer people take them, and in fifteen years we discover they have no effect. People have tried other, effective treatments instead. So we’ve helped people (more by luck than judgment).

    3. We criticise SSRIs and then discover they make depression worse. Fewer people have taken them, and they’ve tried other treatments instead, so we’ve helped people.

    4. We don’t criticise SSRIs, then find out they’re beneficial. We’ve helped people.

    5. We don’t criticise SSRIs, find out they have no effect. People have relied on SSRIs instead of effective treatment. We’ve harmed people.

    6. We don’t criticise SSRIs, find out they make depression worse. People have taken harmful SSRIs instead of effective treatment. We’ve harmed people.

    And then, of course, you have to add in the layers of complexity which arise from all the different SSRIs there are, which I’m not even going to attempt.

    So on the balance of probabilities, we might be better off criticising, and that, I think, is a big part of evidence-based medicine – it’s why you assume something doesn’t work unless there’s evidence that it does, rather than the reverse. But since we have no idea whether SSRIs are good, neutral or bad, we can’t make a sensible decision about whether to criticise or support them. Certainly, criticising them isn’t the wrong approach, with the amount of information we currently have. And that’s very, very frustrating, especially for someone who’s had a good experience with SSRIs, because we’d all rather be able to say “Try it, it might help!” than “Don’t try it, it might or might not help but it will probably reduce your chances of recovering overall”; especially with an illness as crippling, and as difficult to treat, as depression.

    Sorry about the novella. I have some statistics work open in another window which I’m sick to the back teeth of.

  17. johewitt said,

    May 15, 2014 at 2:36 am

    I am currently studying towards a Chemistry degree in my spare time and so am finding this book a fantastic introduction to picking my way carefully through research. I particularly like your passion for what you do and the real efforts you are putting in to try and improve things.

    I was interested in whether the real problems with medical studies are also seen in other areas of science. For example, in the development of new materials?

    My husband works as an engineer within the nuclear industry where things have got to be right. The UK nuclear industry is one of the most heavily regulated in the world and they have on site inspectors with warrant cards who can shut down the plant at a moments notice. So safety really does come first. The company he works for actively encourages everyone to question things and there are numerous checks and balances in place.

    I asked my husband how they assessed, for example, a shiny new piece of equipment that a company is claiming is the best thing since sliced bread. He told me that they are encouraged to try and ‘break’ the equipment in testing and to see for themselves where the limits are before it is fitted.

    Now imagine if medicine could work like this? If a questioning attitude were encouraged and pharma companies could be closed down at a moments notice if they caused harm?

    My theory is that the nuclear industry, despite being largely safe is perceived to be unsafe because of the confusion with likelihood /severity of risk in people’s minds. For example, I read some figures which suggested that of people who lived around Chenobyl at the time of the accident, the majority of their long term health problems were mental health problems. The majority of deaths at Fukashima were from the tsunami, not the nuclear incident.

    This perception though has forced the nuclear industry into being highly regulated, which is a good thing. It improves practice and raises standards as well as being safe. So it’s a win/win.

    I wonder if how the general public perceives the risk of problems in medicine is contributing to the problem. Although the risks are likely, they never seem that severe because it happens to someone else.

    I do think it’s not just up to the medical profession or Big Pharma to improve things. The general public have a responsibility to question things and push for change. Public perception can be a powerful motivator.

    Good luck with the rest of the campaign. I feel that in the last few years there’s been a real shift in power away from the old notion of never questioning authority. The public in general now demand more transparency and accountability. Look at the scandals with MPs expenses, Hillsborough, the Catholic Church. So I feel like Big Pharmas time will come.

    Jo

You must be logged in to post a comment.