Washing the numbers, selling the model

January 26th, 2008 by Ben Goldacre in adverts, bad science, big pharma, medicalisation, neurostuff, regulating research | 57 Comments »

Ben Goldacre
The Guardian,
Saturday January 26 2008

If there’s one thing I love, it’s academics who take on the work of investigative journalism, because they are dogged. This has been a bad week for the SSRI antidepressants.

First there’s the stuff you already know: bad data got buried. In a cracking new analysis of the “publication bias” in the literature, a group of academics this week published a paper in the New England Journal of Medicine which listed all the trials on SSRIs that had ever been formally registered with the FDA, and then went to look for the same trials in the academic literature.

37 studies were assessed by the FDA as positive, and with one exception, every single one of those positive trials got properly written up and published. Meanwhile, 22 studies that had negative or iffy results were simply not published at all, and 11 were written up and published in a way that described them as having a positive outcome. You’re a sophisticated reader, so you understand this doesn’t mean that they’re necessarily rubbish drugs, but you also understand that this is dodgy behaviour, all the same.

That’s the easy bit.

The second paper is more interesting. Over the past few decades, we have been subjected to a relentless medicalisation of everyday life by people who want to sell us sciencey solutions. Chancers from the $56 billion international food supplement industry want you to believe that intelligence needs fish oil, and that obesity is just your body’s way of crying out for chromium pills (“to help balance sugar metabolism” etc).

Similarly, quacks from the $600 billion pharma industry sell the idea that depression is caused by low serotonin levels in the brain and so – therefore – you need drugs which raise the serotonin levels in your brain: you need SSRI antidepressants, which are “selective serotonin reuptake inhibitors”.

That’s the serotonin hypothesis. It was always shaky, and the evidence now is hugely contradictory. I’m not giving that lecture here, but as a brief illustration, there is a drug called tianeptine – a selective serotonin reuptake enhancer, not an inhibitor – and yet research shows this drug is a pretty effective treatment for depression too.

Meanwhile in popular culture the depression/serotonin theory is proven and absolute, because it was never about research, or theory, it was about marketing, and journalists who pride themselves on never pushing pills or the hegemony will still blindly push the model until the cows come home. Which brings us on to our second new study on antidepressants. Two academics, a lecturer and an associate professor of neuroanatomy, decided to chase journalists, in the style of this column – or rather, in the style of this column on crack – and fired off multiple emails, demanding unrealistic levels of referencing from doubtless irritated and baffled hacks. They proudly document their work with an excessive number of examples, and I will pick just a few.

“In the New York Times (12/31/06), Michael Kimmelman wrote about the life and work of Joseph Schildkraut, one of the founders of the chemical theory of depression. The Times reporter stated, ‘A groundbreaking paper that he published in 1965 suggested that naturally-occurring chemical imbalances in the brain must account for mood swings, which pharmaceuticals could correct, a hypothesis that proved to be right [italics added].'” The prof’s gave chase. “E-mails to the author requesting a citation to support his statement went unanswered.” A victory for the noble pedantry.

“In another New York Times article (6/19/07), ‘On the Horizon, Personalized Depression Drugs,’ Richard Friedman, the chairman of Psychopharmacology at the Weill Cornell Medical College, stated: ‘For example, some depressed patients who have abnormally low levels of serotonin respond to SSRIs, which relieve depression, in part, by flooding the brain with serotonin.'” They chased, and they give no quarter. “For his evidence he supplied a 2000 paper by Nestler titled, “Neurobiology of Depression,” which focuses on the hypothalamic pituitary system, but not on serotonin.”

The serotonin hypothesis will always be a winner in popular culture, even when it has flailed in academia, because it speaks to us of a simple, abrogating explanation, and plays into our notions of a crudely dualistic world where there can only be weak people, or uncontrollable, external, molecular pressures. As they said in the Pittsburgh Tribune Review (4/2/07) “It’s not a personal deficit, but something that needs to be looked at as a chemical imbalance.”

The real world is more complicated than this simple dichotomy. But when you probe the evidence for simple fables about serotonin stories in popular culture, you’ll find “the quote was attributed to a psychiatric nurse practitioner, the author did not respond to e-mails, and the nurse’s e-mail was not available”.


Here is the NEJM paper on the missing SSRI trials.


Here is “The media and the chemical imbalance theory of depression”


You can read an older Leo and Lacasse paper for free here if you haven’t got an academic login (it’s ridiculous that this stuff is behind a paywall).

Serotonin and Depression: A Disconnect between the Advertisements and the Scientific Literature. Lacasse and Leo 2005.

The relative ropeyness of the serotonin hypothesis is nicely reviewed in this paper from Nature Reviews Neuroscience


Contemptibly that is behind a paywall too but there is a lay summary from my chums MindHacks here


And if this sort of thing interests you, then you can listen to a looong lecture from me about medicalisation called “more than molecules”, at the British Pharmacology Society meeting last year, here.


If you like what I do, and you want me to do more, you can: buy my books Bad Science and Bad Pharma, give them to your friends, put them on your reading list, employ me to do a talk, or tweet this article to your friends. Thanks! ++++++++++++++++++++++++++++++++++++++++++

57 Responses

  1. inspiros said,

    February 18, 2008 at 4:02 pm

    Add the latest publication bias to the 2002/2003 studies by Kirsch and Moncreif showing that placebo effect had been understated in the public studies.

    In 1998 Kirsch et al found placebo was 75% as effect as SSRI. In 2002 a further analysis found it to be 82% effective as SSRI.

    Add this to the concerns about most studies not including active placebos (Moncrief at el 2001) – and those that do showing NO difference between SSRI and active placebo. (www.cochrane.org/reviews/en/ab003012.html)

    Here is a somewhat dated (2003) and fairly conservative summary of the studies and media coverage (with a notable mention of the Guardian… possibly pre-Ben G?)

    Comments on comments:
    WONKO: you don’t explain the improvements people experience. I’m assuming you are saying that SSRI and CBT both are effective due to placebo. People do experience real improvement. That isn’t under question here.

    guvno0or: “Do they work or is it just placebo?” They work – there is no doubt about that. However it is likely that your belief and expectation that they are going to work may be the “active ingrediant”.
    Does knowing that weaken the “active ingredient”?
    In SSRI/drug treatment possibly. In the case of CBT acquiring a meta-belief about the effectiveness of CBT – should be part of CBT treatment anyway.
    If you undergo CBT and persist in the belief that CBT doesn’t work then I would predict lower outcomes.

    The real question here is whether a complex mix of poor trial design, media hype, desperate GPs and patients, and unpublished studies have allowed GPs to prescribe substances where there is a very high expectation of positive outcome (with matching side effects) in an area where self-fulfilling prophecies hold sway (our psyche). It has worked and helped millions. But perhaps we can do better.

  2. banshee said,

    February 21, 2008 at 7:47 pm

    And the latest studies from the UK cast doubt on any relationship between SSRI use and suicide rates – at least in the UK. (See my previous entry)


    Joanna Moncrieff has published some good material and we certainly need questioning professionals but her work attracts a lot of valid critiscism.

    The Cochrane report quoted by inspiros was on TCA’s vs placebo – although I wouldn’t discount the validity entirely it wasn’t on SSRI’s – and the data they looked at looks very incomplete – 1966-2000 but only including 9 studies with 751 participants? Very curious.

    I’m very impressed with the SMHP page – thanks inspiros! – very well balanced and making the point borne out in many trials that the more severely ill you are with depression the more difference the medicines make.

    Consent to trials in mental health tends to lead to milder illness being trialled more commonly – somebody with really serious psychomotor retardation can’t consent to anything!

    CBT and other talking therapies are not tested to anywhere near the same degree as medication.

  3. Robert Carnegie said,

    February 26, 2008 at 10:52 pm

    Y’know, you’d expect an important issue like this to get more news coverage.

    Or, to put it another way, Boom!

    I -thought- I’d just -been- reading about this at BadScience… deja vu.

  4. chewywater said,

    May 24, 2008 at 7:33 pm

    I once took part in a study to see if zoloft would cure night hunger (I can’t remember exactly but I’m pretty sure it was funded by pfizer). When my data showed that zoloft did not have any effect, they sent me more forms to fill out and I was asked to continue taking the medication. If I said no I would be counted as someone who dropped out of the study. In this way they could string people along and wind up with the data they really wanted. I finally took an online poll of most of those involved in the study and found that the actual number of people who claimed zoloft helped with night eating was no better than placebo. Here is the study:

    I now have very little faith in corporate funded science, unless the studies are replicated by people with opposing interests.

  5. NeilHoskins said,

    September 18, 2008 at 10:19 am

    Is there such a thing as an anti-placebo? No, really. Has anybody ever run a trial in which a drug with known, measurable, beneficial properties, is given to patients who are told, “This one’s not very effective and won’t do you much good”?

    All I know is that I’ve been leading a normal life for ten years on fluoxetine, and since you bastards started publicly slagging off SSRIs I’ve been feeling like shit. Causation or coincidence?

    Another, separate question: if placebos are so effective, why do you – apparently – want them removed from the GP’s toolbox?

  6. Paul Murray said,

    July 13, 2009 at 4:54 am

    “The serotonin hypothesis will always be a winner in popular culture … because it … plays into our notions of a crudely dualistic world where there can only be weak people, or uncontrollable, external, molecular pressures.”

    Or demons. It wasn’t me – the devil made me do it. It was witches. It was caused by the moon being in Mars.

    Part of the problem is that “self” is a bit of an illusion. We have these competing drives, and people want to know, which is “me” and which is other. The answer is that they are all “me” – me is precisely the interplay of these parts. My hormones, my self. No excuses.

  7. Astonmartin said,

    January 2, 2011 at 4:14 pm

    Anti-depressants has helped alot of people to help them function in everyday life I agree, but im just woundering if I offered any scientist £5000 to give me concrete physical proof to say that depression is down to the chemical imbalance whats the chance of me keeping my money?