When journalists do primary research

April 9th, 2011 by Ben Goldacre in bad science | 18 Comments »

Ben Goldacre, The Guardian, Saturday 9 April 2011

This week some journalists found a pattern in some data, and ascribed a cause to it. “Recession linked to huge rise in antidepressants” said the Telegraph. “Economic woes fuel dramatic rise in use of antidepressants” said the Daily Mail. “Record numbers of people are being handed antidepressants” said The Express. Even the Guardian joined in, and it seems to have come from a BBC report.

The journalists are keen for you to know that these figures come from a Freedom of Information Act request, which surprised me, since each year – like you – I enjoy reading the Prescription Cost Analysis documents, which detail everything that has been prescribed over the previous year. The 2009 data was published in April 2010, so I guess the 2010 data was due about now.

But are the numbers correct? Yes. From 2006 to 2010 there was a 43% increase in number of prescriptions for the SSRI class of antidepressants. Does that mean more people are depressed in the recession?

Well. Firstly, this rise in scripts for antidepressants isn’t a new phenomenon. In 2009 the BMJ published a paper titled “Explaining the rise in antidepressant prescribing”, which looks at the period from 1993 to 2005. In the 5 year period from 2000 to 2005 – the boom before the bust these journalists are writing about – antidepressant prescribing also increased, by 36%. This isn’t very different to 43%, so it feels unlikely that the present increase in prescriptions is due to the recession.

That’s not the only problem here. It turns out that the number of prescriptions for an SSRI drug is a rubbish way of measuring how many people are being treated for depression: not just because people get prescribed SSRIs for all kinds of other things, like anxiety, PTSD, hot flushes, and more; and not just because doctors have moved away from older types of antidepressants, so would be prescribing more of the newer SSRI drugs even if the number of people with depression had stayed the same.

Excitingly, it’s a bit more complicated than that. A 2006 paper from the British Journal of General Practice looked at prescribing and diagnosis rates in Scotland. Overall, again, the number of prescriptions for antidepressants increased from 1.5 million in 1996 to 2.8.million in 2001 (that is, it almost doubled).

But they also found a mystery: looking at Scottish Health Survey, they found no increase in the prevalence of depression; and looking at the GP consultations dataset, again they found no evidence that people were presenting more frequently to their GP with depression, or that GPs were making more diagnoses of depression.

So why were antidepressant prescriptions going up? This puzzle received some kind of explanation in 2009. The BMJ paper above found the same increase in the number of prescriptions that the journalists have found this week, as I said. But they had access to more data: their analysis didn’t just look at the total number of prescriptions in the country, or even the total number of people diagnosed with depression: it also looked at the prescription records of individual patients, in a dataset of over 3 million patients’ electronic health records (with 200,000 people who experienced a first diagnosis of depression during this period).

They found that the rise in the overall number of antidepressant prescriptions was not due to increasing numbers of patients receiving antidepressants. It was almost entirely caused by one thing: a small increase in the small proportion of those patients who received treatment for longer periods of time. Numerically, people receiving treatment for long periods make up the biggest chunk of all the prescriptions written, so this small shift bumped up the overall numbers hugely.

I don’t know for certain if that phenomenon explains the increase in prescriptions from 2006-2010, as it does for the period 2000-2005 (although in the absence of work examining that question, since the increase in scripts was so similar, it does seem fairly likely). And I’m not expecting journalists to go to academic research databases to conduct large complex descriptive studies.

But if they are going to engage in primary research, and make dramatic causal claims  – as they have done in this story – to the nation, I don’t think it’s too much to ask that they familiarise themselves with proper work that’s already been done, and consider alternative explanations for the numbers they’ve found.


++++++++++++++++++++++++++++++++++++++++++
If you like what I do, and you want me to do more, you can: buy my books Bad Science and Bad Pharma, give them to your friends, put them on your reading list, employ me to do a talk, or tweet this article to your friends. Thanks! ++++++++++++++++++++++++++++++++++++++++++

18 Responses



  1. Matthew Cline said,

    April 9, 2011 at 5:57 am

    “Recession linked to huge rise in antidepressants” said the Telegraph.

    How long before someone claims that antidepressants caused the recession?

  2. keristor said,

    April 9, 2011 at 8:22 am

    I saw the BBC’s articles (two at least: “Money woes ‘linked to rise in depression'” and “How the recession is causing depression”). Well, part of it is obvious, people worrying about money are going to be stressed and often depressed. Duh. But there’s a difference (which the media often overlook) between being unhappy because of a real world condition and being clinically depressed, the latter generally has biological causes.

    But as often happens the BBC contradicts itself. Not long before they had another of the BBC’s report, “When taking pills can be better than talking” , which points out that NHS depression ‘help’ often consists of mass “being talked to” sessions even bigger than average classroom sizes (the article mentions “Sitting in a village hall with 50 other people listening to a Powerpoint presentation about coping with stress”). If it’s that or the pills I can guess what most people (and doctors) would be more likely to choose.

    Take your pick which theory you like. But correlation does not imply causation, as you point out, no matter how hard the media and politicians would like to believe it does.

  3. timelady said,

    April 9, 2011 at 3:42 pm

    How about a few more potential reasons? Availability of new drugs generally (what I’m taking wasn’t available 15 years ago so I took nothing), better awareness of depression by GPs and other health workers so possibly just higher treatment levels of the same % of the population, availability of more alternative treatments such as online counselling (which in my case helped me decide, after 20 years, to try a prescription).

    I’m a longtime depression sufferer who is now a new prescription. The amusing (or not so amusing) thing to me in this reporting is that my own health workers (except my GP) seemed determine to find reason for my state of mind in my work, my finances and the recession simply because I’m self employed and have lower income. Despite my repeated insistence that this was not the case – either that or I’ve been suffering a continual recession for a couple decades. So I think the problem of false blame is starting somewhere earlier in the chain than the reporting.

  4. bhaji said,

    April 9, 2011 at 6:32 pm

    Maybe the above newspapers would like to print your article in their next issue, preferably on the front page?

    As the guardian has already printed it, maybe they could run a new article pointing out and apologising for the false claims in their original article?

    Hang on, just saw a vaguely porcine shape wizz past my window and zoom up into the sky…

  5. emen said,

    April 9, 2011 at 7:36 pm

    “…caused by one thing: a small increase in the small proportion of those patients who received treatment for longer periods of time.”

    But this increase is real and something must have caused it. It could be the recession as much as anything else.

  6. TonyTheGoat said,

    April 10, 2011 at 8:15 am

    ” It could be the recession…”
    Yes, it could, but…

    “… as much as anything else”
    Precisely.

  7. anyachaika said,

    April 10, 2011 at 1:10 pm

    A few years ago a few national newspapers printed some stories about a highly intrusive treatment for children in care called Holding Therapy (also known as Attachment Therapy). The articles unquestioningly presented the claims of the organisation providing this as factual. In practice this therapy was completely unvalidated, and undertaken without any kind of meaningful governance in place.

    This practice is still going on and seems to be causing serious trauma to the most vulnerable children in society. I can’t think of a better example of when a little research by journalists could have made such a difference. Please see -http://anyachaika.wordpress.com/2011/04/05/a-first-hand-account-of-holding-therapy-in-the-uk/

  8. gilly.cooper said,

    April 11, 2011 at 8:29 am

    I am a GP and will be one of those responsible for this increase in prescribing. This phenomenon, of a small number of patients being prescribed antidepressants for longer, may very well be a consequence of NICE guidelines which, for the first time in my experience, gave specific guidance on how long to treat with antidepressants. 6 months for first episode and 2 years or more if recurrent. This meant that GPs will be much more specific about length of treatment and will encourage patients in the knowledge that there is evidence that depression is less likely to recur when treated in this way.

  9. danfaller said,

    April 11, 2011 at 1:43 pm

    I too am a GP and would echo the comments made above. Exactly the same thing was postulated by a Professor of psychiatry on the Today programme on the morning of the story. Perfectly reasonable alternative explanation. But it makes no difference to editorial decisions. 13 hours later on the 10pm news the same line was being trotted out about the link to the countries’ finances.

    If the media was really interested in the increased use of antidepressants, rather than using it as a surrogate marker for whatever political conclusion they want to draw, then this point, as well as the excellent ones about other uses, could have been made.

    Interestingly, normally the media likes to bash GPs. In this case we are praised for ‘better recognising the signs of depression’. The cynics amongst us might argue that an increase in the use of antidepressants could demonstrate a deterioration in our ability to properly diagnose depression as we lazily and inappropriately using antidepressants to treat non-specific sadness and low mood (for which the evidence of antidepressant efficacy is limited, and which constitutes a large patient group).

    Like all data and journalists, it appears to just depend who they have in their sights – then they can manipulate conclusions to suit the story!

  10. David said,

    April 11, 2011 at 2:46 pm

    Ironically I think it is the media that depresses me the most. They are the invisible gatekeepers of that currency of the 21st century – knowledge. But like a toddler jealously guarding a plastic toy, their power is embedded in the restriction of information rather than clarity and freedom. It seems like the freedom of the press is the freedom to make up as much cak as they possibly can in order to peddle emotive and ill informed sensationalism. Of course there are some superb journalists out there, but we are talking about the whole machine here. The media is not just an extension of the entertainment industry, it has become a key component.

    But all this is an aside from the matter in hand. It seems to me that the problem here is the idea that depression is a simple cause and effect condition: recession = depression = increase in SSRI prescription. Mental ill health is not a linear issue, recovery from mental illness or even sadness is a narrative, which involves multiple interventions, actions and changes. What worries me about this article is not only the misrepresentation of the facts (poor data analysis), but the way that these figures have been used to effectively trivialise mental health and turn it into headline to whip up yet more banker hatred and bitterness.

    Good mental/emotional health is fairly straight forward for most people. For some, SSRI intervention is useful. But so what? Do we see articles when Ibuprofen sales are up? Would this indicate that the recession is responsible for more sprained ankles? Mental health will always be a headline maker as long as it is kept as some kind of separate issue from “physical” health.

  11. CliveHill said,

    April 11, 2011 at 4:02 pm

    How does a ‘small increase’ in _any_ proportion of the patients result in almost doubling?

  12. DanB said,

    April 12, 2011 at 11:19 am

    @CliveHill

    Pretend I’m a doctor. I have 10 patients who I have prescribed antidepressants this year. 9 have only one prescription each, and then their condition improves. But the 10th needs repeat prescriptions as his condition doesn’t improve, so has 10 prescriptions for antidepressants in that year. Total of 10 patients, 19 prescriptions.

    Another person who has a long-term condition which isn’t responding to treatment turns up. I again have 9 people who have only short-term conditions who only need one prescription each, but I now have two people who have 10 prescriptions a year. I now have 11 patients for this year on antidepressants, a modest 10% increase, but write 29 prescriptions, 20 for my 2 long-term depressives and 9 for the short-term patients.

    Adjust the numbers as you can double the number of prescriptions with virtually any increase in the number of patients that you want, for the simple reason that the number of patients having anti-depressants prescribed is not proportionate to the number of prescriptions.

  13. CliveHill said,

    April 12, 2011 at 1:29 pm

    @DanB
    That’s not really a ‘small increase’, you’ve doubled your numbers going from 1 long term patient to 2.
    But, many thanks anyway! I now know what Ben meant.

  14. ronanos said,

    April 13, 2011 at 10:01 am

    @CliveHill

    That is a small increase – in the total patient population. The number of prescriptions went from 19 to 29 (approx. 50% increase). The number of TOTAL patients went from 10 to 11 (approx. 10% increase).

    You say he doubled his number of long-term patients, and that is the precise point of this article, there is no way of telling whether these new prescriptions belong to a population with a big increase and a small need for prescriptions or to a population with a small increase but a BIG need for prescriptions.

  15. ferguskane said,

    April 14, 2011 at 9:44 pm

    It’s still worrying given that given the total failure to detect meaningful separation from placebo means that antidepressants are essentially active placebos (see STAR*D, and the various recent meta-analyses on the subject e.g. Kirsch (2008, doi:10.1371/journal.pmed.0050045).

    One might have hoped, that given the increasing evidence for the above opinion, antidepressant prescriptions would be going down, not stabilising or going up.

  16. notajournalist said,

    April 17, 2011 at 3:04 pm

    It seems suicide rates (a closer measure of the meaningful public health outcome) ARE correlated with the business cycle, at least in the US:
    www.cdc.gov/media/releases/2011/p0414_suiciderates.html

  17. Veronica said,

    April 20, 2011 at 3:56 pm

    I thought the whole thing about clinical depression is that it doesn’t need a reason. In fact, if you do have poor life circumstances of some sort it is often called “reactive depression” as a cause can be ascribed. For the majority of depressive patients a cause cannot be ascribed. Could be recession, could be bad weather, most likely to be wobbly brain chemistry without an external cause. What do people think?

  18. darren27k said,

    January 6, 2012 at 1:32 pm

    I agree that looking at prescriptions may be misleading, so I have looked at a longitudinal study of 150 GP practices (not peer reviewed, but nontheless valid) which shows that the number of patients treated with an antidepressant has increased by 28%, and the number of prescriptions has increased by 47% between Dec 2006 and Dec 2011. This is the equivalent of an extra prescription per patient per year. The adjectives you want to use to describe that increase is up to the individual journalist!