Here’s an interesting problem with data analysis in general, and so, by extension, data journalism: you have to be careful about assuming that the numbers you’ve got access to… really do reflect the underlying phenomena you’re trying to investigate.

Today’s Guardian has a story, “Antidepressant use in England soars“. It’s much more overstated in the Independent. They identify that the number of individual prescriptions written for antidepressant drugs has risen, and then assumes this means that more people are depressed. But while that’s a tempting assumption, it’s not a safe one.

Thinking off the top of my head, it could be – for example – that doctors are writing more frequent prescriptions for the same number of patients, but with each prescription for smaller amounts (to reduce overdose risk, say). These potential alternative explanations are the sort of thing that comes up all the time in data analysis for medical research.

In fact, this specific question – what does an increase in antidepressant scripts mean? – has been researched in some detail before. I wrote about it in April 2011, the last time this rise was written up as a big story, in several major newspapers, including the Guardian. I guess nobody listens to me, and fair enough.

www.badscience.net/2011/04/when-journalists-do-primary-research/

…Firstly, this rise in scripts for antidepressants isn’t a new phenomenon. In 2009 the BMJ published a paper titled “Explaining the rise in antidepressant prescribing”, which looks at the period from 1993 to 2005. In the 5 year period from 2000 to 2005 – the boom before the bust these journalists are writing about – antidepressant prescribing also increased, by 36%. This isn’t very different to 43%, so it feels unlikely that the present increase in prescriptions is due to the recession.

That’s not the only problem here. It turns out that the number of prescriptions for an SSRI drug is a rubbish way of measuring how many people are being treated for depression: not just because people get prescribed SSRIs for all kinds of other things, like anxiety, PTSD, hot flushes, and more; and not just because doctors have moved away from older types of antidepressants, so would be prescribing more of the newer SSRI drugs even if the number of people with depression had stayed the same.

Excitingly, it’s a bit more complicated than that. A 2006 paper from the British Journal of General Practice looked at prescribing and diagnosis rates in Scotland. Overall, again, the number of prescriptions for antidepressants increased from 1.5 million in 1996 to 2.8.million in 2001 (that is, it almost doubled).

But they also found a mystery: looking at Scottish Health Survey, they found no increase in the prevalence of depression; and looking at the GP consultations dataset, again they found no evidence that people were presenting more frequently to their GP with depression, or that GPs were making more diagnoses of depression.

So why were antidepressant prescriptions going up? This puzzle received some kind of explanation in 2009. The BMJ paper above found the same increase in the number of prescriptions that the journalists have found this week, as I said. But they had access to more data: their analysis didn’t just look at the total number of prescriptions in the country, or even the total number of people diagnosed with depression: it also looked at the prescription records of individual patients, in a dataset of over 3 million patients’ electronic health records (with 200,000 people who experienced a first diagnosis of depression during this period).

They found that the rise in the overall number of antidepressant prescriptions was not due to increasing numbers of patients receiving antidepressants. It was almost entirely caused by one thing: a small increase in the small proportion of those patients who received treatment for longer periods of time. Numerically, people receiving treatment for long periods make up the biggest chunk of all the prescriptions written, so this small shift bumped up the overall numbers hugely.

I don’t know for certain if that phenomenon explains the increase in prescriptions from 2006-2010, as it does for the period 2000-2005 (although in the absence of work examining that question, since the increase in scripts was so similar, it does seem fairly likely). And I’m not expecting journalists to go to academic research databases to conduct large complex descriptive studies.

But if they are going to engage in primary research, and make dramatic causal claims  - as they have done in this story – to the nation, I don’t think it’s too much to ask that they familiarise themselves with proper work that’s already been done, and consider alternative explanations for the numbers they’ve found.

Incidentally, if you’re missing the column, I’m procrastinating on Twitter, and posting occasionally on posterous. I’ll stick a round-up of the most interesting things from there onto here occasionally, and there’s also a backlog of columns to pop up here too, from when I was too busy to breathe. The new book is in fighty form, thanks for asking, out in August 2012.