Here’s an interesting problem with data analysis in general, and so, by extension, data journalism: you have to be careful about assuming that the numbers you’ve got access to… really do reflect the underlying phenomena you’re trying to investigate.
Today’s Guardian has a story, “Antidepressant use in England soars“. It’s much more overstated in the Independent. They identify that the number of individual prescriptions written for antidepressant drugs has risen, and then assumes this means that more people are depressed. But while that’s a tempting assumption, it’s not a safe one. Read the rest of this entry »
Ben Goldacre, The Guardian, Saturday 20 August 2011
What do all these numbers mean? “‘Worrying’ jobless rise needs urgent action – Labour” was the BBC headline. They explained the problem in their own words: “The number of people out of work rose by 38,000 to 2.49 million in the three months to June, official figures show.”
Now, there are dozens of different ways to quantify the jobs market, and I’m not going to summarise them all here. The claimant count and the labour force survey are commonly used, and number of hours worked is informative too: you can fight among yourselves for which is best, and get distracted by party politics to your heart’s content. But in claiming that this figure for the number of people out of work has risen, the BBC is simply wrong.
Read the rest of this entry »
Ben Goldacre, The Guardian, Saturday 29 January 2011
If science has any authority, it derives from transparency: you can check the claims against the working. Sometimes you hit a brick wall. Sometimes you might consider a shortcut. Let’s look at 3 types of checking. Read the rest of this entry »
Ben Goldacre, The Guardian, Saturday 18 December 2010
It’s been a marvellous year for bullshit. We saw quantitative evidence showing that drug adverts aimed at doctors are routinely factually inaccurate, while pharmaceutical company ghostwriters were the secret hands behind letters to the Times, and a whole series of academic papers. We saw more drug companies and even regulators withholding evidence from doctors and patients that a drug was dangerous – the most important and neglected ethical issue in modern medicine — and that whistleblowers have a rubbish life. Read the rest of this entry »
Ben Goldacre, The Guardian, Saturday 16 October 2010
You will be familiar with the Daily Mail’s ongoing project to divide all the inanimate objects in the world into the ones that either cause or prevent cancer. Individual entries are now barely worth documenting, and the phenomenon is best appreciated in bulk through websites such as the Daily Mail Oncological Ontology Project and Kill Or Cure, with its alphabetised list: from almonds, apples and artificial light; through horseradish, hotdrinks and housework; to wasabi, water, watercress, and more.
But occasionally one story pops up to illustrate a wider issue, and “Strict diet two days a week ‘cuts risk of breast cancer by 40 per cent’” is a good example. It goes on: “a strict diet for two days a week consisting solely of vegetables, fruit, milk and a mug of Bovril could prevent breast cancer, scientists say.” Read the rest of this entry »
Ben Goldacre, The Guardian, Saturday 28 August 2010
For simpletons and amateurs, there are good research methods, and bad research methods. In reality, different tools are valuable in different situations, and sometimes, even very tiny numbers of people can give you a meaningful piece of information: even an anecdote can be informative. Read the rest of this entry »
Ben Goldacre, The Guardian, Saturday 7 August 2010
According to the Home Office this week, Sarah’s law – where any parent can find out if any adult in contact with their child has a record of violent or sexual crimes – has “already protected more than 60 children from abuse during its pilot“. This fact was widely reported and was the headline finding. As the Sun said: “More than 60 sickening offences were halted by Sarah’s Law during its trial”. Read the rest of this entry »
You might be amused by this piece from the Independent’s health reporter Jeremy Laurance today. It’s about what a bad man I am for pointing out when science and health journalists get things wrong. Alongside the lengthy ad hominem – a matter of taste for you – there are a number of mistakes and, more than that, a worrying resistance to the idea that anyone should dare to engage in legitimate criticism. He also explains that health journalists simply can’t be expected to check facts. This worries me. Read the rest of this entry »
Ben Goldacre, The Guardian, Saturday 5 June 2010
“Fish oil helps schoolchildren to concentrate” was the headline in the Observer. Regular readers will remember the omega-3 fish oil pill issue, as the entire British news media has been claiming for several years now that there are trials showing it improves school performance and behaviour in mainstream children, despite the fact that no such trial has ever been published. There is something very attractive about the idea that solutions to complex problems in education can be found in a pill. Read the rest of this entry »