I made this Radio 4 documentary on randomised trials on government policy

January 6th, 2013 by Ben Goldacre in bad science, evidence based policy, government reports, podcast, politics | 4 Comments »

Here’s a documentary I made for BBC Radio 4 (with producer Rami Tzabar) about evidence based social policy, and why we should do more randomised trials in government. It’s good fun, 40 minutes, with contributions from Dean Karlan (who wrote this book and is behind all these excellent trials on reducing poverty), Prof Sheila Bird, Jonathan Portes from NIESR (his excellent blog here), the man they call GOD, and many more.

www.bbc.co.uk/programmes/b01phhb9

It’s also Radio 4 documentary of the week (woo!) which means you can download it as an mp3. The link is live for another 5 days, and it works for some countries outside the UK too.

www.bbc.co.uk/podcasts/series/r4choice

And now that it’s expired on iPlayer, you can listen to it on FigShare here:

figshare.com/articles/R4Doc_04_Jan_13_Ben_Goldacre/105918

If you’re interested in reading more evidence based policy, I highly recommend this Cabinet Office paper that I co-authored a few months ago, downloadable for free online. As explained here, it’s brief, and very much designed to be the Ladybird Book of RCTs in Government. If you want more on the uses for randomised trials in criminal justice, I wrote this in the British Medical Journal with Sheila Bird and John Strang in 2011 (sorry it’s not open access, I’ll try to fix that soon). More to come on this topic soon. Read the rest of this entry »

Is this the worst government statistic ever created?

April 23rd, 2012 by Ben Goldacre in economics, evidence based policy, government reports, politics, pr guff, statistics | 23 Comments »

I forgot to post this column up last year. It’s a fun one: the Department for Communities and Local Government have produced a truly farcical piece of evidence, and promoted it very hard, claiming it as good stats. I noticed the column was missing today, because Private Eye have published on the same report in their current issue, finding emails that have gone missing through FOI applications, and other nonsense. That part is all neatly summarised online in the Local Government Chronicle here.

Is this the worst government statistic ever created?

Ben Goldacre, The Guardian, 24 June 2011.

Every now and then, the government will push a report that’s so assinine, and so thin, you have to check it’s not a spoof. The Daily Mail was clear in its coverage: “Council incompetence ‘costs every household £452 a year’“; “Up to £10bn a year is wasted by clueless councils.” And the Express agreed. Where will this money come from? “Up to £10bn a year could be saved … if councils better analysed spending from their £50bn procurement budgets.” Read the rest of this entry »

We should so blatantly do more randomised trials on policy

May 23rd, 2011 by Ben Goldacre in africa, evidence based policy, politics | 22 Comments »

Ben Goldacre, The Guardian, Saturday 14 May 2011

Politicians are ignorant about trials, and they’re weird about evidence. It doesn’t need to be this way. In international development work, resources are tight, and people know that good intentions aren’t enough: in fact, good intentions can sometimes do harm. We need to know what works.

In two new books published this month – “More Than Good Intentions” and “Poor Economics” – four academics describe amazing work testing interventions around the world with proper randomised trials. This is something we’ve bizarrely failed to do at home. Read the rest of this entry »

I’d expect this from UKIP, or the Daily Mail. Not from a government leaflet.

April 16th, 2011 by Ben Goldacre in bad science, politics | 37 Comments »

Ben Goldacre, The Guardian, 15 April 2011

HM Government have issued a new leaflet to justify their NHS reforms: Working Together For A Stronger NHS. It was produced by Number 10, appears on the Department of Health website, and many of the figures it contains are misleading, out of date, or flatly incorrect.

It begins, like much pseudoscience, with uncontroversial truths: the number of people over 85 will double, and the cost of drugs is rising.

Then the trouble starts. In large letters, alone on one entire page, you see: “If the NHS was performing at truly world-class levels we would save an extra 5,000 lives from cancer every year.” The reference for this is a paper in the British Journal of Cancer called “What if cancer survival in Britain were the same as in Europe: how many deaths are avoidable?” Read the rest of this entry »

Pretending that evidence is difficult and complicated

February 19th, 2011 by Ben Goldacre in bad science, politics | 42 Comments »

Ben Goldacre, The Guardian, Saturday 19 February 2011

For the past two weeks we’ve followed the government’s misuse of evidence on NHS reforms, remembering that they’re perfectly permitted to reform things with no evidence at all, like everyone else does, they just shouldn’t pretend to have evidence. On Thursday health minister Simon Burns appeared before a BMA meeting in London. Read the rest of this entry »

Why is evidence so hard for politicians?

February 12th, 2011 by Ben Goldacre in bad science, politics | 60 Comments »

Ben Goldacre, The Guardian, Saturday 12 February 2011

One thing you hope for, with politicians, is that they won’t make the same mistakes over and over again.

Last week we saw that the government has overstated the problems in the NHS by using dodgy figures (to be precise, they used misleading static figures instead of time trends). We saw that Andrew Lansley’s repeated claim that his reforms are justified by evidence was untrue: the evidence doesn’t show that his price-based competition improves outcomes (if anything it makes things worse); and the evidence also doesn’t show that GP consortia improve outcomes (unless you cherry pick only the positive findings). It’s okay if your reforms aren’t supported by existing evidence: you just shouldn’t claim that they are.

Now Lansley’s junior minister Paul Burstow MP has kindly responded, repeating the exact same mistakes again, only more clumsily. I find this, in all seriousness, genuinely frightening from a minister, so I’ll explain how he does it. Read the rest of this entry »

Andrew Lansley and his imaginary evidence

February 5th, 2011 by Ben Goldacre in bad science, politics | 50 Comments »

Ben Goldacre, The Guardian, Saturday 5 February 2011

I have never heard one politician use the word “evidence” so persistently, and so misleadingly, as Andrew Lansley defending his NHS reforms. Since he repeatedly claims that the evidence supports his plan, let’s skim through what we can find on whether GP consortiums work, the benefits of competition, and the failures of the NHS.

Are GP consortiums better than PCTs for commissioning? There have been 15 major reorganisations of the NHS in 30 years. We’ve had GP fundholders, GP multifunds, primary care groups, primary care trusts, family practitioner committees, purchasing consortiums, and more. After all this change, lots of data should have been gathered on the impact of specific strategies.

In reality, few were properly studied. Here are 4 papers on GP fundholding, which is broadly similar to Lansley’s GP Consortiums. Kay in 2002 found it was introduced and then abolished without any evidence of its effects. In 2006 Greener and Mannion found a mix of good and bad but no evidence that it improved patient care. In 1995 Coulter found nothing but gaps in the evidence and no evidence of any improvement in efficiency, responsiveness, or quality.  Petchley found there was insufficient data to make any judgement. Lansley says he is following the evidence. I see no evidence to follow here.

Next, competition. Andrew Lansley has repeatedly denied that he is introducing competition on price. This is disturbing behaviour: his bill explicitly introduces price-based competition, it’s in paragraph 5:43 of his NHS Operating Framework.

Does variable-price competition work in healthcare markets? It’s hard to measure, but the evidence even on fixed-price competition – where you compete on quality – is mixed. There are various ways to assess it: often people choose an outcome – like the number of people who survive a heart attack – and compare this outcome in areas of more intense or less intense competition. Sometimes competition makes things worse, sometimes better.

Working from first principles, markets where people compete on price as well as quality will probably make quality worse, because prices are easy to measure, while quality is not. The evidence seems to support this theory. The introduction of variable price competition in New Jersey in the 1990s was associated with a worsening death rate from heart attacks, while in the UK, stopping variable price competition was associated with improvement. It’s hard to measure either way, but despite his using the word repeatedly, again, the “evidence” does not support Lansley here.

Lastly, there is the justification for reform. Both Lansley and Cameron overstate our mortality figures to claim that the NHS is failing. Everyone wants more improvement, but money does not produce an immediate and visible reduction in mortality from one thing. Interventions take time to have an impact, especially on things that kill you slowly, and treatment isn’t the only factor affecting how many people die of something. But to take just two things, mortality from cancer has fallen every year since 1995, and heart attack deaths have halved since 1997.

The government claims that our rate of death from heart attacks is double that in France, even though we spend the same on health. Health economist John Appleby instantly debunked this claim in the BMJ, and his piece will become a citation classic. From static 2006 figures in isolation the government is right: but the trajectory of improvement in the UK is so phenomenal that if the straight line continues – as it has done for 30 years – we will be better than France by 2012.

I’m not in favour of, or against, anything here: all health service administrative models bore me equally. But when Andrew Lansley says all the evidence supports his interventions, as he has done repeatedly, he is simply wrong. His wrongness is not a matter of opinion, it is a fact, and his pretence at data-driven faux neutrality is not just irritating, it’s also hard to admire. There’s no need to hide behind a cloak of scientific authority, murmuring the word “evidence” into microphones. If your reforms are a matter of ideology, legacy, whim, and faith, then like many of your predecessors, you could simply say so, and leave “evidence” to people who mean it.

Nerds, rise up! Science cuts protest tomorrow

October 8th, 2010 by Ben Goldacre in bad science, politics | 15 Comments »

I’m speaking tomorrow at the protest against science cuts, it’s 2pm outside the Treasury, wear something that looks like your field, maybe a white coat, or a telescope, or a field if you’re a botanist.

Details of the protest:

scienceisvital.org.uk/ Read the rest of this entry »

Pornography in hospitals

September 25th, 2010 by Ben Goldacre Tags:
in bad science, politics | 67 Comments »

Ben Goldacre, The Guardian, Saturday 25 September 2010

The Sun, of all people, are angry about pornography: “THE hard-up NHS is blowing taxpayers’ cash on PORN for sperm donors, a report reveals today.” The Telegraph immediately followed suit. Some clinics provide pornography for men masturbating in clinic rooms to produce sperm for IVF with their partners. Read the rest of this entry »

“Exams are getting easier”

August 21st, 2010 by Ben Goldacre in bad science, evidence, numerical context, politics, schools | 123 Comments »

Ben Goldacre, The Guardian, Saturday 21 August 2010

Pass rates are at 98%. A quarter of grades are higher than an A. This week every newspaper in the country was filled with people asserting that exams are definitely getting easier, and then other people asserting that exams are definitely not getting easier. The question for me is always simple: how do you know?

Read the rest of this entry »