Mark Easton has been covering this story over on the BBC News website, for the last week or so. He’s revealed that the Minister in charge was the current Defence Secretary, Bob Ainsworth.

The Academics who gave the advice, regret now that they didn’t push their reservations about the design of the trials/evaluation further than they did. They believed, perhaps naively, that when they gave advice back in 2002 (stating that the research would be pointless) that that would be the end of the evaluation. As we have seen, it wasn’t.

Why do these things happen? I suspect that there’s often a toxic mix of (a) basic ignorance and (b) incentives to do the wrong thing – it’s cock-up and conspiracy. If you’re a manager and you negotiated your project a £6m budget, you’ve little incentive to give up “your” money, or to even listen to arguments that you could achieve the same outcomes with less cash. Your budget is of course a measure of how important you are amongst your peers and a bigger budget can mean faster career progression – it looks good on your CV.

And, back in the boom, when this one started, a big budget was better for ministers – spending was always investment, and investing £6m means you’re committed to this important issue. It would hardly have been worth an announcement (an end in itself) if it had been a few hundred grand on a tiny pilot. (One good side to the fiscal crisis is that these incentives for both civil servants and ministers to maximise spending must now be reducing somewhat.)

In this case, maybe the advice from Prof Bird arrived at the wrong time. Once a minister has given something the nod, it would be pretty counter-cultural to re-visit that and also bad for your career to say, look, we got this wrong, we should do it this way instead. And the ignorance just exacerbates things; the briefing for Coaker was probably written by someone who thought that if you’re trying something then that’s a trial.

The perverse incentives might not matter so much if there were just a little more basic scientific literacy and numeracy – so if a minister asks “how many people will this trial help?” someone in the room has the minimal amount of brains and balls required to say “Well minister, we don’t know how many people it will help because we don’t yet know if it will actually help. That’s why we’re doing a trial and having a control group. But we’re looking at a sample size of X”, or whatever, rather than “We’re looking at running this exciting new programme in 23 schools minister, which means that X children would be helped”

The policy leads often lack the basics and the analysts often don’t get in the room. I suspect that, often, if they’re considered at all, they’re condemned as too purist and kept out of the loop until after decisions have been made – so they can’t advise but can only mop up and push for caveats to be appended to rotten reports.

Perhaps it’s culturally seen as heartless to suggest spending less on doing stuff that might help and spending more on finding out what stuff would help. Perhaps part of the problem is that we live in a democracy and people like the stuff that might help, and ministers know they like it.

I’ve no idea how the civil service science problem can be fixed – although there are, to be fair, multiple programmes ongoing.

The only reason I can think of for continuing the study after the commentary on its design had been received is that there were people deciding the question with a wistful hope that the very low significance comparisons would produce propaganda material in favour of the initiative. (Instead, the result moved the Bayesian prior very slightly – insignificantly – in the direction of discrediting the Blueprint approach.)I suggest that the people who took the decision to continue need to pass a compulsory course in the basis of statistical reasoning. Darell Huff’s booklet ‘How to Lie with Statistics’ would be suitable text.

Ben Goldacre’s Bad Science gets it wrong shock? What’s all this about then?

From The Guardian Corrections column, Saturday 19 September 2009 www.guardian.co.uk/theguardian/2009/sep/19/corrections-clarifications

Bad Science:
Magnetism, mystery and plain muddle, 20 June, page 16, we wrongly identified The Times as the publisher of an article that appeared in The Sunday Times on 14 June with the headline: Oceans charge up new theory of magnetism. Bad Science criticised the way The Sunday Times reported research by Professor Gregory Ryskin, of Northwestern University in the US, because his paper did not, as The Sunday Times claimed, say that Earth’s magnetic field may be produced by ocean currents. Prof Ryskin suggested, instead, that small fluctuations in the field may be related to the movement of oceans. Unfortunately, when we edited Bad Science we removed a sentence, included in the copy submitted to us, which reported that The Sunday Times said Prof Ryskin had approved its coverage. We apologise for this error.

In the same Bad Science column we made the mistake of saying that the Scottish Daily Express, rather than the Scottish Sunday Express, published a story about survivors of the Dunblane school shooting based on material taken from social networking sites. Bad Science made an analogy between the Scottish Sunday Express’s use of private information about the lives of the Dunblane teenagers and The Sunday Times’s use of a science paper posted by Prof Ryskin, for discussion by the science community, on a pre-publication internet archive several years ago. The Sunday Times has complained that this was unfair. We accept that the extent to which that comparison was open to argument would have been clearer if we had included the response from The Sunday Times. The Sunday Times interviewed Prof Ryskin in connection with its report; it also showed him a draft (though not the final version) of its report before publication and took some of the changes he requested into consideration.

The next lot of “legal highs” to be banned are being banned because of their “potential harm” (meaning they haven’t killed anyone yet, or at least not without copious amounts of alcohol and other illegal drugs in their system first), while alcohol and tobacco kill thousands of people each year. I’m sure the cost to the UK from alcohol is something like £3bn a year.

It’s hard to take anything they say about drugs seriously when it’s clear they don’t value evidence *at all*.