Ben Goldacre, The Guardian, Saturday 2 April 2011
Here are two fun ways that numbers can be distorted for political purposes. Stop me if I’m boring you, but each of them feels oddly poetic, in its ability to smear or stifle.
The first is simple: you can conflate two different things into one number, either to inflate a problem, or confuse it. Last weekend, a few hundred thousand people marched in London against the cuts. On the same day, there was some violent disturbance, windows smashed, policemen injured, and drunkeness.
The Sun said: “Police have charged nearly 150 people after violent anarchists hijacked the anti-cuts demo and brought terror to London’s streets.” The Guardian republished a Press Association report, headlined: “Cuts protest violence: 149 people charged”. And from the locals, for example, the Manchester Evening News carried “Boy, 17, from Manchester among 149 charged over violence after anti-cuts march”.
In reality, a dozen of these charges related to violence, while 138 are people who were involved in an apparently peaceful occupation of Fortnum and Masons organised by UKUncut, who campaign on tax avoidance.
You will have your own view on whether people should be arrested and charged for standing in a shop as an act of protest. But describing these 150 people as “violent anarchists… who brought terror to London’s streets” is not just misleading: it also makes the police look over 12 times more effective than they really were at charging people who perpetrated acts of violence.
The second method of obfuscation is even simpler. After London was chosen to host the 2012 Olympics, Labour made a series of pledges, including two around health: to use the power of the games to inspire a million more people to play sport three or more times a week; and to get a million more people doing more general physical activity.
Politicians seem keen on the idea that large multi-sports events can have a positive impact like this, so the area has been studied fairly frequently, and last year the BMJ published systematic review of the literature. They set out to find any study that had ever been conducted looking at the real-world health and socioeconomic impacts of major multi-sport events on the host population.
They found 54 studies. Overall, the quality was poor (it’s a fairly difficult thing to measure, and most studies used cross-sectional surveys, repeated over time). The bottom line was this: there is no evidence that events like olympics have a positive impact on either health or socioeconomic outcomes.
Here is what they reported. One study looked at Manchester before and after the 2002 Commonwealth Games: overall sports participation (4 times or more in the past month) fell after the games, and the gap in participation rates between rich and poor areas widened significantly.
Another study in Manchester suggested there were particular problems around voluntary groups being excluded from using Commonwealth branding, and that new facilities tended to benefit elite athletes rather than the general population.
There was a vague upward trend in sports participation in Barcelona between the early 1980s and 1994, and they had the Olympics in 1992. Volunteers in the Commonwealth Games showed no increase in sports participation.
You will have your own views on whether the cost of hosting the Olympics is proportionate to the benefits, and where those benefits lie. From this systematic review, however, there’s no evidence for large multi-sports events having a positive health or socioeconomic impact overall, so only an optimist would make promises to the contrary.
This week, it emerged that both of the government’s targets for improving healthy activity after the 2012 Olympics are now being quietly dropped. By walking away from outcome indicators that will not be met, a government can create a false impression of success: if prespecified outcome indicators are ever to mean anything, after all, it’s because you report on each of them clearly, whether success is achieved or not.
But more than that, governments around the world spend billions of pounds on these events: by quietly dropping these outcome indicators, rather than carefully documenting our success or failure at meeting them, our current politicians pave the way for ever more false and over-optimistic claims by their colleagues, all around the world.