Ben Goldacre, The Guardian, Saturday 23 January 2010
Like many people, you’re possibly afraid to share your views on animal experiments, because you don’t want anyone digging up your grandmother’s grave, or setting fire to your house, or stuff like that. Animal experiments are necessary, they need to be properly regulated, and we have some of the tightest regulation in the world.
But it’s easy to assess whether animals are treated well, or to assess whether an experiment was necessary. In the nerd corner there is another issue: is the research well conducted, and are the results properly communicated? If it’s not, then animals have suffered – whatever you believe that might mean for an animal – partly in vain.
The National Centre for the Replacement, Refinement and Reduction of Animals in Research was set up by the government in 2004. It has published, in the academic journal PLoS One, a systematic survey of the quality of reporting, experimental design and statistical analysis of recently published biomedical research using laboratory animals. It’s so not good.
The study is pretty solid. They describe the strategy they used to search for papers, which is important, because you don’t want to be like a homeopath, and only quote the papers that support your conclusions: you want to have a representative sample of all the literature. And the papers they found covered a huge range of publicly funded research, behavioural and diet studies, drug and chemical testing, immunological experiments, and more.
Some of the flaws they discovered were bizarre. Four per cent of papers didn’t mention how many animals were used in the experiment, anywhere. The researchers looked in detail at 48 studies that did say how many they used: not one explained why they had chosen their particular number of animals. Thirty-five per cent of the papers gave one figure for the number of animals used in the methods, and then a different number of animals appeared in the results. That’s pretty disorganised. They looked at how many studies used basic strategies to reduce bias in their results, like randomisation and blinding.
If you’re comparing one intervention against another, for example, and you don’t randomly assign animals to each group, then it’s possible you might unconsciously put the stronger animals in the group getting a potentially beneficial experimental intervention, or vice versa, thus distorting your results.
If you don’t “blind”, then you know, as the experimenter, which animals had which intervention. So you might allow that knowledge, even unconsciously, to affect close calls on measurements you take. Or maybe you’ll accept a high blood pressure reading when you expected it to be high, knowing what you do about your own experiment, but then double check a high blood pressure measurement in an animal where you expected it to be low.
Only 12% of the animal studies used randomisation. Only 14% used blinding. And the reporting was often poor. Only 8% gave the raw data, allowing you to go back and do your own analysis. About half the studies left the numbers of animals in each group out of their tables.
I grew up friends with the daughters of Colin Blakemore, a neuroscientist in Oxford who spoke out to defend animal research at great personal risk. My first kiss – not one of these sisters, I should say – was outside a teenage party in a church hall, in front of two special branch officers sitting in a car with their lights off.
People who threaten the lives of 15-year-old girls, to shut their father up, are beneath contempt. People who fail to damn these threats are similarly contemptible. That’s why it sticks in the throat to say that the reporting and conduct of animal research is often poor; but we have to be more grown up.