Read and search all the Bad Science Columns from the Guardian and more here.
Thursday February 19, 2004
Hold your breath and forgive me: this week things are a tiny bit more complicated than usual. According to the Daily Mail, cod liver oil is “nature’s super drug”. I don’t doubt that for a minute. Although I can’t help remembering that almost every time I read a story like that, and then go and check the original published journal article, it shows nothing of the sort.
But this time I can’t even do that: because the paper has not been published yet. All there is on this research is a press release from Cardiff University. “The findings from this study will be published in a medical journal later this year.” When? Where? Has it passed peer review? Has it in fact been accepted? Was the study any good? I don’t know the answer to any of these questions. There’s no way of knowing.
What I can tell you is that there were only 31 people in the study, which really isn’t very many, but I can’t tell you how significant or meaningful the results were, on this small sample, because they don’t tell us the figures. What I can also tell you, for all Cardiff University’s press release hyperbole about this being unique human in vivo evidence, is that it wasn’t even about real people getting better, or about their pain levels, or range of movement, or how far they could walk. All the study seems to show was a small change in a laboratory measurement of an obscure enzyme, what scientists call a “surrogate outcome”. That’s a hell of a long way from deserving five major stories in national newspapers, and even further from what Michael McCarthy wrote in the Independent: “They’re not yet saying it can enable you to stop a bullet or leap tall buildings, but it’s not far short of that.”
GPs this week will, quite rightly, be inundated with patients asking for advice on cod liver oil. How can they give the right advice? And, in any case, it’s not just a question of being right; how can they give sensible advice to pensioners, who may be almost living on the breadline, about whether to spend what little money they have on these pills? To do that you need a proper paper, with all the data. Because science is about data, and papers, which scientists spend their careers finding flaws in; it isn’t about taking things on faith, or on the word of experts. That, all too painfully, is at the heart of the public’s misperception of science, and of bias in research, and that’s something to be fought, as the House of Lords report on science and the media recommended: by training scientists to deal with the press appropriately.
This article has been followed up on here.