Thursday January 20, 2005
Here’s an interesting question, and I’d be interested to hear your thoughts. Can a published research paper ever be Bad Science? I think not. This week, we publish a letter from Raxit Jariwalla, the author of a study on vitamin C and HIV, who feels done down for being mentioned here two weeks ago. He wasn’t singled out, but Patrick Holford, of the branded vitamins, misrepresented the paper in his Optimum Nutrition Bible. Holford said: “AZT, the first prescribable anti-HIV drug, is potentially harmful, and proving less effective than vitamin C,” and referred to the paper as proof. In fact, the paper was a lab study on what happens if you tip lots of vitamin C onto HIV-infected cells and measure a few things related to HIV replication. It did not compare vitamin C with AZT and was not a clinical trial where tablets were given to people to see how well they did. Read it here: tinyurl.com/4l7vz.
Patrick Holford was guilty of at least incompetence in claiming that this study demonstrated vitamin C to be a better treatment than AZT. But is the Jariwalla paper Bad Science? No. I don’t think a paper ever can be. The meat of a paper is the methods and results section. In the discussion at the end, granted, things sometimes go a bit weird. But the meat of a paper honestly and accurately describes an experiment and its results. We all know that all experiments could have been done better or worse, to a greater or lesser extent. That’s the point of critical appraisal, of learning to pick papers apart. Maybe the sample size could have been bigger; maybe what you measured, the surrogate marker, turns out not to be as valid as you thought. Scientists know this. And by scientists, I mean people who know a bit about science and are capable of thinking it through, no white coats required.
A paper, an experiment, is merely one bit of evidence, and how you choose to interpret it, how you fit it into your understanding of whether a theory holds, with lots of other evidence, is a thing that a person does, with varying degrees of fallibility: you weigh it up, come to a verdict, personally, on the evidence, accounting in your own imperfect way for the flaws in all the bits of evidence you happen to know.
Jariwalla’s paper is useless as supporting evidence to Holford’s statement. It is excellent evidence for lots of other statements. So Jariwalla I have no opinion on, his paper is just a paper, and Holford is a fool or worse. Or am I wrong?
Follow up to this.