Ben Goldacre
Thursday January 20, 2005
The Guardian

Here’s an interesting question, and I’d be interested to hear your thoughts. Can a published research paper ever be Bad Science? I think not. This week, we publish a letter from Raxit Jariwalla, the author of a study on vitamin C and HIV, who feels done down for being mentioned here two weeks ago. He wasn’t singled out, but Patrick Holford, of the branded vitamins, misrepresented the paper in his Optimum Nutrition Bible. Holford said: “AZT, the first prescribable anti-HIV drug, is potentially harmful, and proving less effective than vitamin C,” and referred to the paper as proof. In fact, the paper was a lab study on what happens if you tip lots of vitamin C onto HIV-infected cells and measure a few things related to HIV replication. It did not compare vitamin C with AZT and was not a clinical trial where tablets were given to people to see how well they did. Read it here: tinyurl.com/4l7vz.

Patrick Holford was guilty of at least incompetence in claiming that this study demonstrated vitamin C to be a better treatment than AZT. But is the Jariwalla paper Bad Science? No. I don’t think a paper ever can be. The meat of a paper is the methods and results section. In the discussion at the end, granted, things sometimes go a bit weird. But the meat of a paper honestly and accurately describes an experiment and its results. We all know that all experiments could have been done better or worse, to a greater or lesser extent. That’s the point of critical appraisal, of learning to pick papers apart. Maybe the sample size could have been bigger; maybe what you measured, the surrogate marker, turns out not to be as valid as you thought. Scientists know this. And by scientists, I mean people who know a bit about science and are capable of thinking it through, no white coats required.

A paper, an experiment, is merely one bit of evidence, and how you choose to interpret it, how you fit it into your understanding of whether a theory holds, with lots of other evidence, is a thing that a person does, with varying degrees of fallibility: you weigh it up, come to a verdict, personally, on the evidence, accounting in your own imperfect way for the flaws in all the bits of evidence you happen to know.

Jariwalla’s paper is useless as supporting evidence to Holford’s statement. It is excellent evidence for lots of other statements. So Jariwalla I have no opinion on, his paper is just a paper, and Holford is a fool or worse. Or am I wrong?

Follow up to this.

Vitamin deficiency – Patrick Holford

Ben Goldacre
Thursday January 6, 2005
The Guardian

You’ll be pleased to hear that my new year’s resolution is to stop going on about nutritionists and find some new targets to bait. However, due to the curved nature of spacetime in newspapers I’m writing this in 2004, so by my reckoning I get one more pop, not at dear Gillian McKeith, but at Patrick Holford. Lots of people seem to like him. He’s a clever guy. I thought I’d grab his book, The New Optimum Nutrition Bible, because it would be handy to have a desk reference, instead of always going to Medline to check wacky claims.

Now, I absolutely swear blind, the first thing I did was open it at a random page: HIV Infection and Aids. “Leading researcher Dr Raxit Jariwalla … found that with continuous exposure to ascorbic acid (Vitamin C) … the growth of HIV in immune cells could be reduced by 99.5%.” That’s 99.5%. Wow. But there’s no reference. You’d have thought, in a book with no less than 241 academic references, that this astonishing fact would be something worth referencing, but hey, it’s Christmas.

So I hunt through the references section at the back, and finally find one paper by Jariwalla. Then, like a young Sherlock Holmes, I find the place in the book, sorry, the “Bible”, where this mysterious paper is referred to. Holford’s sentence, on page 208 reads like this: “AZT, the first prescribable anti-HIV drug, is potentially harmful, and proving less effective than Vitamin C.” Then there’s a little superscript 23, referring you to this Jariwalla paper. Just like in a proper academic article! So, vitamin C is better than AZT. Obviously I had to read that paper. The abstract is at tinyurl.com/4l7vz. The paper is free online. It doesn’t compare vitamin C to AZT for efficacy. It’s a laboratory study. It doesn’t look at whether Vitamin C treats HIV in humans. It measures a few jolly complicated things like extracellular reverse transciptase activity, p24 antigen, giant cell syncytia formation. It has nothing to do with AZT. If anyone can read that paper and tell me how it backs up Holford’s sentence about AZT, then I would like to know how. The paper doesn’t even contain the word AZT. Not once.

Everything Holford writes is plastered with references. He’s almost impossible to argue against, because he’s constantly pulling these references out of the bag. Each one takes about an hour to check – so if you’d like to join the struggle, his book is only £12.99. I hope some of them are better than this one.

Followed up here.