As someone who is nerdishly fascinated by the systematic analysis of health risk data – check me out, ladies – I sometimes look at the health pages and try to work out what they’re supposed to do, what kind of information they offer, and for who.
This week, for example, you’ll have found: “Teenager helps his twin brother by donating a piece of his back“; “In pain? Take one Botticelli three times a day“; “Taking antibiotics to prevent premature birth can ‘increase risk’ of cerebral palsy“; “Woman dies in agony after eating toadstools“, “Why drinking water to shed weight is a waste of time“; “More men suffering from ‘Manorexia’, health experts warn“; and “Cheap bone-strengthening drug cuts chance of breast cancer relapse ‘by a third’“.
Are people around the nation carefully clipping these stories out, and pasting them in indexed box files, ready for the day when they develop the condition in question, or encounter the opportunity to modify an unusual health risk exposure? And how will they know if the data they are gathering is complete, or just an arbitrary patchwork of newsworthy and self-serving information, multiply filtered through a range of imperfect agents with diverse interests and allegiances? In fact, how does anybody know that?
This week the media collectively ignored a study looking at that exact question, but in a high stakes environment. It was also one of the most important papers to be published this year: only one in five trials on cancer treatment, it turns out, actually gets published; the rest are simply missing in action. And it gets worse: only 5.9% of industry-sponsored trials on cancer treatment get published. Later, it will get worse again.
For decades now people have known that negative results tend not to get printed in academic journals, and it can happen for all kinds of reasons (eg here but many more): they’re not newsworthy, so journal editors reject them; they’re not much fun to write up, and they don’t look good on your CV (although they should get plaudits); and they might not flatter your idea or product, so you might not want them out there.
One bright suggestion which I bang on about incessantly was that all clinical trials should be registered before they begin: then people can at least stand a chance of noticing if and when a trial goes missing in action. This took about 20 years to be put into practise, half-heartedly, and it solves half the problem. But there is a problem: who will chase up the missing studies?
Scott Ramsey from the Fred Hutchinson Cancer Research Center in Seattle and John Scoggins from the University of Washington took on the role of investigative journalists. In a world where not one person from the world of alternative therapies can bring themselves to criticise even vitamin pill entrepreneur Matthias Rath for his dangerous practices in South Africa – indeed some still actively support him, as we may soon see – critical self-appraisal is simply business as usual in academia.
They went to clinicaltrials.gov, the register run by the US National Institute for Health, and found all the trials on cancer: 2,028 in total. Then they went to Pubmed.gov, the searchable database containing almost all the proper medical journals out there.
The overwhelming majority were missing.
Only 17.6% were published at all, but better than that, 64.5% of those reported positive results. How impressive. Meanwhile only 5.9% of industry-sponsored trials were published, but in those 5.9%, golly did they do well: 75.0% gave positive results. Just lucky fingers I guess.
We may never know what was in all that unpublished data, but for all that this study is nerdish and lacking in newsworthiness, those missing numbers will cost lives in quantities larger than any emotive health story covered in any part of any newspaper this week. Doctors need negative data to make rational prescribing decisions. Academics need to know which ideas have failed, so they can work out why, what to study next, and which ideas to abandon. In their own way, academic journals are exactly as selective as the tabloid health pages. God help us all.