Saturday March 17, 2007
In the pharmaceutical industry there are people called “drug reps“, who travel around doctors trying to “educate” them about their products. They actively foster an ignorance of scientific methodology, and much of what you get taught in medical school is about how to spot their complex fluffs. Luckily, when pill peddlers market directly at consumers, the fluffs are much simpler.
In the media this week – by which I mean in the half of the British news media who weren’t trashing that dodgy global warming documentary – a company making VegEPA was wiping the floor thanks to an amazing study. Basant Puri, a GMC-registered doctor from Imperial College London, performed the research. “The results of this study were astonishing,” he said. “After taking VegEPA daily for just three months, the children showed an increase in reading age of well over a year.” Prof Puri, who is on the patent as the inventor of VegEPA, also scanned the four children’s brains. “The results were astonishing,” he said: “It was as if these were the brains of children three years older.”
Now oddly, this research was funded by TV company Endemol – home of Gillian McKeith – for a Channel Five documentary for last Thursday. As I read it, the media now love these “pill solves complex social problem” stories so much that they’re willing to pay for the research to base them on.
The show was mysteriously pulled by Five at the last minute after “points were raised”, but it hit the front page of the Daily Mail, with a headline: “Pill that can boost young brain by three years”, and got proper space in the Sun, the Evening Standard, the Times (by a science correspondent no less), the Metro, ITV, the BBC, the Scotsman, and more. The coverage was what you might call “uncritical”.
But let’s approach these VegEPA promotional pieces like you would a story from a drug rep. There was no placebo, no control group, it was “before and after”, and with only four children, whose entire diet and lifestyle was changed. These are old tricks for big pharma, but too foxy for science journalists to spot, apparently.
Next, you might think about whether the population being examined in the study is the same as the population you are “treating”: in this case, perhaps, your child. On the standard growth charts – which I have in front of me – the four children in this study are not simply overweight, they are in the highest possible weight category on the charts, beyond the heaviest one in 200 of the population for their age. They are very unusual children with, you might guess, exceptional diets.
The same issue comes up all the time with misleading drug trials. As any doctor would ask: does the population in the trial reflect the population you treat? The answer here is probably not.
Then you might look at the outcomes measured. Behavioural outcomes, in a study of four children, with no control, and lots of extra attention for the subjects – including TV cameras pointing at them – are meaningless. “One boy who previously scorned books and was hooked on TV developed a love of reading and declared he was ‘bored’ with television” said the Daily Mail. I bet he did.
But the action, surely, is in this brain imaging data? “The most striking finding emerged from the brain scans, which all suggested they had denser nerve fibres,” said the Mail. This is what you call a “surrogate outcome”: they feel all sciencey, drug reps love them, and conjurers would call it a misdirection.
The scan measured something called NAA in the brain. It’s been noticed, in other research, that as you grow, especially as a foetus or a neonate, you show more NAA in your brain. To call that a marker of brain development, that you measure and then make a sales claim on, is a very big leap. It is not a reliable or valid predictor of intelligence, or behaviour, or anything meaningful. It is a speculative research finding.
This individual pill is a sideshow. There is a far greater issue at stake here, beyond even the misrepresentation of the scientific method by the media: the nation’s children are being systematically re-educated to believe that they need to take pills every day to lead a normal, happy, productive life. Pill peddlers of all varieties, supplements and pharmaceutical, must be rubbing their hands with glee.
Â· Please send your bad science to firstname.lastname@example.org
There’s more to come on this story, as you will hopefully see next week.
On that subject, could the second nice person with the leaked documents please get back in touch? We’re good to go with one bit but just need to check one tiny thing and then we can go three pronged, and if you think you know someone who might think they know something about what I’m talking about here then do encourage them to get in touch too. In fact, if you ever know anything about anything then get in touch. Immediately. As you can see circularity and discretion is very much assured.
And if you’re feeling cheated that this week’s column is related to an earlier blog post, then (a) blame the circumstances that pulled the other column and (b) my new discovery for this evening is Significance, the journal of the Royal Statistical Society. It really is a properly ace read. The most recent issue is free:
…and if you have an Athens Login for academic journals then you can get the lot, Iain Chalmers in 2006(4) is particularly good as ever:
and so is the one from Bland and Altman about left handed people dying young.
and this is the most read articles list: