Ben Goldacre
Saturday March 17, 2007
The Guardian
In the pharmaceutical industry there are people called “drug reps“, who travel around doctors trying to “educate” them about their products. They actively foster an ignorance of scientific methodology, and much of what you get taught in medical school is about how to spot their complex fluffs. Luckily, when pill peddlers market directly at consumers, the fluffs are much simpler.
In the media this week – by which I mean in the half of the British news media who weren’t trashing that dodgy global warming documentary – a company making VegEPA was wiping the floor thanks to an amazing study. Basant Puri, a GMC-registered doctor from Imperial College London, performed the research. “The results of this study were astonishing,” he said. “After taking VegEPA daily for just three months, the children showed an increase in reading age of well over a year.” Prof Puri, who is on the patent as the inventor of VegEPA, also scanned the four children’s brains. “The results were astonishing,” he said: “It was as if these were the brains of children three years older.”
Now oddly, this research was funded by TV company Endemol – home of Gillian McKeith – for a Channel Five documentary for last Thursday. As I read it, the media now love these “pill solves complex social problem” stories so much that they’re willing to pay for the research to base them on.
The show was mysteriously pulled by Five at the last minute after “points were raised”, but it hit the front page of the Daily Mail, with a headline: “Pill that can boost young brain by three years”, and got proper space in the Sun, the Evening Standard, the Times (by a science correspondent no less), the Metro, ITV, the BBC, the Scotsman, and more. The coverage was what you might call “uncritical”.
But let’s approach these VegEPA promotional pieces like you would a story from a drug rep. There was no placebo, no control group, it was “before and after”, and with only four children, whose entire diet and lifestyle was changed. These are old tricks for big pharma, but too foxy for science journalists to spot, apparently.
Next, you might think about whether the population being examined in the study is the same as the population you are “treating”: in this case, perhaps, your child. On the standard growth charts – which I have in front of me – the four children in this study are not simply overweight, they are in the highest possible weight category on the charts, beyond the heaviest one in 200 of the population for their age. They are very unusual children with, you might guess, exceptional diets.
The same issue comes up all the time with misleading drug trials. As any doctor would ask: does the population in the trial reflect the population you treat? The answer here is probably not.
Then you might look at the outcomes measured. Behavioural outcomes, in a study of four children, with no control, and lots of extra attention for the subjects – including TV cameras pointing at them – are meaningless. “One boy who previously scorned books and was hooked on TV developed a love of reading and declared he was ‘bored’ with television” said the Daily Mail. I bet he did.
But the action, surely, is in this brain imaging data? “The most striking finding emerged from the brain scans, which all suggested they had denser nerve fibres,” said the Mail. This is what you call a “surrogate outcome”: they feel all sciencey, drug reps love them, and conjurers would call it a misdirection.
The scan measured something called NAA in the brain. It’s been noticed, in other research, that as you grow, especially as a foetus or a neonate, you show more NAA in your brain. To call that a marker of brain development, that you measure and then make a sales claim on, is a very big leap. It is not a reliable or valid predictor of intelligence, or behaviour, or anything meaningful. It is a speculative research finding.
This individual pill is a sideshow. There is a far greater issue at stake here, beyond even the misrepresentation of the scientific method by the media: the nation’s children are being systematically re-educated to believe that they need to take pills every day to lead a normal, happy, productive life. Pill peddlers of all varieties, supplements and pharmaceutical, must be rubbing their hands with glee.
· Please send your bad science to bad.science@guardian.co.uk
There’s more to come on this story, as you will hopefully see next week.
On that subject, could the second nice person with the leaked documents please get back in touch? We’re good to go with one bit but just need to check one tiny thing and then we can go three pronged, and if you think you know someone who might think they know something about what I’m talking about here then do encourage them to get in touch too. In fact, if you ever know anything about anything then get in touch. Immediately. As you can see circularity and discretion is very much assured.
And if you’re feeling cheated that this week’s column is related to an earlier blog post, then (a) blame the circumstances that pulled the other column and (b) my new discovery for this evening is Significance, the journal of the Royal Statistical Society. It really is a properly ace read. The most recent issue is free:
www.blackwell-synergy.com/toc/sign/4/1
…and if you have an Athens Login for academic journals then you can get the lot, Iain Chalmers in 2006(4) is particularly good as ever:
www.blackwell-synergy.com/doi/pdf/10.1111/j.1740-9713.2006.00200.x
and so is the one from Bland and Altman about left handed people dying young.
www.blackwell-synergy.com/doi/pdf/10.1111/j.1740-9713.2005.00130.x
and this is the most read articles list:
www.blackwell-synergy.com/action/showMostReadArticles?journalCode=sign
Dandelion said,
March 17, 2007 at 2:45 am
Hi Ben, long time reader first time poster.
I have a severe chronic illness and about two years ago I tried taking large amounts of fish oil as recommended by Puri and others. (Yes, I know it’s stupid but when you’re sick you get desperate. You’ll eat pooh on toast if enough people sing it’s praises). I found his internet sales division(http://www.vegepa.com/vegepa.php?n=books&indi=2&ti=Products) and very nearly wasted a lot of money shipping the stuff over here to Australia. However, there are plenty of other people selling fish oil formulations so I found a cheaper supplier.
After a couple of months consuming large quantities of this stuff, I wound up in hospital sicker than I had ever been and I have not yet returned to the level of health I was at. I still wonder if it was the fish oil.
If four people make a study, we only need three other people like me and we might have cancelled out his four kids and their (alleged) sudden love of book learning…
Bob O'H said,
March 17, 2007 at 6:57 am
Haven’t you joined the RSS yet? We must do something about that…
Of course, only 5% of the articles in Significance are at all interesting: the rest are just noise.
Bob
tms said,
March 17, 2007 at 8:28 am
Don’t Imperial College London have, like, rules and stuff about using your institutional affiliation to promote you own product?
censored said,
March 17, 2007 at 10:30 am
Personally, I like the more concise and broader-aimed writing in the articles compared to the blog posts (though the column is invariably too short).
I *thought* I understood the earlier post (turns out I did) about this, but I don’t always follow the details of the science and don’t always have the time to thoroughly read the references. It’s like the blog is more hard-core and the column for more lay-people, where I fall somewhere in the middle.
tregenza said,
March 17, 2007 at 10:39 am
For more about VegEPA with respect to Dyslexia and ADHD, have a look at www.myomancy.com/2007/03/vegepa_does_it_.html
Chris
swift said,
March 17, 2007 at 12:38 pm
This old article from the Observer about vegEPA is interesting, especially with regard to your comments about people believing they have to take pills every day to lead a normal life.
observer.guardian.co.uk/magazine/story/0,,1221157,00.html
evidencebasedeating said,
March 17, 2007 at 1:09 pm
media bias is an interesting aspect of reporting, as is obvious permission by researchers to render it so….
Only in a couple of articles, later in the week, did it appear that the children were , in fact , given nutritional advice by Paul Sacher RD, a senior ninja dietitian who works at the Institute (yes, a proper one) of Child Health/ Great Ormond Street Childrens Hospital, and – unique in the field of dietetics – a dietitian who had managed to wheedle serious mega-monies out of both the DoH and Jamie’s favourite supermarket to extend his ‘holistic’ MEND programme (targeting childhood obesity) into the community.
www.mendprogramme.org/
So, assuming the ‘weight loss’ in these obese kids was mainly a result of ‘Mend’ style dietary recommendations – with a vague but potential margin of enhancement by omega-3 intake, why did Prof Basant Puri allow full credit to his VegEPA product alone?
And why no rebuttal / caveat from Mr Sacher about the limitations of accrediting VegEPA-devised-and-patented-by-Puri = brainier kids?
The dynamics between these two researchers must surely be an interesting one…..
evidencebasedeating said,
March 17, 2007 at 1:16 pm
I’m sure that earlier in the week there was actually some text about said Professor on the Imperial College website. My search is drawing a blank, now…..
www1.imperial.ac.uk/medicine/people/basant.puri/
Tom P said,
March 17, 2007 at 1:38 pm
Er, Ben… Endemol isn’t “home to Gillian McKeith” – You Are What You Eat is made by Celador, an entirely different media behemoth. Stupid mistake to make in an otherwise great article.
Tom P said,
March 17, 2007 at 1:51 pm
By the way, the reason that this story got such wide coverage in the press can be largely traced back to the fact that it was reported, entirely uncritically, on the PA wire. PA’s coverage of science is generally appalling, re-write the press release guff – but the fact that they’re a huge, generally trustworthy news agency means that a lot of papers accept their reporting as reliable.
It’d be interesting to do an analysis of every science story PA file over, say, one week – to see just how much of it is unsubstantiated, PR hype. Certainly, newspapers are failing to do their job by uncritically publishing what the news agencies feed them – but if you want to find the root cause of a lot of Britain’s bad science in the media, go after PA, and (to a lesser extent) AP and Reuters.
ChrisR said,
March 17, 2007 at 3:35 pm
Thanks for the link to Significance. My attention was immediately drawn to an article asking whether rail safety had got worse after privatisation. A bit odd, I thought, given that is axiomatic in the media today that evil profit-grabbing big rail has “wrecked the railways”. Imagine my surprise when the short, well-written and eminently comprehensible (to a non-statistician) article showed the opposite.
Now I am starting to wonder if other bastions of big capitalism, possibly even drug companies, are not the evil b******s they are repeatedly portayed as in the media ….?
jackpt said,
March 17, 2007 at 4:09 pm
Hi ChrisR, that may very well be the case(I hope it is, I like good news) but the usual caveat of not taking it in isolation, because it’s just one paper. The model, data, and/or conclusions may or may not be disputed.
Generally the media portrays things in stark black and white terms. There has to be bad guys and good guys, and the stuff in between gets confused. So that’s as likely to apply to big capitalism as much as anything else. It’s always the case that if you want to be even half sure of your views it requires thinking critically about what the press says – as much so as the verbal spew that emanates from the politically motivated.
Robert Carnegie said,
March 17, 2007 at 4:34 pm
Jasper Carrott is with Celador, I think, or was. Who does “Hitler’s Holocaust”?
Robert Carnegie said,
March 17, 2007 at 5:04 pm
And #6: Dominic Moore apparently is never heard of again, assuming he isn’t the violinist or the ice hockey player with the Pittsburgh Penguins. Can we make sure he hasn’t topped himself? Which would be bad news for makers of vegetarian fish vitamins. But anyway, he should have seen his doctor and got the pills that work. Your judgment is off when you’re depressed. The pills put you right. Would he feel less of a man if he was diabetic? (For all I know, he is.)
And #11: an argument is presented at www.guardian.co.uk/transport/Story/0,,2025671,00.html that railway safety culture currently is defective, still, since privatisation. On the other hand, one person died last month in a railway accident, and presumably several other people on the same day in cars – around 9 or 10, I think. Then again, the latest accident at Grayrigg was said to have had its impact reduced by the bravery of the driver staying at his post and by the greater safety of modern train carriage design. That’s not a matter of privatisation except insofar as private money paid for new rolling stock. If “rolling stock” is not one of those specific terms that laypeople never use correctly, just like “food chain”.
BobP said,
March 17, 2007 at 5:37 pm
Ben – I didn’t get usual email alert of the last 2 blogs. If you’re hoping for leaked docs, make sure you ask loudly!
(mind you, big problems with my ISP talktalk at the moment. But the gas bill got through!)
BobP said,
March 17, 2007 at 6:16 pm
Basant Puri is described on the Imperial website as either “visiting” or “honorary”. He has co-published a number of papers with Alex Richardson. Richardson was involved with the early Durham trials but has dissociated herself from devleopments over the last year or so.
Richardson AJ, Cyhlarova E, Puri BK. Clinical and biochemical fatty acid abnormalities in dyslexia, dyspraxia, and schizotypy: an overview. In Glen AIM, Peet M, Horrobin DF. (eds.) Phospholipid Spectrum Disorder in Psychiatry and Neurology. Carnforth: Marius Press; 2003: 477-90.
Puri BK and Richardson AJ. Long-term follow-up of a single patient with schizophrenia treated with ethyl-EPA alone. Phospholipid Spectrum Disorder in Psychiatry and Neurology. Carnforth: Marius Press; 2003: 377-89.
Puri BK and Richardson AJ. Magnetic resonance spectroscopy in the assessment of brain phospholipids metabolism in dyslexia. Phospholipid Spectrum Disorder in Psychiatry and Neurology. Carnforth: Marius Press; 2003: 501-8.
stever said,
March 19, 2007 at 9:46 pm
hang on – I thought Ben was a pawn of the pharmaceutical companies. how come hes slagging the off?
j said,
March 19, 2007 at 11:36 pm
interesting- one novelty with vegepa compared to other fish oils tabs is it also has added omega 6 fats. So you’re paying extra to ‘supplement’ your diet with something that most ‘normal’ diets in the UK contain too much of. Also, if you eat a load of junkfood (like the kids in the ‘study’) you’re likely to be eating even more omega 6 fats than most people.
CaptainKirkham said,
March 20, 2007 at 10:33 am
As to rail safety – there is an argument out there that it is overall less safe to increase safety on the railways. Increased safety measures make rail fares go up, which pushes people onto the roads, where you are far more likely to be injured than on any train, with added safety features or not. Counter-intuitive, but sort of sensible.
Apologies for continuing the off-topic conversation.
Delster said,
March 20, 2007 at 11:29 am
so this guy Puri invents and patents this vegEPA stuff. Then when the media want a piece on it he does a, quite frankly, p*ss poor study for them.
Would it not have been better for him to provide the original research test findings which must surely, to get approvals, have been carried out in a much more controlled fashion?
Or is this another of those “supplements” that anybody can market without getting drug approvals?
simongates said,
March 20, 2007 at 11:36 am
Do we know yet why the programme was pulled at the last minute?? Or rather, I know that some people know, so can they tell me yet?
LizardKing said,
March 20, 2007 at 2:13 pm
#10 – With regard to PA, I used to work for them and can confirm that their science articles are mostly rewrites of press releases. Their coverage of political and legal affairs is pretty damned good, with some very experienced and dedicated editors, but they simply don’t have the qualified staff to vet science stories.
ACH said,
March 20, 2007 at 2:46 pm
www.bbc.co.uk/radio4/youandyours/items/01/2007_12_mon.shtml
An item on You & Yours yesterday. Not the whole story, I’m sure, but you can admire Dr Puri’s ability to back-pedal (or peddle ) with the statements his “representative” quotes from him!
Ben Goldacre said,
March 20, 2007 at 3:06 pm
i believe the truth will out tomorrow.
wewillfixit said,
March 21, 2007 at 8:47 am
Anything in the news about it today? I can’t find anything.
stephenh said,
March 21, 2007 at 1:37 pm
Thanks for the pointer to Significance, Ben, it really *is* an ace read. I’ve just spent a few hours reading through the archive. Heartily recommended.
I particularly like Dr Fisher’s case notes.
Nickynockynoonoo said,
March 21, 2007 at 11:27 pm
Ben,
Weds has come and Weds has gone.
We’re waiting cha cha cha
We’re waiting cha cha cha
We’re waiting cha cha cha
We’re waiting cha cha cha
We’re waiting cha cha cha
Eric T Zipper said,
March 30, 2007 at 12:46 pm
BobP (post 16) – You have noted the collaboration between Prof Puri and Alex Richardson.
What a fishy story then from yesterdays Mail – All pupils should be given fish oil, says experts – by Sean Poulter:
“Health experts want compulsory fish oil supplements for all school children to improve their intelligence and behavious. British and U.S. experts told a Parliamentary inquiry yesterday the nation needs to massviely increse its consumption of the Omega-3 fatty acids found in fish”.
“Dr. Alex Richardson of Oxford University told the first hearing of the Inquiry on Diet and Behaviour, organised by the Parliamentary Forum on Food and Health, that the Government should start providing Omega-3 supplements in schools. She said a trail in Durham had shown they can reduce disruptive behaviour and improve learning.”
So now we have OXford academics reccomending unilateral supplementation to our children through a school feeting programme! One wonders where the data is coming from to support this?
Robert Carnegie said,
March 30, 2007 at 8:01 pm
-Is- there a “Mansfield College, Oxford”? Hmm. It appears that there is.
“and Founder Director of FAB Research … www.fabresearch.org/ ” – “Food And Behaviour” apparently.
Evidence-biased said,
April 15, 2007 at 11:28 pm
Whoops.
‘…even pointed readers to Bad Science while trying to help them see why this was pure hype’.
Everyone will of course recognise that ‘this’ was referring to the media coverage of Puri’s miracle pills – and not to the Bad Science website
(won’t they?)
diudiu said,
December 21, 2009 at 6:33 am
free shipping ugg
free shipping ugg