Ben Goldacre, The Guardian, Saturday 10 July 2010
This week the food and nutrition pills industries are complaining. They like to make health claims about their products, which often turn out to be unsupported by the evidence. Regulating that mess would be tedious and long-winded, the kind of project enjoyed by the EU, and so the Nutrition and Health Claims Regulation was brought in during 2006. Since then, member states have submitted tens of thousands of health claims on behalf of manufacturers about cranberries, fish oil, and every magical ingredient you can think of. This week it turned out that 900 have been examined so far, of which 80% have unsurprisingly been rejected.
“The regulation is killing this industry and the job losses are already being felt,” says the head of the International Probiotics Association. Even “established claims like cranberry for urinary tract health” are being rejected, say pill company PowerHealth: “there will be no information on packs for the consumer to assess what the product is supposed to do.” SlimFast, that great British institution, may have to change its name. All complain that the bar for evidence is being set too high.
Trusting nobody, and as a very boring man, I decided to read some adjudications. Excitingly these are all available in full on the EU website. I can only apologise, but the first thing I typed in was fish oil. Picture this as a long-running pseudoscience soap opera (and they are the best selling food supplement product in the UK, from the global $55bn food supplements market).
The pharmaceutical company want to claim that Eye-Q fish oil capsules improve working memory in children, and so they sent in references to 6 studies (the deliberations are in full online). Now firstly, the fact that they sent 6 studies in is interesting, because Equazen, the manufacturers of Eye Q, told me in 2006 that they had 20 positive studies showing that Eye-Q improves school performance and behaviour in children, although they refused to let me see them, so there is no possible way to find out what they did, what they measured, what kind of children participated, or indeed anything at all about these studies. If ever there was a time to roll out those well conducted trials, in which thousands of children volunteered their bodies for experiment, surely it was now, for the EU? Apparently not.
So they sent in 6 studies. 2 of these were conducted in children with developmental coordination disorder and ADHD. Whatever you might think about the value of these labels, and the extent to which they are overused – as with so many diagnostic categories in medicine and psychiatry – these are not mainstream children, and the results may not generalise, so it seems reasonable to take those with a pinch of salt.
Next, 3 of the 6 studies submitted – to support an application for a claim about working memory – did not look at working memory. I hope it’s not repressive or unfair of me to suggest that these can be fairly disregarded.
Lastly, there is a positive result for one sub-type of working memory, verbal working memory, tested with digit span (how many numbers you can remember): this comes from an unpublished trial of the company’s product. I don’t know how many trials they have today, but from a company that has 20 hidden trials in which to hunt for an isolated positive result in one subscale from one variable, this does not feel like compelling evidence.
I don’t think this is a regulator being unfair. What is unfair, I think, is taking these claims at face value. These companies want to make these claims to sell their products, and they will find a way, making them through journalists if need be, rather than through labels. What’s more, people want to buy these products: many of us enjoy the game, if you like, of pretending to ourselves that pills have medical effects, even though often we kind of know, somewhere inside our head, that the claims aren’t quite for real.
orpheus66 said,
July 10, 2010 at 2:00 am
You have to love a reaction that boils down to, “Yes, we KNOW our products don’t do anything, but you can’t let that kill an industry!”
I wish the US followed up things like this, but since Reagan, it’s been all too easy for the fish oil merchants to climb out from under their rocks.
Matt Keefe said,
July 10, 2010 at 2:01 am
I’m a writer. Some people buy expensive writing software because they think it will make them a better writer. It won’t, but it might give them the enthusiasm to write at all. I know what you mean about people placing a certain kind of faith in a certain kind of bullshit, but ultimately we need to be able to compare even bullshit to the truth.
People pay money for these things. Supposed Bullshit Pills Brand A are £2 a pop. Bullshit Pills Brand B are £5 a pop – not because they work any better, but because they’ve got even bigger bullshit claims written on the label. I don’t see any reason they can’t put a statement of belief on the label – ‘the manufacturers believe that bullshit pills can be part of a healthier lifestyle when used in combination with various other bullshit’ – to deny them that would probably be to deny them the voicing of their opinions, but I do think it’s necessary to say that they can’t claim to know for certain, or to have proven it.
That to me would seem to serve both to separate their assertions from claims supported by evidence, and to level the playing field between the various different strains of bullshit. Bigger bullshit, bigger prices is the worst part of it all to my mind. Buying your way to health with magical cures like those just undermines too many other parts of the health message, I think.
And some of them plainly are just in it for the money. Maybe we should tax their bullshit by weight.
T said,
July 10, 2010 at 8:57 am
as someone with a very low score “digit span” and working memory (dyslexia)……would I take fish oil if I thought they would improve this? No. I may have the working memory of an (old school catagories) of an “imbacile”, and my spelling is crap. Does this make me less of a person. No.
dvavasour said,
July 10, 2010 at 10:07 am
I think we should have a simple labelling scheme for claims made on labels, maybe a traffic light scheme so that claims with evidence are printed in green, and claims which have no evidence must be printed in red.
Or maybe a warning, at least as big as the claim itself, which says “we just made this up”.
Of course, it would be too lengthy to put in place a regulatory framework, so it would need to be industry self regulation of its own made up claims.
Beaverman33 said,
July 10, 2010 at 10:35 am
Irrespective of what you are selling, you should be able to substantiate any claims you make about your product. Be it the fuel efficiency of a car, the mega pixel resolution of a camera or the health benefits of a food product or supplement.
Vilding1 said,
July 10, 2010 at 12:01 pm
@Keefe
I think allowing companies to add unsubstantiated assertions next to verifiable claims would be needlessly confusing. Better to make it simple: If it can verifiably treat ailment A, you can put it on the box, otherwise not.
Consumer choice depends on access to accurate information.
Trinoc said,
July 10, 2010 at 12:19 pm
I think it’s time we rehabilitated the term “snake oil” as a generic term for a dodgy medicine supported more by marketing hype than evidence. Snake oil was actually fairly effective, by the (rather low) standards of its day, and the people who derided it were the real charlatans selling genuinely ineffective or dangerous medicines.
From now on, I suggest peddlers of evidence-free medicines should be called “fish oil salesmen”.
frnknstn said,
July 10, 2010 at 12:37 pm
Forgive me for stating the obvious, but I believe this statement from PowerHealth bears repeating: “there will be no information on packs for the consumer to assess what the product is supposed to do.”
Is there nobody in their organisation that realises that it doesn’t matter what the prodduct is /supposed/ to do? We need to know what the product ACTUALLY does.
Guy said,
July 10, 2010 at 1:03 pm
Good write up Ben. How can the industry expect to get this licensed when the only relevant study was unpublished and not available to the general public.
As well as not being peer reviewed, we have no idea how many primary endpoints it had, or whether they went on a fishing expedition after the evidence was collected.
We are also told
“Results showed a significant improvement in the Digit Span test in the treatment group as compared to placebo after the first three months of the study.”
There can be a big difference between statistically significant and clinically significant results. In reality we have no idea if this really was statistically significant because we don’t know how many trials they cherry picked this from.Did they check if the blinding was effective? Did they state before hand what a clinically significant result should look like and whether this was achieved? I doubt it!
No doubt the Daily Mail Morons will put this all down to the EU interfering with their rights to con ignorant people.
CoralBloom said,
July 10, 2010 at 3:48 pm
“The regulation is killing this industry and the job losses are already being felt,”
Look the banks didn’t like the regulation, lobbied for change and got it. Result was a bit of a mess.
The oil companies didn’t like regulation and lobbied hard to have the regulations changed in the US. The regulations were largely ignored and look at the mess.
I’m currently reading one of the best books I’ve ever come across – it is about the anti-slavery movement in the UK. Guess what? The slave traders cried the same song – it will harm our industry, it will harm the economy, it will cost jobs….
The industry would be creating jobs if they set about testing the stuff they sell and then made appropriate claims because that would require someone to do the work….
Wouldn’t we rather people worked creating things that were effective, useful, rather than working at a job creating at best Scots Mist?
At the end of the day the regulations protect us, the public and help us make informed choices.
SimonW said,
July 11, 2010 at 4:39 am
>> Even “established claims like cranberry for
>> urinary tract health” are being rejected
This was probably one of the easier claims to reject. Cranberry juice and extracts have been repeatedly trials for UTI, and shown to be ineffective. We can be extremely confident that Cranberry juice isn’t useful in treating UTI.
I’ve used it as an example of a case where people have over investigated a claim because the alternative medicine folk say it works, and there is a financial interest, and it is relatively easy and cheap to study.
The depressing thing is that Cranberry juice has antibacterial properties, and some of the studies suggested it might be useful, but there has been little work studying the effects of Cranberry juice on bacteria it can come into contact with by drinking which would have been equally easy to do, and likely would have satisfied the financial interests of those who profit from sales of Cranberry juice and its extracts.
memeweaver said,
July 11, 2010 at 7:16 am
@orpheus66. I heard similar complaints from the Australian gambling industry a few weeks back. Never mind the destroyed lives, families and such – think of the jobs lost if the gambling industry were in any way curtailed!
apothecary said,
July 11, 2010 at 10:46 am
The other thing is that there is positive evidence of [b]harm[/b]from some antioxidant supplements:
Antioxidant supplements for prevention of mortality in healthy participants and patients with various diseases. Goran Bjelakovic et al, Cochrane review DOI: 10.1002/14651858.CD007176, www.mrw.interscience.wiley.com/cochrane/clsysrev/articles/CD007176/frame.html
“We found no evidence to support antioxidant supplements for primary or secondary prevention. [b]Vitamin A, beta-carotene, and vitamin E may increase mortality.[/b] Future randomised trials could evaluate the potential effects of vitamin C and selenium for primary and secondary prevention. Such trials should be closely monitored for potential harmful effects. [b]Antioxidant supplements need to be considered medicinal products and should undergo sufficient evaluation before marketing [/b]”
Of course, given the heterogeneous nature of the trials and conditions examined, this is a very big picture summary. But I wonder how widely this is known. Or indeed the fact that NICE says people who’ve had an MI should be advised not to take beta-carotenes supplements (increased mortality)
And another thing! I went to my optician recently and they were selling a product for ‘eye health’. I don’t want to put the name in case I’ve remembered it wrongly, but it was clearly linked to macular degeneration (AMD). I looked at the literature. There is reasonable evidence from the US NIH-funded AREDS trial for a mix of vitamins and zinc to prevent further progression of AMD in people with moderate AMD already (lots of products available, but only 2 or 3 which match the AREDS formula, btw). But another Cochrane review found “There is no evidence to date that the general population should take antioxidant vitamin and mineral supplements to prevent or delay the onset of AMD”.
www.mrw.interscience.wiley.com/cochrane/clsysrev/articles/CD000253/frame.html
Put the two Cochrane conclusions together and selling these ‘ocular health’ products is not good, IMHO
(apols if the formatting in this post is not good – not my strength)
pv said,
July 11, 2010 at 12:12 pm
And if the product does nothing supported by evidence then there will be no information on the pack. Simple, eh!
But won’t somebody THINK OF THE CONSUMERS?
Big M said,
July 11, 2010 at 12:54 pm
Personally I think something like dvavasour said, with a two-tier labelling scheme should be brought in.
People would be permitted to peddle all the fish oil, glucosamine, st john’s wort etc. that they want, and make whatever claims about it they want, but a big MHRA stamp (the UK equivalent of the American FDA, for those from across the pond) should be smacked across the packaging. As long as the product is not harmful in itself, it gets a stamp, but the colour would be different depending on whether the claims are backed up. It would get a green stamp if the product has been proven to the satisfaction of the regulator to do everything it says (rather than for each separate claim), and a red one if it hasn’t.
But frankly if firms make claims that are either demonstrably untrue or for which there is no evidence one way or the other, I see that as tantamount to fraud, and they should be prosecuted as such (fines for the companies, criminal sanctions for individuals making the decisions).
topcat said,
July 11, 2010 at 1:50 pm
Well, at least Danone actually seems to be making an effort to do some proper research into its claims:
www.newsbiscuit.com/2010/07/07/danone-announce-discovery-of-digestifus-flirtius-bifidus-jocularis-others/
“In a development sure to send shockwaves through both the scientific and probiotic communities, research scientists from Danone, makers of Activia yoghurt and Actimel drink, today announced the discovery of several new types of bacteria that were neither friendly nor unfriendly, but often completely indifferent.”
IMSoP said,
July 11, 2010 at 10:45 pm
My favourite riposte to the industry directly from the EU comes at the bottom of this FAQ-style Press Release: europa.eu/rapid/pressReleasesAction.do?reference=MEMO/07/267
“It should be noted that health and nutrition claims are voluntarily put on products by producers as a marketing tool. If positive claims cannot be established, the Regulation does not oblige anyone to make negative claims about the product.”
In other words, “stop whingeing and come up with a marketing strategy other than lying through your teeth!” 😀
JaredBond said,
July 12, 2010 at 8:05 am
I just have to remind everyone that the drug companies are not that much better. Maybe they have more money to make it look more legit, but I believe a lot of their products, such as antidepressants for starters, are unfoundedly hyped and mostly worthless. Bullshit seems to be the way of the world.
Trodamus said,
July 12, 2010 at 3:00 pm
JaredBond, that’s an entirely separate issue and I think you’re letting your personal views influence your outlook on legitimately researched claims. The snake oil part of pharmaceuticals is when the patent on their formula expires, so they make a minor change to the molecule in order to maintain their branding with a “new” product.
Inevitably, people will come down with the idea that these supplements should remain with their claims on the box, and that consumers can purchase them if they wish. It’s their right, after all, and who wants a nanny state?
What I think people fail to realize is that these claims are alluring, and these companies will never be forthright in the research into the claims. In the US, all they place on the label is a disclaimer that their claims have not been verified, though in many cases their claims have actually been disproven outright. That never goes on the box — “This product has been empirically proven to not do what we are claiming.” As long as it’s allowed in the first place, they’ll find the best way of packaging it so it doesn’t matter.
People in general aren’t necessarily stupid, but when you’re surrounded by products promising you a better, healthier, smarter life you can’t help but be tempted. I worked in a drug store for a few years, and while I referred to the aisle with the supplements as the “fake medicine” aisle, years of staring at products promising healthy, pain-free joints made me start to consider actually buying whatever product it was as I’d destroyed my knees some years previous. I never purchased it, but imagine the people that actually use these in leiu of seeking legitimate medical attention?
This is why harsher regulation is needed.
Guy said,
July 12, 2010 at 3:18 pm
“JaredBond, that’s an entirely separate issue ”
No it isn’t. The drug companies do very similar things, they just spend more money on them and have been doing them for longer, so are more superficially convincing.
Many drug company claims don’t stand up to scrutiny and yes, they are a bit better than the snake oil salesman but not by much. You didn’t answer his point about antidepressants???
Jon d said,
July 13, 2010 at 4:53 am
Dunno how much difference this is really going to make. If you have a look at the stuff on the shelves in Holland and Barretts a lot of it doesn’t have any claims on the packaging. People find out what the stuff’s supposed to do from asking the staff, reading the H&B magazine or from the ‘new miracle berry’ style articles in the health and beauty sections of newspapers.
Craig said,
July 13, 2010 at 8:36 am
Unfortunately, stealing money from gullible sick people is harmful in itself, even if the fake medicine isn’t directly poisonous. Peddling snake oil is not a victimless crime.
lasker said,
July 13, 2010 at 1:45 pm
Surely the problem here is, if you have a child for whom you wish to optimise the diet in order to promote intellectual ability then apart from the provision of a balanced nutritious diet then what would you do? Fish oils seem to have benefits in children with defined deficits so in the abscence of anything proven they seem worth a try.
Trodamus said,
July 13, 2010 at 3:05 pm
Guy > Drug companies funding trials that are biased towards producing favorable results is a well-documented phenomena on this very blog, separate from supplement peddlers that do no research when they aren’t, on occasion, producing conclusions out of thin air. In the former, they’re more likely to play up the benefits while downplaying the side effects, while in the latter they are stating something that is patently untrue.
And the point of “Antidepressants are largely worthless” is a loaded statement, considering the flak psychiatry gets from…certain groups. Antidepressants aren’t a miracle-cure for depression, and they are over-prescribed for an over-diagnosed condition. However, combined with other medication and careful monitoring, they can help manic-depressives get out of bed on a bad day; hardly worthless.
Guy said,
July 13, 2010 at 4:32 pm
Trodamus, I wish I had as much faith in big pharma as you do. The file drawer effect where you only admit to and publish positive trials is rather more than bias towards favourable results!
Modern antidepressants is a great example. When a Freedom of Information request was put in in the states and ALL the trials, not just those published were analysed, then anti-depressants were found to be no better than placebo. A sad fact that I find it hard to come to terms with but undeniable.
He actually said “such as antidepressants for starters, are unfoundedly hyped and mostly worthless” which about sums it up.
Trodamus said,
July 13, 2010 at 9:55 pm
I have no faith in big pharma. I know absolutely they’d rather more people were using their products than less, and were they able to break with the 7-year patent, they’d happily gouge everyone eternally.
Businesses pick up on this; drugstores such as Osco, CVS and Walgreens can literally be on every street corner due to the profit margin provided by Rx. Little is actually being done to prevent Rx abuse, as filling vicodin or oxycotin for an addict or legitimate sufferer of pain offers the same boost to the bottom line.
And then we accept trials they fund.
Those antidepressant studies show that they tend to have the greatest, non-placebo-mirrored affect on the most depressed, which basically supports what I said about them being over prescribed. The mid-range is where you’ll find them working about as well as a placebo, but all in all this isn’t the same as them having “no effect.” But, given what I’ve said above, it’s certainly in the companies’ best interests to overdiagnose and prescribe for something that’s probably best treated through other means.
Xobbo said,
July 14, 2010 at 8:21 am
So they did 20 studies, and found a statistically significant positive result in exactly one of them?
What are the chances of THAT happening?!
Big M said,
July 14, 2010 at 11:05 am
Craig said
Peddling snake oil is not at all harmful in and of itself. What is harmful is misrepresenting what your product is and does, thus persuading sick people to forgo treatment proven to be effective, in favour of your ineffective alternative.
Introducing a clear bold labelling system that lets consumers know at quick glance whether the claims are backed up with science would immediately end any notion of ‘stealing money from gullible sick people’, because those people would know from the start that the product with the bright red stamp on the box wouldn’t help them, whereas the product next to it with the bright green stamp would.
Nothing is certain in life, so we should simply accept that some people are going to do stupid and self-destructive things despite our best reasonable efforts to help them. If consumers choose to buy something that essentially states on the packaging that the claims are bogus, then frankly it’s entirely their own fault if they remain sick and I won’t lose any sleep over it. They will have had fair and reasonable warning. That’s all we can, and should, give them.
quietstorm said,
July 14, 2010 at 6:57 pm
Poor alternative health industry, losing all those jobs. Perhaps these companies could instead hire some people to figure out what *does* work…?
[yes, naive, I know 😉 good article though, and it’s great to know that EU stuff is all online]
Makemonsters said,
July 15, 2010 at 11:33 am
My favourite comment in the article:
“It can take three years to get these kinds of human studies together but in the meantime the claims are going to be wiped away,” Ioannis Misopoulos
Perhaps they should have undertaken the studies before publishing the claims….
suggsygirl said,
July 15, 2010 at 10:07 pm
Personally I’ve become so cynical of any claims made by large pharmaceutical, or food companies, or in actual fact by any large company, that I simply don’t believe anything they tell me. Cuts out the need to legislate against lies. Sort of the opposite of innocent until proven guilty.
red5 said,
July 17, 2010 at 3:58 pm
Your article is further supported here:
“The danger of science denial”
www.ted.com/talks/michael_specter_the_danger_of_science_denial.html
elflojo84 said,
July 26, 2010 at 4:58 pm
“there will be no information on packs for the consumer to assess what the product is supposed to do.”
That is brilliant! SUPPOSED to do! That’s really the best argument they’ve got?
pberry said,
July 27, 2010 at 12:37 pm
Thanks for posting that lecture, red5. Pretty illuminating and contains a pretty strident catchphrase: “You’re not entitled to your own facts.” Definitely a phrase stored for future use.
gwpurnell said,
August 16, 2010 at 4:07 pm
@CoralBloom
As someone born, raised and living in Edinburgh, I can assure everyone that Scotch Mist is much more substantial than most of these claims.
On the subject of our gullibility to healthcare chicanery, Rose Shapiro’s book ‘Suckers: How Alternative Medicine Makes Fools of Us All’ is a jolly good read. It will also shame people like Trodamus (if they ever read it) from ever using the phrase ‘big pharma’ again.
joey89924 said,
November 16, 2012 at 7:35 am
Great post. I like it very much HT1621 Circuit