Hhahahahhahahhahahaaa they can’t help themselves, they love it. Two features in the same day! Apparently there is a third to come. I haven’t been able to get out to the papers yet today, are these actually on different pages or are they part of the same feature? Please tell me they’re not two separate pieces. Hehehhahahaaaaa…
www.guardian.co.uk/g2/story/0,,1920102,00.html
education.guardian.co.uk/schools/story/0,,1920302,00.html
Dr Aust said,
October 13, 2006 at 9:48 pm
Ben wrote:
“For example, they introduce their sceptical take on omega-3 and intelligence with: “The scientific evidence that eating Omega 3 is beneficial for heart health is well established. But… †(The Money Programme).
This is patent nonsense. The Cochrane review reprinted in the BMJ (below) is very clear: there is not evidence for omega 3 fats having an effect on total mortality, combined cardiovascular events, or cancer. This was even reported in the popular press at the time. If you’re a journalist, and you don’t know enough about the background, if you don’t even know enough to check Cochrane, then it’s meaningless for you to be writing on the subject. Go and write about Blair or Britney instead. You’re not informing anyone.
bmj.bmjjournals.com/cgi/content/full/332/7544/752
…………”
Tricky… it is a little less clear-cut than that, I think. The rapid responses to the Hooper et al BMJ review are worth scanning through, and make an interesting intro to the difficultues of doing meta-analyses!! There is a lot of debate about which trials the authors included (or not) and how they weighted them, and how this “loads” the conclusions. The response by the authors (about 60% of the way down) is particularly interesting.
The general thrust of what they are saying is that omega-3s are probably good to have in your diet (e.g. from eating fish), but that the evidence for supplements to boost omega-3 levels doing anything for you is weak.
So the Money Programme’s statement is probably too strong, but the articles that appeared at the time claiming the Cochrane review totally blew away omega-3s were also over-statements.
The real summary of the trial and the authors’ follow up might be re-phrased as:
“Lots of suggestive evidence omega-3s are good things to have in your diet, but evidence from RCTs w supplements not as conclusive as earlier studies had suggested, indeed marginal, but there are some problems with this too. More bigger RCTs with all relevant safeguards and measurements are needed, and in the meantime it would be sensible to keep eating fish.”
Dr Aust said,
October 13, 2006 at 9:56 pm
The right has always been keen on “mysticism” in the wider sense. Did anyone say “blood and honour”? True even in the 20th century when they were also into portraying themselves as powered by leading-edge technology.
Being a bit fanciful, one can still see this in GeorgeW, where the American belief in super-high-tech solutions (including bombs) as the key to solving problems is married to a belief in the power of prayer to fix things.
Of course, it may also be about being all things to all men, a tendency not confined to the right – see our current Dear Leader and his Tory doppelganger DaveyCam.
Ben Goldacre said,
October 13, 2006 at 10:23 pm
yeah, adorno’s take on this is that it’s all about fascists craving order (from eg astrology and tarot) where there is chaos and diversity. to me it’s also about individualism: woo is rarely about community, it’s about how a pill can solve a complex social problem, or an individual can have special healing powers. social inequality is the primary determinant of ill health, but the whole nutritionism project revolves around the notion that complex systems for interpreting individuals’ diets are the key.
perfectly happy with your formulation of the cochrane review but the money progs statement was foolish.
Dr Aust said,
October 13, 2006 at 10:37 pm
Well, hardly needs stating that insurance-based (esp. US-style) healthcare vs. universal trad. old-NHS healthcare is another paradigm of the individual vs. community and right vs left thing.
In the freeat-point-of-access universal system, SOME sort of rationing will always occur since “community” needs (e.g. effective early detection of cancer, perhaps via screening) will have to be prioritized over “individual” ones (£ 35K / yr monoclonal Ab drugs to extend survival times in metastatic colon cancer).
In the “individual” system you pay upfront to be absolute no. 1 priority, cost no object – Thatcherite look after you own-ism in microcosm.
The fact that the second system, esp. as practised in the US, systematically excludes the poor and disadvantaged and makes rapacious private corporations rich is just an unfortunate side-effect… or maybe Darwin in action, as I suspect many on the right see it after a few G&Ts.
Incidentally, when you look at it that way , in the “individualist” paradigm it is your (PAID) right to insist on being sent for homeopathy if that’s what you want (you’ve PAID!), regardless of the waste of scarce resources. In contrast, in the NHS resource-limited model there is necessarily something better to spend the cash on
Barnacle Bill said,
October 14, 2006 at 11:02 am
Just wondering if Matthew, Mark, Luke and John were arts graduate journalists. Headline, “Man walks on water”, “Man feeds 5,000 with leftovers”.
Leading nutritionist said,
October 14, 2006 at 12:15 pm
Ref #51/52
While I am in total agreement with you that it is unwise to conclude that fish oil intake does not reduce CHD risk (several, but not all, large scale studies have shown significant effects plus meta analysis is not a foolproof method for determining an effect) there are difficulties with showing a benefit of nutrient supplementation using a RCT. In a drug trial, you would randomize your participants to a treatment and placebo group and look for differences in the outcome measure. Crucially, you can be pretty confident that your placebo group is not going to be consuming any of the treatment drug. Unfortunately, using the same paradigm in a nutritional study, you cannot be sure that your placebo group is not already consuming adequate amounts of your treatment substance (individual requirements can differ widely and tools to determine habitual food intake are nothing short of woeful). Indeed, it is likely that once you reach a threshold of intake for most nutrients additional intake is not going to have much of an effect (thus the null hypothesis is unlikely to be rejected). This is certainly not an attack on RCTs, clearly they are currently the gold standard. It is just nutrition studies have a certain amount of problems associated with them that need to be addressed.
With regards to supplementation, I think the there is potential for its use among targeted groups of the population but it is unlikely to have significant population effects. Of course, getting these nutrients from the diet is perhaps optimal, but getting people to eat a ‘balanced diet’ is very difficult.
Dr Aust said,
October 14, 2006 at 1:03 pm
Re post 56:
This problem with “comparability” between groups in nutrition trials is one of the things hashed over in the rapid responses to the Hooper et al. BMJ meta-analysis.
One extension of this, again argued on the BMJ responses, is that what is needed in the trials is a clear and standard “index” for total dietary omega-3 intake / levels in the body so that people either taking (or not) “nutritional advice” can be “graded” according to real levels, rather than just divided into nominal sub-groupings.
Like anything else, one thing that research gradually reveals is ways to do the research better, e.g. in avoiding / controlling for confounding factors. See also Ben’s column today about the controls (or lack of them) in the “hunting measles virus mRNA in gut specimens” studies.
doctormonkey said,
October 14, 2006 at 2:07 pm
ref #56
i think this exactly and precisely WRONG and is the same argument made by CAM about why RCTs don’t work for them.
the whole point is that everyone in a randomised placebo controlled trial is that everyone thinks they are taking the supplement and you correct so that this is the only intervention seperating the two groups.
participants in the RCT won’t have a pill or not, they all have an identical pill, and then researchers find out what happens to them (not knowing which they have taken) and only then another researcher, seperated from any direct contact with the people taking the pills, looks at how people have done and whether they took a real pill or a fake/placebo one.
Leading nutritionist said,
October 14, 2006 at 3:06 pm
# 57
Yes, I am aware of how an RCT works and, as I said, I think it is the gold standard. However, if I select two groups at random from the population and give them drug x, I can be pretty sure that the only difference (if they are selected completely at random) is that one group is consuming drug x and the other isn’t. The difference in outcome measure can then be attributed to drug x. For nutrition studies, this is, perhaps, not possible. If I select two random groups from the population and give one supplement x (say fish oil to be topical) and the other the placebo, how can I be sure the placebo group is not consuming an amount of the active ingredient contained in supplement x that is sufficient for health benefits? I can suggest ways, but I don’t think they would be acceptable to any ethics boards. If it was for a specific food, you could proscribe use of that food, clearly there are problems with that in terms of blinding etc. But with specific nutrients that are widespread in foods it becomes more difficult.
Clearly, this a weakness in nutritional studies that needs to be acknowledged, addressed or accepted. My personal view is that it should be addressed. My comment was on an issue that faces nutrition research (serious nutrition research), not a defence of indiscrimanate supplement use. For many people, supplements are not going to have any effect at all, but there could be sub-groups who benefit.
doctormonkey said,
October 14, 2006 at 4:47 pm
#58
surely the lack of difference between intervention and placebo groups is as significant as any difference as it might show that supplementation is not needed as the “normal” diet (whatever that is) contains plenty of what is being supplemented OR that it will not effect what is observed, even if it does have good effects on something else
this is the problem with the publishing bias against good negative studies
sorry if the above felt like a personal attack, it wasn’t meant to be, i just get frustrated by the complaints about problems with RCTs that seem to me to be poor study designs rather than a flaw in the whole system of RCTs
the other point i was making was that in a good study the placebo group should not be taking other supplements and should be told not to, explaining it might spoil the study, and should anyway think they are taking the supplement (possibly) and so people taking the supplement studied are at as great a risk of such doubling due to the over-the-counter nature of the products studied
Leading nutritionist said,
October 14, 2006 at 6:14 pm
# 59
No apology needed! With your first point I agree. Supplementation is probably not needed in the general population. Obviously there are caveats, but if an experiment is designed well, no effect would suggest no need for widespread supplementation (or even fortification). But, care also needs to be exercised to conclude that the intervention might not be useful for some sub-groups.
I wasn’t complaining about RCTs per se and I genuinely believe it is the gold standard for intervention studies. Sometimes it is not possible to use such a system. For instance, I have recently completed studies that fall short of the RCT ideal. Funding limited the sample size so a cross-over design weas used. As the intervention was an actual food, it was difficult to blind anyone or use a placebo. Is the evidence provided by such a trial as strong as an RCT, definitely not. But it is evidence nonetheless. It requires a more cautious interpretation. This is completely lost on the press and I have been misquoted and selectively quoted by journalists…egged on by our industry funders PR company. It makes you look a twat when a solid study, with limitations, is reported as the final word.
I agree with your sentiment. But the problem comes is if the supplement is available in the diet. If I wanted to look at the effect of vitamin C supplementation on disease risk (because of good theoritical or epidemiological evidence) I could give the treatment group Vitamin C tablets and the placebo group, well a placebo. But the problem is, the placebo group are likely to consume a significant amount of Vitamin C each day from the diet. Short of making them deficient, it is impossible to get around this (if we randomaly select from the population). As this study would now appear to favour the null hypothesis, the conclusion may be reached (or reported in the press) that vitamin C is not important to health. If we selected a group that was deficient in Vitamin C and then split them into treatment/placebo, the effects may be more notable. That is just a hunch mind!
Clearly, there are issues with some clinical nutrition research which means you have to be careful in interpreting it and making recommendations based upon it.
I apologize if I sound overly defensive. Nutrition research really seems to polarize people and there is a balanced view!
doctormonkey said,
October 14, 2006 at 8:01 pm
i think you have defended nutrition research and the difficulties well
there is a problem that RCTs are a medical research tool and nutrition crosses the boundary between medicine and “life” which is far harder to fix in place as one can medical research.
the problem comes when nutritional research is abused to provide medical-type claims (such as the fish oil pills) without the medical-type back up to the claims (such as the fish oil pills)
this is why there is legislation to protect the role of doctor (among others) from people claiming to be one without the training etc but there is not the same thin (rigorously enough) for treatments. you could argue that a lot of CAM practitioners fly rather close to the wind against the legislation protecting health professionals’ status
Dr Aust said,
October 14, 2006 at 9:09 pm
All the above is why something you can actually MEASURE on people which effectively tells you something about their background / baseline intake is needed. For the Vitamin C example in #61 one would at least want to MEASURE blood and perhaps tissue Vit C levels, and maybe Antioxidant indices, periodically, so that one would have a fighting chance of seeing if (for example) supplements were really only benefitting people who effectively STARTED with deficient levels, or if people in the “non intervention arm” actually supplemented themselves on the sly.
A paradigm for how one does science – a lot of the time the research ends up not getting you closer to “the answer”, in the sense of “do we now have a better idea whether X helps or not”…. EXCEPT that the attempts end up telling you how better to ASK the question next time/
…Another thing that journalists who write about science mostly do not understand.
Ben Goldacre said,
October 14, 2006 at 9:55 pm
mmm i think there’s a much simpler issue here that some of you are kind of missing. in appraising any study, you have to ask the question: “is the population in the study the same as the population i want to apply its findings to?”
if a trial shows that a population of average normal everyday fairly well nourished western people do not benefit from a supplement, then there is no sense in recommending that supplement to that population. when i see someone recommending a supplement to the general population, i go to trials performed on that population, if they are negative, then i say the advice is nonsense.
you can talk about people who are undernourished to your hearts content, if you want to see if they benefit, then decide on a way to select them, and do a trial on them (or find a trial where someone else already has). perfectly simple, not a limitation of the RCT design at all, it’s just a question of which population the trial is in, and then which population you want to apply the findings to.
Robert Carnegie said,
October 15, 2006 at 3:15 am
Re perils of castor oil. On tonight’s “QI” panel game Stephen Fry brought up the subject of aeroplane “dogfights”, mentioned an affection for Biggles and sketched the learning curve described by Biggles’ author in how planes in the Great War began to fight other planes by dropping heavy weights on one another. Mr Fry also claimed that pilots in that war had extraordinarily loose bowels because the engine bearings were lubricated with castor oil. Presumably the pilots breathed it in, but I don’t recall Biggles mentioning it much.
You also don’t hear a lot about astronaut toilet arrangements. On the way up they can’t really stop and get out, so as far as I recall they fly in nappies. I suppose acceleration isn’t kind either.
doctormonkey said,
October 15, 2006 at 10:51 am
sorry to not respond to #65 (i missed the programme)
#64 etc
i think Ben makes a very valid point that renders much “good” research unusable, the use of experimental groups too different from the populations we would look to apply them to. there are various reasons for this and it is not a case of “blame” but the problems of doing a study in the real world.
it was a frequent topic of debate at most journal clubs (geeks getting around and picking apart a paper) – not only is it a good paper but can we say we see the same thing in sufficiently similar people. often the answer was no.
examples include a big German study showing St John’s Wort works for mild to moderate depression as well as conventional antidepressants, a good study BUT none of the drank alcohol (certainly not to excess) and there are also the problems with the interactions of St John’s Wort with almost everything else (due to cytochrome P450 upregulation, if i remember correctly) – we decided we might well not be able to use it on most of our patients as many self-treat low mood/depression with alcohol before seeing a doctor about it.
therefore, Leading nutritionist, i think it is just that you are more concious about these confounders to your nutrition studies and honest about them!
raygirvan said,
October 15, 2006 at 11:59 am
lubricated with castor oil
Yes, specifically rotary engines like the Gnome. (Google confirms). I remember reading this way way back in an Analog SF story, with the added detail that pilots drank blackberry brandy, believed to be an antidote.
Leading Nutritionist said,
October 15, 2006 at 12:41 pm
# 62
There is an issue with journalists taking results from a study and extrapolating them unreasonably. Medical claims on products are a different and I believe are protected by law. I think every ones beef here is about poor reporting of scientific results by the press.
# 63
Certainly, and this frequently happens. Two issues there, if you are selecting people on baseline characteristics you no longer have a random sample. There are also issues with post hoc subgroup analysis. Recruiting a huge sample so that you can look at coorelations between baseline characteristics and effect may be prohibitively expensive. The other, some people may have very plasma levels of a nutrient but is more than sufficient for them. It is not exactly known what a deficient level is! In addition, antioxidant status…we could use ORAC or TEAC to measure that, but there is substantial debate about what the results actually mean. Modern nutrition science is reasonably young science and it may be some time before we can really determine its role in health and disease.
Ben
The point I was trying to make (badly) is that rather than be enthralled with p values and confidence intervals and the type of study done (although these are all important), it is still the interpretation of the data that has precedence. That will require some knowledge of the strengths and weaknesses of the study type and whether there are any confounding factors. My point merely was that baseline intakes could confound RCTs of nutritients and is something to bear in mind.
If a supplement is given to a population and it has no effect I think we can conclude that advice for everyone to use this supplement is unwarranted. But to conclude that that supplement has no use, or even worse, that nutrient is unimportant to health is wrong. Newspaper reports go one of two ways (in general) – something is the greatest or it is useless. I get the impression it is this hyperbole that upsets most people on this site.
I agree with you about targetting interventions to groups. However, there are some caveats. If I took a group that was deficient in a nutrient, that strong epidemiological data, animal studies and maybe short-term intervention studies on disease risk markers show positive effects of that nutrient, and we randomized people to treatment and placebo there may be ethical problems. Certainly if you were to look at disease progression or mortality. While I think good science is important, if you found that the placebo group exhibited a higher prevalence of disease or had higher mortality than the treatment group this would be tantamount to negligence. this doesn’t always apply of course.
My comment wasn’t really sugesting a limitation of the RCT per se (although it does have limitations), like any experiment you have to be aware of confounders. If there are confounding factors, you have to interpret the data more carefully.
# 66
Thank you, confounders of experiments cause many sleepless nights!
ceec said,
October 16, 2006 at 12:13 pm
NB as a footnote to the “problems with RCTs for nutritional supplements” discussion, another problem might be that during the informed consent etc. procedures, lots of info about fish being natural source of omega-3 etc. might encourage both groups to increase intake of those foods “just in case” they are in the placebo group. If the benefit comes from removing nutritional deficiency rather than increasing intake per se, any benefits would be masked.
Then again, if telling people they need fish in their diet made them eat fish, McDonalds might not be doing such a roaring trade.
orangejo said,
October 18, 2006 at 9:56 am
I’ve just noticed Netmums are promoting Equazen’s products….
www.netmums.com/lc/fishoils.php#offer
Loads of mums use this site and they even have a little box saying, here’s the science bit.. and hey presto.. it takes you to a load of publicity from Equazen’s site. And that’s weird, it tells you that it’s a really good idea to buy a load of their stuff and give it to your kids because science says you should….
So, not only have they got the press on their side, they’ve also got a website used by many mums who are often too busy to read a newspaper, to be converted into their way of thinking.
I think we can’t underestimate the power of mums’ websites and forums. The whole MMR scepticism was allowed to grow because people ranted and raved about it on mums’ forums and allowed it to get into the national psyche and become institutionalised paranoia.
The damage may well have been done already with the Omega3 stuff – mums will never UNbelieve it now.
Also – is it possible nowadays to buy anything from marks and spencers that doesn’t say Contains Omega3 ? Do I want fish in my cheese?!? Oh, but it’s good fish that a man in a lab coat told me was good for my children. oh, that’s ok then (!)
kim said,
October 19, 2006 at 9:08 am
Here’s what Felicity Lawrence has to say:
commentisfree.guardian.co.uk/felicity_lawrence/2006/10/omega3_junk_food_and_violence.html
At the end she says she’s planning a response here, so should be interesting.
Dr Aust said,
October 19, 2006 at 11:13 am
Re post 71: On the Guardian’s “Comment is Free” page Felicity Lawrence writes:
“In the paper version of yesterday’s Guardian we reproduced a graph showing the correlation between the rise in murder rates in the UK and the increase in consumption of omega-6s from vegetable oils mainly in processed foods….
..Someone wrote to our letters page to say our use of this graph was “sensationalist”.
Fame at last — that person was me.
I eagerly await Felicity Lawrence’s response. In the meantiime, here is what I actually wrote to the Guardian:
“Dear Editor
Felicity Lawrence’s extended spread on omega-3s (“Omega-3, junk food and the link between violence and what we eatâ€, October 17th) alludes slightly coyly to the “backlash…from scientistsâ€â€“ including Ben Goldacre’s Bad Science column and its readers – over “the hype surrounding omega-3â€.
She blames the “eagerness of… supplement companies to suggest that fish oils work might wonders even on children who have no behavioural problems†but does not mention the other thing that annoys the scientists – the over-promotion of these claims in the media and the credulousness of much of the reporting.
Lawrence’s story uses as a “Case Study†the progress made by nineteen Wiltshire special school children given a fish oil supplement. At least this time Lawrence identifies Dr Jackie Stordy, quoted here and in a News story on the same topic last week (“Severely troubled boys “soothed by fish oils†October 12th), as a former paid consultant to fish oil supplement companies. However, there is still no mention that Stordy is a long-time advocate of fish oil supplements and author of a book plugging them (“The LCP solution: The remarkable nutrition treatment for ADHD, Dyslexia and Dyspraxiaâ€).
Having given all the caveats – “not a trial… tiny numbers… no placebo or control [group]†– Dr Stordy then gives us the punchline – “[giving fish oil supplements] obviously had an impactâ€. Paraphrased: even though this was done in a way that ensures it could provide no reliable indication of anything, giving fish oil “obviously workedâ€. No wonder scientists are grinding their teeth.
Finally, Lawrence’s article showed a graph of the UK murder rate plotted against the omega-6 fatty acid content of the diet. Although this diagram came from a scientific review, its use here seems deliberately sensationalist. I could probably have plotted the UK murder rate in the same way against the mean number of video recorders per household, or even the weekly salary of footballers at top clubs, and got a similar graph. But none of these correlations would prove anything about causation.”
Dr Aust said,
October 19, 2006 at 12:02 pm
While we’re here, something curious about ferlicity Lawrence’s latest FishOil double-pager.
www.guardian.co.uk/medicine/story/0,,1924089,00.html
It quotes prisoners and experimenters in what it says is a “Double blind trial” of fish oils in a US prison.
But…. it quotes the nurse hading out the pills as saying he has seen major changes IN THE TREATEMENT GROUP.
…..??????!
If he knows who is getting the pills and who is getting the placebo, it’s not a double blind trial.
..and if he knows, almost certainly the subjects do to, as the experimenters cannot really help tipping them off, intentionally or otherwise.
[There are also quotes from the prisoners about how much better they are doing, again suggesting no blinding of the study]
All this strongly suggests that this trial, which occupies about a third of Felicity Lawrence’s article, is not what she says it is. If it is not properly double-blinded, its scientific value is marginal at best.
This just serves to re-emphasise what Ben and the rest of us Obsessives have been saying here: if non-science literate journalists write “features” about scientific subjects, you get problems because they do not understand the nature of scientific evidence.
(Thank to erasmus from the forums and others for pointing out the mistakes in Lawrence’s article)
Dr Aust said,
October 19, 2006 at 12:17 pm
PS A curiosity:
If you look back through the Guardian’s Comment is Free you will find Sarah Boseley, the Guardian’s senior health writer, telling us back in March this year (on the basis of the BMJ meta-analysis discussed about in posts #47 and #51) to:
“Throw away your fish oil pils”
commentisfree.guardian.co.uk/sarah_boseley/2006/03/throw_away_your_omega_3.html
Interesting to see how many different writers have written about fish oil pills in the Grauniad:
Felicity Lawrence (consumer journailst)… was skeptical, now getting pro-oil
Lucy Atkins (womens’ health issues journo with arts B/G) neutral-ish
Tim Radford (veteran hard bitten journo) neutral-y “reporting the debate”
Sarah Boseley (science correspondent) anti-ish
Note that Sarah Boseley in the CiF bit favours the “eat a sensible diet, as that’s what your body is optimized for” line, effectively what the doctors and scientists (at least those without a paid interest in omega-3s) say.
felicity lawrence said,
October 19, 2006 at 4:52 pm
Where to start? Having been accused here of laziness, it seemed appropriate not to rush, though actually as well as finishing the long piece on nutritional deficiencies and antisocial behaviour for the Guardian
lifeandhealth.guardian.co.uk/health/story/0,,1924356,00.html
to give you all something more to blog about. I also wanted to check with colleagues on G2 what the thinking behind their piece on fish oils had been.
Ben tells me he really want to know why fish oil stories are so attractive to journalists. So G2 first – I’m afraid that one is actually down to you, Ben – they commissioned the piece because you had written yours and readers had told them they now felt confused, so they asked Lucy Atkins to summarise the various views. I thought she did a reasonably balanced piece .
Dr Aust said
Lucy Atkins is a health journalist with an Oxford first in English.
How far do you want to take this, Dr Aust? Should only trained priests be allowed to be religious affairs correspondents? Should only those GPs with qualifications in nutrition be allowed to comment on nutrition?
Anyway, my pieces. I write about the politics of food and have been exploring all the implications of the industrialisation of the diet in the last 50 years or so. The idea that we might have altered the architecture and functioning of the brain by radically altering the composition of the diet is boggling – of course we’re interested.
I proposed a piece on the links between diet and antisocial behaviour, based on new work at NIH in the US and older research here at Oxford and in Aylesbury prison. I had a discussion with my editors about the quality of the science involved. We all agreed that the main piece shd focus on the serious science but we were also keen to have some case studies, ideally including some in the UK, that would help ordinary readers see the implications while bringing the story to life. I had been told about the use of essential fatty acid supplements at the Cotswold school at a specialist seminar on fatty acids at the Society of Chemical Industry that I had attended while doing several weeks’ background research for the piece. My editors and I discussed the fact that a commercial supplement manufacturer was involved and that the treatment at the school essentially provided only anecdotal evidence. We agreed that anecdotal evidence had a place so long as we made the limitations clear. So I approached Efamol, visited the school and wrote the piece as a case study from my interviews. Any press release came later. I asked Jackie Stordy whether she was being paid by Efamol and she said that in this case she was not and was working pro bono, although she had worked for them as a consultant before. The fact that Efamol was involved was made clear enough, I thought. If I’d mentioned her book too you would probbaly have accused me of giving it a plug!
What happened then is that news events dictated timing. The big piece incorporating the Cotswold school interviews had to be pulled out of the paper at the last minute – General Dannatt needed the two pages back for his explosive statements on Iraq. But by then we had wind that one of the tabloids was planning to cover the Cotswold school story. We wanted to protect our case study from what my editor that day predicted would be a tabloid “yobs cured by pills†story – these rapid editorial decisions are made all the time; it is the business of newspapers to be exclusive and first wherever possible. So that part of the story got detached and we ran it as a short piece on its own with comments from leading academics that made clear what its limitations were. Would I have covered the Cotswold school story if I had not been writing the longer piece? Almost certainly not. Were we wrong to make the last minute editorial decision to run it preemptively? No, I don’t think so.
Jonno said
What about parental consent?
Yes, it was obtained from social workers who are in loco parentis. What is and isn’t allowed in dealing with these boys is about as regulated as it could be. Quite rightly.
And that leaves me with a last point yr discussion has prompted. I always feel humbled when I do-on-the-ground reporting on some of these most desperate cases. Staff at the school seemed to me to be taking a pragmatic approach in the boys’ best interests. There’s long historic evidence that fish oil supplements are safe. The head would rather improve the boys’ diets and had done so, but it’s tough changing the habits of children even in easier circumstances. For three boys the head said the supplements seemed to have made a really remarkable difference to their lives, when nothing else had changed to his knowledge. Can he prove they worked? No. Can you base policy on such anecdotal evidence? No. Should you dismiss his action and tell him to stop until you have the proof from a double blind placebo controlled trial, which might on the home office’s current form take another ten years to fund? What would you do if you were their GP, Ben?
doctormonkey said,
October 19, 2006 at 9:09 pm
felicity lawrence – hi and please take this as constructive criticism or counter points, not personal and not meant to be vindictive etc
i am afraid i have not managed to sort my responses in the same order you have given them:
if I were their GP i don’t think i could prescribe these dietary supplements with little or no evidence of benefit because:
a) i don’t think they can be prescribed, especially as most people prescribe from set lists of approved drugs from local formularies rather than anything you want, the panels that write these formularies would need to be shown that these would both work and be the best value for money
b) i don’t think they can be prescribed, they are not drugs or medicine, they are food supplements, if they were drugs none of this would be happening for various reasons
i am not sure that fish oil tablets do have confirmed safety, it has been a running point and while we have seen links to lots of nice websites from these companies showing us cartoons about how clean and pure they are there has been no mention of any safety data, we know there are heavy metals and all kinds of other toxins in the fish food chain and we have suggestions on limiting their intake in our diets. i would also argue that anyone can write a nice website but it does not mean anything and we need FSA or similar data on safety
there has been a massive move in medicine away from anecdotal evidence because it is so often wrong or misleading , although it often makes excellent journalism (MMR could be an example of this). it can often be a good trigger to perform good research and i suspect that there is a lot of anger that such an opportunity is being squandered in durham.
as a sort of scientist i must defend dr aust’s comments about your background. it is always important to know someone’s background when you read their work, an article in the guardian is different to one in the sun, one written by a scientist is different to one written by a non-scientist. as someone with a scientific background: i would not expect to be taken seriously in a formal debate about the finer points of english literature. the issue at stake is the tension between science, with high standards of evidence before making claims, and marketing, with almost an expectation of duplicity and pushing a point to breaking-point. the reason ben goldacre has raised this issue it that a marketing campaign appears to be trying to take on a mantle of scientific respectability and honesty.
you are writing about the politics of food yet now you are entering the murky world of evidence based medicine and (from a point of view of some ignorance) i think that it is as difficult to pick apart a scientific paper as it is a piece of literature as there are a lot of levels in this (hence the peer review) – for example dr aust and some others have been criticising someone’s pcr techniques which is way over my head. i think it is important to recognise one’s own ignorance (this last is the nub of badscience i think)
Dr Aust said,
October 19, 2006 at 10:54 pm
Felicity
The problem is that these long “balanced” reports give the impression that the issue – fish oils – wonder drug or snake-oil? – is poised on a knife-edge, or that fish oils are about to be recognised as a real breakthrough, rather than the reality, which is
“Fish oils – interesting but pretty much completely unproven”.
– people will read the stories and think “wow – all we need is a couple more studies, which this article says are coming soon, to confirm that this does miracles – so lets’s give little Jimmy the tablets now”.
Personally I am perfectly happy to say that the evidence that the oils help people with severe developmental / behavioural problems, who probably also have dreadful diets, is worth following up, though it is far from convincing. There are studies, like the Oxford Durham one, which say fish oils seem to work for kids with developmental difficulties. There are other published ones which say different.
However, the evidence – any evidence – that the supplements do anything for developmentally normal kids – which is where the wider story started, with the Durham / Equazen initiative – does not exist. It is that simple.
The manufacturers – like Efamol and Equazen – can , do and will cherry pick the positive outcome studies in the first scenario (kids w developmental problems) to “sell” the stuff hard into the “normal kids with worried aspirant parents†market, as that is where the money is. As the manufacturers are barely handicapped by ANY regulation, most of the advertising will be misleading at best, downright rubbish at worst. It will use articles like your one as testimony and evidence, in the best (worst?) traditions of unproven “alt medicineâ€.
Ditto, evidence-wise for the adults. Or not quite, as there seems to be less evidence for fish oil in adults than in kids. Prisons may well be a special case, with some inmates having a nutritionally-disastrous diet. Interesting point. Should the Home Office run a bigger, properly-constructed, study of diet and/or supplements in prisons? Why not. I am more than happy for my taxes to help fund it.
BUT: Evidence that a wider adult population than violent prisoners can be helped by fish oils – there isn’t any.
Journalistic articles repeatedly blur these kind of boundaries, because it gives the article more “legsâ€. Scientists hate this because it misleads about exactly what you can, or can’t, reasonably hypothesise from THE EVIDENCE. The public is poorly equipped to make this distinction, and (as I see it) the scientific journalists, who should be helping them, prefer to spin things up in order to sex up their stories.
A further recurring problem in these reports is that the skilled writing of professional journalists, making eloquent testimonies out of statements by prisoners and special school staff, loads the “reader take” on the story. Perhaps it is a trite line that “the plural of anecdote is NOT evidence”. But it is largely true, AND worth repeating here, because the possibility of “treatment†or “intervention†effects is such a bugbear in uncontrolled studies. Time after time in medical research we hear these “miracle” stories which, on hard-nosed placebo-effect-avoiding investigation, disappear into the wind.
With Jackie Stordy, I think you are being disingenuous. “Jackie Stordy has made her living for a decade by promoting fish oilâ€. If you had printed that sentence in your article, instead of nothing (first time) or “she has worked for them but this was independent work†the second time), I don’t think the impression given of her independence (or not) would have been quite the same.
Using the Wiltshire children as a “case study  hmmm. In a medical journal setting Case Study has an agreed meaning; a single case, or perhaps a few, which may offer clues, or exemplify some wider point. The people reading these studies typically have the training and experience to understand that such case studies do not constitute evidence that would, say, persuade someone that the treatment used is, on balance, the best one for any patient in this situation. That would require a proper TRIAL. Are the Guardian’s readers as capable of making this distinction? Or will they read your story and take away “there must be something to it�
One issue you haven’t mentioned in your response it the question about the US prison study being (or not) double blind. This is absolutely critical. If the study IS double-blinded, it is important scientific evidence. If the nurse really knew who was getting which pill, as the article later suggests, the study pretty much isn’t any use at all. By telling people it was a proper trial, you are clearly implying it can be relied on because it is being done “rightâ€. If this is not true, surely your argument is completely undermined? To a scientific reader, this is a key point, the kind of thing a legal case would hang or fall by.
My comments on Lucy Atkins’ background were not meant to imply that science stories should be covered exclusively by professional scientists, or by journalists with a science background. It WAS intended to suggest that having people writing about these sort of things who mostly do NOT have this kind of training makes misstatements of the science more likely – see the confusion about the US trial referred to in your article. Another example is the statement in Lucy Atkins’ piece that “most children are therefore officially deficient in Omega 3â€, which is nonsense. No person with scientific training, I suggest, would have written that statement.
Re. your final line about what Ben would do if he were the Wiltshire kids’ GP:
This is a cheap shot, and I suspect you are deliberately misreading what it is we are objecting to. It’s not that we object so much to people TRYING these things. The reasons why they are desperate for hope from somewhere are quite clear, as exemplified by the stories in your piece.
What we object to is JOURNALISTS “cheating†by presenting this kind of story in a way that says “something IS happening here… why can’t it be the miracle pills?â€
The answer, which any journalist covering health issues surely must know, is that there are lots of other well-known possible explanations of why it seemed to work, mostly based in the many forms of the placebo effect.
That is why we have proper trials, to work out whether it is real, or fishful winking, sorry, wishful thinking.
jdc325 said,
October 20, 2006 at 9:00 am
There doesn’t seem much point posting on this topic after #77, as Dr Aust has pretty much said everything that needs to be said. However, there were a couple of points I was interested in.
“Should only those GPs with qualifications in nutrition be allowed to comment on nutrition?” – actually not a bad idea. Some (not all) GPs seem as clueless with regards nutrition as many nutritionists are with regards science.
doctormonkey referred to the lack of safety data on fish oils with regards (a) heavy metals and (b) ‘other toxins’ (PCBs, Dioxins etc…) – I would have thought it was in the interests of the supplement companies to publish safety data for their fish oils and I would certainly rather rely on results of tests run by an independent lab than a cartoon on a website. WHO / EU limits on heavy metals, PCBs, Dioxins etc could be published along with the results so people can see what the limits are, what the actual results are and make a more informed decision on whether to buy fish oils or not. Perhaps it would also be a good idea to do the same for fish as well as oils derived from fish?
felicity lawrence said,
October 20, 2006 at 11:55 am
Joe HJibbeln has asked me to post this comment from him:
This comment is written by CDR Joseph R. Hibbeln, MD, NIAAA/NIH.
I have conducted research in the role of lipids and essential fatty acids in psychiatric disorders for 15 years serving as a Commander in the US Public Health Service at the National Institutes of Health. My research is entirely supported and overseen by the US Federal Government and me which requires my documentation that I have no personal financial interests in this field of science.
I originated the field of omega-3 fatty acids in the treatment of depression and other affective disorders and have done my best to assist the press in an appropriate and measured reporting of the development of that field. In adults, this field has matured to the development of treatment recommendations by the American Psychiatric Association which will be published next month. The American Heath Association has several publications recommending the use of omega-3 fatty acids. On Oct 17, 2006 the National Academy of Sciences, Institute of Medicine released a statement confirming the benefits of consuming seafood based substantially of the delivery of omega-3 fatty acids. Thus, it is inappropriate to state that these nutrients have no effects. That said it is clear that the field of study in the effects on behavioral effects in children is still in it’s infancy (if you will) and I feel that it is very important to keep a clear distinction comparing the scientists in this field and judge the scientific work and integrity on it’s own merit. Clearly there is substantial press interest in this area due the hope for a non-pharmacological treatment to treat of prevent behavioral problems in children. As far as I have read the press literature, no treatment recommendations have been made by these journalists.
I consider the use of a so called “blog” to be an interesting and useful tool to debate the merits of the science and the presentations to the media. However, regardless of the informal nature of this media, it seem to me that this public discussion would be not be best served by rejecting all the carefully conducted science in this field and rejecting the careful reporting by skilled and balanced journalists, because of some studies and reporting for which very appropriate scientific criticisms exist. I have considered your appropriate debate of the question of scientific independence from commercial interests for the data presented for the open trial in the Durham School system and for the open labeled trial presented by Dr. Stordy. I would also concur that that state of the science ant this time would make it difficult to conclude that the majority of children are deficient in omega-3 fatty acids. While this is possibly quite likely, and frightening if it is so, more data is clearly needed.
It is, however, quite inappropriate to summarily lump all scientists in this field and all reporters communicating the science into a “vast omega-3 oil conspiracy”. To me science is a process of dissecting our best approximation of what is correct and true and not a process to prove that one’s views are correct. Many of the comments here seem to take a tone of summarily proving the predetermined view that all the all studies and reporting of studies in this field are corrupt. One example is the summary assumption that double blind procedures of the NIH were not properly followed. In this case, the blind is broken only to the nurse and the patient only after completion of the study by that participant, as mandated by the ethical review committee, so that participants do not have to wait three years to find out what they were consuming. All investigators performing data handling or analyses remain blinded. Regarding the presentation of the graph of correlation data between linoleic acid consumption and homicide mortality: I feel that this was an appropriate use of correlation data with appropriate caveats regarding it not being causal. Rejection of this data on the basis of a belief that similar correlations can be made is simplistic and would likely be clarified by a reading of the primary manuscript. Thus, I think that summary assumptions such as these are unfortunate because this field holds both great promise for contributing to the prevention and treatment of childhood behavioral disorders (not a sole cause, but as a modifiable risk factor), and vulnerability to exploitation to commercial interests. It is important to proceed with criticism carefully and neither accept conclusions too readily nor reject the work of the entire field too readily.
Dr Aust said,
October 20, 2006 at 2:26 pm
Hi Joe
I think you are misreading us a bit. We are not trashing all studies on omega-3s- we are mainly making the point that many things – the vast majority? – in this field referred to by the supplement companies and journalists as “studies” turn out NOT to be studies at all as most scientists would recognise them – e.g. the work in Wiltshire referred to by Felicity in her article, the Durham initiative, and so on. What we are asking for is science reporting BASED ON THE PROPER PEER-REVIEWED SCIENTIFIC DATA, presented in an appropriately balanced way. In the UK we are crying out for this from our newspaper science writers, but mostly we aren’t getting it. I say this as a PhD-qualified scientist and University Faculty member who has spent too many years grinding my teeth at the UK press science coverage.
With respect to your prison study, thanks for filling in the blanks about the question of blinding. But wouldn’t it have been better if Felicity had made this clear in her piece? It would have saved her audience a lot of confusion. As to being prepared to take it on faith without being told – well, given the endless reporting of anecdote as evidence that we get in the newspaper science coverage, you can probably appreciate why we tend to the “default-sceptical”.
Just for a relevant example, take the Durham initiative where this started – it was talked up by Durham and the company as “the biggest ever serious trial†of omega-3s, and widely reported as such in the media here, including by the big newspaper specialist correspondents. When people started asking specific science-based questions, it turned out the “trial†they were proposing wasn’t what they had said it was.
I doubt you would find most of us here disagreeing that there are links between nutrition and health. I certainly believe there are, and that they are an appropriate subject for scientific investigation. The idea that dietary interventions might be a way of targeting certain disease states is also fascinating, and clearly holds out hopes for some conditions. But recommending “eat fish†is not the same as saying “take omega-3 supplementsâ€. One is an important basic piece of dietary advice which we could probably all agree on. The other is medicalising diet and opening up a sales-fest for a largely unregulated supplements industry.
I do think science and medical journalists have a special responsibility, because we don’t do people any service by giving them unrealistic hopes, and I feel a lot of the current reporting does just that. I think this is true of many scientific fields, not just this one. Gene therapy and stem cell therapy are two examples where the media reporting has been ludicrously over-optimistic, and left many people with the misleading impression that cures for things like cystic fibrosis and spinal injury are just around the corner.
You say the journalists never make specific treatment recommendations, and that is true, though I think the overall tenor of the articles will give many people a good steer towards the omega-3 supplements. It is notable that the journalists typically don’t say “eat fishâ€, either, although this is the well-publicized specific dietary advice of the UK Food Standards Agency. I suspect the reason is that writing a story that says “eat the fish the standard dietary advice already advises you to eat – it’s probably important†isn’t nearly as eye-catching as what Ben Goldacre calls a “pills for ills†story. And since that is where the fish-oil stuff started on this blog, we have come full circle.
doctormonkey said,
October 20, 2006 at 2:27 pm
The problem is not that the studies (such as the NIH one in prisons) are bad it is that conclusions drawn may not be supported by the data. The classic example of this in a wider context is Wakefield et al 1998 in the Lancet. In this case I think the danger would lie in ASSUMING that a prison population or children with special needs/ADHD/etc are representative enough of the general population so that any study conducted on them can be immediately translated. Another example of this would be safety data from animal studies (including pregnant animals) says thalidomide is safe, studies on non-pregnant humans says it is safe therefore it is safe for pregnant humans to take…
I do not think that scientists in this field are being ignored or accused of anything more inappropriate than others would (such as drug company sponsored trials) but some people have been reticent about their links to manufacturers and the public debate appears to be going beyond the areas covered by the data currently available. There is also a lack of journalistic reference to specific pieces of research when they draw their holistic conclusions/opinions.
One area that may also bear research or having the research done brought into the argument is whether the omega-3 oils etc need to come from fish or can they be from any source? Are there other factors in the fish oils of benefit other than omega-3?
There also does not appear to be any standardisation of the amount of fish oil that needs to be consumed to get any benefit (I would rather eat an equivalent amount of fish) and there still remains potential safety concerns regarding high levels of fish oil consumption and as a physician I would want hard independent safety data before I could recommend it.
kim said,
October 20, 2006 at 3:20 pm
Dr Aust – I expect you spotted it, but your letter does appear in today’s Guardian.
Dr Aust said,
October 20, 2006 at 8:45 pm
Yeah, spotted it, thanks Kim. My mum has rushed out to buy up all the copies…
Dr Aust said,
October 24, 2006 at 4:11 pm
An interesting / depressing coda: if you go to the Guardian’s Science “start page”:
www.guardian.co.uk/science/0,,921066,00.html
The link to Felicity Lawrence’s Fish Oil piece at the top (complete with graphic) is headlined / captioned
“Fish Oils: the truth they tried to hide”
….!??!
Er…. which truth would that be? And who is “they”? Are we they? Is the Sub who did this a situationist wind-up artist? Or just a plonker?
jdc325 said,
October 24, 2006 at 5:07 pm
Thanks Dr Aust – I needed a good laugh. Tried to hide by reporting twice weekly in national newspapers??
Dr Aust said,
October 24, 2006 at 8:59 pm
indeed. To borrow a phrase: you couldn’t make it up.
jiangjiang said,
December 8, 2009 at 2:18 am
ed hardy ed hardy
ed hardy clothing ed hardy clothing
ed hardy shop ed hardy shop
christian audigier christian audigier
ed hardy cheap ed hardy cheap
ed hardy outlet ed hardy outlet
ed hardy sale ed hardy sale
ed hardy store ed hardy store
ed hardy mens ed hardy mens
ed hardy womens ed hardy womens
ed hardy kids ed hardy kids ed hardy kids
Carol Moore said,
February 15, 2012 at 11:53 am
I am the creator and owner of ABCfastphonics.com and Children’s Storybooks Online (popular children’s stories websites in schools). Children’s Storybooks Online has been on the web for 16 years and is on page 1 of Google for the search “childrens books.” Having struggled to learn to read, I overcame my problem to become a bookworm and author. Interestingly enough my mother gave my twin sister and me cod liver oil. She was an avid and devoted reader of the nutritionist Adelle Davis. Upon reading B. Jacqueline Stordy, Ph.D.’s book “The LCP Solution,” I was very impressed by its depth of research. I plan on prominently advertising her book for the benefit of all students and parents who need to try a nutritional adjunct rather than simply accepting failure after trying all the other modalities. Anecdotal stories can be powerful as I’ve learned after finding a few important solutions for myself in other areas. Most people do not realize that pharmaceutical companies have derived more than a few products from the discoveries first made by natives living in the Amazon. I try not to forget that monetary agendas affecting individuals and small companies are amplified by the huge sums involved in big pharma.