I’m sure there’s some data in here somewhere…

October 7th, 2006 by Ben Goldacre in adverts, bad science, equazen, fish oil, nutritionists, references, statistics | 67 Comments »

Ben Goldacre
Saturday October 7, 2006
The Guardian

It is often unfairly assumed that I am a tenacious obsessive who refuses to let go. So at Durham council – as reported all over the newspapers and television – they’ve done loads of research on omega-3 fish oils making kids clever. It’s all very well saying that, but I need to see the data, to be sure there are no flaws.

Science has a certain authority, which makes it attractive to journalists and salesmen alike, but the authority comes from the transparency: it’s not about taking things on faith, or newspaper articles, it’s about openly publishing your data and your methods, so everyone can check your working. That’s why papers are published. That’s why you should not go to the media with unpublished data, unless you are ready and willing to go through it in detail.

So what happens when a bloke like me comes along, who knows how to read a piece of scientific research, and wants to find out some real information about the trials?

Well, your technical questions get directed to Madeleine Portwood, senior educational psychologist. She has been on all the television news and given interviews to all these journalists on the wonders of omega-3 and Durham’s fabulous research: she has a lot of time for the story. But she won’t even return my phone calls, or my emails.

If I want to find out about the data, beyond inadequate answers with a huge delay, the press office tells me, I have to travel 275 miles to Durham, to do it in person. Obstructive, but I will eventually make the time, just to prove a point.

But in the meantime, what can you do? All I want to do is find out the science behind their endless headlines. I’ve been communicating through the Durham press office. A week ago I asked simple, basic questions such as: for all these trials, what’s the story? Who were the kids? How old? How many? What were they given? For how long? What was measured? What were the results? A week later, no joy.

So then what do you do, to try and get some useful data? I go to the website, and the data is a farce. It features testimonials that would not be out of place on bonkers alternative therapy websites (and I am familiar with that genre). One child says: “Now I am not so interested in the TV. I just like reading books. The best place in all the world is the library. I absolutely love it.” It’s a miracle, honey.

But what about all those complicated-looking graphs? This site is the perfect illustration of how important it is that data is properly published in peer-reviewed academic journals before making dramatic claims, so that people can simply see exactly what you did.

It’s hard to tell what’s there. There is some data from a published trial by some Oxford researchers, but apart from that, I can’t find any sign of Durham’s own placebo controlled trials that they keep banging on about.

There are plenty of graphs, but the graphs are just reporting more of those classic Durham “trials”, with no placebo control group, that cannot give useful data (you know, the ones where they change their minds about whether they’re trials or not, depending on circumstances). So they report improvements, for example, with sciencey looking graphics to illustrate them, but there are no statistics to say if the changes were statistically significant.

I’m trying to think of a way to explain to the lay person just how much data is missing from this site, and how useless that renders the information there. How’s this: nowhere on the page – nowhere on the entire site, as far as I can see – does it tell you how many children were in this study. I can’t think of a single more basic piece of information about a study than how many subjects there were, and it’s simply not there.

All in all, it’s incredibly difficult for me to establish what has been done and where. My interpretation of all this running about is very simple: Madeleine Portwood is falling over herself to make time for journalists who know nothing about science, with her “research”, and her “limbic system”, and her graphs; but as soon as someone who knows about things like t-tests and the CONSORT guidelines for presenting clinical trial research data comes along, she shuts up shop.

Journalists of Britain: if a woman from Durham fitting this description approaches you with a “science” story involving children, you know where to come first.

· Links, references, and an explanation of what on earth CONSORT guidelines are, at badscience.net. Please send your bad science to bad.science@guardian.co.uk

Links and References:

The site in question is:

www.durhamtrial.org/

It’s very confusing so I’ll talk you through what seems to be on it. Firstly, and crucially, none of the randomised controlled trials Durham/Portwood claim to have done, or the positive results that are discussed, can be found anywhere there.

There are two of those “trials” which don’t have a placebo group. Durham, Portwood, Ford, and Equazen still can’t make their mind up about these. First they called them trials and talked about the positive results they were hoping for. Then when people like me pointed out that this is a rubbish way of collecting data, they said, oh no, they’re not trials, we never said they were, and rather preoccupied with the use of the word “trial” for some reason. But in the same breath they still go on about collecting data from this kind of useless study design, and in fact, 2 of the 3 things on Durham Trials website are reporting exactly that kind of data. They just can’t make their minds up. It’s a stupid way of collecting data, and we don’t really care what you call it.

Anyway, that’s the “secondary” and “pre-school” parts of the Durham Trials site.

Then there’s all this stuff which looks like a placebo controlled randomised controlled trial. Brilliant, you think: here’s the action, they’re presenting us with some real data. No. In fact, it took a fair while for me to even spot it (and I know this literature backwards now) but these graphs and results are all taken from the “Oxford-Durham Dyspraxia Trial” designed, performed and published by two researchers from Oxford University. This study was certainly done with some research assistance from Durham Council, as the study was done on Durham children (although if you read it eg Portwood’s name is nowhere on the paper), and it’s very good of them to have helped on it: ownership of this paper (on this site and elsewhere) seems to have been rather grandly adopted by the Durham fish oil posse. This is pretty much neither here nor there as far as I’m concerned, but it’s a shame to do it in ways that create further unnecessary confusion in the minds of those you are trying to communicate with.

There is no citation to or acknowledgement of the Oxford researchers’ published paper, so no way of knowing that it is a properly published study, designed, performed and published by other people, where it was published, by who, etc (The Oxford-Durham study: a randomized, controlled trial of dietary supplementation with fatty acids in children with developmental coordination disorder. Pediatrics, 2005. 115(5): p. 1360-6 since you asked).

I would say this confusing lack of clarity is the dominant theme in the presentation by durham of their research.

EDIT: Oh my god! The site is down! If it stays down during today I’ll post an archived copy of it so you can feel the joy, Durham Council are used to their sites being hosted alongside other peoples (long story).

CONSORT:

The CONSORT guidelines are an internationally well-respected set of standards for the information that should be presented for a clinical trial, and they represent the basic things you would need to know about a study to assess its quality and results.

The quick checklist is here:

www.consort-statement.org/Downloads/checklist.pdf

And there is more on the CONSORT project and its history here.

At the best of times, you would not take it at face value that somebody’s trial simply proved that “X improves Y”: you would want to know how they measured Y, whether it was randomised, whether it was blinded (that is, the subjects and the experimenters didn’t know who got the active treatment), and if so how, was the blinding tested to see if it worked, and a million other things.

CONSORT is very useful, and the benchmark, the gold standard of information you’d want, to assess research claims. If someone made claims, I’d expect them to at least be able to answer the kind of questions in CONSORT, you know, on the phone or something.

I had a phone conversation with the director of Equazen a couple of days ago (genuinely pleasant businessman, glad we spoke) and I sent him an email about the kind of information I’d like to be able to have to assess someone’s positive research claims. By the by, I’m not sure I think that Equazen have made such specific claims about their research as the Durham posse have.

Anyway, we’re chatting about me getting to see more details of Equazen’s many unpublished studies and this is a small extract of what I sent to him on CONSORT, which I post here because it’s late, I wrote it today, I can’t imagine he’d mind, and I think it explains the issue fairly well (forgive the “can’t type with capitals in emails” idiosyncracy):

“…as you know, my main interest is that if people make a claim about scientific research, it’s not enough to have their interpretation of the data, one has to be able to see what was done, to who, how many people, what was measured, how the study was conducted, what the results were, what the stats on those results were, whether it was randomisd and blinded, if so how, and so on. that’s because the details are where things that might make you have concerns about a study are likely to arise.

“to take an extremely extreme example: somebody might say to me, “i have research that shows green children are more intelligent than blue children”. so i go to read their full study, and when i do, it turns out that they measured intelligence as IQ, perhaps using some sub-tests that are known to be more culturally determined than others, so there’s a problem already. then i look at who they’ve compared with who, and the children were from completely different backgrounds, and that was not accounted for. and then maybe it turns out the differences in results between the two groups were only very small, albeit that they were statistically highly significant, and so on. i could still see why they summarised the research as they did, but i am not so sure i agree with the interpretation, or the confidence they have in the results, even if they are still an honest account of a positive finding.

“anyway, the full list of what most people would consider to be enough info to make a sound judgement on a clinical trial is enshrined in a very famous and internationally recognised document called the CONSORT statement.

“here is the checklist:

“http://www.consort-statement.org/Downloads/checklist.pdf

“if anyone came along making an assertion about their research, i would go and find the paper in the journal and look at the kind of stuff that’s in the CONSORT statement to judge it. if a research team approached the media in advance of publication of a trial – which many would say should only be done in cases of big public interest, even though it does happen – it would certainly be very reasonable to expect that you would be able to approach them and ask about the kind of stuff in the CONSORT statement, and they’d have it on the tip of their tongues, as it were.”


++++++++++++++++++++++++++++++++++++++++++
If you like what I do, and you want me to do more, you can: buy my books Bad Science and Bad Pharma, give them to your friends, put them on your reading list, employ me to do a talk, or tweet this article to your friends. Thanks! ++++++++++++++++++++++++++++++++++++++++++

67 Responses



  1. AitchJay said,

    October 7, 2006 at 3:33 am

    The site seems ok now; just the “trial formats” page is down – no surprise really.

  2. Frank said,

    October 7, 2006 at 3:43 am

    the simplest questions are always the most incisive.

  3. cribbins said,

    October 7, 2006 at 8:25 am

    It’s possible I’ve missed some steps in your reporting of this story, but is there any hint of Durham officials/councillors having unreasonably close business links with Equazen?

  4. Jeremy Zeid said,

    October 7, 2006 at 10:11 am

    It’s a miracle…. Surely Omega-3 fish oils will win the war on terror and stop the asteroid crashing into the earth…. praise be o’ Lord of junk science. Brilliant site and about time too.

  5. kim said,

    October 7, 2006 at 11:02 am

    I suggest you make a request under the Freedom of Information Act – they are legally obliged, I believe, to give you the info within 30 days.

  6. monkeychicken said,

    October 7, 2006 at 12:13 pm

    I thought the act only applies to government.

  7. bad chemist said,

    October 7, 2006 at 1:00 pm

    If you want someone to get the data in person, I may be able to. I’m also off to the launch of a new chemistry/science initiative for north east schools on the 19th. It’s aimed at teachers and is held in a school, so I may be able to find out the general feeling towards this “trial”.

  8. BobP said,

    October 7, 2006 at 1:43 pm

    Durham trial also mentioned as “published” on www.dyslexic.org.uk/nr2.htm – but no reference.

  9. kim said,

    October 7, 2006 at 1:53 pm

    monkeychicken – FOI Act applies to local govt and all public bodies, even schools.

  10. stever said,

    October 7, 2006 at 2:11 pm

    “Who were the kids? How old? How many? What were they given? For how long? What was measured? What were the results? A week later, no joy.”

    dont know if its new or not but there is some info, albeit rather vague, on some of this here

    www.durhamtrial.org/primary%20main.htm

    it says there were ‘more than 100′ for example.

  11. ceec said,

    October 7, 2006 at 2:51 pm

    Stever – I think that was the one which actually was “a trial”, rather than the current trial-oh-no-i-mean-not-a-trial-initiative – it says it was done in 2002.

  12. stever said,

    October 7, 2006 at 3:21 pm

    ah – ok

  13. Dr Aust said,

    October 7, 2006 at 3:27 pm

    kim

    - re 9, but NOT to UK Universities….? Maybe?

    Not specifically relevant to current debate, but I have heard of Universities refusing to show people course info requested under FOI because of “commercial” considerations.

  14. Phil said,

    October 7, 2006 at 4:35 pm

    Attn Dr Aust

    FOI does apply to Universities, but the act does give them scope to refuse to respond to a request if it may prejudice some commercial interests. See the link below:

    www.opsi.gov.uk/ACTS/acts2000/00036–h.htm#43

  15. Ben Goldacre said,

    October 7, 2006 at 3:26 pm

    stever etc: that is very confusing aspect of the site, i’ve posted more details on it above. essentially what you are seeing there, the bit of the site that looks plausible, is unattributed data from the Oxford researchers proper published trial.

    FoI: yup, done it, waiting for the results.

  16. David Mingay said,

    October 7, 2006 at 5:37 pm

    As an aside, it may amuse (or appal) you to know that a couple of years ago the British Psychological Society banned the use of the word “subjects” to describe the people who take part in a study, on the grounds that they might be offended. We have to use “participants” instead. I could imagine that I might be offended if I were called an “object”, but it’s hard to see what’s wrong with “subject”. And everyone – the researcher, technicians, and even the cleaner – “participate” in the study in one way or another. It’s political correctness gone mad, I tell you. (Or madness gone politically correct, depending on the nature of the psychological study.)

  17. Ben Goldacre said,

    October 7, 2006 at 5:40 pm

    as an ardent royalist i have to say that is a disgrace.

  18. pv said,

    October 7, 2006 at 9:26 pm

    “It’s possible I’ve missed some steps in your reporting of this story, but is there any hint of Durham officials/councillors having unreasonably close business links with Equazen?”

    Cribbins, I’ve posed that question in differents ways more than once. So far only silence.
    So I’ll take that as a yes?

  19. Ben Goldacre said,

    October 7, 2006 at 9:51 pm

    i think a financial link would actually make the story much less interesting and i hope i don’t find one. there is already a very real practical pay off for all the players in this story: equazen get to sell more pills, newspapers get stories involving children and science, and various individuals in durham get to prance around on TV and in newspapers being special science experts doing great work for the kids, receiving adulation and accolades. money is much less interesting than that.

  20. Heywoope said,

    October 7, 2006 at 11:01 pm

    kim / monkeychicken – Here goes. The Freedom of Information Act applies to all public authorities. Requests for information must be made in writing (e-mail will do and you don’t even have to give your real name). The information requested must be provided in the format you request within 20 working days. The authority can refuse to provide information by citing one or more of 23 ‘exemptions’ including ‘commercial sensitivity’ (e.g. the contract to carry out a piece of work / trial is in the process of being tendered / negotiated). The ‘commercial’ exemption can’t be used once the contract has been awarded, i.e. you can ask for details of the reason for the work and how the reason was determined, a copy of the request for proposal, which organisations tendered, copies of the tenders, how the tender process was run, how the decision to award was made, who made it etc. If you don’t get the information you’ve requested, you can appeal to the authority and, if you’re not happy with the result, you can complain to the Information Commissioner. Might be worth coming up with some standard letters to send to public authorities(?)

    By the way, if the information you request isn’t held by – or on behalf of – the authority, you could of course ask what it is doing spending public money without any formally recorded / auditable processes.

  21. pv said,

    October 8, 2006 at 5:46 pm

    “money is much less interesting than that.”
    I think I agree there, especially about the ego trip for the Durham LEA, but money is such a popular motivating factor in getting these things started. It’s often a common interest shall we say.
    Of course everyone is hiding behind the “it’s all about the kids” excuse.
    What I don’t understand is that, if Durham LEA are so keen to do this properly, why the name of the pill manufacturer isn’t kept secret. And why that particular manufacturer? there are others? The publicity aspect with regard to Equazen and Durham LEA jumping about saying “look at me, look at me” – it’s a typical, undignified media circus, and all such a give away, don’t you think?

  22. Ben Goldacre said,

    October 8, 2006 at 6:28 pm

    well obviously i asked about money, just on the off chance, and they said no. in fact they became a bit preoccupied with that, but again no data. i think they’re so far off the “science” page that they don’t even understand what i’m asking for and why, they’re baffled by anyone scientifically knowledgeable from the media coming along and saying anything : “this story, tell me more”.

    in fact, the real reason this has been so interesting to me is just that: how they’ve responded to my probing.

    remember, at first this was just a little “hahaha calling something this scientifically inept a “trial” is farcical” story, and as always i lay most of the blame with the media for uncritically puffing very thin “scientific” material.

    but what i’ve exposed has been a complete failure to dish up any of the normal data one would expect to come behind a science story (unless it’s the raelians claiming to have cloned a baby “but youre not allowed to see it…”).

    if you say you have all these positive trials, endlessly in the media in “news” stories about your research, then you should be able to show them, it’s as simple as that.

    but meanwhile, even more bizarrely, i’ve also received unrepeatable anonymous slurs and innuendo, from unnameable sources, about academics who’ve made scientifiic criticisms of these durham-equazen “projects”.

    i mean that kind of thing’s neither here nor there to me, but in context, from my perspective, i can get anonymous slur – of relative nobodies – but i can’t get information about the science. i mean one or the other, heck, but both? i’ve had journalists contact me since this story started and they had worryingly similar experiences. utterly bizarre.

    any other science story on unpublished data in the media, let’s say it’s just a conference presentation for example, you call up the researchers, ask them about the study, what they did, what they measured, whta the stats were, all that, and they tell you. they just tell you. they do! they don’t hide, you don’t get told funny stories about people you’re not interested in. aarhgh this whole scene is too weird!

  23. spingle said,

    October 8, 2006 at 7:48 pm

    Just a quick note on FoI – certainly in Scotland, and I presume in England, the Information Commissioner tends to err towards disclosure, and so a good case for “commercial confidentiality” has to be made, and is not guaranteed to succeed.

  24. Dr Aust said,

    October 8, 2006 at 9:48 pm

    On a complete fish oil tangent:

    What is the heavy metal – e.g. mercury – content of Eye-Q and similar supplements?

    And therefore what is the mercury “dosage” per week if you take six capsules a day, which is what they recommend?

    They say it meets “the stringent EC and WHO standards” …

    ,,, but these are presumably the same standards that are used to say “don’t eat oily fish more than once a week” (because of worries about heavy metal levels).

    The fish that are the starting material for the supplement preparation come from the same oceans as the fish we eat, so the eyeQ source fish cannot be “heavy metal free”. I guess the extraction processes they use probably do remove some of the heavy metals in the fish (e.g. by not using liver to make the oils, and perhaps by various solvent extraction techniques – if you are interested they describe this on their website)…

    ..but it would be interesting to know exactly HOW MUCH is left.

    You would have thought this would be of interest since it would have sales pitch value – e.g. “You can’t eat too much oily fish during pregnancy becuase of worries about heavy metal contamination…but you CAN take omega-3 fish oil capsules!”

    In fact, surfing fishy websites I have the impression that this is quite a big poart of the Fish-(Ooil) Pitch, especially in the States: Oh No! We can’t eat fish because of the toxins! But never fear, fish oil supplements are here!

    I was also wondering about the comparability PUFA-intake wise between chomping the capsules and actually eating fish once/twice a week.

    One of the points Equazen make on their website (with the aid of a groovy “fishometer” java graphic) is that you would have to eat shedloads of fish “to get the same amount of DHA and EPA as you would from the capsules. But the calculator uses a rather small “unit sized fish”. For instance, it turns out that to get the same content of DHA as ten days worth of EyeQ capsules you would only have to eat one “portion” of salmon, if one takes a portion to be about 150g. For their other “favourite” PUFA, EPA, you would have to chomp your way through rather more fish… but there are other sources of this one, for instance walnuts.

    So your alternatives are: cough up for the capules; or eat a nice salmon steak once a week, have a tinned tuna sandwich for lunch another day, and eat walnuts as an occasional snack. ..or even walnut oil in the salad dressing.

    I know which one sounds preferable to me.

  25. Michael Harman said,

    October 8, 2006 at 10:45 pm

    I think ethics has been mentioned previously, but it may be worth looking at again. Precisely what information was given to the subjects of the trial(s)? (Yes, SUBJECTS – I’m not subject (Freudian pun) to the BPS.) (I was going to bring in Yobbishness here, but it’s a bit strained, so I won’t.)

    And a more general point about ethics. I suspect that most people, if asked to participate in a trial, wouldn’t think to ask about publication of the results. But if the point were put to them – “We do not intend to publish the results of this proposed trial – are you happy to participate?”, I think many of them would think twice. Perhaps the explanation should also explain, if secrecy is proposed, why the secrecy is regarded as necessary – there may be a valid reason, although I can’t think of one offhand.

    I’m pretty sure that if I had been asked to participate in a trial, it wouldn’t have occurred to me that the trial results wouldn’t be published (or at any rate submitted for publication in likely places) – and I’d also have been pretty pissed off if I found that the results weren’t being published.

    This is perhaps more a matter for the future, that trials should include something along those lines. I think it would be difficult (though not impossible) to argue that any trial has an implied undertaking to publish unless explicitly stated otherwise.

  26. Ben Goldacre said,

    October 8, 2006 at 11:02 pm

    i couldnt agree more, people give their bodies to research on the understanding that the results will benefit the wider community, not to be used as part of a media exercise. i would dearly love to see the promotional and consent/ethics material that durham gave out for this most recent “trial” and the others, i’ve asked for it with my freedom of information request.

    Oh, and I’m just clearing out some old links on the fish oil saga, this is an article on the elusive Madeleine Portwood and her brilliant research, from July 2002.

    archive.thisisthenortheast.co.uk/2002/7/21/127472.html

    There are a couple of things that are interesting about it, looking back at it now we know much more about the characters involved. Firstly, there’s Portwood’s use of the alternative therapists trope “we are treating the underlying cause”, in much the same spirit as her Durham Trial website’s testimonials (“now I am not so interested in the TV. I just like reading books. The best place in all the world is the library. I absolutely love it.”).

    Then, it seems that Portwood is strongly influenced by the Horrobin essential fatty acids and eczema line, to the extent of presenting her findings on this to the press.

    Then of course there’s all the talk of how Portwood is running this brilliant trial, and the promise that the data from the trial will be published in September (the article is from July! It must have been literally in press, to be that forthcoming… although Portwood has not published any paper on the subject that i am aware of). I guess she must be referring to the Oxford-Durham trial. It’s funny, she is constantly described throughout the media as the person behind that trial, and I’m sure she did plenty of work for it, but her name doesn’t actually appear anywhere on the paper, and she’s not a published researcher in the field (in fact, I’d be grateful if people could point me towards what she has published, it’s not necessary to be a published trial guru to be a worthwhile human being or anything, but she is flagged up as this great researcher running trials all over the place so it would be good to see a couple).

    Oh, and then there’s talk of setting up projects in Edinburgh: we haven’t heard much about the results of that adventure.

  27. GarySimpson said,

    October 9, 2006 at 10:20 am

    Has Anyone seen this Portwood product now available at large Boots stores?
    Revolutionary New Child Development Programme for children aged 3-5
    only £34.99
    www.boots.com/features/feature.jsp?articleId=1004603&wblinktype=SMA
    Just a DVD and a Book from the product blurb so no Fish OIls then.

  28. nohassel said,

    October 9, 2006 at 11:47 am

    I’ve dropped an email to madeleine.portwood@ukonline.co.uk politely asking her to get back to Ben as I’m genuinely interested in hearing her position. If I hear anything(!), I’ll let y’all know.

  29. Hacker said,

    October 9, 2006 at 12:36 pm

    Given the A level grades required to get into Durham or Edinburgh Universities, I wouldn’t have thought there would be too many students at either of these institutions with learning difficulties. Or is this further evidence that A level standards are falling?

  30. jdc325 said,

    October 9, 2006 at 3:03 pm

    Re #24
    Some interesting points raised by Dr Aust.
    1. I can’t speak for Equazen, but in my (limited) experience, both Cod Liver Oil and Fish Body Oil supplements tend to be ‘low’ in mercury (i.e. well below the maximum permitted levels). The limit for mercury in fishery products is 0.5mg/Kg. Fish oil / Cod Liver oil supplements tend to come in at

  31. jdc325 said,

    October 9, 2006 at 3:07 pm

    oops, don’t know what happened there. Fish oil / cod liver oil tends to come in at

  32. jdc325 said,

    October 9, 2006 at 3:12 pm

    Bugger!
    … less than 0.02mg/Kg

    2. Comparisons of n3 PUFA – try nutritiondata.com
    149g sardines provides 1463mg n3 PUFA
    88g mackerel provides 1059mg
    178g salmon provides 3821mg
    Each Equazen capsule provides 122mg

  33. David said,

    October 9, 2006 at 3:26 pm

    Just to lend my support to Micheals’ comments (comment 25)

    Kids are being used in a (non)’trial’ of fish oil tablets. Presumably their parents have given consent based on full disclosure as to what will happen in the (non)’trial’.

    If not then this surely must be exploitation and possibly fraud as well. There seems to be a clear case here where a number of organisations and individuals are using children in a process designed to look like an open scientific study (but which is nothing of the sort) in order to gain reputation and recognition (either personally or at an organisational level).

    This is surely a child welfare issue is it not? The kids are being given pills – let’s not even delve into the psychological issues surround the message that popping little pills every day is good for you – and there appears to a be an active attempt now to avoid disclosing any real information about the purpose of this activity.

    A few weeks ago I read this story (and Ben – top work on this) and uttered many a grrr at both the crap reporting and the sad co-opting of the principles of the scientific trial into a bit of marketing speak. Now however, I find myself quite disturbed at what I read. Ben, you are having to resort to the freedom of Information Act to get information on what is actually being done to these kids – AND you seem to be on the receiving end of some attempts at character assasination? It’s not weird – it’s worrying.

  34. Ben Goldacre said,

    October 9, 2006 at 3:45 pm

    I very much agree that one of the more interesting aspects of this case is how the children are being sold the idea, early in life, that taking pills is an important and necessary part of a healthy existence. most unwise.

  35. Dr Aust said,

    October 9, 2006 at 3:52 pm

    Thanks jdc

    …so it appears the “extraction” procedures do remove mercury. Not that that would persuade me to take the FishOil, personally.

    The issue of “fish vs. fish oil” is quite interesting. Taking the numbers you quote, PUFA intake from chomping eye-Q would be

    122 x 6 x 7 = 5124 mg / week for 6 pills/day (“take 6 a day for the first three months”)
    122 x 2 x 7 = 1708 mg / week for 2 pills /day (“take at least 2 a day thereafter”)

    So note that eating 150g (“one portion”) of oily fish only ONCE a week (on the figures jdc quotes) provides 1500-3500 mg PUFAs.

    – and there are also other dietary sources; some nuts, as mentioned before, some types of vegetable oil, and even meat (which contains some PUFAs as the other animals you are eating have similarly had to make them).

    So all in all if you follow the usual good, but unexciting, dietary advice:

    “mixed diet, plenty of fresh produce, fish once or twice a week” etc

    You would be taking in as much PUFA as by chomping the pills.

    Again, this brings one back to the “sensible eating habits and good school meals” comparison. If people can be taught or shown (by eating it, even) what proper food is, then they will hopefully be able to feed themselves sensibly for the rest of their lives. This strikes me as a much better “investment in the future” than a few months course of iffy magic pills

    Much harder to present through a fog of smoke and mirrors as a scientific trial, though.

  36. Dr Aust said,

    October 9, 2006 at 4:02 pm

    So…

    …if there is going to be a deficiency in PUFA levels in “the population”, of sectors of it (and we still don’t know if there is), it would appear to be man-made. To whit:

    Eat sensible mixed fresh food-rich diet: get all the PUFAs you need (see above)

    Eat a diet of chips, tinned shite, processed PizzaBurgers and ready-meals:

    - make the processed food industry happy AND eat too much salt, trans-fats up the wazoo and have lousy intake of various things, perhaps including PUFAs.

    But… FEAR NOT!!! We can take high-tech “scientifically designed” SUPPLEMENTS to put back all the stuff this diet is lacking…

    …and eat “Lo-salt Lo-fat” processed shite to not overload of sodium, etc etc.

    Result: rich food industry, rich supplements industry, bamboozled exploited cash-cow consumers (get ‘em young).

    As the immortal John Lydon put it:

    “Ever get the feeling you’ve been cheated…?”

  37. jdc325 said,

    October 9, 2006 at 4:26 pm

    If anyone wants an extra argument in favour of food over pills – fish also contains vitamins, minerals and trace elements as well as being a decent source of protein and phospholipids.
    One point I wanted to pick up on is that EPA is not present in walnuts (as far as I can ascertain). However, several nuts (inc. walnuts) and seeds (esp. pumpkin, flax) are rich in Alpha Linolenic Acid, a precursor to EPA and DHA.
    Another point (and another digression) is that we don’t (in general) seem to eat enough foods rich in Omega 3 or in monounsaturated fats but we eat more than enough of foods rich in saturated fat and Omega 6 fatty acids. As this is the case, not only is there no reason to supplement Omega 6 fatty acids (just say no to EPO) but we need to look at REPLACING saturated and omega 6 fatty acids with more monounsaturated and Omega 3 fatty acids – for example a decrease in Omega 6 PUFAs would mean less competition for the enzymes that convert n3 PUFAs such as alpha linolenic acid into long chain n3 PUFAs and therefore make the conversion of ALA to EPA and DHA more efficient. No more long-winded digressions today, I promise.

  38. Dr Aust said,

    October 9, 2006 at 5:05 pm

    Sorry about the ranting in the last couple of posts.

    Think what one might call the CULTURAL elements of this whole business are interesting.

    On one level the whole FishOil thing is another “magic bullet” science story (what Ben originally called a “Pill solves complex social problem” story).

    Further back, the whole FishOil business grows out of a “food scare” story (mercury and other toxins mean EATING FISH MAY KILL YOU).

    Then there is the “Kids – Want to stay healthy / properly prepared? Pop pills!” aspect alluded to above.

    And finally the ever-popular Alt health “Bind ‘em with (apparent) science” technique.

    The tragedy of this is that, taking Alex Richardson’s Pediatrics paper at face value, there MIGHT be something to the idea that kids with certain behavioural / developmental problems AND (probably) a nutritionally-poor diet MIGHT benefit from a few months of Fish Oil as a kind of “accelerated” way to boost their PUFA levels.

    Note: “might” and “might”.

    This is the scientists’ “might” as in “there’s some evidence here but not nearly enough to be sure”

    However, I doubt now we will EVER know for sure, as the whole FishOil saga is so mired in bullshit that the chance of the proper science to find out the answer EVER getting done is minimal.

    This reminds me of something. A year or two back I heard a BBC R4 prog about the prospects for “nutriceuticals / smart foods”. This was largely uncritical, chock full of food industry sales persons and nutriceutical company CEOs giving the whole thing their very best boosting. There was only one “voice of caution” on it. The thing I remember this lone naysayer coming out with was something like:

    “How far off are real neutriceuticals / smart foods? Ten to twenty years. But I’ll bet you one thing – the food and supplements industry will be trying to sell you something they SAY is a nutriceutical in a couple of years time.”

    How true, he mused, how very true….

  39. Ben Goldacre said,

    October 9, 2006 at 5:35 pm

    yup. if there is any value in the pills, it is only as a research tool, because you can’t be blinded to whether you’re eating fish. if omega 3 is proven to be valuable then the resulting advice will be “eat fish”, not “buy pills”: the manufacturers will have served their purpose and can move on to make something else.

  40. doctormonkey said,

    October 9, 2006 at 6:42 pm

    can we assume that the equizen fish oil pills do meet standards?

    should public health be involved in this? it sounds public (lots of people potentially poisoned by heavy metals) and health related, or someone else not part of the education department

    alternatively, if this is not a trial is it just a marketing stunt, should Durham trading standards be asked to investigate?

  41. Dr Aust said,

    October 9, 2006 at 9:38 pm

    Hi Dr M

    From what I understand the Fish Oils have removed a lot of the heavy metals – see #24, 30 and 32 above – BUT the general point about oversight is valid. Is there any oversight of any kind INDEPENDENT of Durham CC’s Ed Dept? Any imvolvement by anyone with any experience of trials?

    As BEn has stressed several times, as a trial it wouldn’t make it past an ethics ctte OR the MHRA regulators.

  42. Ben Goldacre said,

    October 9, 2006 at 10:04 pm

    “As BEn has stressed several times, as a trial it wouldn’t make it past an ethics ctte OR the MHRA regulators.”

    i don’t think i have said that.

    although as a piece of science it is a farce, yes: although what is unethical in my view is to exploit good will for participating in trials but then perform nonsense, and then not even publish your nonsense, but to go on t’ TV news all the same, singing your own praises, and those of your wonder pills, and worst of all, to teach children that they need to take pills to lead normal lives. there are other issues but those are the ones that spring to mind as i run a bath.

  43. Dr Aust said,

    October 9, 2006 at 10:39 pm

    Sorry for taking your name in vain, Ben

    What I was getting at was that I reckon even as a PROPER trial with effective double-blinding it would be regarded as ethically iffy because:

    - wanting to give it to loads of children who have NO treatable medical condition – why?
    - no evidence of long-term safety
    - some known downsides of the supplements reported

    - given the above, would be unlikely to be approved unless the “likely positive outcome” was

    (a) going to be very important (in terms of “things that could be treated” NOT just for profits) and
    (b) relatively LIKELY in view of GOOD QUALITY scientific evidence.already in hand (er…)

    MHRA would have probably wanted detailed analysis of “potential benefit of knowledge gained” vs. potential for harm to participants.

    Agree about “goodwill”, BTW. Reminds me of that example a few years back where patients having open chest thoracic surgery were asked “mind if we take a skin strip before we close you up – for research purposes, of course.”

    Some time later it transpired that the hospital sold the skin strips, for hard cash, to a commercial company for skin testing of consumer products.

    The punters were understandably mifffed, mostly because they felt (with some justification) that they’d been lied to.

  44. j said,

    October 9, 2006 at 10:47 pm

    “Hacker83 said,

    October 9, 2006 at 12:36 pm84

    Given the A level grades required to get into Durham or Edinburgh Universities, I wouldn’t have thought there would be too many students at either of these institutions with learning difficulties. Or is this further evidence that A level standards are falling?”

    Hacker- firstly, afaik the trials have been (and have been planned to be) in schoolkids, so I’m not sure how this applies to universities. Secondly, the idea that there won’t be many students with learning difficulties with the academic record to get into ‘good’ universities is BS: for example, Durham had 458 dyslexic students in 2003-4 (www.dur.ac.uk/dussd/aboutus/historyoftheservice). Despite a ‘learning difficulty’, my academic record is better than most, and more than adequate for most purposes; of course, that could just be proof of declining educational and assessment standards.

  45. jonnyjazz said,

    October 10, 2006 at 6:04 pm

    From the equazen website:

    Equaquirk
    Calories consumed each time you laugh: 3.5

  46. doctormonkey said,

    October 10, 2006 at 9:59 pm

    do we have safety data for this fish oil capsule though? is it published? is it available? i assume it is food safety rather than medicine safety but still…

    what about any nastiness from the processing, I was taught to avoid margerine due to the traces picked up in the processing to make it butter-like being worse than the (predictable) problems of butter

    to put it another way, are we boosting with fish oil (possibly) but reducing with heavy metal poisoning both their brain power and potentially that of the girls’ children (not to mention possible effects on boy’s sperm)?

  47. jdc325 said,

    October 11, 2006 at 8:54 am

    doctormonkey – the figures I gave for mercury content of fish oils relate only to results I have actually seen and do not include any Equazen products. The only way to find out how much mercury Equazen capsules contain would be to see results of testing. It may be worth asking Equazen if they have tested for heavy metals in their capsules or raw materials and, if so, whether or not they would be willing to divulge the actual results to you. If not, I suppose you could always buy a pack and send them off to a lab yourself. Apparently, some (but not all) fish oils are molecularly distilled to remove contaminants such as heavy metals and PCBs.

  48. Dr Aust said,

    October 11, 2006 at 11:57 am

    The oil supplement will, of course, only have to meet food safety standards.

    Equazen make a big song and dance about the “processing” of their oils – see the animated bit at:

    www.equazen.com/default.aspx?pid=143

    - or the PDF version at :

    www.equazen.com/gfx/ul/files/process.pdf

    ..indeed, a significant bit of Equazen’s web “pitch” is based on saying:

    (i) Our oils are more “scientifically” formulated and carefully produced than other suppliers; and

    (ii) They are better supported by proper research

    In the light of this thread and the previous ones the irony of part (ii) is rather cherishable….

    On mercury/PCBs, I would expect there to be less in the oil than in actual fish as once it is in oil form they can (as jdc says) no doubt process it to remove things.

    There is some info on contaminant nasty levels in food grade fish oils at:

    www.oceansalive.org/eat.cfm?subnav=article&contentID=4362

    ..which amazingly actually indicates some “real” references.

    BTW Dr Monkey, I suspect the margarine vs. butter debate (debacle?) you are referring to is about avoiding this year’s baddies, trans fats in margarine. But the history of the butter vs. marge thing is interesting in the context of the “nutrional advice for optimum health” debate, and nicely exemplifies why many of your punters probably despair of advice about What To Eat.

    Back in the 80s, when cholesterol was the Big Baddie, and pre-statins, margarine was heavily promoted on “avoid butter – nasty bad cholesterol and saturated fats!” – margerine good! – unsaturated fats!”. This carried on into the 90s, and no self-respecting health freak in the 80s or early 90s would have eaten butter.

    More recently, since DIETARY cholesterol has been somewhat down-graded as a source of body cholesterol (and the “debate” has moved on more to LDL/HDL) and since hydrogenated trans-fats have become Big Bad Nasties, butter has made a come-back since you can now say “Arrgh! Trans-fats! Margarine BAD!”

    Result – people are totally confused. Enter “mystify-’em” nutritionists saying “Oh, what you need to do is read the FULL list of contents to ask EXACTLY how much saturated fat / trans fat / unsaturated fat / cholesterol…

    …or alternatively buy the one that has my picture on it! Ker-ching!

    No wonder many people just say “oh stuff it” and go back to eating the one whose taste they prefer.

  49. coracle said,

    October 11, 2006 at 1:47 pm

    Dr Aust,

    Butter vs Marg fear goes further than simple transfats, some will scream ‘it’s plastic I tell you, plastic!!

    See spreading the fear (and links therein) for details.

    ps thanks!

  50. ZackDavies said,

    October 11, 2006 at 2:14 pm

    I put this on the forum too – sorry for the repitition.

    Equazen have started putting leaflets in Holland and Barrett – a location they clearly thought of as impervious to scientists owing to the prevalence of Gilliam McKeithenalia – talking about Durham’s “…randomised, double-blind, placebo controlled trial”. It takes some careful reading (through a red haze of anger) before one realises that they’re not talking about the very large upcoming trial but the smaller trial partially detailed on www.durhamtrial.org.

    The intention is clearly to mislead fools such as myself. I bought fifteen crates of fish oil before I realised my error. Death to Equazen!

  51. Dr Aust said,

    October 11, 2006 at 2:35 pm

    Glad to see that linseed oil has inflitrated your polymerisations, coracle. My mad chemistry A level teacher (blogs passim) would be pleased.

    I used to eat polyunsaturate-rich marge myself, but have defaulted back to butter since ‘Er Indoors (the medical half of the family but born and raised on a dairy farm) flaty refuses to touch “plastic marge”.

    Like many other diet / “lifestyle” interventions, it isa question of degree. All in all I suspect that:

    (i) not smoking;
    (ii) staying physically active;
    (iii) not being a huge fat bloater and
    (iv) not chowing through shedloads of high saturated fat high-salt processed shite

    …are so dominant in the “cardiovascular disease risk” sweepstakes that butter vs. marge on your morning toast is unlikely to be a biggie, except insofar as it relates to (iii).

  52. wewillfixit said,

    October 11, 2006 at 2:56 pm

    You have to have butter. Marge tastes awful spread really thick….

  53. Aspiring Pedant said,

    October 11, 2006 at 3:39 pm

    Not only does margarine taste better than butter but it spreads more easily & thinly.

  54. wewillfixit said,

    October 11, 2006 at 3:44 pm

    Who wants it spread thinly on a freshly toasted crumpet?

    Mmmmmmmmmmmm

  55. doctormonkey said,

    October 11, 2006 at 4:20 pm

    hmm, we are now debating margerine vs butter, can we get back to basing homeopaths again?

    back on the point, Dr Aust’s link to oceans alive says that the mercury binds to protein and the oil should be fat/lipid so i suppose it should be ok… unless some nasties are lipid soluble (as many things are in pharmacology) such as dioxins, PCBs and pesticides (same source) and so will be in highest concentrations in the oil/lipid/fat of the fish

    still sure they are safe?

  56. pv said,

    October 11, 2006 at 5:32 pm

    This is the point (and it’s not very nice):
    www.badscience.net/forum/viewtopic.php?p=14856#14856

  57. Gordon said,

    October 11, 2006 at 6:07 pm

    I was surfing and trawling “Durham fish trial” on google. I found this quote attributed to an Equazen spokesperson that seems relevant to doctormonkey’s concern:

    “We use a highly sophisticated six-stage process to ensure that the oil used in the finished product is the highest quality and free from contaminants. This process starts from the careful sourcing of fish (sardines, pilchards and tuna) from local fisherman fishing in the world’s cleanest seas.

    The oils are then washed, chilled and filtered so that contaminants are removed — well in excess of FSA requirements — whilst maintaining the integrity of the oil itself.”

    It was on this site:
    www.naturalproductsonline.co.uk/industry_news.asp?ItemID=2113&rcid=71&pcid=65&cid=71

    So it appears that Equazen’s oils are full of natural goodness without the nasty man-made pollutants.

  58. Gordon said,

    October 11, 2006 at 6:07 pm

    On another note, and on the Durham council site there was mention of a pre-trial assessment of some children, 94% of whom “had moderate or severe ratings for Attention Deficit-Hyperactivity Disorder (ADHD), 94 per cent had the same Inattention Scale rating and 89 per cent were rated as having severe impulsivity.” I’ve just listened to a talk by a postgraduate student in the psychology department where I study/work who stated that 1-5% of children would have ADHD. So either Durham has a huge ADHD problem or the children tested were all selected because they showed signs of, or had been diagnosed with, ADHD. The point is, as Ben and others have said above, that this 94%, 89% or whatever percent showed improvement on whatever scale of measuring they used is useless information because it tells us nothing about who the participants (subjects) were.

    In answer to Ben’s post (#26) I trawled Web of Science for any reference to M* Portwood (search term to find all authors surname Portwood who have an initial M). There were 26 hits. All of them, as far as i could determine, were for MM Portwood who specialises in something completely different. The only published work for Madeliene Portwood I could find were the books she has published on dealing with dyspraxia.

  59. Dr Aust said,

    October 11, 2006 at 6:20 pm

    Dr M

    As I said above somewhere, I think they do have some tricks for removing “chemical nasties” like PCBs.

    Apart from anything else, before they were telling you the stuff would make you dead brainy, a lot of the “Fish(Oil)Pitch” – esp. in the US – was that the Fish oils would give you “the goodies of fish without the EVIL Hg / PCBs”

    Although it would take more than that to make me trade my pan-fried salmon steak for a fistful of fish-pills.

    I do wonder how much of their “clean-up” is basically just NOT using the tissues where nasties are most concentrated, which would be guts (esp liver), gills and skin, applying some fairly basic toxicology.

    But once you have a “liquid product” there will always be SOME nifty chemical way of cleaning it up. Just depends how much you are prepared to spend doing it

  60. doctormonkey said,

    October 11, 2006 at 7:17 pm

    the problem seems to be (if the source i mentioned before is accurrate) that many of these nasties (although not mercury) are specifically in the oil from the fish as this is almost pure lipid and these nasties are lipid-soluble and although you may be able to remove them it sounds a little difficult and i would worry about the processes to remove them as well or instead

    btw i tend to use cartoonish descriptions of things, including to patients, such as “nasties” rather than PCBs, dioxins, pesticides… but i can speak “science” when i have to

  61. Dr Aust said,

    October 11, 2006 at 9:18 pm

    Agree lipid-soluble contaminants would be the biggest worry, DrMonkey.

    Equazen also have this line about “using fish from the world’s least polluted oceans” -though not sure which ones those would be these days. I can believe coastal fish from (e.g.) polluted estuaries would have more toxins/pollutants than ocean fish.

    Which reminds one that the people eating the MOST polluted fish are likely to be the 3rd world poor living near industrial zones, who presumably rely disproportionately on local coastal fish pulled from heavily polluted waters. Double whammy.

    As was mentioned about, the FishOilersall mention “molecular distillation” to remove organic impurities like PCBs – for another example see

    www.authentic-breathing.com/omegarx.pdf

    My n yrs back Chemistry degree didn’t mention “molecular distillation”, although I remember plain old “distillation” well enough. I don’t know how well this removes lipophilic PCBs, aryl hydrocarbons, pesticide residues etc. etc. but maybe someone else hereabouts does.

    Of course, it’s rather a pity we can’t NOT pollute waters with all this shite in the first place. On that at least I’m with the Greenies.

  62. doctormonkey said,

    October 11, 2006 at 10:19 pm

    equazen have a nice pdf flyer with their own processing, not a formal rundown but quite informative and as they are the focus of the attention and we seem to be managing to damn them and Durham LEA perfectly well with their own words…

    www.equazen.com/gfx/ul/files/process.pdf

    essentially, they seem to remove the protein (where the naughty mercury lives) and remove the “bad” fats from the oils but i’m not sure what they do that removes all of the lipid soluble badness

    they also give their source fish and oceans: sardines, pilchards and tuna from the south pacific and atlantic

  63. jdc325 said,

    October 12, 2006 at 10:29 am

    #61 – From reading the PDF you referenced, it seems they don’t explicitly state that any one particular process will clear dioxins, PCBs and furans from the oil. Apparently, however, the filtration stage uses a natural source of calcium carbonate (from the ‘special clay’) that “marries with any unwanted contaminants”. To me that this implies that this treatment stage will remove ALL contaminants (including the ‘nasties’ such as PCBs etc…). Does anyone know if this would actually work? Or by contaminants do they just mean heavy metals?

  64. doctormonkey said,

    October 12, 2006 at 7:37 pm

    #61 / #63 – but the pdf i referenced in #62 is from equazen’s website about their process and it does not have such a section

    so far as i know, some such clays and things will absorb some amount of some nasties but i am not sure it is a complete panacea, especially as it might also cleanse the product of the good things too!

  65. Michael Harman said,

    October 15, 2006 at 11:18 pm

    The illustration is interesting, but don’t you think the guy is taking a hell of a risk? What if someone crept up behind the fish and shouted “Boo”?

  66. jdc325 said,

    November 14, 2006 at 1:26 pm

    Ridiculously late, I know but I came across a link to this today (refers to safety concerns over fish that several people have raised on this thread).

    JAMA 2006; 296: 1885-1899

    Refers to fish consumption rather than fish oil consumption, but thought it may be of some interest.

  67. FrogJacuzzi said,

    May 25, 2012 at 12:00 pm

    Dr Aust,

    What on Earth is wrong if people DO ‘just say “oh stuff it” and go back to eating the one whose taste they prefer.’?

    I wish they would! I wish people would explain themselves for eating butter and drinking red wine by saying, “Because I like them!”

    Instead they have righteous anger against margarine, because they feel they were mislead for years, they spit the word ‘hydrogenated’, and they claim that magarine actually tastes foul!

    Unobtrusively bland, more like! Talk about trying to win an argument by overstatement. I wish people would shut up about butter AND margarine. Neither is causing all the fat bastards.

You must be logged in to post a comment.