The trial that never was.
Ben Goldacre
The Guardian,
Saturday March 29 2008
And so an epic saga comes to a close. You will remember the Durham Fish Oil tale - don’t switch off now, the punchline’s funny. The county council said it was doing a “trial” of fish oil pills in children, but the trial was designed so that it couldn’t possibly give useful information - not least because it had no placebo group - and was very likely to give a false positive result.
The idea was to give 3m pills to 2,000 children over eight months and see if their GCSE results improved. Unfortunately the GCSE results for Durham were rather disappointing this year, as we pointed out at the time. This fact was not press-released by the county council.
But would Durham publish the apparently disappointing data from the “trial”? Would Equazen - recently chastised by the Advertising Standards Authority for its misleading use of the Durham data - publicise it?
I can now answer this question for you. Durham has finally announced, in a formal response to a written question to the county council, that the trial in 2,000 children was never intended to produce any data on children’s performance. Specifically they said this: “It was never intended, and the county council never suggested, that it would use this initiative to draw conclusions about the effectiveness or otherwise of using fish oil to boost exam results.”
That’s funny. Because in the Daily Mail article from September 5 2006 headlined “Fish oil study launched to improve GCSE grades“, Dave Ford, the council’s chief schools inspector, says: “We will be able to track pupils’ progress and measure whether their attainments are better than their predicted scores.”
Dr Madeleine Portwood, senior educational psychologist at Durham county council, who ran the “trial”, says: “Previous trials have shown remarkable results and I am confident that we will see marked benefits in this one as well.”
Durham county council’s own press release from the beginning of the “trial” reads: “Education chiefs in County Durham are to mount a unique back-to-school initiative today which they believe could result in record GCSE pass levels next summer.”
It says that children are being given pills “to see whether the proven benefits it has already brought children and young people in earlier trials can boost exam performances too”. The council’s chief schools inspector is “convinced” that these pills “could have a direct impact on their GCSE results … the county-wide trial will continue until the pupils complete their GCSE examinations next June, and the first test of the supplement’s effectiveness will be when they sit their ‘mock’ exams this December.”
“We are able to track pupils’ progress and we can measure whether their attainments are better than their predicted scores,” says Dave Ford, in the press release for the trial which, we are now told, was not a trial, and was never intended to collect any data on exam results. Perhaps in future journalists will regard the grand promises of Dr Madeleine Portwood, Dave Ford, the Equazen pill company, and of course Durham county council, with appropriate cynicism.
Please send your bad science to bad.science@guardian.co.uk
Convergent evolution with the excellent Holfordwatch on this, and more good digging into the bizarre and implausbile opacities from Durham here.
MataHari said,
March 29, 2008 at 2:15 am
What about parents who opted out of their children taking the pills?
Was there an opt-out opportunity? Was there ethical committee approval?
What about the children’s reaction, and how many complied after the first week? Fish oil pills cause disgusting fishy burps, and the children will have discovered this soon enough. Disposing of an unwated pill is easy enough, and I wonder how many carried dodn taking them to the end?
And, while I am in ranting mode:
Equazen’s boos is Adam Kelliher. He was son-in-law of David Horrobin, the crook who brought evening primrose oil to the world and made a few £bn from the NHS for this worthless remedy.
There was a critical obituary of Horrobin for the BMJ. Keiliher organised a protest, declaring he was the distraught son-in-law with no competing interests.
He’d forgotten he founded and owned Equazen.
What will be his next sales pitch?
Are there any more credulous and uncritical education directors?
Dr* T said,
March 29, 2008 at 7:00 am
What is it about Fish Oil Companies and dubious research?
From an FOI request re: risk analysis for the Durham ‘trial’:
There was no requirement to undertake risk analyses for fish oil, which is regarded as a food, not a medicinal compound. Fish oil is governed in the UK by food law. Abroad it has a similar status: in the USA it is categorised as GRAS (Generally Regarded as Safe) and within EC regulations on RDAs for supplements, there is no RDA for fish oil.
The daily intake that is being supplied in the current initiative in Durham (0.7 gram/day) is well within the guidelines issued by leading health and scientific authorities…
Kess said,
March 29, 2008 at 7:25 am
It’s sad that there seems to be little effective action that can be taken against the wealthy quacks, nor the Durham councillors and educators who now seem to be trying to wriggle off the hook.
I wonder if the local media in Durham is hot on the case?
And what do the parents think about their offspring being used as guinea pigs in a “trial” that may have damaged their exam performance? (I know it’s unscientific, but had the children’s results improved then the pills would’ve undoubtedly got all the credit so why not also seek to blame them for the poor results? It’s only fair.).
Paul Crowley said,
March 29, 2008 at 9:53 am
If they didn’t do it because they had some reason to think it would work, and they didn’t do it because they wanted to know whether it would work, then why the hell did they do it?
Teek said,
March 29, 2008 at 12:53 pm
hilarious. ceci n’est pas une trial.
bloody awful backtracking, as if they thought nobody would be paying any attention - but the didn’t count on your ninja skills of, err, memory and typing…
what will it take, other than this fiasco, for the press and other enthusiastic pill-lovers to realise that fish oil ain’t all that, especially when it comes to “trials” that aren’t anything of the sort?!
j said,
March 29, 2008 at 1:57 pm
Like the post - and thanks for the kind words about HolfordWatch - but I think you may be underestimating the stubbornness/tenacity of some bloggers (yourself included) when you say that “And so an epic saga comes to a close.”
Although any hope of scientifically valid results is long dead, the Durham fish oil zombie is still swimming - and it will swim again
Kess said,
March 29, 2008 at 2:29 pm
Although the “trial” was a failure, you would think the Durham team would wish to act professionally and publicise the outcome with urgency to prevent anyone else falling for the fish oil scam.
Why are they so sheepish? Is there a conflict of interest somewhere - are any of the Durham people also linked to Equazen, I wonder…
JQH said,
March 29, 2008 at 3:53 pm
It would appear that at the time they told parents that Durham had entered into a partnership with Equazen.
I have not been able to obtain any other documentation on this. Durham claimed they did not make the decision to push the pills and thus there were no reports to Senior management or Elected Members. Presumably therefore the partnership spontaneously generated without any decisions being made about not having a trial in order not to improve exam results.
Ford, Portwood, Equazen and DCC need to be reminded of this farce every time they open their mouths on the subject.
Dr Aust said,
March 29, 2008 at 4:24 pm
I think that when “naming and shaming” them it is also important to point out that the reason the ScienceGeek-osphere is cross is not just the misrepresentation; it is also that they COULD have constructed a proper mass trial of fish oil pills.
Although one cannot be sure, Adam Kelliher’s statements at the time -at least as far as I remember them - sort of suggested that Equazen would have been amenable to this. But instead, we got a bunch of bullshit, now disowned by all, and telling us nothing. Nada.
Anyway, shame on Durham Education, on Ford and Portwood (in particular) and indeed on all involved.
Kess said,
March 29, 2008 at 4:48 pm
18 months ago a school near me also jumped on the bandwagon and started a fish oil “trial”. Like Durham it was highly simplistic and a wasted opportunity to show children how a scientific experiment should be conducted (see http://badscience.net/forum/viewtopic.php?t=962&highlight= ).
Anyway, also like Durham it’s all gone very quiet since which suggests it didn’t produce any improvements either. Let’s hope those involved have learned some useful lessons and won’t be so easily misled in the future.
pv said,
March 29, 2008 at 5:06 pm
The winners here, temporarily at least, are the fish oil supplement companies who’ve been able to cash in on the flood of press and publicity in the run up to this debacle. Once again, as with the fabricated MMR scare, the press hacks have been working overtime to milk the public’s fears and aspirations with their partial, uncritical diatribes about the purported “benefits” of fish oil. Not once, as with MMR and Wakefield, did they ask if their might be something amiss with the whole scheme.
The press bear a deal of responsibility here. They are happy to have fish-oil product advertising, making whatever claims, within their pages and then are happy to fuel the process of deception further by their time-honoured uncritical reporting.
I think the parents of Durham school children have a right to feel abused and aggrieved, and I sincerely hope they don’t fall for the current bout of hand-wringing and denials by those who were at the centre of promoting the fish-oil fiasco. Duplicitous is I think the word to describe their behaviour.
marcdraco said,
March 29, 2008 at 7:58 pm
Oh dear. I was part of this trial and was hopeful it might just have worked. I would hang my head in shame, but then, I didn’t see the results until now.
Robert Carnegie said,
March 30, 2008 at 9:50 am
If it wasn’t a trial then I suppose they didn’t need consent or ethics committee approval or anything.
One still can’t help thinking of the recent Doctor Who story where aliens took over a school and fed chips fried in a special alien brain-improving oil to the kids, but that wasn’t a good scientific trial either because students who performed poorly were taken out of class and eaten. But I believe this was fictional.
ceec said,
March 30, 2008 at 6:05 pm
This may be obvious but there is also another possibility re this “initiative” and the ethics of it. The fish oil supplementation may have been actively harmful, which could have caused lower results than expected.
One plausible mechanism would be this: “I’ve been told that with this pill I can increase my grades, therefore I don’t need to work so hard”. It’s fairly easy to imagine a scenario where people prefer not to work hard, so suggesting, even indirectly, that fish oil per se can increase grades is potentially damaging.
Of course, unfortunately, as discussed ad nauseam, we can’t tell because this wasn’t a trial, and also presumably had no qualitative component in which a “real” harmful result could be investigated. It seems depressingly likely that even the possiblity of harm was not considered at the time (or perhaps even after ethical concerns began to be raised with them) - a terrible failure on the part of Durham Council.
pv said,
March 30, 2008 at 6:40 pm
I’m sure this has been discussed before, either in this blog or in the forum. Anyway, I agree. A variation on the placebo effect, eh!
Why don’t we just call it auto-suggestion and have done with it? It’s much easier to understand that way.
Pathos said,
March 30, 2008 at 11:53 pm
Can someone explain how you get fish oil into a pill? They did say pill and the pic has a pill. Oil is well… oily. Omega3 is also very delicate and messing about with may even make it harmful. Is this something else the organisers “forgot” about? Should have taken some caps themselves!!!
Martin said,
March 31, 2008 at 4:32 am
Pathos,
I don’t think the picture at the top of the page is representative. I assumed that the fish oil ‘pills’ are actually capsules (although I may be wrong, in which case, yes the action of making them a pill could be detrimental to the fish oil efficacy!)
McCruiskeen said,
March 31, 2008 at 9:58 am
JQH said:
It would appear that at the time they told parents that Durham had entered into a partnership with Equazen.
I have not been able to obtain any other documentation on this. Durham claimed they did not make the decision to push the pills and thus there were no reports to Senior management or Elected Members. Presumably therefore the partnership spontaneously generated without any decisions being made about not having a trial in order not to improve exam results.
I have documentary evidence dated 13/11/06, showing the following:
Durham County Council initially approached Equazen.
There were no financial inducements involved, in cash or in kind.
The initiative was planned by David Ford.
The aim of the initiative was to support the LA’s existing Key Stage 4 strategy.
The aim has not changed.
The initiative was offered as an opportunity for schools. Individual head teachers (in their capacity as managers of autonomous institutions) agreed that the initiative should take place in their schools.
A project plan was prepared as an information pack for schools which was shared with them ata a briefing meeting. Similar packs were distributed to all parents and there were information meetings for them.
The evaluation approach has been discussed with head teachers. It will focus on uptake, teacher’s (sic) perceptions and outcomes compared with predictions.
David Ford is leading the initiative.
The evaluation will be managed through the participating head teachers.
Regarding the involvement of members of Durham County Council, in response to questions about their role, I received this response 0n 20/03/07, from the then Leader of the Council, Councillor Ernie Foster:
“As I understand the position, a recommendation to use Omega 3 supplement was made by officers in line with the normal advisory relationship between the county council and its schools. In that sense, Members were not directly involved in the decision, although key Members were made aware of and supported providing this opportunity to schools.”
I am currently awaiting the answer to two questions:
1. Who are the “Key members” and when were they made aware?
2. What was the nature of the “Support” provided by these Key members?
The question that I feel needs to be asked of this farrago, with all its chnaging plots and backtracking, is simply, why?
What on earth was the reason that persuaded David Ford to plan and lead it?
It started to fall apart almost from the first press release (as it always would have done, given the appalling structure)but one has to ask how some thirty seven very single-minded Durham secondary school heads were persuaded to participate in this utter dog’s breakfast of an initiative (It barely qualifies for that appellation).
The word from the chalk face is that it was an utter shambles. I have heard of biscuits and juice being given as inducements to take the capsules!
So in spite of the ethical issues of Durham County Council entering into a partnership with Equazen, the totally flawed initiative structure, the hype of the press releases and media coverage, the lost opportunity to conduct a meaningful study and the craven rewriting of the script as it started to unravel, the lack of any apparent detailed scrutiny by Members of DCC and the willing acceptance by Durham’s secondary school head teachers of the package, we are left with one big unanswered question……….
Why did David Ford (Chief Inspector of Education, Durham LEA before the initiative; now Head of Achievement Services, Durham LA) do it?
BSM said,
March 31, 2008 at 10:16 am
So, the Education officials in County Durham are incapable of describing a “trial” as a trial and are incapable of seeing the need for a control group to draw any useful inferences about the effectiveness of an intervention. And these buffoons are responsible for educating children in science and good government. Presumably they have decided to do this by providing powerful counter-examples in their own actions.
Can we have a campaign, please, to force the their department to wear quotation marks around its name, you know, sort of like a dunce’s cap?
So, it would be the Department of “Education”. I think that looks perfect.
But, wait, it seems they have got there before me. In a whizzy new idea, Durham no longer has an Education Service, or even an “Education” Service. It has;
“Children and Young People’s Services
The new Children and Young People’s Services brings together:
Building Schools for the Future team
The existing education services from what was the Education Service
Emergency and Family Support Services
Social care services Children and Family Care
Young People’s Service
Youth Engagement”
Clearly education has become subservient to servicing children and young people. I can see how pushing fish oil pills fits into their scheme now.
Jamie Horder said,
March 31, 2008 at 11:03 am
If this were a scary drug, rather than a friendly natural product, the media would be outraged. I can see it now :
“Heard of cervonic acid? Probably not. Also known as 22:6(ω-3), all-cis-docosa-4,7,10,13,16,19-hexaenoic acid, this compound is not naturally produced by the human body, but it can be harvested from the bodies of certain animals. Cervonic acid has widespread effects on the brain. Some scientists believe that giving the chemical to prisoners as part of their food could be an effective way of keeping them calm and preventing violent behavior. But no-one knows what the long-term effects might be.
Now YOUR CHILDREN could be about to find out, because cervonic acid, along with other similar compounds, is currently being tested in SCHOOLS - and all because some scientists think it could improve exam results. Find out the shocking truth about cervonic acid inside today’s Daily Mail…”
Bass Tyrrell said,
March 31, 2008 at 12:33 pm
Ben, are the figures right? 3 million pills to 2000 children = 1500 pills each over 8 months = 6 pills per child per day approximately? Just checking.
outeast said,
March 31, 2008 at 2:48 pm
lol @ Jamie Horder!
Pathos said,
March 31, 2008 at 4:53 pm
Jamie LOL
Ben Goldacre said,
March 31, 2008 at 5:08 pm
haha excellnet
bass: yes gave 6 pills to every child per day. it’s a lot of pills. i think the main lesson the children will have learnt is that you should take lots of pills everyday to live a normal healthy life which, as i keep saying, is good for all pill industries of every complexion, and somehow rather bad for our collective outlook.
McCruiskeen said,
March 31, 2008 at 5:14 pm
I wonder how the oil in the capsules interacts with custard creams?
Wonk411 said,
March 31, 2008 at 5:38 pm
@ Jamie
I would change ’scientists’ to ‘”Educators”‘ then print it up on official looking paper and send it out. Await results in a week or two. Could be good fun.
feline said,
March 31, 2008 at 6:10 pm
What a shame, as I’d heard otherwise…this was from the BBC
Dear Ms …
Thank you for your email regarding the coverage of the Durham trial of Omega 3 supplements.
I note that you feel this coverage is premature, given that the results of the trial have not yet been published, and that you are even wondering if this is a real trial at all.
If I may explain, although the results are yet to be published, the trial has ended, and the BBC was able to report on what the published results are expected to indicate.
Nonetheless, I realise that you may take a different view, and accordingly, I would like to assure you that we have registered your comments on our audience log. This is the internal report of audience feedback which we compile daily for all programme makers and commissioning executives within the BBC, and also their senior management. It ensures that your points, and all other comments we receive, are circulated and considered across the BBC.
Thank you, once again, for taking the time to contact the BBC.
Regards
BBC Information
__________________________________________
Have your say about the complaints process in the BBC Trust’s current public consultation - www.bbc.co.uk/bbctrust/consult/open_consultations/complaints.html
Would you like FREE tickets for BBC TV and Radio shows? Call us on 0870 901 1227 or visit www.bbc.co.uk/tickets
—–Original Message—–
>{Date:} 13/01/2008
>{Comments:}
>Hi, I wondered why I’m watching Dr Robert Winston on ‘The Human Mind’ quoting the apparently very successful trial at the school in Co Durham, involving children and Omega 3 supplements. I have been following it since coverage began, and as far as I am aware the results have yet to be published. How Dr Winston knows the outcome already is beyond me, and I think it may be somewhat premature to have coverage on your site that is so biased. Unless, somehow, this isn’t really a trial and that’s how you all know the results already? That’s not how the peer review system works though. Or it’s not supposed to. Until you bring back the trustworthy ‘Tomorrow’s World’ I shall continue to rely on pubmed and Ben Goldacre for new developments.
Ben Goldacre said,
March 31, 2008 at 6:19 pm
hahah good work.
it’s so funny, it really does just seem to depend on who asks them.
quietstorm said,
March 31, 2008 at 6:21 pm
Jamie, that’s superb
I wonder how the local press are treating this - letters pages etc? Does anyone know the name of a good local paper? Do they have an online presence? You’d think that something like this would have extensive local coverage…. I’m just interested in current local opinion - do people still think it might have helped their children? What about the headteachers involved? Have any of them spoken out? People have spent a lot of (misguided) time and effort, I presume, trying to encourage kids to ingest six capsules a day - are they angry that the results of this effort have been conveniently forgotten? Even though the results are not scientifically useful, I think the people involved deserve the chance to discuss the whole process etc., with the hope that lots of good lessons might be learnt.
At least this whole debacle can now be used as a teaching aid - the mistakes that can occur when designing a trial. I hope teachers get the opportunity to use it!
Ben Goldacre said,
March 31, 2008 at 6:25 pm
there’s some bloke called barry something who had written about 30 positive articles about the trial, in the local paper, back to back. maybe he could use an alerting email?
what i particularly enjoy about all this is how closely it parallels the kinds of nonsense you get at a much more technical level in the world of academic literature. it’s almost as if it was all specifically set up as a kind of pantomime performance piece by “public engagement with science” activists to illustrate the weirdness you sometimes get in academia, but without using any difficult technical concepts.
McCruiskeen said,
March 31, 2008 at 10:09 pm
I notified the Newcastle Journal, with appropriate references on 30/03/08. They have been critical of Durham County Council’s handling of this issue in the past, but it remains to be seen whether or not they regard the story as worth carrying.
McCruiskeen said,
March 31, 2008 at 11:30 pm
What chance does reasoned debate stand and what future has “education” in Durham when primary school head teachers do this sort of thing?
http://www.journallive.co.uk/north-east-news/todays-news/2007/09/13/fish-oil-boost-for-youngsters-61634-19792368/
I wrote to this head teacher, deploring his actions and received a reply acknowledging receipt of my letter. I wrote to all twelve of his governors pointing out the ethical problems in using their pupils, accepting free supplements from equazen etc etc. - no reply from any of them! So I contacted the Newcastle Journal.
http://www.journallive.co.uk/north-east-news/todays-news/2007/09/15/challenge-over-fish-oil-benefits-61634-19793291/
One tries!
Ben Goldacre said,
March 31, 2008 at 11:40 pm
seriously, bloody good on you. it is astonishing that nobody in durham will take any meaningful responsibility for what has happened here. my suspicion is that most of the people involved lack the intellectual horsepower even to grasp the nature of what they’ve been involved in.
NuttyBat said,
April 1, 2008 at 1:53 pm
Ironically, the side panel to the two Journal Live articles features sponsored links for fish oil supplements.
e.g.
http://www.journallive.co.uk/north-east-news/todays-news/2007/09/15/challenge-over-fish-oil-benefits-61634-19793291/
McCruiskeen said,
April 1, 2008 at 3:42 pm
The Newcastle Journal has taken up the issue, following my e-mail:
http://www.journallive.co.uk/north-east-news/todays-news/2008/04/01/silence-on-effect-of-fish-oil-for-students-61634-20699670/
Their comment on page 10 reads as follows:
“It all Sounds Fishy”
“Families in County Durham may be miffed to learn that they must wait even longer for the County Council to deliver its verdict on fish oil supplements.
Until then, they will have to content themselves with Councillor Michelle Hodgson’s categoric statement with regard to the scheme.
She insists that it was never intended to “draw conclusions about the effectiveness or otherwise of using fish oil to boost exam results.”
Which may leave parents asking: what was the point of the exercise then?”
The Northern Echo has picked up the Journal’s story and, I understand, will carry a feature tomorrow.
In the meantime, do visit the Durham County Council Public Questions website where my three questions and Councillor Hodgson’s answers to them are published:
http://www.durham.gov.uk/durhamcc/usp.nsf/pws/281A783A0DAF3B47802573E200458C72?opendocument
tonyy said,
April 2, 2008 at 9:57 am
If one school in County Durham has had unprecedented success at football after putting its kids on the fish oil, then the teams they play against, from all the other Durham schools that are also taking the pills, must have done worse on average.
Surely if all the schools took the fish oil, every team should have won all their games.
Arrowheed said,
April 2, 2008 at 2:34 pm
Or finished in the same positions as last season but all played a little bit better.
McCruiskeen said,
April 2, 2008 at 4:42 pm
The school in question was Toft Hill Primary School. The big fish oil farce was with the county’s secondary schools.
McCruiskeen said,
April 2, 2008 at 6:06 pm
You can let the people of Durham know what you think by leaving comments in the Durham Advertiser at:
http://www.theadvertiserseries.co.uk/search/display.var.2164706.0.0.php
Kess said,
April 3, 2008 at 8:04 am
There’s an odd comment in the Durham Advertiser by one Steve Jones who supports the Durham trial, slags off Ben’s “vendetta” against them, and states that “Durham results were the best ever and their performance improvement was substantially above the average” - directly contradicting Ben’s statement that “the GCSE results for Durham were rather disappointing this year”.
Very strange.
McCruiskeen said,
April 3, 2008 at 5:10 pm
Check out the BBC Look North streaming video of David Ford and Ben Goldacre speaking on the fish oil study at:
http://news.bbc.co.uk/player/nol/newsid_7320000/newsid_7328800/7328844.stm?bw=bb&mp=wm&news=1&bbcws=1
Ben Goldacre said,
April 3, 2008 at 5:25 pm
haha 15 seconds of me with “bike hair” as i dashed through london at lunchtime.
Ben Goldacre said,
April 3, 2008 at 5:39 pm
here’s the deal with Durham’s results:
http://www.badscience.net/?p=538
“The preceding year, with no fish oil, the results – the number of kids getting 5 GCSE grades A* to C – had improved by 5.5%. And now? After the fish oil intervention? This rate of improvement seems to have deteriorated spectacularly. I chased the results myself through Durham press office: this year there was only a 3.5% improvement. And this is against a backdrop of a 2% increase in GCSE scores nationally.”
So, regression to the mean? Extra support from outside? Noise? Fish oil retarded the improvements? Who knows. Well, Durham might have some more data, but frankly…
Kess said,
April 3, 2008 at 10:45 pm
Ben - you really must either carry a comb or get a haircut, else how can you possibly be taken seriously compared to those charismatic, tanned, smooth-talking snake-oil salesman.
Or stick to radio.
Ben Goldacre said,
April 3, 2008 at 11:01 pm
mm trouble is on radio i sound like a girl. rock and hard place.
McCruiskeen said,
April 3, 2008 at 11:46 pm
And you can watch the evening version of Look North with McCruiskeen having his two-pennyworth at the end.
http://news.bbc.co.uk/player/nol/newsid_7320000/newsid_7329600/7329659.stm?bw=bb&mp=wm&news=1&bbcws=1
ACH said,
April 4, 2008 at 9:38 am
“trouble is on radio i sound like a girl. rock and hard place”
Try 40 fags a day. That should give you a nice deep voice. (May have side effects)
Regarding the results - the TV clip says “Durham schools recorded their best ever GCSE results” clearly implying it’s the fish oil wot dun it. From the figures you quote above regarding rate of improvement vs noise, it sounds as if they might be busy massaging what data they have to try to get the required answer. If they’re bothering at all - they might just be hoping that it all quietly goes away.
Lafayette said,
April 4, 2008 at 10:27 am
I think what is more important even than the fish oil story is that Durham seem to have discovered pupils get the same results in their final exams as they have in the mocks. If that is the case, they might as well just sit the exams instead of the mocks, saving children months of hard work!
Asides from the missing control group, if any valuable data were to come from this, one would have to ensure that the children and the teachers did not know the results of the mocks, otherwise you cannot compare the mocks and finals to judge improvement. Knowing you’re likely to fail in a subject is going to have such a huge effect (positively or negatively) on your efforts in that subject that the effects of a few miligrams of oil will become impossible to judge.
pete d said,
April 4, 2008 at 12:35 pm
The Northern Echo has run a lot of puff about this in the past but seems to be hedging its bets somewhat now.
http://www.thenorthernecho.co.uk/display.var.2167279.0.0.php
Who is this Steve Jones character, surely not the ageing Sex Pistols “guitarist”?
Robert Carnegie said,
April 6, 2008 at 10:53 am
Hang on though. You’re saying that the 2006 exam results pre-fish-oil were 5.5% better than the 2005 exam results. And nationwide the 2007 exam results are 2% better than 2006, whereas the fish oil kids were 3.5% better. Surely, if the teachers did the same let’s suppose best practice teaching in 2007 as in 2006 then you’d expect 0% improvement, the same outcome, although nationally it is actually 2% better for some reason. (Don’t say it’s because they made the exam easier - well, if you must.) Doesn’t that leave 1.5% of genuine academic improvement due to fish oil? And isn’t that worth swallowing some oily pills for? (As long as they don’t poison you or nothing.)
If you improve a school between 2005 and 2006 then you get better results in 2006, but why would that give you even better results than that in 2007? I suppose that one argument is that the 2007 cohort have spent 2 years in the improved regime in lower classes and the 2006 cohort only had one year after the improvements and one year before, but please show your working.
mhudson said,
April 6, 2008 at 5:28 pm
I admit that I’ve not been following this Durham fish oil story as closely as many others, but my general feeling with regard to these “take pills -> be a genius” stories is that the supplements would only be of use to those children whose diets are extraordinarily poor. The pills would just then be putting them back into “balanced diet” territory, which could more easily and cheaply be achieved by simply, er…, eating a balanced diet.
Anyway, despite all this, and my natural scepticism towards any of these health supplement stories (like Q10 etc), I’ve been to equazen’s website to have a browse around.
About the constituents of the eye-q pills, they say:
“Testimonials from parents and children have indicated real benefits in learning ability and concentration. eye q has become one of the leading supplements in the UK and European markets for anyone with concerns about fatty acid deficiency in learning and behaviour. The formulation has been scientifically tested, most notably in the ground-breaking Oxford-Durham schools trial, which investigated how fatty acids may be of benefit in the classroom.”
Now… I didn’t like this paragraph because they take “anecdotal evidence says…” and “we’ve done some tests” and juxtapose them to make it sound like the tests back up the anecdotes. Anyway, they provided a link to a page about eye-q research:
http://www.equazen.com/default.aspx?pid=145
This page states:
“The Oxford-Durham Schools Trial
Full details are explained on www.durhamtrial.org, but in short this is the most comprehensive and thorough trial yet of how fatty acids may be applied in the classroom. This trial was run by the Durham LEA, and we contributed free active and placebo capsules. The first results published in the American journal Pediatrics indicate significant benefits for concentration and behaviour.”
So: explicit mention of placebos, mention of some published research. Is this the same trial that is now being referred to as not having been a trial, etc? I am now rather confused.
Ben Goldacre said,
April 6, 2008 at 5:32 pm
as you rightly suggest you’re supposed to be confused.
trials have never been done in mainstream children, only special groups, adhd, dyspraxia, etc.
the durham trial they are referring to there is richardson et al 2005, done in kids with Developmental Coordination Disorder.
i did a quick plain language summary of all the lit a while ago here, it is missing the recent Sinn paper, again not done in mainstream children. as John Stein said in my radio show, the companies are afraid to fund a trial in mainstream children in case it doesnt give them the lucrative result they want:
http://www.badscience.net/?p=299
Academic References
These are the five published trials looking at what happens in children (with various diagnoses) when you give them fish oil supplements. I do not wish to undermine these studies in any sense, but it is worth noting, along with your other readings around them, that in most only a small number of the many variables measured were changed by fish oil, and that the p-values in the variables that were found to be changed were only just below 0.05, that is, they did just reach statistical significance. If you disagree with any of these brief summaries or have anything to add to them then do please let me know. In general, you will see if you get the original papers that they were methodologically meticulous and reported to a high standard. Top Jadad scores all round.
Voigt, R.G. et al., A randomized, double-blind, placebo-controlled trial of docosahexaenoic acid supplementation in children with attention- deficit/hyperactivity disorder. Journal of Pediatrics, 2001. 139(2): p. 189-96.
Kids with ADHD, found no significant differences in objective or subjective ADHD measures between treatment and control group. 63 subjects, 14.3% dropped out.
Richardson, A.J. et al., A randomized double-blind, placebo-controlled study of the effects of supplementation with highly unsaturated fatty acids on ADHD- related symptoms in children with specific learning disabilities. Progress in Neuro-Psychopharmacology & Biological Psychiatry, 2002. 26(2): p. 233-239.
Kids with LD, improvements in Conners ADHD score, inattention, and psychosomatic symptoms (P = 0.05, 0.03, 0.05 respectively) (3 out of the 14 things measured). 41 subjects, 22% dropped out.
Stevens, L.Z. et al. EFA supplementation in children with inattention, hyperactivity, and other disruptive behaviors. Lipids, 2003. 38(10): p. 1007-21.
Kids with ADHD or other disruptive behaviours, only a pilot study, improvements in fish oil group for parent rated conduct problems (p=0.05) & attention teacher rated symptoms (P=0.03) (2 out of the 16 things measured). 50 subjects, 34% dropped out.
Hirayama, S. et al., Effect of docosahexaenoic acid-containing food administration on symptoms of attention-deficit/hyperactivity disorder - a placebo-controlled double-blind study. European Journal of Clinical Nutrition, 2004. 58(3): p. 467-73.
Kids with ADHD, no difference between placebo and fish oil group (oh, except the placebo group, rather than the fish oil group, showed a significant improvement in visual short term memory and continuous performance). 40 subjects.
Richardson, A.J. and Montgomery, P., The Oxford-Durham study: a randomized, controlled trial of dietary supplementation with fatty acids in children with developmental coordination disorder. Pediatrics, 2005. 115(5): p. 1360-6.
Kids with Developmental Coordination Disorder, no significant differences between placebo and fish oil groups for motor skills, but improvements for the fish oil group in reading and spelling (P= 0.04 and <0.01) and CTRS-L global scale (P<0.05) and some subscale improvements (P<0.05) for the fish oil group. 117 subjects, 6% dropped out.
mhudson said,
April 6, 2008 at 5:57 pm
Cheers, Ben. I couldn’t get to the abstract when I was writing my comments, but between then and coming back to see your reply, I managed to read it and see that it was, indeed, from 2005.
So… isn’t there a danger that people might (like I did) confuse the two sets of trials?
What I really don’t like, I suppose, is that research is mostly slow, hard and built up in layers over time. But “the press”, and perhaps the general public, like to have very simple messages to present. “X causes Y, without doubt”, or “worried about being too stupid to bring up your children? give them some of these pills, and you can continue to ignore them”.
Grr (sorry, I *may* have gone off on a rant, there)
ps. I’m hoping to make it to London on the 19th to listen to you, Randi et al.
Dr Aust said,
April 9, 2008 at 4:53 pm
Talking of fish oils and Durham, I was wondering what had become of Dr Alex Richardson (”Richardson, A.J.” in Ben’s post above), who did the “proper” bit of the older Durham study. She doesn’t seem to work in the Physiology Dept at Oxford any more, as far as I can tell from their website.
She is listed to speak, specifically on the evidence about omega-3s, at a couple of upcoming conferences
here and here. Is it too much to hope we can send Ben and his notebook to at least one of them? The second one in Cardiff looks particularly promising.
On a completely different subject, I was wondering about “Steve Jones” from Hull too. Hull is, of course, the home of Seven Seas Ltd. who make a bunch of omega-3 products. Do you think we can start our very own “fish oil lobby” conspiracy theory?
I should say I am wholly in favour of eating fish… as opposed to fish oil. Delicious stuff. Good for you too.
banshee said,
May 1, 2008 at 10:30 pm
Yeah -I’ve been copied in on these adverts for this charity. Odd that some of these talks are on Ethyl EPA which might be an active constituent of “Omega 3 oils” for some illnesses/bahaviours.
Includes Malcolm Peet as a speaker who has a good reputation and has published in this field (and also viva’d my MPhil!) http://apt.rcpsych.org/cgi/content/full/8/3/223
FRB - could not find any clue as to who funds this organisation on their site?
Does everyone know about PharmaCom supporting pressure groups? Seeding the market? Holding Conferences in exotic places? Well maybe not the latter but there could be some marketing similarities going on here!
student77 said,
May 7, 2008 at 3:21 pm
Hi, I was just wondering if anyone could post the reference for the recent ‘Sinn’ paper that has been referred to in 52?
Thanks