You’ll remember the Durham fish oil “trial” story, possibly the greatest example of scientific incompetence ever documented from a local authority.
Initially they said – to blanket media coverage – that they were running a trial on fish oils, giving pills to 3,000 children to see if it improved GCSE performance. I pointed out, along with several academics, that their experiment was incompetently designed, for no good reason, and so would only produce false positive results. They responded that this was okay, as they hadn’t called it a “trial”. This was very simply untrue: they had, repeatedly, in press releases, and interviews, and who cared anyway, because whatever you called it, this was still a stupid experiment. Durham’s response was to edit the online version of their press release to remove the word “trial”.
Then I asked what they were going to do in this experiment, how they were going to measure results, and more. They refused to give me this information – for an experiment by a public body performed on thousands of children – so I used the Freedom of Information act. They still refused. Then hundreds of you wrote to their information commissioner, using the FoI, and they refused again, accusing us of running a “vexacious campaign”.
Then the GCSE results for Durham came out: they weren’t too great, so I asked for the results of the “trial”. Durham refused to give me this information. Then they announced, bizarrely, untruthfully, in a formal response to a written question: “It was never intended, and the county council never suggested, that it would use this initiative to draw conclusions about the effectiveness or otherwise of using fish oil to boost exam results.”
This was, once again, very untrue. Durham’s own press release had clearly said they were giving out the pills “to see whether the proven benefits it has already brought children and young people in earlier trials can boost exam performances too”. All the press coverage said the same. The council’s chief schools inspector Dave Ford said “the county-wide trial will continue until the pupils complete their GCSE examinations next June, and the first test of the supplement’s effectiveness will be when they sit their ‘mock’ exams this December.” Suddenly this trial did not exist.
Now, bafflingly, suddenly, in defiance of their previous denials, Durham have released some results. “Detailed analysis of the outcome of the initiative,” they say “shows that pupils who took the Omega-3 supplement did better than those who did not.” Hardly. Let’s try to disentangle what they think they’ve done.
“Initially, just over 3,000 Year 11 pupils began the study, taking the Omega-3 tablets at school and at home. By the time GCSE examinations came around, 832 pupils had 80 per cent or greater compliance.” This is appalling. 2,168 of their subjects dropped out of the trial: they must count these people in the results. They do not. This makes the rest of their claimed results even more meaningless.
“Mr Ford and his colleagues then sought to identify the same number of Year 11 pupils who had not taken the supplement and match them to those who had, according to school, gender, prior attainment and social background.” They originally said they were going to compare childrens’ predicted GCSE performance (whatever on earth that means) against actual performance. Of course, they refused to say how they would analyse this, despite hundreds of requests: and it is vitally important that people performing experiments are clear what they are measuring, and how they will analyse it, before they begin, otherwise they can move the goalposts and get a false positive result afterwards, by slicing the cake a dozen different ways.
“The GCSE results of 629 ‘matched pairs’ – fish oil takers and non-fish oil takers – were then analysed.” Who are these 629? I thought it was 832? But more importantly, by selectively only looking at the results from the pupils who were most highly adherent to the capsules regime, they have skewed their sample, entirely unnecessarily. They have, in fact, simply discovered that school performance is better in children who are more highly adherent to a school regime involving pills, and who are, in all probability, also more adherent to everything at school, harder working, better performing, from completely different families, with higher aspirations, and so on. If you wanted to design an experiment to produce a spurious false positive result, you could not do any better than this.
This result has nothing to do with the pills, it is laughably incompetent science, in an experiment performed on thousands of Durham children. It has been widely reported in the local press, and in the Telegraph (doubtless with more to come) as “perhaps there is something in this preliminary stuff”. Every journalist is baffled by the details. Some report that there is “controversy”, in the classic style of journalists who simply report “two sides”.
But there are not two sides here: there is nothing in this work. It is not “controversial”. A crap unpublished result presented in a press release from an incompetent experiment analysed incompetently in god knows how many different ways by incompetent people who have shrouded themselves in secrecy and demonstrated themselves to be incapable of making reliable true statements about their own research – or even whether it exists – is not a “maybe”.
Fish oil pills are now the biggest selling food supplement product in the UK, with a market size of £110m. Nobody has ever tested them. Most damningly of all, Durham council had the children, and the pills, necessary to perform a decent piece of research. The only thing they were missing was the rigour, and help was offered. The only real question now is this: why has Dave Ford performed a incompetent experiment on thousands of children? And more importantly, why has Durham Council let him?
pv said,
September 30, 2008 at 11:53 pm
Is that not a tad disingenuous?
If it wasn’t supposed to be a scientific trial (“experiment” has Dr Mengele overtones here), the only other thing that could possibly have been intended was a marketing exercise for the company that supplied the product. After all they were mentioned enough in press releases and the national press.
It does prompt a couple of questions:
Is it really the job of David Ford on behalf of DCC to conduct a product placement exercise on behalf of a commercial entity?
And, is this a case of corruption or rank incompetence?
McCruiskeen said,
October 1, 2008 at 11:27 am
If it was never a “Trial” or “Study” then David Ford and DCC should have been much more circumspect in their press releases and quotations to the media. If David Ford reents the criticism he has received, he only has himslef to blame, it seems to me, and he continues to be evasive in his repsonses.
There is no doubt in my mind that this was in part a gigantic marketing exercise for a certain company who mentioned Durham County Council in their advertising. Commercial reality dictates that the company would hardly supply supply £1m worth of “free” capsules to DCC for a properly conducted study, the results of which could have been indifferent or even negative. That would have been expensive commercial suicide.
Desperation may also have played a part. There is tremendous pressure on local authorities such as Durham to raise their game in their schools, as measured by GCSE scores. David Ford may well have thought that the capsules may have had some effect and so there was nothing to be lost in promoting his “Brainchild.”
Equally, he may have felt that the Hawthorn effect might have assisted even if the capsules themselves did not.
In any event, one cannot fault his desire to assist some of the most socially disadvantaged children in the country (with the government breathing down his neck!) by rolling out his “Brainchild.” In my opinion, where he went wrong was in seeking publicity for it!
It was not necessary, so why did he do it? Pressure from Equazen who clearly needed it and were footing the bill? Raising his own and DCC’s profile?
Whatever the reasons, it all went disastrously pear shaped and attracted all kinds of legitimate criticism, which David Ford now resents.
Things have changed at Durham County Council. It now has a leader with a brain, in Councillor Simon Henig. Hopefully scrutiny by members will improve. How can you ask appropriate questions about something you do not understand?
I doubt that David Ford will ever get away with another “wheeze” like this!
Dr* T said,
October 2, 2008 at 11:03 pm
I agree. There are a lot of questions that a decent full-time investigative journalist should be sniffing out answers for. I reckon there is a cracking scoop still to be had.
Stinks like fish.
McCruiskeen said,
October 3, 2008 at 12:51 am
I forgot to add that the media just don’t seem interested.
McCruiskeen said,
October 4, 2008 at 8:28 am
Robshorrock – I, for one, had not considered that option. Referral to the Local Government Ombudsman must be made within one year of the event about which one complains. The Director of DCC Children and Young People’s Services “bullshits” an enquiring MP and David Ford is interviewed on BBC NE television, coming across as aggrieved about being misunderstood and defending his actions – even smiling. They obviously think they have it “sewn up” and they probably have. Some one has to be respomsib;e for education (if it can be called that) in County Durham; it is just a pity that integrity and an ethical conscience do not appear to be among the criteria required for the role.
Sili said,
October 4, 2008 at 8:51 pm
Thank you, bob sterman.
I think I’ve partly misunderstood what the HD is, and partly expressed myself badly.
There is some kind of provision in the law against experimentation on people (and for that matter animals) without ethical approval, isn’t there?
Or is it nothing but guidelines and professional contracts? I know Wakefield stands before the GMC and not a court, but still … Does this mean that it’s only a problem if a registered doctor does it, while Joe Q. Public can go ahead and do as he pleases?
McCruiskeen said,
October 13, 2008 at 4:11 pm
Episode 6 of my Blog (David Ford Replies to Me) is now available.
http://maccruiskeen@wordpress.com
DrFred said,
October 14, 2008 at 1:47 am
Just thought I’d share a bit of similar test improvement foolishness from across the pond. Today I was given a package of mint chewing gum to share with my students to ‘improve their performance’ on their NECAP tests required under the NCLB mandates.
This evening, while searching for other examples of ‘Dumbo’s feather’ in education, I stumbled on an on-line study run by the Univ. of Northumbria psychology department to assess the effect of Omega 3/Fish Oil on memory performance.
(Link is: www.omega3research.org.uk )
I decided to participate in the study, playing both parent and child roles. I proceeded to the end, wondering when I would be asked for some type of validating information. I know you’ll never guess – there was none! The only potential trap might have been when I selected a weight and height for my 14 year old child – couldn’t remember what a stone was equal to in pounds – had to spend a moment looking it up.
In any case, I’m sure this study’s results will someday be publicized, yet it’s results are totally bogus. Any idiot, including this 62 year old Vermont chemistry teacher, can participate!
Keep up the great work exposing bad science!
McCruiskeen said,
October 14, 2008 at 12:21 pm
Chapter 7 of my blog is now available at:
maccruiskeen.wordpress.com
(Sorry about the earlier mistyped link!)
Chapter 8 (in a couple of weeks) will be about the questions I asked in the DCC council chamber and DCC’s pathetic response(s)!
McCruiskeen said,
November 7, 2008 at 5:59 pm
Chapter 8 is done!
EleanorC said,
November 8, 2008 at 10:16 pm
Hi.
McCruiskeen: “Have you noticed how far back Equazen have stepped in this? […] I doubt they wish the product they gave so generously and philanthropically to be sullied by any of this criticism.”
Then again… I’ve just picked up a leaflet in a health food shoo, advertising Eye Q. Front cover: “Look Mum! The fish oil used in the largest ever omega-3 schools trial was eye q.” Inside it refers to “the Durham Schools Trial” four times and also bangs on about the “Middlesbrough Trial” and how robust, unbiased, etc that is. It finishes with the delightfully fallacious statement that “eye q was the only [fatty acid supplement] used by the Middlesbrough and Durham LEAs, and therefore the only one worthy of your consideration … To find out more about the Durham School trials please visit www.durhamtrial.org.”
I guess any publicity is good publicity.
(Sorry if this is all old news, by the way.)
athene317 said,
November 10, 2008 at 9:12 pm
Ah, wonderful. Thanks to the BBC’s coverage of the Durham “trial”, this nonsense is now being implemented in New Zealand classrooms! www.nzherald.co.nz/health/news/article.cfm?c_id=204&objectid=10542321 Hawthorne effect, anyone?
Dr* T said,
November 17, 2008 at 1:22 pm
I’ve just this NZ nonsense and I see MacCruiskeen is still hammering away at Durham!
YYY said,
January 15, 2009 at 2:26 pm
Hammering away is not always that smart neither. Your book is very interesting, ideal to freshen the critical brain!
But for PUFAs there’s still some hope, so instead of hammering alone, some extra research wouldn’t be bad!
Referenties :
Richardson AJ, Puri BK. Prog Neuro-Psycho-Pharmacol 2002; 26: 233-9
www.fabresearch.org/view_item.aspx?item_id=201
Starobat-Hermelin B, Kozielec T. Magnes Res 1997; 10: 149-56
Van Oudheusden LJ, Scholte HR. Prostagl, Leukotr Ess fatty Ac. 2002; 67(1): 33-8
anonymouse said,
March 5, 2009 at 11:10 am
This might be a clue as to why Dr Portwood didn’t exercise proper oversight–she may have been developing a product with Equazen. I saw her speak at an autism conference in Newcastle in 2004. If memory serves, the results of the original “trial” had not been released at that time but she basically said that while they are embargoed til publication I can tell you that they were really great.
Here’s something I do remember quite clearly: She started her talk by saying that some people in the audience might not recognise her, because her appearance has greatly improved due to using an amazing new EFA-based anti-wrinkle or skincare product that she was involved in developing and would be bringing to market in future. I wonder what company was involved in that development, hmmm? Because as far as I know, Dr Portwood does not have a home chemistry lab.
Haven’t heard anything about the product since so I guess the transformation was not as miraculous as she thought.
janetholden said,
September 4, 2009 at 10:04 pm
This sort of pseudo science is creeping in everywhere. I just came from some training in my college we were supposed to be updated on the current responsibilities of classroom teachers on recognising students with dyslexia. The “consultant” rapidly got into all kinds of quasi scientific nonsense, but when fish oil was mentioned I had to respond! The alarming asertion was that fish oil causes an increase in myelination of neurones in the brain of dyslexic children. I asked what the evidence was and was told it was on the “web” !!! One study…………. Are these people cynical, or just ignorant? A little bit of knowledge can be a very bad thing. We have to fight back. If anyone knows anything about this I’d love to know.
dystalk said,
September 23, 2009 at 1:40 pm
Completely agree, Janet, though we recently filmed Oxford Prof. John Stein, who argues for the benefits of experimentation when it comes to solutions for dyslexia. See this, esp., on fish oils:
www.dystalk.com/talks/88-the-use-of-fish-oils-for-children-with-dyslexia-and-adhd
What do we think?
Will
chazmike said,
October 9, 2009 at 4:28 pm
anyone wandered onto this site by mistake
ezinearticles.com/?Fish-Oil-and-Children–The-Durham-Trials&id=471285
its worth a look for a laugh.
I posted a comment a week ago pointing out one or two salient things, and guess what? My comments have disappeared and not been published. Wonder why.
Stegs said,
October 21, 2009 at 9:53 am
I’m confused.
The following article from BBC Science & Nature talks about the Durham Trial using an ‘experimental’ method called a randomised double-blind controlled trial. Where half of pupils got the Omega 3 and the other half got Placebos. Are they talking about the same “Durham Trial” or is this article completely incorrect?
I note that the article still doesn’t say how many children took part in this amazing, spectacular ‘experimental’ randomised double-blind controlled trial.
www.bbc.co.uk/science/humanbody/mind/articles/intelligenceandmemory/omega_three.shtml
jiang said,
December 22, 2009 at 5:22 am
ed hardy ed hardy
ed hardy clothing ed hardy clothing
ed hardy shop ed hardy shop
christian audigier christian audigier
ed hardy cheap ed hardy cheap
ed hardy outlet ed hardy outlet
ed hardy sale ed hardy sale
ed hardy store ed hardy store
ed hardy mens ed hardy mens
ed hardy womens ed hardy womens
ed hardy kids ed hardy kids ed hardy kids
Philip123 said,
December 26, 2009 at 11:37 pm
Interesting article on review of the literature that indicates, on average people with High cholesterol live longer.
healthjournalclub.blogspot.com/2009/10/do-people-with-high-cholesterol-live.html#more
Also an interview with the MD Phd who first did much of this research
healthjournalclub.blogspot.com