I think it’s clear now that Madeleine Portwood and Dave Ford, the leading figures behind the Durham fish oil "trial", will be providing us with comedy and teaching opportunities for many years to come.
First they said they were doing a "trial".
Then when everyone pointed out it was incompetent research, they said that didnt matter, because it wasn’t a "trial", it was an "initiative" (and even retrospectively changed the wording of their press release online…).
Then when people asked for details of what they were doing, how they were going to measure success, they refused to say, even when hundreds of people asked, using the freedom of information act.
Then when the GCSE results came out, everyone could see a slowing in the rate of improvement.
Then Durham announced that they weren’t going to release the results after all, and claimed – defying all reason – that they’d never said they were doing to, and it was never their intention to analyse the results, or tell anyone what they were.
Now they have released the results.
You really, seriously, couldn’t make this up.
Initially they said they were going to measure GCSE performance of individual children against what they predicted the children should have attained (ingenious!). Their current analysis is even more incompetent. I’ve got to run out of the door, will add links later. Below is what I sent to the Northern Echo, and Barry Nelson, after he sent me their press release. I’m posting it so you can see what happens behind the news. After receiving it he wrote these two articles:
SCHOOL FISH OIL STUDY GIVES SCIENTISTS FOOD FOR THOUGHT
A north east council’s study into the effects of a fish oil dietary supplement on pupils’ GCSE exam results may have given scientists food for thought.
Detailed analysis of the outcome of the initiative, launched by Durham County Council in advance of last year’s GCSE exams, shows that pupils who took the Omega-3 supplement did better than those who did not.
First they said they were doing a trial, then when it was over they refused to release the results, and claimed – in defiance of everything they’d already said at the time – that it was never their intention to do a trial. Now they claim to have some results. This is extraordinarily irresponsible and unprofessional, Durham Council have performed an experiment giving 6 capsules a day to thousands of children, and they have a responsibility to behave ethically and transparently. They have failed in this regard.
And the council’s education chiefs believe that although the results of the study are not definitive, they could now justify more clinically-based scientific trials
This is a meaningless term, a revealingly incompetent "jargony" misuse of basic scientific terminology.
to determine once and for all whether fish oil supplements can and do boost educational attainment among young people.
They could easily have determined if fish oil capsules are beneficial in this "trial", if they had performed it competently, which they chose not to, despite all offers of help and criticism at the time.
Dave Ford, Head of Achievement for Durham County Council’s Children and Young People’s Services said : " We have always maintained that if the outcome was positive, it would then be for the scientists to examine in more detail.
Dave Ford said he knew the results would be positive before it even began. I’m not surprised: this "trial" was flawed by design from the outset.
" The findings of our study suggest it may now be worth them following it up in more depth through proper clinical trials."
Durham could very easily have performed a proper trial were it not for the incompetence of Dave Ford and Madeleine Portwood.
Initially, just over 3,000 Year 11 pupils began the study, taking the Omega-3 tablets at school and at home.
By the time GCSE examinations came around, 832 pupils had 80 per cent or greater compliance.
This is appalling. 2,168 of their subjects dropped out of the trial. They must count these people in the results. It is incompetent not to do so. This makes the rest of their claimed results even more meaningless.
Mr Ford and his colleagues then sought to identify the same number of Year 11pupils who had not taken the supplement and match them to those who had, according to school, gender, prior attainment and social background.
This is completely different to what they claimed they were going to do and measure two years ago.
Mr Ford and his colleagues repeatedly refused all requests from myself and dozens of other academics and interested members of the public, including under the freedom of information act in exasperation, where we simply asked what they were planning to do and what they were going to measure in their experiment on thousands of Durham children. They refused to tell us.
It is vitally important that people performing experiments on children are clear what they are doing to them, and what they are measuring, before they begin, otherwise they can move the goalposts and get a false positive result afterwards, simply by measuring things lots of different ways until they get the positive result that they want.
We gave them every opportunity to do the right thing and describe what they were going to do, and what they were going to measure. They refused to tell us, entirely as predicted, they have moved the goalposts and are measuring a "positive result" in a completely new way. Perhaps they have done this because the way they initially said they were going to measure outcomes did not give them the result they want?
This is incompetent and means that the results are entirely meaningless.
The GCSE results of 629 ‘matched pairs’ – fish oil takers and non-fish oil takers – were then analysed.
In selectively only looking at the results from pupils who were highly adherent to the capsules regime, they have skewed their sample, and are simply measuring children who are likely to be more adherent to work at school, harder working, better performing, different backgrounds, from completely different families, and so on. This is laughably incompetent science, in an experiment performed on thousands of Durham children.
" To reach comparative levels of their attainment prior to the study, we used a nationally accepted system which took into account the results for each pupil at Key Stage 2 and Key Stage 3," said Mr Ford.
" At both stages, the difference in predicted GCSE outcomes between the groups was on average less than three points.
" However, following the fish oil initiative, the difference in Key Stage 4 (GCSE) results between those who had taken the supplement and those who had not rose
to 17.7 points.
They are not comparing like with like, so their results are meaningless in any case, but it is impossible to understand from this statement what they have measured, how they have matched samples, and so on. That is the problem with incompetent people doing science as PR, and communicating their results in press releases. I will be contacting them to clarify.
" If there had been no difference in attainment between the two groups, we would be tempted to dismiss the benefits of Omega-3," said Mr Ford.
I very much doubt that Mr Ford will ever dismiss the benefits of omega-3.
" However there seem to be some very clear indications that pupils taking the supplement do significantly better."
Rubbish. Mr Ford has produced meaningless data from an incompetent "trial" performed on thousands of children. I believe it is unethical to perform incompetent research on children, as it wastes goodwill, exposes people to risks unnecessarily, and misleads parents and children into believing that they are participating in a project which will produce useful information for everyone. In reality this was nothing more than a marketing project for some pills and, I’m sorry to say, a vanity project for some people in Durham council. It is scientifically worse than worthless, because it has been used, and will be used, to mislead the public.
Mr Ford said the Council made no claim that the results of its GCSE study could be attributed only to Omega-3 supplementation.
This is laughable. Everything Mr Ford has said in public for several years now has been to promote the use of Omega-3 supplementation.
" Other factors may be responsible for the difference in performance – for instance, the benefit may be a placebo effect, or it may be that those students who achieved 80 per cent compliance were better organised and had families who provided support at home and so might have done better anyway," he added.
Glad to see he acknowledges this. Then why did he decide to perform his experiment on thousands of Durham children incompetently in this way? He was told before he began that this experiment was wasteful and scientifically meaningless. He could have done a proper study for no more money, he chose not to. Why?
" But taking all this into account, it is our view that this study has produced some interesting and possibly exciting issues for further investigation that could be the basis for future scientific trials, and that is all we ever set out to achieve.
" The road to determining whether there are education benefits to be had from taking fish oil supplements was always going to be a long one, but I hope the findings of our study have taken the journey one step further."