They still don’t seem to understand the problems with the one in 58 figure, and they still don’t seem to be able to understand the report they keep going on about (but won’t let anyone see because they think their scientific evidence is top secret), and they are still covering up their mistakes.

At first, when it was a one off story, this was a slip. Now, after two failed “clarifications” this story is just blowing me away. I simply don’t understand why they haven’t printed a full, clear retraction and apology, and I don’t see how they can possibly gain from continuing to half defend this story with further inaccuracies.

Stop me if this is too much detail: writing tight takes time and I’m really over this story now. I think people often rely on the boringness of details and repetition to cloak their obfuscation. Here in Anal-land, that’s not an option. Since the Observer piece today is the only one on the entire comment pages which doesn’t permit any comments underneath, I felt obliged to write one here.

Autism?

Here’s a simple example: the Observer still go on about this study measuring the prevalence of autism. In the version I have of the interim report, which we think is the closest to what the Observer has, the opening line is “The present study provides a novel approach to active identification of all autism spectrum disorders in a population based sample of children aged 5 to 9 years in Cambridgeshire, UK. ” Not autism, as the Observer says, still, in its article, but autistic spectrum disorder: autism, aspergers syndrome, atypical autism. A far wider net.

Stott and Wakefield:

Their efforts on Stott are mealy mouthed in the extreme.

“Dr Stott, one of the authors of the Final Report and described by The Observer as believing that there maybe a link in a small number of cases between MMR and autism, does some work for Thoughtful House, the autism centre in Texas that treats children from all over the world. Dr Wakefield works at Thoughtful House.” etc.

For chrissakes. “Some work”? Dr Wakefield “works at” Thoughtful House? Dr Wakefield founded Thoughtful House!

www.thoughtfulhouse.org/founders.htm

He doesn’t “work at” Thoughtful House. He is the Executive Director.

www.thoughtfulhouse.org/board.htm

This is beyond childishness. They were wrong. They should have clarified the closeness of the relationship in the article, and if they’re making amends now, they should do so properly. But instead they’re still trying to cover up (rather ironically for a story effectively claiming a cover up about autism).

In the Observer’s world, Wakefield “works at” a place in America where Stott also does “some work”: in the real world, Stott and Wakefield have even issued joint press releases answering critics of Thoughtful House.

They still somehow don’t understand the perfectly simple concept of an unfinished analysis:

I have no idea if this is defensiveness, or if there is still something that’s not clear about this to people at the Observer, but the analysis wasn’t finished. I don’t see why they’re going on about it, or suggesting it might have been, or defending that point. It was an incomplete analysis. It doesn’t matter what they think they’ve seen. They were wrong. But no, they’re still defending it:

“the report from the ARC was entitled the Final Report of a three-year research project for the Shirley Foundation, a private charitable trust that has an interest in the issue of autism. The foundation paid almost £300,000 for the study which Dr Scott, one of the authors, described in an internal email as ‘very thorough’.”

And here – really rather bravely to my mind – they are quoting Dr Scott again. What exactly did she say, in what context, and to whom? Was Fiona Scott simply saying that the text of the write-up was “very thorough”? Was she saying it was “very thorough… for an interim report”? Or was she saying the statistical analysis of the collected data was “very thorough” and complete, as they suggest? Did they ask Dr Scott what she meant by “very thorough”? Did they call her before they quoted her again this week? Are they absolutely certain that this time they’re not misrepresenting her views, again, and that they’re not misquoting an email, again?

We’ve already seen, remember, how the Observer have distorted emails from other people to make this story stand up. This from my BMJ piece on the Observer’s article, for example:

According to the Observer, Baron-Cohen “was so concerned by the one in 58 figure that last year he proposed informing public health officials in the county.”

But Professor Baron-Cohen is clear: he did no such thing and this was simply scaremongering. I put this to the Observer, which said it had an email in which Baron-Cohen did as the paper claimed. Observer staff gave me the date. I went back to the professor, who went through his emails. We believe that I too now have the email to which the Observer refers. It is one sentence long, and it is Professor Baron-Cohen asking if he can share his and the other researchers’ progress with a clinical colleague in the next door office. This dramatic smoking gun reads: “can i share this with ayla and with the committee planning services for AS [autism services] in cambridgeshire if they treat it as strictly confidential?”

Professor Baron-Cohen told me, “That’s not saying I’m concerned, or that we should notify anybody; these are just the people who run the local clinic, who I share a corridor with, who said they were interested to hear how it was going so far. They are not public health officials, and it’s not alarmist, it’s not voicing concern, it’s simply saying: ‘am I allowed to share a paper with a colleague in the next door office?’ It seems very important to me that we discuss clinical research with clinical colleagues, and I only stressed confidentiality because the paper was not yet final.”

Failing to understand the central point: there was never any suggestion of an increase in autism prevalence over time:

They then try to justify their use of the one in 58 figure, by saying it was described as the “primary analysis” in the paper. Again, as I have said many times before, this paper was specifically designed to cast the widest possible net.

They still don’t get this at the Observer. They still think they found an increase. They even quote favourably – because they still clearly stand by this – the incomprehensible Readers’ Editor piece from last week:

The central point, in my view, is that the leaked story of the apparent rise in the prevalance of autism was a perfectly legitimate and accurate story in its own right, which did not need the introduction of the MMR theory.

No. The central tenet of the Observer piece was wrong. There was no rise in prevalence found. They were fed a paper – god knows how, or by whom, god knows whether these schoolboy misunderstandings of basic methodology were made first by the Observer or by the person who fed the story to them – but they were fed a paper that used a different way to measure autism and autistic spectrum disorders, aspergers, atypical autism, and this wider net got a bigger number, as you would expect, and even that bit is not even certain, because the analysis is not complete.

How can they still not understand this? After ruminating on the subject for two weeks? It was very decent of them to finally call Dr Fiona Scott, after misrepresenting her opinions for two weeks, and clarify her views. I don’t imagine she’d be too pleased with them. But could they not also have phoned someone who understands basic research methodology, and could explain the paper to them? Are they so puerile that they think that every single person who is capable of reading and explaining that paper is part of a conspiracy to cover up MMR?

The “primary analysis”

Lastly, looking at the paper, the term “primary analysis” is one they are eager to quote as exxoneration, as if it justifies their using that “one in 58” figure alone.

The meaning of the phrase “primary analysis” as used in the version of the paper I have is very simple. They described three different ways of estimating the prevalence of autism spectrum disorders (aspergers, autism, atypical autism, etc), their “primary” is counting all possible cases from all the different sources of data, various screening tools, school records etc; the “secondary” is using a subset of those; and the “tertiary” is counting only existing confirmed diagnoses and cases reported by schools, without including any cases from their extra screening tools.

The “primary analysis” only had special status in that it was explicitly a new way of measuring prevalence that cast its net far wider than before: therefore it is absolutely no surprise that it gave a larger figure, it is simply a different way of measuring not just autism, but lots of other conditions on the autistic spectrum.

“New health fears over big surge in autism” my arse. I’ll post the picture again. The story was wrong. Wrong. No surge. A different method of measuring it. You misunderstood a very simple research paper. Get over it. Retract!

Without this crucial context their entire news story was meaningless. And screening tools, anyway, in a general population, raise fascinating statistical issues like positive predictive value, because the predictive value of a screening tool changes as the prevalence of the condition in the population you are screening changes.

It’s very weird maths but there you go, life is complicated sometimes, that’s why medical statisticians write and interpret papers instead of sports journalists like the Observer’s Denis Campbell, who wrote the front page story (you’ll notice I’ve deliberately kept his name out of this until now, and advisedly so, it’s clear this is a systematic failure, and that is even more clear now that we have had two failed “clarifications”).

The wider issue – a lack of basic skills

To me this is the key problem. They want to write about some research. They want to disagree with the lead authors. Fine: but they need to understand the research to do that.

It’s not difficult. They would be welcome to give me a call. God knows I’ve been calling them and leaving messages, so they have my number. I’d be happy to help out. Lots of people ring me for informal advice on stuff like this. I’m always happy to chat, and this isn’t showing off: being able to read a paper is not a mandatory life skill, but it’s not difficult either.

If they’re going to get their sports correspondent Denis Campbell to write a front page story on MMR with an interpretation of figures that specifically and rather bravely goes against the interpretation of the data by the same study’s own lead authors, then their sports correspondent should know how to read a medical academic paper too.

Journalists often say, well, all we can do is report concerns, we can’t be experts, and that’s fine: but that simply doesn’t apply here. Denis Campbell wants to go beyond “reporting”, in his first week after moving from sports to health, so he’ll be needing some next level skills. Easy.

Not getting in touch with Dr Fiona Scott

In the original articles they misrepresented her as believing that MMR causes autism, and that MMR caused that “increase” which the Observer still fantasise this study found. Scott protested about this to everyone who would listen. Then the next week, the Observer again, bafflingly repeated this false claim in their first non-clarification.

Now they say:

Although we attempted to contact Dr Scott by email before publication, we were unable to speak to her. We should have made greater efforts to speak to Dr Scott directly and apologise for this, and for suggesting that she links rising autism prevalence figures with the use of MMR.

What efforts did they make? Did they email her? On what address? I Googled Dr Scott, the moment I saw the Observer’s first article: I found her ARC email immediately, I emailed her immediately, she replied within hours, almost before I had finished writing one blog post. No other newspaper has had any difficulty getting hold of her. I asked Dr Scott if she had received any emails from the Observer, and she said no.

And i confirm again that i received no emails whatsoever from them. First contact I had from them was yesterday when the cheif news editor telephoned my mobile (and hey – got me immediately) to say that the piece was going in the Observer today.

and

They SAY they emailed me but i got nothing, and all my work address emails are forwarded automatically to my home address so it would have come through. I believe on the phone “Thursday before the article” was mentioned as the dya I was emailed(I did not respond one way or the other to the comment that they had emailed once). I have checked records – nothing. So IF they emailed the address must have been wrong or it was bounced back to them. Either way I did not receive anything, and they made no further attempts.

Now one email can go missing, but after someone was protesting that you’d misrepresented their secret views, surely at least then you’d get in touch with them? But no.

And of course, the Observer still even now have the bravery to quote an email they say they have from Dr Scott (“very thorough”) to defend their position. And they’ve still not asked her about that email at all. Very brave indeed, given the circumstances.

I could go on.

I am amazed they are sticking at it. This “on the one hand on the other people have said…” nonsense is misleading, confusing to readers, and absurd. The front page story was based on unfinished research which the reporters were clearly unable to read and interpret. They thought they knew better than the people who wrote it. They were wrong. End of issue.

They should stop beating about the bush, and fully retract this entirely bogus story.

So you can remember the joy, here is their original front page article:

www.guardian.co.uk/medicine/story/0,,2121542,00.html

Here is my delayed bad science column on their story:

www.guardian.co.uk/comment/story/0,,2128807,00.html

Here is my longer and juicier British Medical Journal article on the affair:

MMR: the scare stories are back

Here is the Observer’s incomprehensible readers’ editor defense of their piece:

www.guardian.co.uk/commentisfree/story/0,,2126649,00.html

And here is their latest attempt, pasted below.

www.guardian.co.uk/medicine/story/0,,2132076,00.html

The Observer and autism: a clarification

Sunday July 22, 2007
The Observer

On 8 July, The Observer published a news report under the headline ‘New health fears over big surge in autism’. The article revealed details of an unpublished report by the Autism Research Centre (ARC) at Cambridge University which showed that a statistical analysis of autism prevalence among primary schoolchildren in Cambridgeshire had produced a figure that as many as 1 in 58 children could be suffering from forms of the disorder. This figure is nearly double the presently accepted prevalence of autism of 1 in 100.

Article continues
The news report also said that two of the authors of the report believed that in a small number of cases the triple measles, mumps and rubella (MMR) vaccine could be linked to the incidence of autism.

The news report has been the subject of a number of comments since its publication. Critics have said that The Observer should not have published figures from a report that had not been finalised, that we failed to detail other figures from the report that showed a lower prevalence of autism, that we did not reveal the links between one of the authors, Dr Carol Stott, and Dr Andrew Wakefield, who has made controversial claims of a link between autism and the MMR vaccine, and that we did not accurately reflect the views of another of the authors, Dr Fiona Scott, on the possible links between MMR and autism. There are a number of points in The Observer report that should be clarified:

The status of the report

The report from the ARC was entitled the Final Report of a three-year research project for the Shirley Foundation, a private charitable trust that has an interest in the issue of autism. The foundation paid almost £300,000 for the study which Dr Scott, one of the authors, described in an internal email as ‘very thorough’. As such The Observer believed it legitimate to report its findings, given the apparent status of the work. Professor Simon Baron-Cohen, the director of the ARC, has subsequently said that the data in the report is still being analysed and is therefore incomplete.

The 1 in 58 figure and other statistics in the report

The 1 in 58 figure was described by one of the authors as ‘our primary analysis’ and was the only figure presented in the Final Report’s summary. It was therefore highlighted by The Observer. In the body of the ARC’s report the figures 1 in 74 and 1 in 94 were also published.

The Observer should have reported these figures in the news story so that readers were aware that there were different interpretations of the findings. That they were left out was due to a reporting and editing error.

Dr Carol Stott

Dr Stott, one of the authors of the Final Report and described by The Observer as believing that there maybe a link in a small number of cases between MMR and autism, does some work for Thoughtful House, the autism centre in Texas that treats children from all over the world. Dr Wakefield works at Thoughtful House. Dr Stott’s links to Dr Wakefield should have been made clear in The Observer news report.

Last week, in addition to a number of letters critical of the paper’s reporting, The Observer’s Readers’ Editor wrote about the coverage of the autism issue. He concluded: ‘The central point, in my view, is that the leaked story of the apparent rise in the prevalance of autism was a perfectly legitimate and accurate story in its own right, which did not need the introduction of the MMR theory.’ In response to his piece, Dr Scott posted her views on the Guardian Unlimited website. We republish that posting here:

‘I feel, given that I was one of the two ‘leaders in the field’ (flattering, but rather an exaggeration) reported as linking MMR to the rise in autism, that I should quite clearly and firmly point out that I was never contacted by and had no communication whatsoever with the reporter who wrote the infamous Observer article. It is somewhat amazing that my ‘private beliefs’ can be presented without actually asking me what they are. What appeared in the article was a flagrant misrepresentation of my opinions – unsurprising given that they were published without my being spoken to.

‘It is outrageous that the article states that I link rising prevalence figures to use of the MMR. I have never held this opinion. I do not think the MMR jab ‘might be partly to blame’. As for it being a factor in ‘a small number of children’, had the journalist checked with me it would have been clear that my view is in line with Vivienne Parry of the JCVI [Joint Committee on Vaccination and Immunisation]. The ‘small number’ was misrepresented by being linked inappropriately and inaccurately with ‘rise in prevalence’, leading readers to arguably infer that it is in fact NOT a small number!

‘I wholeheartedly agree with Prof Baron-Cohen, and many of the posts and responses received to date, that the article was irresponsible and misleading. Furthermore I reiterate that it was inappropriate in including views and comments attributed to me and presented as if I had input into the article when I had not (and still have not) ever been contacted by the journalist in question.’

Although we attempted to contact Dr Scott by email before publication, we were unable to speak to her. We should have made greater efforts to speak to Dr Scott directly and apologise for this, and for suggesting that she links rising autism prevalence figures with the use of MMR.

I am pretty jaded and sceptical, but this front page story has completely stunned and astonished me. The misrepresentations and errors went way beyond simply misunderstanding the science, and after digging right to the bottom of it all, knowing what I know now, I have never resorted to hyperbole before, but I can honestly say: this episode has changed the way I read newspapers.

Two failed “clarifications” later that clarify nothing, and I am even less impressed. Retract. Delete. Apologise.