This article was massively cut in the paper at the last minute, below is the last version I touched…
Ben Goldacre
Saturday October 14, 2006
The Guardian
Think back into the mists of MMR: in 2002, Professor John O’Leary’s group in Dublin reported finding measles virus in the intestine of children with autism and bowel problems. The anti-MMR movement were almost delerious with excitement, and so were the media. Andrew Wakefield, working with Kawashima et al in Japan, had already reported finding measles virus in blood cells in similar children.
What if they were mistaken? How would you know? Well.A major paper published in the leading academic journal Pediatrics this month
strongly suggests that these earlier results were in error, false positives. This study has been unanimously ignored by the media: it has been covered, by my reckoning, in one Reuters piece, and in one post on the lead researcher’s boyfriend’s blog. Nowhere else. Although the papers can find space for multiple stories about unpublished “studies” on how you should buy fish oil pills for your children. I digress.
This new MMR study, by D’Souza et al, replicates the earlier
experiments pretty closely, and in some respects more carefully, in 54 children with autism (80% also had gastrointestinal symptoms), and 34 controls. All but 6 had received the MMR vaccine.
All these studies, old and new, used PCR, the same process used in
“genetic fingerprinting”. PCR works by using enzymes to replicate
RNA, so you start with a small amount in your sample, but then it is
“amplified up”, copied over and over again, until you have enough that you can measure and work with it.
Beginning with a single molecule of genetic material, PCR can generate 100 billion similar molecules in an afternoon. Because of this, the PCR process is exquisitely sensitive to contamination – as numerous innocent people languishing in jail could tell you – so you have to be very careful and clean up as you go. One substance used to prevent this contamination is called “UNG”. As an example of their
meticulousness, the new study used used 50 times more UNG than the original O’Leary paper, because research has since shown that you need these higher concentrations, to prevent contamination.
The researchers were also careful to use the very same primer
sequences for the measles virus genes as their predecessors. Primers
are what tells the PCR process where to start and stop copying RNA,
and they are what picks out the genetic material you are hunting for.
The results were striking. Firstly, using the Kawashima primer pairs,
they simply got negative results, where Kawashima et al reported
positive results. Go figure.
The replication of the O’Leary work was more interesting. Looking only at the results of the PCR, at first it looked like the O’Leary primers did indeed produce RNA strands that matched measles virus: the primers designed to pick out measles virus found and amplified up some genetic material. However, when they looked more carefully at the size of the strands, and the “melting curves”, and then actually sequenced the genetic material, they discovered that what had been amplified was not actually from measles virus at all: they were false positives. The original O’Leary paper did not pursue these extra “double checking” steps.
The authors are are quite clear: there is good reason to suspect that
the earlier studies produced false positive results, because of suboptimal contamination control, and because the O’Leary primers can accidentally amplify bits of normal human RNA. Let’s be clear: this is absolutely not about criticising individual researchers. Techniques move on, results are sometimes not replicated, not all double-checking is practical: what is odd is that the media rabidly picked up on the original frightening data, but has completely ignored the new reassuring data. Too many fish oil pill stories, perhaps.
So how could an anti-MMR campaigner criticise this new study? In two ways. Firstly they might take up minor differences in PCR technique: although the onus would be on them to explain why these minor differences invalidate the results. Secondly, they might say: for the Kawashima blood cell replication, okay, fair enough; but O’Leary found measles virus in gut biopsies, not blood cells… this study, they might say, should have looked at gut biopsies.
This is interesting. Firstly, people currently argue that it is unethical to take gut biopsy samples simply to refute a scare (I disagree, now that people are dying from diseases preventable with MMR, but I guess we all have our opinion). And there is no reason to imagine that measles RNA should persist in gut only, and not blood. But lastly, and most crucially, remember: this new study found just the same positives identified by the O’Leary primers, but checked, and found that they were apparently false positives. This is likely to be just as true in gut as it is in blood.
This ignored negative finding is not even an isolated event. Afzal et
al performed a similar replication study, and also reported negative
results, and were also ignored. So in a week where newspapers found
room for even more “research” stories promoting fish oil pill manufacturers, this study – of major significance, published in a major academic journal, on a topic of major interest, on a topic where “more research” was “demanded” – was ignored. In a country where unpublished and unfinished “new research” on the dangers of MMR can become headline news in almost all the broadsheets; in a country where MMR is still only accepted by 83% of parents; this is contemptible.
References:
The paper (sorry, full manuscript needs cash or academic sub):
pediatrics.aappublications.org/cgi/content/abstract/118/4/1664
The Reuters piece which no-one picked up:
www.emedicine.com/news.asp?name=20061002epid005.xml&page=eMedicine%20Today
A fun looking video on PCR and MMR (not watched it all myself yet):
bartholomewcubbins.blogspot.com/2006/10/bc-on-autism-17-pcr-primer.html
Some pictures of cute furry animals to cheer you up:
doctormonkey said,
October 14, 2006 at 1:14 pm
yay, you went with MMR and it was a good one –
public demand more research, research done, public/press/media not interested in research as it says government and establishment right, vaccine safe, Wakefield et al bunch of gibbering morons or cynical profiteers (what with the research seemingly by or supported by groups with a financial gain from not using MMR vaccine)
amoebic vodka said,
October 14, 2006 at 1:19 pm
The PCR product was the wrong size? The wrong size?
Now that’s an obvious thing to miss. And it doesn’t need expensive technology to figure out either, just a set of size markers and/or a *gasp* positive control. Not sequencing the PCR product or doing some other check it’s the right sequence is pretty bad too. We can see why the referees of the orginal papers would have not noticed though – it’s a given assumption, just as you assume they didn’t spit in the samples while they were at it.
It’s easy to get a set of primers to amplify any old crap if you change the conditions to be non-specific enough. We’ve managed it with just 4 base pairs in common between the primer and the product before now (very low annealing temperature and a too long extension time in case anyone was wondering).
Dr Aust said,
October 14, 2006 at 1:54 pm
Hear hear, DrMonkey and Ameobic V.
Re. PCR, when referee-ing papers I don’t let anything through that identifies a PCR product solely on “size”…
….and wrong size…???!! Jeez.
To prove it’s what you say it is I would want to see either (i) sequencing of the PCR product; or (ii) high stringency DNA-DNA hybridization of the PCR product to the “authentic” sequence.
This has been fairly standard practise since the end of the 90s, so the 2002 paper was shoddy even then.
“Amplifying any old crap” w “permissive” PCR conditiosn is dead right, AmV. Had one friend in the late 90s who was “fishing” for related genes with PCR primers derived from “gene family consensus sequences”. He spent 2 yrs at it and amplified >40 different things with roughly the right product size. Sequencing showed NONE were related to what his primers were designed to hunt down. Amazingly he did get a PhD without topping himself.
BTW, did you see that (tertiary syphilitic) organ the Daily Mail was bleating on this wk about how “kids should be taught the scientific controversies…like MMR”?
Pop over to the forum at:
badscience.net/forum/viewtopic.php?t=837
…to have a look, but take your blood pressure-lowering medication first.
Melissa said,
October 14, 2006 at 4:13 pm
Standing ovation for this one, Ben! Glad you covered this. Contemptible, indeed, that mainstream press has paid so little heed to this life-threatening issue.
Richard said,
October 14, 2006 at 4:52 pm
Excellent article, but I do wish the newspaper would stop cutting your articles down to size. I thought it finished rather abruptly when I read it in the Grauniad this morning.
cenderis said,
October 14, 2006 at 8:13 pm
Yeah, chopped. As it is on the Grauniad website, it reads very much like a Private Eye “sadly, there wasn’t space to mention…”. Odd how they happened to chop the criticism of newspapers rather than anything else in the piece.
pseudomonas said,
October 15, 2006 at 2:23 am
I can see why they have to trim the print version, but why on earth not put the full spiel on their website?
stever said,
October 15, 2006 at 2:33 am
I was suppose to get a quote in the drugs article on the smae page, but clearly squeezed by the same gigantic ad that took up two thirds of the page. Bit cross since I helped Sample put the article together, but more importantly it would have been good forum double, and excuse for some camden ales.
stever said,
October 15, 2006 at 2:36 am
Blimey just read the original version above – and they’ve really edited the hell out of that. I did actually think the print one was not your best – but now see why. Id make a bit of fuss and defo insist that the full version goes online. That’s pretty outrageous – I hope not because of your implied criticism of the guardian.
martin10 said,
October 15, 2006 at 10:22 am
Ona realted matter, I have been surprised at the lack of media coverage of the Israeli study Archives of General Psychiatry which identified a strong correlation between father’s age and autism – it’s almost as if it’s too boring because there isn’t much anyonecan do about it. If the anti-MMR campaigners were really concerned about causation of autism, they should have been greeting this news wildly, but I haven’t heard that, and perhaps as a result the media have simply ignoredit (I heard one brief mention on BBC radio and have subsequently located a Washington Post article, but apart from that, silence).
It may be that the study was inconclusive or flawed but it was at least thoughtprovoking.
billgibson said,
October 15, 2006 at 11:06 am
I suppose the problem is that “widely-used vaccine is safe” isn’t the kind of story that sells tabloids (or berliners). Similarly, it’s a continual pisser for me (and I expect the rest of the badscience MBChB crew) that despite the fact that the NHS treats literally thousands of people very very well and very very cheaply avery day, you’re never going to get the Daily Mail screaming “Old lady feels better and goes home” on pages 1 through to 9.
Bastards.
stever said,
October 15, 2006 at 12:33 pm
or
‘person has enjoyable drug experience and fails to go mad or die’
apothecary said,
October 15, 2006 at 2:04 pm
Re 10 – But surely there are only three medical stories – new drug brings hope, old drug is risky, and the novelty (and often titilating) story (eg the woman who said she had an orgasm every time she sneezed, since taking a certain SSRI). Just as there are only three general science stories – The March of Progress story (electricity too cheap to meter), the We’re All Doomed story, (rising sea levels, nanotechnology will create a universal grey mush) and the novelty story (the mathematical formula for a funny joke). Anything which doesn’t fit stock story formula 1, 2 or 3 gets spiked. Of course newspapers have to find interesting stories, otherwise no-one would buy them. But even to a non-journo like me there are enough angles in the fish oil and MMR stuff to run some great copy.
pv said,
October 15, 2006 at 9:30 pm
“Odd how they happened to chop the criticism of newspapers rather than anything else in the piece”
This, I believe, is what’s known as a code of ethics. Otherwise known as honour among thieves.
Dr Aust said,
October 15, 2006 at 9:40 pm
Newspapers seem very reticent of printing criticism of newspapers, and are violently averse to printing criticism of themselves.
An example from the herceptin arena.
Some while back, after weeks and weeks of ludicrous fact and science-free “they’ve sentenced me to death” coverage, Sarah Boseley finally wrote something sensible in the Guardian about the selling of herceptin.
www.guardian.co.uk/science/story/0,,1742168,00.html
I wrote in and pointed out that, although it was a good article, everything in it – notably real numbers and sensible discussion of costs and risk – had been common currency in the specialist medical press months before, and in the meantime the Guardian (along with every other paper) had been printing endless fact-free etc etc.
They printed the letter, heavily edited. It came out reading as congratulating Sarah Boseley AND the Guardian for doing such a bang-up job.
Another case study for my course on “Scientific Journalism”, methinks.
Ben Goldacre said,
October 15, 2006 at 9:38 pm
nah, gutted though i am, i think they chop the end off because there’s a hidden wdanger of changing the sense if they tinker with the middle. it’s an act of caution and kindness.
Filias Cupio said,
October 15, 2006 at 11:15 pm
Apothecary (13):
The “better” papers (e.g. New York Times) have another two science stories: popular scientific theory resoundingly proved, and popular scientific theory resoundingly disproved. Back when I had regular access to the NYT (over 10 years ago now) it would be about a month between evolution/big bang proved/disproved stories. I think they were proved and disproved with about equal frequency, but I didn’t take statistics.
Teek said,
October 16, 2006 at 9:53 am
nice work, good to see MMR back in the fold – altho a juicy follow-up on the fish oil story would be appreciated!
not surprising that the media that so ‘rabidly’ applauded the ‘studies’ showing measles in autistic kids’ guts now ignores better research that pretty well proves them wrong. for ages now newspapers have printed stories based on presumptions, innuendo, hear-say and anecdotes (not just talking about science/medicine here, same applies to politics/current affairs etc) but present them as HARD FACT, under headlines that scream we KNOW that XYZ in 100% the case, no doubt, you’re mad if you disagree. to do a volte-face and to admit that ‘ooops, we might have got the wrong end of the stick there chaps, sorry’ is not a strong point amongst editors. whils they may print small corrections regularly (mis-spelling, occasionally mis-quoting) it would be bloody embarrasing for e.g. Daily Wail to come out after years of saying ‘MMR causes autism, it says it right here in the Wakefield/Kawshima/O’Leary papers’ and say ‘acutally it’s the fathers’ age or something we dont know of, because those other papers didnt do their research properely.
on a slightly different note, should we at least credit reuters for reporting this…?
crana said,
October 16, 2006 at 10:17 am
I’m being picky, but I don’t think the statement “PCR works by using enzymes to replicate RNA” is right.
Using current technolohu, you have to reverse-transcribe the RNA into DNA and then perform PCR on the DNA, so you are replicating DNA not RNA. The whole process is called reverse transcriptase polymerase chain reaction, or RT-PCR.
Kells said,
October 16, 2006 at 11:31 am
Yes you are being picky Crana!
Ben the Article was badly chopped and omitted one of, if not the main point of the story. Very poor indeed. Complain and campaign to have your own page!
You were also way too kind to those researchers. A simple probe or some sequencing would have confirmed their ‘positives’. No-one should publish without that kind of back up especially when the results are controversial and relate to such an important vaccine programme. I suspect there are lab books lying around somewhere showing how they tried and failed to confirm these results.
Why has no-one else reported this before?
Tony Jackson said,
October 16, 2006 at 11:46 am
Crana (no.18) is right, but we forgive you!
I agree with Kelly (no.19). It’s astonishing that even the basic controls were apparently not done. The referees must have been asleep.
superburger said,
October 16, 2006 at 1:09 pm
Did anyone have the misfortune to read the Mail on Sunday on 15th October? There was a three page spread by Andrew Wakefield’s missus, explaining how they’d be shunned and vilified by the ‘establishment’ etc etc.
It really was piss-poor one sided journalism.
Dr Aust said,
October 16, 2006 at 1:41 pm
“It really was piss-poor one sided journalism”
Business as usual for The Wail, then.
All the doctors I know think Andrew Wakefield was / is a disgrace to the profession because his personal ambition, vanity and lack of judgement led him to initiate (on no evidence), and then sustain, a media shit-storm which has set vaccination in the UK, and arguably public health generally, back a decade.
Are all my friends (GPs, hospital doctors, community paediatrician etc etc) part of “The Establishment”?
Or is it just that they possess a certain amount of insight (professional and personal) and humility lacking in Dr Wakefield, then and now?
Delster said,
October 16, 2006 at 2:57 pm
I think i have an idea for a juicy article which a dedicated science orientated reporter could do… i can see the headline now
“hundreds killed and disabled by poor science reporting” although possibly in bigger, bolder and more interesting font.
After all death / scandle / children in danger (delete as required) sells so why not do a piece, using these latest research pieces, on exactly how many kids have died needlessly or suffer serious health problems due to this scare preventing certain parents from giving MMR.
Use the same scare tactics but with the science right…. it might even work.
ceec said,
October 16, 2006 at 4:47 pm
Trouble is, it’s hard to get good estimates of that kind of thing without stooping to er… bad science.
doctormonkey said,
October 16, 2006 at 6:57 pm
as a junior member of the MBChB/MBBS crew, i haven’t often been asked about MMR etc but my uncle who fairly recently had a child was astonished when i told him i wouldn’t forgive him if he did not give his child MMR – he assumed that due to the media hype there must be something behind it, i happily reassured him i had not seen anything.
on another note, is it worth doing a retrospective to pull apart the 1998 Wakefield et al from the Lancet? it was such amazing rubbish i could not believe it was published in any journal
Dr Aust said,
October 16, 2006 at 8:55 pm
The secret to the 1998 Lancet paper is (media) “splash value”, doctormonkey.
As in “Wow – this will be BIG. It’ll be on the NEWS!”
I would bet no-one in the process was immune – not the Royal Free bigwigs who suppressed any doubts about Dr W, not te Lancet manuscript screeners, and not the Lancet editor who tied himself in such knots later trying to explain why he had printed it .when it was so iffy.
Years back I went for an all-day interview for a job as a manuscript screener / evaluator (“Assistant editor”) at a very famous science journal. It was quite clear that “how big a splash this will make” was a significant consideration in the decisions about what would actually get passed through for referee-ing.
Which is fair enough – they can only referee one in ten submissions, say, so there must be a mix of criteria to decide which. But there are obvious dangers, to put it mildly.
Ben Goldacre said,
October 16, 2006 at 10:23 pm
The O’Leary data was also called into question at the time of the 2004 MMR court case. If I had an eight million word long column obviously I’d mention this kind of thing in the actual column.
FRESH DOUBTS CAST ON MMR STUDY DATA
The Sunday Times (London) April 25 2004
By Brian Deer
SERIOUS concerns have been raised about a key study cited by Dr Andrew Wakefield and campaigners as crucial evidence that the MMR vaccine causes autism in children.
The High Court in London has requested that the Irish courts order Professor John O’Leary, the Dublin pathologist who carried out the study, to hand over all the raw data so it can be re-examined by experts.
It follows claims made in the High Court of anomalies in O’Leary’s laboratory reports on samples from hundreds of autistic children who are allegedly victims of the MMR vaccine. Along with earlier research by Wakefield, which was discredited in February after a Sunday Times investigation, O’Leary’s tests have been seen as critical to the claim linking the vaccine with autism.
Unigenetics, O’Leary’s private company, found that 80% of the 91 autistic children it tested had traces of measles in their bodies, presumably as a result of being given MMR. This data is being relied on by parents who believe MMR caused autism in their children and are suing three manufacturers of the vaccine  GlaxoSmithKline, Merck and Aventis Pasteur.
Experts for the three companies, who have had access to some of O’Leary’s raw data, have claimed in court that the samples could have been contaminated and were incorrectly reported. If this is so, it would be a big blow to the families’ case and to Wakefield’s campaign for single jabs to replace the MMR triple inoculation.
O’Leary’s data provided Wakefield with the linchpin evidence that was central to his theory,†said Dr Stanley Plotkin, of the Wistar Institute in Philadelphia, who is a world expert on vaccines. “If that falls away, all you have is a hypothesis unsubstantiated by anything . . . the whole thing falls into the water.â€Â
Wakefield first sparked controversy over a possible link between MMR and autism six years ago with the publication of a study in The Lancet. The paper’s key finding was withdrawn last month following a Sunday Times investigation that revealed Wakefield’s involvement in the vaccine lawsuit.
The O’Leary study, co- authored by Wakefield, was published in the journal Molecular Pathology in April 2002. They claimed to have found fragments of the measles virus in samples from the intestines of children with autism. This research and further tests by O’Leary were described by the judge presiding over the MMR litigation as “pivotal†to the case.
However, questions were raised over the research when two other laboratories, testing samples from the same children, apparently failed to confirm O’Leary’s findings. One, at Edinburgh University, had been commissioned by the drug companies, and the other, at Queen Mary medical school, London, by the children’s lawyers.
O’Leary’s results were generated from a machine called TaqMan PCR, which amplifies molecular DNA sequences.Since 2000, Unigenetics has been paid £800,000 for this work by the UK legal aid fund.
Further experts were asked to investigate O’Leary’s methodology and results. Led by Dr Stephen Bustin, of Queen Mary, they visited O’Leary’s laboratory last year and were given 20% of the raw data recorded by the PCR machine.
But after comparing this data with O’Leary’s reports, Bustin’s team claims it found instances of different results. “These findings are raising very serious concerns,†Charles Gibson QC, counsel for one of the MMR manufacturers, told a court hearing this month.
Bustin claims the PCR machine used by O’Leary found traces of measles in control samples  such as those containing distilled water  that should have been negative. He also claims some of the controls were wrongly reported as negative when the machine had found positive traces of the virus.
“Bustin is beginning to find a mismatch between the raw data which he is examining and the experimental reports that relate to that,†claimed Gibson.
Measles is among the most infectious agents known. In tests such as O’Leary’s, some materials are intentionally infected  acting as positive controls to be sure the machine is working  and it is thought that this may have been a cause of the contamination.
In a statement issued on Friday, O’Leary strongly disputed the claim that contamination might have occurred and said that positive controls used in the research were “prepared in a separate buildingâ€Â. He said that had contamination occurred “all the samples would have tested positive, and that clearly was not the caseâ€Â.
O’Leary claimed that any differences between the reports and raw data were because of “the use of different software†by those checking his data.
The lawyers in the MMR litigation had agreed informally last July that O’Leary should allow Bustin to collect the remaining 80% of data from the machine. Bustin says the exercise is as simple as copying a home computer disk.
But Bustin, a specialist in PCR technology, and a fellow expert claim they have met a number of obstacles and Unigenetics put forward numerous reasons why the data could not be produced. O’Leary insists he has gone to “considerable lengths†to help Bustin retrieve the data.
Yesterday Dr Richard Smith, editor of the British Medical Journal, which publishes Molecular Pathology, said: “If it turned out to be true that the results were contaminated, then I imagine that O’Leary himself would want us to retract that paper, because clearly it would be delivering a false message.â€Â
Atactic said,
October 16, 2006 at 9:29 pm
I decided to check out where John O’Leary’s group is based, and came across this which is pretty interesting:
http:// whale.to/a/a6.html
While the group’s work certainly seems to be seriously flawed, I think this shows how the media and various action groups really jumped on this and turned it into the media frenzy it became.
Love the website.
DrB said,
October 17, 2006 at 10:50 am
Surely what’s needed here is for the secret 4th science story – unscrupulous scienctist dupes innocent public and journalists – to explode across a certain tabloid…
doctormonkey said,
October 17, 2006 at 12:26 pm
hmmm so both BMJ and Lancet have published bad studies on this stuff, it sounds as though the BMJ one is more forgivable as they may have been mislead by the data rather than wild claims “based” upon it.
now all we need are JAMA and NEJM to publish and we get the full set…
and i agree with #30 – we need a Daily Wail “splash” that it was all a hoax by unscrupulous profiteering scientists, like the MRSA-man they used… because that will happen around the time they spontaneously combust
Kells said,
October 17, 2006 at 5:27 pm
Unigenetics has been paid £800,000 for this work by the UK legal aid fund.
This is the crux of the problem – this work should have cost £15k to £25k max. Nice little earner, who would want to really test those results?
Gleamhound said,
October 18, 2006 at 9:03 am
I wonder if the Wakefield article made a significant difference to the Lancet’s citation index?
DrGlenn said,
October 19, 2006 at 5:56 pm
…well i only hope that in going all out to maintain the credibility of the science behind some vaccinations that the public is also made more aware of the potential risks of some already well established vaccines which most medics seem quite happy to pretend don’t exist and so people having the vaccination are never told about – especially for those people who have an auto-immune condition. The problem is that side effects from some vaccinations have been for a long time ignored and played down. You can’t blame a public or media for ignoring the ‘hard’ science when the ‘hard’ science is often first presented in an unbalanced risk free way.
jiangjiang said,
December 8, 2009 at 3:37 am
ed hardy ed hardy
ed hardy clothing ed hardy clothing
ed hardy shop ed hardy shop
christian audigier christian audigier
ed hardy cheap ed hardy cheap
ed hardy outlet ed hardy outlet
ed hardy sale ed hardy sale
ed hardy store ed hardy store
ed hardy mens ed hardy mens
ed hardy womens ed hardy womens
ed hardy kids ed hardy kids ed hardy kids