The Year in Bad Science

December 27th, 2008 by Ben Goldacre in bad science, roundup | 24 Comments »

It’s only when you line these jokers up side by side that you realise what a vast and unwinnable fight we face.There was the miracle pixie dust which made a man’s fingertip grow back, although fingertips do just grow back by themselves. The Telegraph’s ever-reliable health correspondent reported that red wine prevents breast cancer – with the flimsiest of nutritionist-style evidence – just two months after writing that alcohol causes breast cancer (the latter is more correct).

We saw the Sunday Express claiming on its front page that an impressive government adviser called Dr Roger Coghill had performed scientific research, and found that the Bridgend suicide cases all lived closer to a mobile phone mast than average: this was an issue of great public health significance, but when I contacted the researcher, he wasn’t a doctor, he wasn’t really a government adviser, he couldn’t tell me what he meant by “average”, and he had, in a twist of almost incomprehensible ridiculousness, “lost” the data.

He wasn’t alone. Esure and Mischief PR refused to hand over their data on vermin and bins for inspection, although it had been reported credulously in every national newspaper. I got a leaked copy, and it was rubbish. Citigate PR refused to hand over the data on their Carbon monoxide and council flats story until I raised a fuss.

In a world where rigorous evidence from scientific research languishes unpublicised, the media continued to churn out bogus wacky science stories. Britain’s happiest places were mapped by “scientists”, although the differences were just chance findings; there were innumerable “surveys” from unrepresentative populations; and the right wing press claimed that “Lord Nelson and Captain Cook’s ship logs question climate change theories,” although they did nothing of the sort, as the researchers themselves helpfully explained. We saw how the BBC misrepresented the statistics on parents’ choices about keeping a Down’s Syndrome pregnancy, producing their a publicity avalanche on the back of an incorrect story, and learnt along the way about confounding variables, baseline changes, and more.

In the world of evidence based social policy we saw how the government quietly dropped death as an outcome indicator for their drugs policy, the fascinating inconsistencies in food additive judgment calls, and more. We also watched with delight as right-wing thinktank Reform produced a report on the crisis in maths in which they got their maths wrong.

The pointless formulae stories continued unabated. The “Fame Formula” media frenzy was triggered by the Guardian itself: it wasn’t just mathematically stupid, it demonstrably failed to model reality. People like to say that actually you need to be really clever to write for a tabloid (“actually”) because it’s the hardest job, although nobody at the Sun spotted that their Cambridge mathematician’s Britney boobline equation (“0×70x(20×5+32)/75”) gave an answer of zero, not 123.2.

It was an interesting year for the drug companies, with most of our fun revolving around selective non-publication of unflattering data. The SSRIs fared especially badly, with repeated studies showing that evidence of non-superiority over placebo was left unpublished, as was evidence of potential harm. We saw how the drug company Lilly have published strikingly similar data on duloxetine – a new-ish antidepressant drug – twice over, in two entirely separate scientific papers.

We saw how the people running the ENHANCE trial were really rather slow to publish their results, and altered their chosen endpoint after the experiment was finished. The same thing was happening with cancer trials, where researchers showed that only one in 5 cancer trials actually gets published at all (and only 5.9% of industry-sponsored trials, but in those 5.9%, golly did they do well: 75.0% gave positive results…).

Regulation has unforgivably failed to deal with these simple problems, but in a spectacular episode of collective point-missing, at the same time we saw how ethics committees have now made trials so administratively cumbersome that only multinational corporations can perform them.

Other repeat offenders continued to churn out good comedy. The Dore “miracle cure” for dyslexia invented by a paint entrepreneur called Wynford Dore was puffed throughout the media (including Radio 4’s investigative consumer slot You and Yours), until it turned out they’d gone bust, leaving some very distressed customers, at which point journalists suddenly decided they agreed with me about the dubious evidence.

The comedy factory of the Durham County Council fish oil “trial” struggles on. In March they announced – in defiance of everything they had said on the subject for several years now – that there was in fact no trial on childrens’ performance, and they had never intended to release results. In September they released the results. They had analysed their data with such unnecessarily laughable incompetence that the results can only reliably be interpreted as a false positive.

The media continued to mischievously misrepresent the evidence on MMR, ten years on, and lest we forget, vitamin pill entrepreneur Matthias Rath dropped his 15 month libel case against me and the Guardian. We saw quacks in universities and a TV nutritionist who wound up in court after her client wound up in intensive care.

Most importantly I was allowed to sneak onto the news pages of a national newspaper carrying explanations of absolute and relative risks, numbers needed to treat, publication bias, confounding variables, the counterintuitive maths on screening programmes, genius research into the placebo effect and irrationality, corrections for multiple comparisons, selection bias, cumulative meta-analyses, clinical trial methodology and more. For that I salute and adore you all.


++++++++++++++++++++++++++++++++++++++++++
If you like what I do, and you want me to do more, you can: buy my books Bad Science and Bad Pharma, give them to your friends, put them on your reading list, employ me to do a talk, or tweet this article to your friends. Thanks! ++++++++++++++++++++++++++++++++++++++++++

24 Responses



  1. used to be jdc said,

    December 27, 2008 at 4:33 pm

    That’s a fair point JQH. It does seem as if irrationality rules. I don’t think things are getting worse though – mumbo jumbo conquered the world some time ago if you believe Francis Wheen. And at least now we have books like Wheen’s – and Singh & Ernst’s, and teh Bad Science book – and the thirty or so blogs indexed at badscienceblogs.net there is some balance to the masses of bullshit produced by the media, proponents of alternative medicine, politicians, creationists, cosmetics firms, and PR companies.

  2. My Blue Friend said,

    December 27, 2008 at 4:57 pm

    Thanks for continuing to fight the good fight this year. You do God’s work, Sir.

  3. davehodg said,

    December 27, 2008 at 6:15 pm

    I’m about 10% into the most excellent Flat Earth News book. This book shows that everything Ben’s been railing against is down to journalism becoming mere reporting. No-one checks facts any more, they take press releases from AP and regurgitate it unchecked as fact and news.

    The press has been broken.

    The Guardian comes out as the best of a bad lot though!

  4. mike stanton said,

    December 27, 2008 at 8:09 pm

    JQH
    as it says on the Straight Dope site: “Fighting ignorance since 1973 (It’s taking longer than we thought)”

  5. peterd102 said,

    December 27, 2008 at 10:47 pm

    The Woos have not won, they still remain a fringe, an occasional indulgence for some. Do we forget that Ben’s Brillaint Book was a bestseller?

    Its a wonder why anyone reads the papers and belives then;
    The Sun is just juvenile,
    The Express is Diana obsessed
    The Daily mail is not long of printing a Headline: “Muslim Assylum seekers cause a cancer which lowers house prices”, repulsivly right wing.
    The Rest seem mostly drivel.
    The Guardian is probably the best written, though I wont read that either – its depressing.

    Ben seems to have resisted this, as shown by some of the mini-blog links. Hooray!

  6. cshelley said,

    December 28, 2008 at 12:57 am

    I disagree that it is a vast and unwinnable fight. Ten (or maybe even five) years ago there was no such thing as debunking bad, populist ‘science’ in the main stream media. Now there are several national columns, increasing numbers of books and blogs, and organisations such as Sense About Science. The war may not be won but I think the tide is turning. I think one of the things that seems to be creeping into the public psyche is that science is not about absolutes. As I once said to my cat, science is about acknowledging and measuring error bars, not making proclamations of fact.

  7. jackpt said,

    December 28, 2008 at 5:05 am

    I think there’s beginning to be a media awareness of an audience worth chasing. Leading to things like the revival of the token sceptic (or two) in otherwise woo satellite television programmes, and a major newspaper with a me-too version of the Bad Science column. I think there’s money in it. The internet has probably broadened the demographic of intelligent fence sitters seeking info, and given them a way to get to quality with less effort than was required prior to the internet. If that audience grows it will force the media to raise its game or face a shrinking audience among internet users. Columns like yours, and blogs dedicated to similar issues, are the internet competition to science reporting in the old school media. You do so without treating your audience as if it is composed of thick people, which, at the least, is a positive message to send out to the wider media.

  8. randywombat said,

    December 28, 2008 at 12:41 pm

    The Indy has another interesting review of the year – from the point of view of politicians’ misunderstandings of science – here: www.independent.co.uk/news/science/scientific-illiteracy-all-the-rage-among-the-glitterati-1212406.html

  9. M2 said,

    December 29, 2008 at 9:29 am

    It’s not “unwinnable.” While we may never have won, past tense, we are still winning as long as people like you keep writing. Cheers!

  10. stoat said,

    December 29, 2008 at 2:38 pm

    And here’s a last-minute entry from the Telegraph…

    www.telegraph.co.uk/comment/columnists/christopherbooker/3982101/2008-was-the-year-man-made-global-warming-was-disproved.html

  11. The Biologista said,

    December 29, 2008 at 8:58 pm

    Keep on fighting Ben. Our numbers grow daily.*

    * I don’t actually have any evidence to support that assertion.

  12. sudont said,

    December 30, 2008 at 11:49 pm

    Wonder why you’re not covering all the “second-hand smoke” nonsense? Perhaps you don’t care for tobacco yourself, but the increasingly ridiculous claims that the anti-smoker crowd are making in the name of science, tarnish all statistical studies.
    And what does it say about those scientists who remain silent, perhaps because the resulting restrictions benefit them personally, inasmuch as they no longer have to suffer a whiff of tobacco smoke?

  13. The Biologista said,

    December 31, 2008 at 12:09 am

    Studont, the health risks associated with tobacco smoke are very well established at this stage. Even if there were not a good body of evidence to support the notion that passive smoking in and of itself is harmful, what possible reason could there be for the smoke to be harmful when inhaled directly and not so when inhaled by another person 30 seconds later? The only difference is concentration.

    You might want to take a close look at where the data contradicting the scientific consensus on the matter is actually coming from. The tobacco industry is unquestionably involved in the current attempts to discredit the current thinking on second hand smoke. They’re generating the illusion of a scientific debate that’s about as real as the “debate” over creationism.

    Are there some specific articles you think are making unreasonable claims?

  14. Svetlana Pertsovich said,

    December 31, 2008 at 5:14 pm

    HAPPY NEW YEAR, Ben!
    New Year has already come to Russia (Far East)! 🙂 Then – I can congratulate all people 🙂
    Best of the best wishes in 2009!!!
    I wish you new victories in new year!

  15. Rich said,

    January 2, 2009 at 12:00 pm

    The Biologista said, “Even if there were not a good body of evidence to support the notion that passive smoking in and of itself is harmful, what possible reason could there be for the smoke to be harmful when inhaled directly and not so when inhaled by another person 30 seconds later? The only difference is concentration.”

    What happened to, “It’s the dose that makes the poison”?

    HNY!

  16. The Biologista said,

    January 4, 2009 at 10:37 pm

    Rich,

    It’s a traditional saying that is incorrect in many cases. Try lots of tiny doses of arsenic and let us know how you get on with that…

    Plenty of toxic substances are harmful in multiple low doses. Including many of the constituents of tobacco smoke.

  17. Ian Preston said,

    January 5, 2009 at 5:13 pm

    I see that the “most … day of the year” theme about which you have been so eloquent in the past has reemerged in today’s papers. The Guardian (www.guardian.co.uk/news/blog/2009/jan/05/most-stressful-day-big-brother) and Telegraph (www.telegraph.co.uk/news/uknews/4109267/Today-is-the-most-stressful-day-of-the-year.html) report that “research” from the RNLI says “the most stressful day” is today. Elsewhere, “research” suggests it is actually next week (www.thisissouthwales.co.uk/news/January-12-stressful-day-year-research-claims/article-585961-detail/article.html). But weren’t we told just over a year ago that it’s in early December (www.cbc.ca/health/story/2007/12/10/stress-day.html?ref=rss)?

  18. stephenh said,

    January 6, 2009 at 10:48 am

    Interesting post on Mind Hacks about the “Blue Monday” nonsense formula: www.mindhacks.com/blog/2009/01/blue_monday_bullshit.html

    Vaughan invites you to come up with a formula for bullshit formulas.

  19. Moi Un Mouton said,

    January 18, 2009 at 11:16 pm

    Now we can read, from a link on the main BBC News webpage today, that it’s in fact tomorrow, 19th Jan, which this year is “regarded” as the most depressing of the year, Blue Monday. news.bbc.co.uk/1/hi/uk/7836319.stm

    Perhaps the formula by which this is calculated contains a variable that makes the day always be just in the future, in order that the media can continue to publish the article every weekend for the foreseeable (or not) future.

  20. Nutritious Terrapin said,

    January 18, 2009 at 11:36 pm

    [quote]what possible reason could there be for the smoke to be harmful when inhaled directly and not so when inhaled by another person 30 seconds later?[/quote]
    What if, playing Devils advocate here, the lungs in fact filtered out (and trapped inside) all the nasty harmful particles and exhaled smoke was in fact then rendered less noxious? We already know how much tar and particles the lung collects from tobacco smoke so there could be plausibility in this as a reason.

  21. jiang said,

    December 22, 2009 at 5:21 am

    ed hardy ed hardy
    ed hardy clothing ed hardy clothing
    ed hardy shop ed hardy shop
    christian audigier christian audigier
    ed hardy cheap ed hardy cheap
    ed hardy outlet ed hardy outlet
    ed hardy sale ed hardy sale
    ed hardy store ed hardy store
    ed hardy mens ed hardy mens
    ed hardy womens ed hardy womens
    ed hardy kids ed hardy kids ed hardy kids

  22. sudont said,

    May 28, 2010 at 2:23 am

    “You might want to take a close look at where the data contradicting the scientific consensus on the matter is actually coming from.”

    How about the original EPA study? They cherry-picked the data, reduced the confidence level to 90%, and still only showed a risk of 1.19. The American Cancer Society, for instance, will not accept studies as showing any effect below 3.0.
    But what about common sense? Smokers inhale deep lungfuls of the stuff, all day, everyday, for decades before developing lung cancer. It’s absurd to think the small amounts non-smokers are exposed to would cause cancer. I didn’t need the tobacco companies to tell me this, and was suspicious the first time I heard this nonsense:
    www.straightdope.com/columns/000602.html
    As for unreasonable claims, wow, could not go into all of them here! Have a look at Dr. Michael Siegel’s blog:
    tobaccoanalysis.blogspot.com/
    He’s a second-hand smoke believer who’s appalled at his own colleague’s increasingly ridiculous lies, and covers them quite well.

  23. sudont said,

    May 28, 2010 at 2:53 am

    Hmm, is this is the “smoking tweet” that answers my original query?:

    “mm, the lady downstairs seems to smoke so many cigarettes that my flat stinks of fags. there’s no stylish way to deal with this.”
    – Tweet from Ben Goldacre

  24. xintian said,

    June 2, 2010 at 9:59 am

    Turkish women came up ed hardy bikini with a good solution.
    ed hardy
    ed hardy swimwear