A startling lack of critical self-appraisal

July 27th, 2007 by Ben Goldacre in bad science, craig sams, hate mail, stifling criticism | 52 Comments »

One for the small print maybe, but I think this is culturally quite interesting, because to me it tells a small part of the story on how you can maintain a belief system by avoiding appraisal of your ideas.

As you will remember, Craig Sams, a confectionery millionaire, recently wrote an article which I suppose you’d have to describe as an “attack” on me. This was close to what I’m hoping for – which is an attack on my ideas – and I genuinely wish he’d engaged with any single one of my criticisms of McKeith and the wider nutritionism industry. As I have said many times: there’s nothing I like better than people engaging in a discussion about ideas, and criticising mine. If you haven’t read Sams’ article already I’d highly recommend it for sheer enjoyment.

What interested me most about his piece was that this was the first time his billion dollar industry has even come close to engaging meaningfully with any of the key concerns that people have about pseudoscience in their promotional activity, rather than holding their heads in the sand and pretending that everyone who dislikes them is in the pay of the devil.

At first in emails it seemed like Sams was up for having a chat about all of this stuff, which I thought would have been interesting and constructive, for an unedited mp3 podcast, but then he suddenly changed his mind a month later and started sending me (and others) some pretty bizarre and angry emails. I will be nice and not print those in public.

So then I got on to the trade magazine which published it, Natural Products, to offer them a piece, explaining my position, defending myself against the slightly odd claims, and outlining the concerns that people have about their industry. I rather hoped this would move things on a little from the “dolphins-good-thalidomide-bad” world view that seems to dominate discourse within the trade.

The medical literature, as you will know, is full of critical self appraisal. In fact, if there is one crucial feature which distinguishes medicine from quackery, and from hysterical populist criticism of medicine, it is that long, strong, justified, and brutal tradition of critical self-appraisal.

The British Medical Journal is probably the most important medical journal in the UK. It recently announced the three most popular research papers from its archive, according to an audit which assessed their use by readers, number of times they were referenced by other academic papers, and so on. Every single one of these papers was highly critical of either a drug, a drug company, or a medical activity, as its most central theme.

The top scoring paper was a case-control study which showed that patients had a higher risk of heart attack if they were taking the drugs rofecoxib (Vioxx), diclofenac, or ibuprofen. 10.1136/bmj.330.7504.1366. At number 2 was a large meta-analysis of drug company data, which showed no evidence that SSRI antidepressants increase the risk of suicide, but did find weak evidence for an increased risk of deliberate self harm 10.1136/bmj.330.7488.385. And in third place was a systematic review which showed an association between suicide attempts and the use of SSRIs, and critically highlighted some of the inadequacies around the reporting of suicides in clinical trials 10.1136/bmj.330.7488.396.

www.bmj.com/cgi/content/full/334/7593/554-a

Such critical self appraisal is startlingly absent from the academic and commercial CAM literature, on the other hand. I think that’s a shame, and I think it’s partly responsible for them indulging some rather silly ideas, and exposing themselves to justified public mockery.

If you’re interested, this is how the Natural Products trade journal dealt with me offering them responses to their criticisms and concerns. I thought I was pretty patient and good humoured, you might disagree, but either way, one day a very bored medical historian might just use it in a passing footnote.

On Fri, Jun 8, 2007, Ben Goldacre < ben@xxx> wrote:
hi jim,

as you know you recently published a rather personal attack on me, by craig sams, which didn’t really address any of my criticisms or ideas about the role of pseudoscience in nutritionism and alternative therapies. i am always very keen to encourage a free exchange of ideas, and craig sams had set a time to meet to discuss these in an (unedited) interview, but he seems to have now decided not to engage in a discussion after all.

i think it might be useful – i’d shy away from saying appropriate – if i wrote something for your magazine, perhaps addressing some of the drearier ad hominem attacks, but also and more usefully setting out why it is that people become concerned about the rather free and easy use of inaccurate scientific information in your industry to sell their products. i’m assuming you pay contributors for their work, perhaps we could negotiate that.

do please let me know as soon as possible, i do feel that the reluctance to engage in meaningful discussion is one of the root causes of so much unnecessarily woolly thinking in what could otherwise be a very valuable and constructive industry.

ben

Ben Goldacre Thu, Jun 14, 2007 at 6:53 PM
Reply-To: ben@xxx
To: editor@naturalproductsmagazine, assistant@naturalproductsmagazine
hi there

did you get my previous emails? maybe you could reply to this one just to let me know that you did?

thanks,

ben

Jim Manson Thu, Jun 14, 2007 at 8:08 PM
To: ben@xxx
Hi Ben

Yes I did get it and apologies for not getting back to you before now.

I’d be very interested in publishing something along the lines you suggest. But since this may well throw up a bit of political crap could we have a quick chat on the phone to discuss what form your piece might take?

Is there a number I can get you on? My mobile is 07768 xxx and office number is 01273 xxx.

Jim

Ben Goldacre Thu, Jun 14, 2007 at 8:15 PM
Reply-To: ben@xxx
To: Jim Manson

heh

seriously tho, i really don’t think that printing someone who disagrees with you is likely to throw up political crap, and the fact that it might is [suppresses sanctimonious voice] the, er, problem, and the reason there is so much woolly thinking around in your world.

literally trapped in hospitals for a week now but totally on email, what crap, and what sidestepping did you have in mind?

Jim Manson Fri, Jun 15, 2007 at 7:34 AM
To: ben@xxx

Ben

Crap probably wasn’t the wisest choice of words in the circumstances (Freudian even?).

I’m not sidestepping anything in particular. But I do think the personal attacks (which have travelled in both directions by now anyway) are a distraction from the real issue – the polarisation of the debate which is preventing meaningful discussion. So all I was going to say is can you make the points you want to make without dwelling on particular ‘personalities’?

Either way, does this have to be confrontational before it gets going?

Jim

ps I’m out of the office today but picking up emails over the weekend.

Julia Brandon Fri, Jun 15, 2007 at 9:13 AM
To: ben@xxx
Hi Ben,

Please accept our apologies. We did indeed receive your first email, and on having discussed the options this end, I thought Jim (the editor) had already replied to you. Jim is out the office today, but we will be in touch with you next week to discuss options.

Thanks for getting back in touch, and prompting us into action!

Kind regards

Julia

Ben Goldacre Fri, Jun 15, 2007 at 1:57 PM
Reply-To: ben@xxx
To: Julia Brandon

hi jim,

it goes without saying that i want to address the issues, but one of the issues is ad hom attacks, and i think you’ll agree as an editor it is entirely reasonable to expect me to respond to those. they also form a part of the story about the rhetorical style of the natural products scene, and the phenomenon of people dodging legitimate criticisms. i’m not aware of writing ad hom attacks on sams (other than a few pretty moderate and relevant comments in an online blog).

it also goes without saying that much of what i have to say is going to be very challenging for many of your readers, that’s the point, and best crystallised with reference to specific cases – specifically characters such as mckeith, sams, and holford – and i would very much hope that you would be able to publish these criticisms of their ideas, and the things they have said. if you are not, then that again exemplifies the problem, of course.

i think you would be rather surprised by the number of friends i have in the health food and organic world, and moreover by the large number of emails i get from people who feel very embarrassed by the way that pseudoscience and ad hom are used to market their products.

Jim Manson Mon, Jun 18, 2007 at 7:09 AM
To: ben@xxx
Cc: Julia Brandon

Hi Ben

It doesn’t actually surprise me that you have supporters in the natural and organic sector (I even know some). But that could be that they don’t conform to the knackered stereotype sometimes bestowed on them!

Re crystallising your points with references to Patrick Holford, Gillian McKeith and Craig Sams, hasn’t this already been done via a national newspaper (sometimes with entire columns devoted to the subject) – PH (12 references), GK (23)?

I’d still prefer to publish something that concentrates on the issues and gets us away from the current personalised as well as polarised debate. But if the references to the ad hom stuff can be reasonably contained, well then let’s do something.

Re dates and rates: We’re just going to press with a combined summer issue (bit like the Beano), so don’t have space available until our September issue. Re rates, well, they not great (£160-180 ptw).

Alternatively we could always set up an interview. Or a head-to-head with someone like Dr Rob Verkerk at the Alliance for Natural Health?

Jim

Ben Goldacre Mon, Jun 18, 2007 at 4:36 PM
Reply-To: ben@xxx
To: Jim Manson
Cc: Julia Brandon

hi jim

i think you might be confusing criticising peoples ideas with criticising them as people. i certainly dont want to get into non-specific “this kind of claim kind of gets made with kind of not very much kind of evidence”. i’m offering you a critique of the unnecessary use of pseudoscience with concrete examples from the leaders in the field, and a discussion of the unhealthy climate in which science is used as a promotional tool for show, and where there is no possibility of ideas undergoing critical appraisal.

it is this environment in which bad ideas flourish. this is also of course relevant to the genuinely ad hom critique of my work made by craig sams, in which he addressed none of my criticisms, engaged in a bit of personal abuse, and accused me of doing things i havent done. that is a relevant part of the story of how science and ideas are used in the industry. he is after all one of the movements leading lights and of course your columnist.

i would remind you that the three most highly referenced studies in the BMJ last year where, in order, on the dangers of vioxx, the dangers of paroxetine, and the dangers of SSRIs. critical self appraisal is an integral part of medicine. with the greatest respect, if you find that hard to allow in the pages of your trade publication, then that is yet another symptom of the problems i am proposing to write about.

b

Ben Goldacre Mon, Jul 2, 2007 at 2:13 PM
Reply-To: ben@xxx
To: Jim Manson
Cc: Julia Brandon

hi jim/julia,

it’s been two weeks now, and i’ve not heard back from you. perhaps if you feel uncomfortable printing an interesting comment/feature on the problems with using pseudoscience in the industry you represent it might be easier for you to simply give me right of reply to the inaccuracies in your ad hominem attacks on me personally. do let me know what form that would take, and how many words, although i think it would be less interesting and informative for your readers,

many thanks,

ben

Ben Goldacre Thu, Jul 26, 2007 at 2:52 PM
Reply-To: ben@xxx
To: Jim Manson

just to let you know i’m writing about this as part of a wider project on how the alt med industries are undermined by their own resistance to critical self appraisal, let me know if you dont want your emails quoted,

b

Jim Manson Fri, Jul 27, 2007 at 2:48 PM
To: ben@xxx

Hello Ben

I’m perfectly happy to be quoted personally. But then, unless you are going to be very selective about what you quote, I don’t really see that my comments demonstrate the alt med industry’s “resistance to critical self appraisal”.

I ought however speak with my employer. How imminent is the ‘wider project’?

Jim

Ps I’m going to email you one or two further thoughts on this later

Jim Manson Fri, Jul 27, 2007 at 4:13 PM
To: xxxxx; ben@xxx
Carsten

This from Ben Goldacre yesterday. He doesn’t give any clues about what the ‘wider project’ might be. I imagine if he does quote me he’ll do so very selectively. Any thoughts?

Jim


++++++++++++++++++++++++++++++++++++++++++
If you like what I do, and you want me to do more, you can: buy my books Bad Science and Bad Pharma, give them to your friends, put them on your reading list, employ me to do a talk, or tweet this article to your friends. Thanks! ++++++++++++++++++++++++++++++++++++++++++

52 Responses



  1. twisted said,

    September 4, 2007 at 1:53 pm

    Not read come across your site before, but I felt this was worthy of comment:

    So, you’ve come on to this editor, saying you want to write a piece in response to someone else’s attack on you.

    He emails you back and says let’s talk and discuss it.

    You send him another email instead to tell him why he should publish whatever piece you decide to write.

    He comes back and continues to express interest in paying you to write a piece for him, but asks you to tell him it will not just consist of personal attacks.

    You come back to say, essentially, that it’s difficult to separate the message from the messenger, so you’d have to make some sort of personal attack in the piece.

    He comes back again and says that some of what you want to write about has already been written about, but says ‘let’s do something’ if the personal attacks element doesn’t take up too much space.

    You come back and make several points in outline of your proposal and then accuse him of not wanting to print your piece.

    He doesn’t respond, you say that’s because he’s not interested in self-appraisal and then offer to write another piece which would be a right-of-reply and then follow it by saying you’ll publish the email correspondence.

    ****

    I’m sorry, but this just makes you look like an idiot who doesn’t listen to other people or read what they say, which I’m sure isn’t the case.

    I have no truck with the alternative health industry, I’ve never read this magazine and I have no desire to.

    But to say this exchange represents an unwillingness to engage in self-appraisal is just nonsense. Maybe they don’t self-appraise, but this doesn’t demonstrate that.

    I can only hope that the annoyance you must have felt at being attacked personally must have clouded your judgement.

    Given the tone of your emails to this editor, I’m not surprised he suspected a stitch-up at the last. I would have been tempted to say “Send me 1,000 words and we’ll discuss it”, and if it had turned into a libellous piece that would have got him sued by his own columnist or several rich celebroquacks – plainly his concern – then I would have dropped it, although perhaps he was worried about kill fees.

    If you wanted a right of reply, you should have written it, not written about writing it. And to be honest, from your emails I fail to see how the second piece would be different from the first one you proposed. And (another and) if you really wanted a response, you have the man’s mobile phone number. You missed an opportunity to (a) get your message to the fluffy bunny brigade and (b) tax their industry for £160.

    Rest of the site is excellent. Keep up the good work in sticking it to the pseudoscience lot.

  2. diudiu said,

    December 21, 2009 at 5:54 am

    ed hardy ed hardy
    ed hardy clothing ed hardy clothing
    ed hardy jeans ed hardy jeans
    christian audigier christian audigier
    ed hardy t shirts ed hardy t shirts
    ed hardy uk ed hardy uk
    ed hardy bags ed hardy bags
    ed hardy hoodies ed hardy hoodies
    ed hardy mens ed hardy mens
    ed hardy womens ed hardy womens
    ed hardy kids ed hardy kids ed hardy kids