Ben Goldacre
Saturday November 26, 2005
The Guardian
The moment I saw the press release for the new Bristol Homeopathy study, I knew I was in for a treat. This was a fabulously flawed “survey”, no more, in which some doctors asked their patients whether they thought they’d got better a while after having some homeopathy. Not meaningless data in itself, but the action, as ever, is in the interpretation, and the interpretation was at its most cock-eyed in the Daily Telegraph.
“The other reason the survey is causing such delight,” gushed Elizabeth Grice, in her public love note to David Spence of the Bristol Homeopathic hospital, “is that it contradicts a scathing report published in the Lancet recently by Professor Matthias Egger.” Now scathing is not a word I would use to describe a rather sober and – by design – tedious meta-analysis. But she is also, weirdly, suggesting that a large systematic review and meta-analysis of a huge number of placebo controlled randomised trials is somehow contradicted by a survey from some homeopaths of their customers’ satisfaction.
Did she feel there were any flaws in the Bristol study, any need for balance? Yes: “This has led to a medical ding dong in the long-running debate about the value of homeopathy, with Egger (known in the profession as Eggy) accusing Spence and his colleagues of failing to use a ‘control group’ for comparison and Spence retorting that his huge observational study – the largest of its kind ever published – involving 23,000 consultations with no exclusions and no bias, is a pure measure of achievement. ‘It’s what I call a ‘real world’ analysis’, he told me. ‘It’s what happens.’ ”
Now the first lesson for sceptics here is, if you contradict the enemy, they will give you a funny name. And people call me childish. But on to business. Is it bad not to have a control group? Yes. Read the academic paper via www.badscience.net/?p=188: they were looking at a lot of chronic cyclical conditions, or time-limited ones, like the menopause, where people get better with time. If 70% get better that’s meaningless. 99.99% of people who get a graze to their knee will get better and 99% of people who get a cold will get better. It’s not enough to know that they got better. We need to know if they got more better, or better faster, than people who weren’t having homeopathy.
And what about this business of “no exclusions” and “no bias”? These are simple technical terms from evidence based medicine, and in fact there were stark staring heinous examples of both “exclusions” and “bias” in this study, Grice. Where did the patients come from? They were selected as patients who wanted homeopathy, and so were positively disposed towards it: this is “selection bias”, picking subjects who will give you a positive result .
And what about the data collected: did they measure how patients were at baseline, and compare how they were later in time, at follow up? No, they just asked patients later to remember how they were when they first came, and decide retrospectively whether they thought they were any better: this will give you “recall bias”, and also another form of “information bias”, as patients give the doctors the answer they think they want or deserve.
Lastly, a large number of patients never came back after their first appointment: and so they were simply, er, ignored in the analysis. That “exclusion” is the very opposite of a “real world analysis”, otherwise known as an “intention to treat analysis”. Did they get worse? Did they get better? Did they go home and die? We will never know.
Send you bad science to bad.science@guardian.co.uk
Learn more on how to critically appraise research with How to Read a Paper: The Basics of Evidence-Based Medicine
tom p said,
November 30, 2005 at 4:25 pm
I’ve just made a request to the department of health under the freedom of information act 2000 to find out the annual budgets and annual number of consultations at each of the 5 homeopathic hospitals in the UK.
I dunno about anyone else, but I’m rather intrigued to know how much this nonsense is costing us each year.
I’ll keep you all informed whenever I hear anything, so we can make an accurate comparisson with proper medical services
Mark said,
December 5, 2005 at 8:25 am
There is an exchange on homeopathy in the most recent issue of science and public affairs from the BA (British Associaiton for the Advancement of Science). Most of it is well worn territory but I am wondering if there something of interest in Micheal Hyland’s contribution.
www.the-ba.net/the-ba/CurrentIssues/ReportsandPublications/ScienceAndPublicAffairs/SPADec05/_Homeopathy.htm
Cheers
Phil Wills said,
December 6, 2005 at 1:26 pm
I think that’s a reasonably well argued case generally Mark, even if I don’t necessarily agree with it’s conclusions, but why do psychologists resort to the likes of:
“The other is there is some other ‘unconventional’ mechanism, such as quantum entanglement or electromagnetic radiation”
If they don’t know about Physics why make themselves look daft to those who do by inserting random phrases to pretend otherwise? Electromagnetic radiation is largely well understood and any effects would be relatively easy to detect. Quantum entanglement is a fascinatingly strange phenenomena, but it’s only important in very specific circumstances and isn’t the new magical reason for everything it seems to be touted as these days.
Rob Buckley said,
December 9, 2005 at 8:39 am
Can anyone spot the difference between Ben’s article above and this article over on Spiked? www.spiked-online.com/articles/0000000CAEB6.htm
Delster said,
December 9, 2005 at 11:53 am
I’m thinking of setting up as a BScHom myself. i have all the qualifications after all.
I know enough science to throw in the buzzwords, enough medical training to spout fancy names for conditions, easy access to tap water and small brown bottles and a very good line in Bull S***, which should also answer Andrew T’s question as to what the B stands for in BScHom
Dean Morrison said,
December 11, 2005 at 9:00 pm
This type of survey is not a new invention of homeopaths – one from 1999 in Tunbridge Wells generated surprisingly similar results. Perhaps we should use this work as a ‘baseline’ of what people are likely to say when the are treated with sugar tablets:
tinyurl.com/dnrkq
Daniel R said,
December 12, 2005 at 2:06 pm
Hey guys
I was wondering about where you mention that patients are more likely to give the response that the doctor wants, or that the patient thinks the doctor wants. This is something that makes sense, of course – but is there a literature reference to show this effect?
I’m currently doing a literature review of homepathy (it’s not faring very well, in case you couldn’t predict that), and while me saying “patients gave the answers that were wanted” is all well and good, if I can’t back it up, it just sounds like I’m being a bitch.
I’ve been trying to find something on this since Friday, and the closest sorts of things I can find are Milgram’s experiments, or maybe the Asch conformity experiments. But they’re not really saying what I want to.
I don’t know any psychology students, and I have no idea where to look, or what to look for. Any help in finding a source/reference would be greatly appreciated.
Oh, my project supervisor told me about this site, so he’ll probably read this. Hope I’m not breaking any rules >_>
Ben Goldacre said,
December 12, 2005 at 2:45 pm
I’m sure you’ll find something on this in How to Read a Paper: The Basics of Evidence-Based Medicine
by Trisha Greenhalgh. It’s a form of “information bias” I guess, as they weren’t blinded, and didn’t use standardised or more objective measurement tools like questionnaires, although I’m not sure about a term like “blinding” in a trial where there weren’t even two arms…
Matt said,
December 12, 2005 at 3:26 pm
On the topic of side effects of homeopathy, in reporting Bristol Homeopathy study, Elizabeth Grice in the Telegraph made the claim that “Homoeopathic physicians have been prescribing it [homeopathic arsenic] for more than two centuries in Europe and America… apparently without a single case of poisoning”.
Hmm… As I’ve already written to Ben, the idea that homeopathy doesn’t cause side effects isn’t true. Forget “aggravations”, these are bone fide “adverse effects”. Dipankar Chakraborti described three Indian patients in the Journal of Toxicology – Clinical Toxicology (now called Clinical Toxicology) who suffered arsenic poisoning after ingesting homeopathic arsenic [1]. Another study in the same journal nearly twenty years ago by Harry Kerr and Leon Saryan of the Wisconsin ACL Industrial Toxicology Laboratory noted that two of six homeopathic preparations tested contained “notable quantities” of arsenic [2]. Not good, since prolonged exposure to arsenic has been linked with cancer, diabetes, skin thickening, liver disease, digestive problems and nervous system disorders.
The safety of other homeopathic remedies is also questionable: David Anick at the Harvard Medical School has found that commercially prepared homeopathic remedies can contain traces of ethanol, acetate, formate, methanol and acetone [3]; Spanish, German and Mexican researchers have all reported mercury poisoning due to homeopathic remedies, including in one six-month old baby [4-8]. I could find several other cases of severe side effects: homeopathic mandrake caused anaphylactic shock in one patient [9]; homeopathic slimming droplets (LOCO X112). were found to contain thyroid extract and diethylpropione, an amphetamine-like noradrenergic anorectic agent banned in Belgium [10]; in another case (abstract and paper sadly unavailable) a patient seems to have died from acute pancreatitis due to homeopathy [11]. Another remedy caused contact dermatitis [12], and the same remedy was in a course of homeopathic medicine given to a child who developed autoimmune blistering (whether it was caused by the homeopathy or by the failure to treat properly isn’t clear) [13].
What was that about homeopathy having no side effects?
Matt
References:
1. Chakraborti D, Mukherjee SC, Saha KC, Chowdhury UK, Rahman MM, Sengupta MK: Arsenic toxicity from homeopathic treatment. J Toxicol Clin Toxicol 2003;41(7):963-7.
2. Kerr HD, Saryan LA: Arsenic content of homeopathic medicines. J Toxicol Clin Toxicol 1986;24(5):451-9.
3. Anick DJ: High sensitivity 1H-NMR spectroscopy of homeopathic remedies made in water. BMC Complement Altern Med 2004, 4:15.
4. Montoya-Cabrera MA, Rubio-Rodriguez S, Velazquez-Gonzalez E, Avila Montoya S: [Mercury poisoning caused by a homeopathic drug]. Gac Med Mex 1991, 127(3):267-70. Article in Spanish.
5. Audicana M, Bernedo N, Gonzalez I, Munoz D, Fernandez E, Gastaminza G: An unusual case of baboon syndrome due to mercury present in a homeopathic medicine. Contact Dermatitis 2001, 45(3):185.
6. Wiesmuller GA, Weishoff-Houben M, Brolsch O, Dott W, Schulze-Robbecke R: Environmental agents as cause of health disorders in children presented at an outpatient unit of environmental medicine. Int J Hyg Environ Health 2002, 205(5):329-35
7. Olujohungbe A, Fields PA, Sandford AF, Hoffbrand AV: Heavy metal intoxication from homeopathic and herbal remedies. Postgrad Med J 1994 Oct;70(828):764.
8. Spoerke DG: Toxicity of homeopathic products. Vet Hum Toxicol 1989, 31(3):259-60.
9. Helbling A, Brander KA, Pichler WJ, Muller UB: Anaphylactic shock after subcutaneous injection of mandragora D3, a homeopathic drug. J Allergy Clin Immunol. 2000 Nov;106(5):989-90.
10. Mortelmans LJ, Biesemans L, Van Rossom P: Homeopathic products, not as innocent and safe as they seem? A case report. Eur J Emerg Med. 2004 Aug;11(4):242
11. Barquero Romero J, Redondo Lopez JM, Galeano Diaz F, Perez Miranda M: [Fatal acute pancreatitis in a patient who received an homeopathic treatment]. Med Clin (Barc). 2004 Mar 6;122(8):318-9. Spanish.
12. Cardinali C, Francalanci S, Giomi B, Caproni M, Sertoli A, Fabbri P: Contact dermatitis from Rhus toxicodendron in a homeopathic remedy. J Am Acad Dermatol. 2004 Jan;50(1):150-1.
13. Kuenzli S, Grimaitre M, Krischer J, Saurat JH, Calza AM, Borradori L: Childhood bullous pemphigoid: report of a case with life-threatening course during homeopathy treatment. Pediatr Dermatol 2004, 21(2):160-3.
online graduate courses said,
February 14, 2006 at 8:04 pm
Great blog. Found your blog while searching for more information at yahoo about online graduate courses . Your blog has quite a lot of interesting thoughts. Keep up the good work.
kallo said,
October 23, 2006 at 1:09 pm
@ Phil Wills
““The other is there is some other ‘unconventional’ mechanism, such as quantum entanglement or electromagnetic radiationâ€Â
If they don’t know about Physics why make themselves look daft to those who do by inserting random phrases to pretend otherwise?”
Michael Hyland is one of my lecturers, let me assure you he was only joking about that. He’s a funny guy. Insane… but funny. His current work on spirituality and its effect on health is interesting stuff.
It’s Evidence Jim, But Not As We Know It! « A canna’ change the laws of physics said,
November 8, 2007 at 8:52 pm
[…] This debacle has been covered in detail here. […]
Blog Atopowy » Blog Archive » Koniec homeopatii? said,
November 22, 2007 at 10:26 pm
[…] to jakie brednie. JakoÅ›, zupeÅ‚nie niewytÅ‚umaczalnie, ankieta satysfakcji konsumenta z kliniki homeopatycznej jest promowana w mediach, tak jak gdyby […]
Publish and be damned « A canna’ change the laws of physics said,
August 16, 2008 at 5:00 pm
[…] is that for communities of True Believers, the approach is very different. Testimonials and customer satisfaction surveys are hailed as hard evidence. Really silly books are cited. Pseudo-journals controlled by cabals […]
wayscj said,
November 21, 2009 at 7:30 am
ed hardy ed hardy
ed hardy clothing ed hardy clothing
ed hardy shop ed hardy shop
christian audigier christian audigier
ed hardy cheap ed hardy cheap
ed hardy outlet ed hardy outlet
ed hardy sale ed hardy sale
ed hardy store ed hardy store
ed hardy mens ed hardy mens
ed hardy womens ed hardy womens
ed hardy kids ed hardy kids ed hardy kids
iphone revolution said,
December 30, 2009 at 8:30 am
iphone wireless
iphone
Apple iphone
It’s Evidence Jim, But Not As We Know It! | CAMcheck said,
June 25, 2010 at 8:56 pm
[…] This debacle has been covered in detail here. […]