Ben Goldacre, The Guardian, Saturday 27 February, 2010
After the Science and Technology committee report this week, and the jaw dropping stupidity of “we bring you both sides” in the media coverage afterwards, you are bored of homeopathy. So am I, but it gives a very simple window into the wider disasters in all of medicine.
Homeopathy, our first example, is a small sector of the pharmaceutical industry, a few sugar pill companies worth a couple of billion pounds a year in europe. Overall, trials show their pills perform no better than placebo. We therefore know that all claims to the contrary are bullshit, but bullshit being tolerated by plenty of MPs, huge swathes of the media, a fair few GPs, and most worryingly of all, the Medicines and Healthcare Regulatory Authority, who permit homeopathy pill companies to list diseases they say they can treat on the side of the bottle, with no requirement that they come up with any evidence that their treatment works.
This doesn’t only tell you a story about homeopathic sugar pills: this tells you how doctors, politicians, the media, and regulators deal with the issue of evidence in medicine.
How closely do the great and the good, for example, scrutinise the promotional material for medical drugs? The latest paper looking at this question is published this month. Researchers in Holland went through all the biggest medical journals in the world – the Journal of the American Medical Association, the Lancet, the New England Journal of Medicine, and so on – between 2003 and 2005. All adverts were included, once each, if they made a claim about the effect of a drug. For all the claims in the ads, they checked the references, found the trials they referred to, and then gave them out to an easily exploited workforce of assessors: 250 medical students who’d just finished their evidence based medicine teaching.
Each student independently assessed 2 trials, and its associated advert, using a pre-specified questionnaire and a well-established scoring system to assess the quality of trials. Scores were given for things like: whether the method of randomly assigning patients to one treatment or another was adequate, and clearly described; whether patients could know which treatment they were getting; whether drop-outs were appropriately included in the analysis, and so on. These are good measures of whether a trial is a “fair test” of a treatment.
By now you will rightly be worrying that medical students – although cheap and easy to come by – are not reliable raters, so you will be pleased to hear that each trial was scored by between 2 and 6 students, and if there was a discrepancy in scores, that trial was reviewed by a panel of 4 academics.
The results were abysmal. Only half of the claims in the adverts were actually supported by the specific trials they referenced, and of all the trials, only 55% got a score of “high quality”: overall, only 39.2% of these adverts – in the leading medical journals in the world – referenced a high-quality trial which actually supported their claim.
This is not the first time such a study has been conducted. Villanueva and colleagues, in 2003, published a paper in the Lancet assessing claims for cardiac medication adverts in six Spanish medical journals: of the 102 references they could trace, 44% did not support the promotional statement. Similar results have been found in psychiatric drug adverts, and in the field of rheumatology. To offset any suggestion that I am cherry-picking, a systematic review of this question in the open access journal PLoS ONE found 24 similar studies, and overall only 67% of the claims in adverts were supported by a systematic review, a meta-analysis or a randomised control trial.
Quacks see the shortcomings in medicine as a justification for their own even more dubious behaviour, but in reality, the horror is this: homeopathy is the obvious, easy, tip of the iceberg. It is the simplest and clearest story of how bad things are, how dumb doctors and politicians can be about evidence, and how lame regulation has become. But it is only the most obvious illustration of the dark, fearsome depths into which these problems extend. We are in very big trouble.
DrJG said,
March 5, 2010 at 8:03 am
I know that we have drifted slightly off topic here, but given the discussions on the state of the NHS, I thought that this from “M.D.” in this fortnight’s Private Eye is worthy of wider readership:
“The take-home message at Bristol [child cardiac surgery unit] came from Steve Bolsin, the anaesthetist who first raised concerns and paid for them with his NHS career: ‘If you want to avoid another Bristol, never lose sight of the patient.’ This is strikingly similar to the conclusion of Robert Francis QC in the Mid Staffs report: ‘If there is one lesson to be learnt, I suggest is is that people must always come before numbers. It is the individual experiences that lie behind statistics and benchmarks and action plans that really matter, and that is what must never be forgotten when policies are being made and implemented.’…..That “new” labour could contrive a culture where £105 billion goes into the NHS at it’s still no safer than bungee jumping, with widespread staff shortages, overcrowded emergency departments, disillusionment and patient harm, suggests that much of our increased funding hasn’t gone where it is needed. Oh, and we’re desperately short of inspirational managers who’ll stick two fingers up at Whitehall and focus on helping their staff to serve their patients.”
Enders Shadow said,
March 5, 2010 at 8:52 am
I guess my aim here is to encourage us to be pragmatic: to recognise that for some people homeopathy makes a massive positive difference to their otherwise blighted lives for reasons that we don’t understand, and the studying of ‘Why’ is a MAJOR project whose results we must resist the temptation to believe we know the answer to.
Given that for some people it does work where other treatments have not been effective, it IS rational to divert a very small amount of NHS cash to it. However I would agree that “homeopaths [should not] be allowed to get away with lying about the effectiveness of conventional medicine”, though given how often other groups within the mainstream medical profession do it, there’s no shortage of target for that campaign, and every time that a doctor prescribes an antibiotic to get rid of a whinging patient, they are guilty of the same behaviour 🙂
skyesteve said,
March 5, 2010 at 9:41 am
@enders shadow (103) – your conclusion only stands up if you believe the NHS should pay for any quakery that makes a “massive positive difference” to some people – as I’ve listed before, massage, aromatherapy, hot stones, shamanism,etc.
People can spend their money on what they want and if it helps them then that’s good too. But the NHS shold not resources things which cannot be backed up with an evidential base and I would apply that to a lot of things currently funded by the NHS, not just homoeopathy.
@jwm and DrJG – one anecdote from the US – I worked there for a very short spell and was curious to know why even the most minor of head injuries presenting to the ER got a CT scan. So I asked assuming it was to do with fear of litigation – but it wasn’t. It turned out that the senior ER doctors had shares in the CT scanner and received a “dividend” each time it was used! The joys of a private, for-profit healthcare system
Sriram said,
March 5, 2010 at 9:42 am
“to recognise that for some people homeopathy makes a massive positive difference to their otherwise blighted lives for reasons that we don’t understand, and the studying of ‘Why’ is a MAJOR project whose results we must resist the temptation to believe we know the answer to.”
Enders Shadow….I agree with that! A little bit of balance in such matters wouldn’t come amiss.
Cheers.
Sriram
elvisionary said,
March 5, 2010 at 10:21 am
Enders, although I think you’re wrong, you do highlight something very interesting: the placebo effect does bring real benefits to patients, but it will be most effective if the patient believes that it is more than just a placebo. This presents an ethical dilemma: is the prescription of sugar pills labelled as something else a kind of white lie where the potential ends (an improvement in the patient) justify the means (deception of the patient about what they’re really taking)?
Rather than getting into an ethical debate about whether ends can ever justify means (philosophers have wrestled with this for centuries), I would answer this question in a different way by picking up your comment about doctors prescribing unnecessary antibiotics. Would it be better for all GPs to have sugar pills in their toolkit? When prescribing them, they could simply say “We’re not entirely sure why this works for this kind of condition, but the evidence is that some patients get better after taking them”. That statement would be truthful. It would enable doctors to harness at least some of the power of the placebo effect. And if these pills could be manufactured and supplied very cheaply (no need to dilute and bash against the wall multiple times, no need for dubious marketing or branding, no need for expensive trials), then the patient would not be being exploited for financial gain.
So it would be a cheaper and more honest alternative to homeopathy, and a cheaper and more honest alternative to the unnecessary prescription of proper drugs. But to me the critical point is that it would be a medical professional making the prescription on the basis of what was in the best interests of the patient. So if the patient really would benefit from proper drugs, they would get them. And this is where homeopathy is so indefensible: the fact that homeopaths are selling/prescribing it leads to highly undesirable results. Patients who should get proper drugs don’t get them, and instead pay for sugar pills. Patients who might benefit from sugar pills end up getting expensive proper drugs when they don’t need them, which can ultimately undermine the effectiveness of those drugs.
This is why I don’t think it is true to say that homeopathy is “no better than placebo” – it is far, far worse.
Guy said,
March 5, 2010 at 10:31 am
Enders, you state “for reasons that we don’t understand,”. Yes we do understand them. There has been hundred of millions of dollars spent by congress on investigating complementary medicine, so a lot is known about the placebo effect. The evidence is there, plain to see for anyone who can read and critique a scientific paper. You talk as though there is absence of evidence, which could mean there was an effect. It is absence of effect beyond the placebo that is demonstrated everytime a properly conducted trial is reported.
This brings us back to the beginning to the MP’s report showing that it is placebo pure and simple. So no it doesn’t work. It is just placebo. We know that for a fact. No futher investigation is needed or warranted.
Guy said,
March 5, 2010 at 10:39 am
Defenders of homeopathy might like to read Simon Singh’s work and comment
www.1023.org.uk/whats-the-harm-in-homeopathy.php
elvisionary said,
March 5, 2010 at 2:14 pm
…or maybe it’s technically more accurate to say that homeopathy is a misleading, exploitative and damaging application of the placebo effect.
emen said,
March 5, 2010 at 2:19 pm
jwm,
I’m not German and I don’t live in Germany.
Yes, a bigger percentage of GDP spent on healthcare is one reason. The other one is that the German system is a mixed-funded system: everybody in employment contributes to their healthcare costs: about 7% percent of their income is chopped off, and about the same amount of money is paid by the employer. (It is, of course, more complicated than that, because the Germans make anybody above a certain income or working in certain occupations take out private health insurance etc.)
Of course the German system is not perfect, they too struggle with the increasing costs of healthcare, and also, the element of “let’s just do this extra test and claim the fee for it from the state” is definitely there.
The reason why I mention it is because if you look at the history of healthcare in Britain, pre-NHS the system was more or less trying to be what the German is now. Then, of course, the war broke out, war-time healthcare, poverty, rationing, all sorts of reasons, and the NHS was created. The NHS has always been a very controversial system, partly because financially it never worked, that was very clear after the first one or two years. Since then it has always been about how to make it work without changing the basic idea of it being completely taxation funded and free for all at the point of delivery.
I’m happy that you and your family have had exceptionally good experiences on the NHS. My anecdotes are, unfortunately, not so shiny. I spent a year struggling on the NHS, and it completely destroyed me, both physically and emotionally (but not DIRECTLY financially). It was different hospitals, different doctors, different problems , and I can’t pick out one place or one person, and comfort myself that apart from them, everything must be, surely, fine. To me it seems to be a sick, almost dead system, unbearably slow, unresponsive and superficial. I wish I could have paid some money towards it (or top it up) and got quicker, better care and be back at work quickly to earn that money back.
I think here in Britain we pay a very big price for not wanting to pay for healthcare. Either by contributing directly like the Germans, or contributing by taking out private insurance, or or at least have the choice to top up the NHS costs when we think we need to. I’m not saying we should copy the German system, something that works over there might not work here at all for a variety of reasons. But the longer we wait and the longer we leave it, the more difficult it is going to be to change it.
There is not enough money in the NHS and what there is, seems to be very badly managed. At the same time, patients are not in the customer position, more like beggars at the mercy of whoever seems to be there to care for them.
In my experience, that is.
emen said,
March 5, 2010 at 2:41 pm
Dr JG
was that M*** in Solihull by any chance? The one that recommends routine US after coil-fitting?
OK, let’s not train GPs to check everything by scans and tests then (it wasn’t my idea, mind you, but some cancer expert worrying about the low cancer survival rates in the UK).
So they can treat depression instead (they are very good at that, aren’t they).
Let’s have more sonographers then! 🙂
I don’t mind, as long as the waiting is days rather than months, when it comes to suspected cancer.
Regarding the tendency to early discharge from hospitals after operations. In my experience part of the problem is that when they discharge you, you find yourself in the hands of your GP immediately – that is, once you are out of hospital, the communication between you and the people who know what they have done to you and what is normal, what isn’t is completely halted. I think it is unrealistic to expect GPs to be fully prepared to answer every question that you should be asking the surgeon really. So you find yourself in a vacuum of care (like so often), when you aren’t allowed to speak to the person who could help, and the doctor who is willing to see you can’t help.
Ho-hum.
skyesteve said,
March 5, 2010 at 2:48 pm
@emen – GPs are not perfect and, like any other sector, there are good ones and bad ones. But I believe that the vast majority are good. I can only speak for myself, but I do the best I can every day for every patient. I don’t regard anyone as a “heart sink” or “time waster” – I see it as part of my job to inform all my patients of the best and most appropriate way to use the health service. I am happy to answer any questions and sufficiebtly humble to say when I don’t have the answers. But one of the features of a good GP is to know where or how to find the answers. I often sit with my patients during a consultaion and do an internet search for difficult questions then I print off material if that is helpful for them. You might say they could do that myself but I guess my years of training and experience allow me to sort out the wheat form the chaff that they mighht struggle to do.
I will agree, however, that communication between secondary care and primary care is not as good as it should be.
skyesteve said,
March 5, 2010 at 3:10 pm
@emen (again – sorry!) – I have genuine sympathy for anyone who feels that the NHS has let them down but I still think, in Scotland at least, such people represent a minority of users. As I say often, I can’t comment on the NHS in England, Wales or Northern Ireland as I don’t live or work there and NHS Scotland is to all intents and purposes a separate organisation.
Primary care in Scotland has patient satisfaction rates in the high 90s percent and those figures are consistent year on year. The model of primary care we have in scotland remains the envy of most of the world. The cost of looking after one average patient in primary care for a whole year is half what it costs for the same average patient to spend on night in hospital.
The NHS was not established out of nowhere in 1948. It was modelled to a greater or lesser extent on the Highlands and Islands Medical Scheme which started just before the First World War. For the first time this produced a uniformity of primary care provision across the impoverished Highlands and Islands of Scotland. It gave every community, no matter how remote, access to doctors and nurses. It built houses for the doctors and nurses and provided them with transport (including, in one instance I know of, a horse!).
I am very proud to work in NHS Scotland and I think it does a wonderful job. Between 2002 and 2005 I was heavily involved as a user of the service and it was fantastic.
Of course, if something bad happens to you then the rate of that is 100% but anecdote (and even mutliple ancedotes) doesn’t stack up to a large body of evidence (any more than the tens of thousands who claim homoeopathy worked for them or their dog).
But even now if I have someone with something I think my be a pelvic maligancy I can get a scan done locally the next working day just by picking up the phone and having a chat with the sonographer.
In any case throughout Scotland there is a national programme that all suspected cancer will be seen in secondary care withing a maximum of 2 weeks (of course, as a GP, you have to suspect cancer in the first place for that to kick in) and in general we have a target of 18 weeks from referral to treatment – 18WRTT – and that’s the start of TREATMENT not just you basic initial out-patient assessment.
The fundamental principle behind any civilised healthcare system is that access must be based on need not ability to pay. The only way to truly achieve that, IMHO, is through taxation and a service free at the point of access. As soon as you introduce an element of co-payment that will favour those who can afford to pay.
But I agree we live in times where everyone thinks they are entitled to something for nothing. they want free healthcare, free education, free social care, free roads, etc. but at the same time they want to have sufficient disposable cash (i.e. low taxes) for the faster car, the bigger house, the nicer holiday, etc. and that for me is the situation which is not sustainable.
pv said,
March 5, 2010 at 4:45 pm
Enders Shadow said,
March 5, 2010 at 8:52 am
Given that for some people it does work where other treatments have not been effective…
The point is that it isn’t “given” at all. Ascribing the effect of real medicine to homeopathic magic water/pills isn’t what I would call “working where other treatments have not been effective”.
Nor is ascribing to a homeopathic effect a normal self-limiting of such types of condition something that can honestly be described as “working where other treatments have not been effective”.
Claims of efficacy don’t constitute evidence of efficacy.
jwm said,
March 5, 2010 at 7:37 pm
emen,
some very interesting points, and it sounds like youve lived a very interesting (but perhaps unlucky?) life having managed to experience several healthcare systems.
re: “everybody in employment contributes to their healthcare costs: about 7% percent of their income is chopped off, and about the same amount of money is paid by the employer”
Im not an accountant but this sounds like tax by another name, with the only difference being that people are aware that a fixed percentage of their pay is being ringfenced for healthcare. The contribution by the employer is tax-deductable, and therefore essentially just tax as far as they’re concerned. Although it may be that increased awareness of healthcare costs has a benficial effect on population usage of the health system with the knowledge that 7% can go up.
“I don’t mind, as long as the waiting is days rather than months, when it comes to suspected cancer.”
This is thankfully something which is changing, with most hospitals running a 2 week wait system for possible cancer cases, and an 18 week system from referal to final treatment, although over 2/3s of cases currently reach this stage within only 12 weeks, and there is moves to reduce this to 9 weeks in cancer.
Martin said,
March 8, 2010 at 1:14 am
Enders Shadow and Sriram:
“to recognise that for some people homeopathy makes a massive positive difference to their otherwise blighted lives”
No, homeopathy doesn’t do anything. Sitting down with someone who has a sympathetic ear for an hour (rather than the rushed 5 minutes you get with a GP) talking about your problems and the homeopath emphasising and going “Oh, yes” and “I know” makes a massive positive difference to people.
Taking a sugar pill which you think will make you better makes a massive positive difference to people.
Even paying large sums of money to a charlatan can make a massive positive difference to people.
All of these things make people think they’re being proactive about their (self-limiting) condition.
What said,
March 11, 2010 at 7:54 pm
I wonder whether a change to the advertising policies at these journals might be beneficial. ‘You can’t mention any clinical trial that doesn’t score at least X on this standardized scale’ wouldn’t be hard to enforce. (In fact, it would be rather easier than the current discretionary rules, in which an editor can reject any ad as ‘offensive’.) I’m not sure that the unintended consequences (ads that mention no studies at all?) would be worth it.
ips said,
March 21, 2010 at 11:53 am
repply to Skye Steve…..
‘But I agree we live in times where everyone thinks they are entitled to something for nothing. they want free healthcare, free education, free social care, free roads, etc. but at the same time they want to have sufficient disposable cash (i.e. low taxes) for the faster car, the bigger house, the nicer holiday, etc. and that for me is the situation which is not sustainable’.
Steve, I have followed some of the debate and tend to agree with most of your conclusions especially the above!
However, I tend to think that the waters are much muddier than many here seem to suggest (in terms of practically dealing with pain and suffering in the clinic). I work as a NHS Physiotherapist primarily in GP clinics , formerly in teaching hospitals with specialist consultants in both orthopaedics and pain management.
Many people like to think that things are scientifically rational and that care is optimised to treat ‘rational people’ with ‘rational’diseases’…..After nearly twenty years dealing with medically unexplained pain syndromes, chronic pain and the ‘no mans land’ between orthopaedics and psychology I can assure most people that a lot of people out there remain a mystery. Sometimes this is due to fragmentation of care , often it is due to a dualistic philosophy perpetuated by outmoded thinking and failure to think of the person in a holistic context.
There does seem to be a lot of Homeopathic bashing and it is an easy target.
Personally, Homeopathy does not interest me that much neither do most ‘modalities’ that get discussed (acupuncture,manipulation etc etc). What I do believe is that there is nothing mere about the placebo or nocebo phenmenon (which clinics are often very good at inducing. For example a lady with a lifelong hx of trauma,’ptsd’ anxiety etc was sent for ‘breathing exercises’ ….the clinic had told her she may need a lung transplant ….this leads to further catastrophisation .
Many people I see are informed that there discs/hips etc ‘are out’ , that they have spines of an old person etc –iatrogenic disability in our inactive society often caused by misinformation is a big part of my caseload. This nocebo situation needs to be understood by many in the medical profession and Benedetti has done some interesting Fmri studies in this area see’When words are painful: Unraveling the mechanisms of the nocebo effect’….Many people seem to need or have been conditioned into expecting help for self limiting problems or problems of living. There is abundant scientific evidence that many common problems can be assisted through ‘self help’ i.e movement restoration , ‘meditation’ or low cost stress reduction techniques. Indeed, I have tried to set these up myself at no cost to patients or the doctors locally…..These measure have not been acceptable and none of the doctors have expressed any interest in coming along or referring patients…..Self management is not a popular option especially if it is ‘free’ —–placebos work much better if you pay for them and they are delivered by a credible hierarchy figure (perhaps I have to wear the bow tie and inject saline for best effects?
I too think that the NHS is the best way to deliver care and also agree that in Scotland (particularly away from the central belt) that the service is largely excellent. However, I sense on a Monday morning now matter how much money you throw at it or doctors that you employ many people will continue to moan or demand more……
I enjoyed Nic Humphrey’s take on understanding the placebo response here (talking with Richard Dawkins).The debate is interesting re self – cure.
www.youtube.com/watch?v=e1AQPue7FEM
ips said,
March 21, 2010 at 11:53 am
in respone to to Skye Steve…..
‘But I agree we live in times where everyone thinks they are entitled to something for nothing. they want free healthcare, free education, free social care, free roads, etc. but at the same time they want to have sufficient disposable cash (i.e. low taxes) for the faster car, the bigger house, the nicer holiday, etc. and that for me is the situation which is not sustainable’.
Steve, I have followed some of the debate and tend to agree with most of your conclusions especially the above!
However, I tend to think that the waters are much muddier than many here seem to suggest (in terms of practically dealing with pain and suffering in the clinic). I work as a NHS Physiotherapist primarily in GP clinics , formerly in teaching hospitals with specialist consultants in both orthopaedics and pain management.
Many people like to think that things are scientifically rational and that care is optimised to treat ‘rational people’ with ‘rational’diseases’…..After nearly twenty years dealing with medically unexplained pain syndromes, chronic pain and the ‘no mans land’ between orthopaedics and psychology I can assure most people that a lot of people out there remain a mystery. Sometimes this is due to fragmentation of care , often it is due to a dualistic philosophy perpetuated by outmoded thinking and failure to think of the person in a holistic context.
There does seem to be a lot of Homeopathic bashing and it is an easy target.
Personally, Homeopathy does not interest me that much neither do most ‘modalities’ that get discussed (acupuncture,manipulation etc etc). What I do believe is that there is nothing mere about the placebo or nocebo phenmenon (which clinics are often very good at inducing. For example a lady with a lifelong hx of trauma,’ptsd’ anxiety etc was sent for ‘breathing exercises’ ….the clinic had told her she may need a lung transplant ….this leads to further catastrophisation .
Many people I see are informed that there discs/hips etc ‘are out’ , that they have spines of an old person etc –iatrogenic disability in our inactive society often caused by misinformation is a big part of my caseload. This nocebo situation needs to be understood by many in the medical profession and Benedetti has done some interesting Fmri studies in this area see’When words are painful: Unraveling the mechanisms of the nocebo effect’….Many people seem to need or have been conditioned into expecting help for self limiting problems or problems of living. There is abundant scientific evidence that many common problems can be assisted through ‘self help’ i.e movement restoration , ‘meditation’ or low cost stress reduction techniques. Indeed, I have tried to set these up myself at no cost to patients or the doctors locally…..These measure have not been acceptable and none of the doctors have expressed any interest in coming along or referring patients…..Self management is not a popular option especially if it is ‘free’ —–placebos work much better if you pay for them and they are delivered by a credible hierarchy figure (perhaps I have to wear the bow tie and inject saline for best effects?
I too think that the NHS is the best way to deliver care and also agree that in Scotland (particularly away from the central belt) that the service is largely excellent. However, I sense on a Monday morning now matter how much money you throw at it or doctors that you employ many people will continue to moan or demand more……
I enjoyed Nic Humphrey’s take on understanding the placebo response here (talking with Richard Dawkins).The debate is interesting re self – cure.
www.youtube.com/watch?v=e1AQPue7FEM
joey89924 said,
November 16, 2012 at 2:23 am
I care partly because we have all done stupid things, those who claim that they haven’t are the ones deceiving themselves more than they deceive anyone else.
www.hqew.net