Alternative therapists struggle with the placebo and hawthorne effects once more

June 15th, 2007 by Ben Goldacre in alternative medicine, bad science, placebo | 35 Comments »

I just wanted to draw your attention to a pair of rather entertaining papers from the current issue of the Journal of Alternative and Complementary Medicine, edited by Professor Kim Jobst (the man who endorses the Qlink pendant, amongst other things).

The abstract from the experimental paper is here:

THE JOURNAL OF ALTERNATIVE AND COMPLEMENTARY MEDICINE
Volume 13, Number 3, 2007, pp. 317–327
© Mary Ann Liebert, Inc.
DOI: 10.1089/acm.2007.6300
Resonance, Placebo Effects, and Type II Errors: Some
Implications from Healing Research for Experimental Methods
WILLIAM F. BENGSTON, Ph.D., and MARGARET MOGA, Ph.D.

ABSTRACT
Background: Classical experimental design presupposes that subjects, randomly separated into experimental and control groups, are independent and distinct. Treatments given to the experimental group ought to have no effect on the control group, which functions as a baseline to illustrate “what otherwise would have happened.” Any change in the control group is often labeled an “anomaly.” Examples of these types of anomalous phenomena can be found in placebo research, which often shows proportional unexpected and unexplained changes in control subjects.

In four previously reported experiments on anomalous healing using “healing with intent” on mice injected with lethal doses of mammary adenocarcinoma (source, The Jackson Laboratories, Bar Harbor, ME; code, H2712; host strain, C3H/HeJ), a high percentage of both experimental and control mice exhibited an anomalous healing pattern, most often passing through stages of tumor ulceration to full life-span cure.

Objective: In order to explain tumor regression of control animals, I posit the formation of “resonant bonds,” which can link spatially separate groups. Healing given to the experimental animals can result in an unintended treatment to the control animals, producing anomalous healing akin to placebo effects.

Materials and methods: A recently completed experiment at the Terre Haute campus of the Indiana University School of Medicine has produced a successful test of resonance theory. One group of mice (n=30) was injected with mammary adenocarcinoma cells and randomly divided into a treated group (n=15) and untreated control group (n=15). A second group of age-matched controls (n=25) was left uninjected. Mice
from each group were intermittently sacrificed to measure hematologic values and spleen weight.

Results: As predicted by resonance theory, there were few differences between treated and untreated animals from the first group, but there were significant differences between these animals and the age-matched controls.

Conclusions: Some implications for placebo research and the way we normally conceptualize Type II errors will be discussed. Researchers are invited to reanalyze past data in light of resonance theory.

A choice quote from many:

…Conventional thinking and hence explanation,
seems to defy logic. After all, if no active agent is admin-
istered, how can there be an active effect? If no healing tech-
nique is applied, how can there be remissions?
With a conceptual shift toward resonance, placebos be-
gin to take on a new light. Among the great mysteries of
placebos is the fact that their effect is proportional to the
strength of the experimental treatment.

Perhaps this is
so because the placebo group is not independent of the ex-
perimental group, but is actually part of a larger bonded
collective. Perhaps the question needs to be reformulated
in terms of the conditions under which resonant bonds
form and resonant bonds are broken.

There is of course a wealth of fascinating research on the placebo effect and how it can be modulated (I’ve got a 5,000 word chapter coming on the subject) but more than anything, I think it’s incredibly interesting how the CAM movement feels the need to locate the placebo effect in some kind of reductionist biomechanical process, rather than embrace more parsimonious explanations involving ideas like patients’ expectations of outcome.

These papers brought me a great deal of pleasure on a train journey this morning, and I hope they will you too.

Since ideas want to be free, the PDF’s can be found here and here.


++++++++++++++++++++++++++++++++++++++++++
If you like what I do, and you want me to do more, you can: buy my books Bad Science and Bad Pharma, give them to your friends, put them on your reading list, employ me to do a talk, or tweet this article to your friends. Thanks! ++++++++++++++++++++++++++++++++++++++++++

35 Responses



  1. flange said,

    June 15, 2007 at 3:58 pm

    I think we’re going to see a surge in resonnance related treatments/causes/effects because of all the press that MIT power transmission idea is getting. Is this just the first example.

    Actually i had an alternative motive for posting, has anyone seen the department for trade and industry’s public consultation on its “unfair commercial practices” legislation
    www.dti.gov.uk/consultations/page39674.html
    I had a look through the actual legislation and to me (v. limited legal knowledge) it looked like a very good vehicle for tightening up on dodgy medical dealers. It would only take a few tweaks to the commercial part
    schedule 1-11 /1-12 to clamp down on journalists doing advertising (that would otherwise breach normal advert rules) in their articles.
    I also thought there could be changes to i think 1-29 and in teh business section, part 1, 3 ,2a.

    Anyone fancy taking a look, i reckon that electrosmog article by that vapid woman in the Independent last week should be a test case. If that doesn’t fall foul of unfair commercial practice/dodgy advertising, i don’t know what does

  2. pinguin said,

    June 15, 2007 at 4:07 pm

    My first thought was “placebo? mice?”

    But of course they think that the placebo effect is due to something outside the body, isn’t it?

    You’ve got to admire them – they may believe in fairies, but at least they’re consistent in their beliefs.

  3. Trez75 said,

    June 15, 2007 at 4:39 pm

    Did I read that correctly?
    Is that the woo-woo’s saying that the reason theres no difference in the results between the “healed mice” and the unhealed mice because they’re part of a bonded collective

    I always thought that the reason that there wasn’t any difference was because that the healing didn’t actually do anything

  4. JohnK said,

    June 15, 2007 at 5:01 pm

    Dig the way he accepts the contribution of Sheldrake, but goes beyond it. This is a refinement of morphic resonance, which is now so old an idea that it must have become true.

    “While, on the surface, there are parallels to resonant
    bonding, Sheldrake does not address how smaller collections
    of individuals of a species form delimited boundaries,
    so that some gain the necessary knowledge and some
    do not.”

  5. projektleiterin said,

    June 15, 2007 at 5:51 pm

    “rather than embrace more parsimonious explanations involving ideas like patients’ expectations of outcome.”
    But these were mice?! I’m not going to support the resonance theory, because I don’t support anything I have no clue of, but in this case this hypothesis will probably have to be discarded unless you really believe that the mice had any kind self-healing powers triggered by their expectations about the outcome of this experiment.

  6. projektleiterin said,

    June 15, 2007 at 5:56 pm

    If animals are used for the trials and the results show a placebo effect, does that not mean that there are external factors outside the influence of the test object involved?

  7. Paul said,

    June 15, 2007 at 6:55 pm

    Forigive me if I’m being dopey but I can’t get the whole article and have a very basic quesiton about the study design.

    My reading is that 30 mice were injected with Mammary Adenocarcinoma and 25 were left uninjected. The thirty injected mice (15 treated, 15 untreated) did not differ from each other on haemtological and anatomical outcome measures but both groups differed from the uninjected controls.

    Now, surely that cannot be the experimental design (clearly if it was then this would be an extraordinarily flawed experiment). Would you mind clarifying for me what they actually did. I really don’t want to have to purchase the article (not purely parsimony on my part)

  8. Ben Goldacre said,

    June 15, 2007 at 6:58 pm

    The PDF’s can be found here and here.

  9. pv said,

    June 15, 2007 at 7:43 pm

    Forgive my dimness here, but isn’t this just simply inventing the evidence to fit the required result? I’m tempted to speculate on who the target audience for the papers is – surely not scientifically illiterate journalists?

  10. Mark Wainwright said,

    June 15, 2007 at 10:48 pm

    #15: Um, the results don’t show a placebo effect, or any other effect — did you read the abstract? It appears to say “We poisoned some mice. We then tried to cure some of them with crystals. The ones without crystals got better just as fast as the ones with crystals! That just proves how effective our crystals are. As additional proof, if any were needed, — all these mice were much more ill than another lot which we didn’t poison in the first place.”

    I paraphrase. In the real experiment, they didn’t bother with actual crystals; they just used imaginary ones.

  11. freeTheMonkey said,

    June 15, 2007 at 11:38 pm

    This is most excellent. I now have a new data analysis exercise for my first year tutorial students.

  12. LankyT said,

    June 16, 2007 at 9:44 am

    Its a great idea, just think nice things and you can heal the world…Er with resonance bonding…maybe if all the mice, got together, and just thought nice happy thoughts they could banish sickness and disease. We could all live in mouse eutopia.

  13. projektleiterin said,

    June 16, 2007 at 9:49 am

    To #16: I understand the experiment as follows: they had three groups, a group with treated sick mice, a control group of untreated sick mice in an adjacent lab, and later on in the last two experiments a control group of untreated sick mice in another city. The mice off site always died.

    The mice in the control group on site also had mice dying until some of the healers broke protocol (I find it weird that people just break protocols when they do experiments) and went over, the remaining mice in this control group then also also went through remission and survived. So, the mice in the control group on site that actually should not have received treatment and should have died, actually did survive when they did get treatment, while the control group off site that never got any treatment always died (although I don’t understand why this group was always so small compared with the other control group).

    I don’t understand what the fourth experiment is supposed to prove as the control group also got the same kind of treatment.

    Writing this I’m getting confused, where is the placebo effect in this experiment if all the surviving mice got the same treatment?

  14. almagest said,

    June 16, 2007 at 7:18 pm

    Yes, there is much bad science in alternative medicine. But two points (1) there is also much bad science in western medicine;
    (2) the placebo effect works – much better in fact that many western treatments. So it is a pity that western doctors don’t make more use of it. The main reason most people use alternative medicine on such a huge scale is probably because they get a much better placebo effect! This is a big change in my lifetime. GPs used to be good at it. Now they are much worse!

  15. Ben Goldacre said,

    June 16, 2007 at 7:23 pm

    agreed but unfortunately it takes a lot of finessing to give a placebo without contravening basic modern medical principles like patient autonomy and informed consent.

  16. projektleiterin said,

    June 17, 2007 at 7:39 am

    Is nobody going to answe rmy questions and enlighten me? Or are my questions really so unreasonable?

  17. Pastafarianbabe said,

    June 17, 2007 at 12:55 pm

    projektleiterin – apologies if I spelt that wrong.

    i think the answer to your question is not that there is any suggestion that the placebo effect works on mice (although interestingly it does seem to work for domestic pets albeit not as well as in humans, possibly because the pet is picking up vibes from the owner’s belief in the homeopathic remedy – although the chances of the experimenters managing to replicate this kind of relationship for 30+ mice is minimal). No, the point is that no “healing” as recognised by sane medical practitioners happened at all in this experiment. All they did was poison two groups of mice, wave crystals over one of them and then compare how they did. they ASSUMED that any remission in the group that they waved crystals over was down to the crystals and that any remission in the non waved mice was due to the placebo effect. Now I don’t know about you but I’d be more inclined to interpret this experiment as proff that crystal healing doesn’t work since it acheived no better results than doing nothing but this conclusion is one that seems to have escaped the experimenters entirely.

    I’d also agree that this does seem a shockingly unethical use of experiemental animals (and I support medical testing on animals in the main) – are we sure that they actually did have the relevant ethical OKs sicne I thought the authorities were very strict about this sort of thing. Is there any way we can check?

  18. Rob Hinkley said,

    June 17, 2007 at 5:12 pm

    “they ASSUMED that any remission in the group that they waved crystals over was down to the crystals and that any remission in the non waved mice was due to the placebo effect.”

    Remission in the healed group was assumed to be because of the “healing-by-intent” – basically someone thinking really hard about the disease going away. Remission in the non-healed group was then assumed to also be because of the healing: which had somehow leaked into them along a “resonant bond” which had been created with the mice being healed. The paper then presents these “resonant bonds” as an explanation for why tests show control groups receiving placebo seem to get better just as well as groups receiving so many tested therapies.

    “Now I don’t know about you but I’d be more inclined to interpret this experiment as proof that crystal healing doesn’t work since it acheived no better results than doing nothing but this conclusion is one that seems to have escaped the experimenters entirely.”

    Indeed. In order to explain what looks like the ineffectiveness of their healing magic they have invented a whole new kind of magic featuring “macroscopic entanglement of experimental subjects so that a stimulus given to one group also
    stimulates the other group”
    (their own words). This can be used to explain away the failure of any treatment to perform better than placebo.

  19. pv said,

    June 17, 2007 at 10:23 pm

    “Now I don’t know about you but I’d be more inclined to interpret this experiment as proof that crystal healing doesn’t work since it achieved no better results than doing nothing but this conclusion is one that seems to have escaped the experimenters entirely.”

    I don’t think the conclusion escaped them at all. I rather think they hope it will have escaped your normal CAM believer who is (they hope) too stupid to know any better. As I said – charlatans!

  20. pv said,

    June 17, 2007 at 11:32 pm

    “Ye Journal of Alchemy and Compleat Magick”

    Hahaha :-)

  21. Camp Freddie said,

    June 18, 2007 at 2:36 pm

    The treated and untreated groups had the same remission rates. Therefore, the treatment has no effect.

    Oh, but of course that would be ‘presupposing’ that basic logic applies to the universe.

    Wow. I’ve got a theory that the dead rats from the woo-treated group actually travelled back in time and told their past-selves to go into remission. The resultant paradox caused the universe to explode, so God got a bit annoyed and quickly rebuilt everything as if nothing had happened. It’s got the same amount of evidence as the resonant bond “hypothesis”.

    And how the hell did they get that past an ethics committee? I’ve seen a research plan which could potentially save people’s eyesight and save millions of pounds (or at worst improve PPE recommendations). But I can’t do it because various authorities think the current (rather inaccurate) animal tests are sufficient. I’ve got no problem with that research plan being blocked, but when crap like this gets through…

  22. Pastafarianbabe said,

    June 18, 2007 at 7:12 pm

    Camp Freddie

    I agree, in fact looking at the paper I’m wondering if they actually bothered with an ethics committee at all – let alone a licence to actually run experiments on animals, particularly ones this cruel. Is there a way we can check out whether they did or didn’t?

  23. Dr Aust said,

    June 18, 2007 at 9:44 pm

    PS As an aside, one of the things that makes my brain hurt generally about Alt Nuttery is the enormous amount of highly complicated thinking that has to go into inventing implausibly uber-complex theories for things that have obvious and perfectly plausible (though non-alternative) explanations.

    For a good example look up the published output of Ben’s sometime Bete Noire Homeopathique Lionel Milgrom:

    Search Pubmed

    www.ncbi.nlm.nih.gov/sites/entrez

    with “Milgrom LR”

  24. Robert Carnegie said,

    June 19, 2007 at 1:57 am

    Indeed if I’m following (haven’t read the PDFs, only comments here), they take it as given that healing by intent works (for some of the mice but not the sceptical ones). So how to account for the same success rate in the so-called control group? Why, unintended consequences! Healing passes mysteriously from one mouse to another!

    Do you think this was really done with actual mice at all, or was it made up, or is it a 21st century Sokal?

  25. pv said,

    June 19, 2007 at 1:29 pm

    “As an aside, one of the things that makes my brain hurt generally about Alt Nuttery is the enormous amount of highly complicated thinking that has to go into inventing implausibly uber-complex theories for things that have obvious and perfectly plausible (though non-alternative) explanations.”

    Dr Aust, surely it’s a darn sight easier to invent the stuff than actually do any meaningful research and be honest about the results – especially where, with the aid of a credulous press, there’s a buck to be extracted from a gullible public. And even if Milgrom isn’t indulging in some elaborate fraud (which I don’t believe at all), wasn’t it Feynman who said that the easiest person to fool is oneself?

  26. Dr Aust said,

    June 19, 2007 at 3:09 pm

    Yes, take your point pv… though in doing research, one is often following a reasonably well-trodden path methodology-wise… It is hard going, but there is less “abstract thought” involved than people often believe.

    And making stuff up is probably harder than you would think.

    Anyway, one could hardly fail to see that hundreds of hours went into things like (e.g.) Milgrom’s papers.

    Sometimes I just think about all the hours (years? lives?) that intelligent (though crazily misguided) people put into alternative nuttiness and a sense of despair overwhelms me…

    Thanks for mentioning Feynman. The quote is apt!

  27. Dr Aust said,

    June 19, 2007 at 9:49 pm

    Yes, I see what you mean, pv. I always understood there was a touch of this fame+celebrity with Jacques Benveniste, of homeopathy-in-Nature infamy, together with plenty of ego, of course.

    What I don’t get about Homeopathy in particular is why the practitioners can’t just admit it is a sort of “stealth psychotherapy with added cultural overtones”, which seems blindingly obvious to me. I can see why people can’t be “unconvinced” once they have spent years as believers – as John Diamond said years back in “Snake Oil”, you are effectively asking them to “renounce” years of their life and work – but how the smarter ones get to the point of believing in it in the first place…. that I can’t figure.

    BTW, if you are in mainstream science and are pretty unscrupulous, you don’t have to make up all that much – just be a bit dishonest with the interpretation, cherry pick or “edit” the data, do no proper controls and then call a press conference. You too can be Andrew Wakefield, as it were.

  28. projektleiterin said,

    June 19, 2007 at 10:43 pm

    Thanks, Pastafarianbabe. I think I finally got Ben’s placebo comment, he wasn’t talking about the people who did the research, but about the CAM movement in general. I still have this question though:

    “I understand the experiment as follows: they had three groups, a group with treated sick mice, a control group of untreated sick mice in an adjacent lab, and later on in the last two experiments a control group of untreated sick mice in another city. The mice off site always died.

    [...]

    So, the mice in the control group on site that actually should not have received treatment and should have died, actually did survive when they did get treatment, while the control group off site that never got any treatment always died (although I don’t understand why this group was always so small compared with the other control group)”

    The mice in the third control group off site always died. How is this to be explained? I assume the sample of the third control group is too small to draw any significant conclusions?

  29. pv said,

    June 20, 2007 at 1:37 am

    Dr Aust, I suggest that there are homeopathists who believe 100% in what they are doing, while there are almost certainly those who know it’s a deception. However, for any homeopathist to admit it doesn’t work would be like inviting oneself to be called a liar, a fraud or whatever. They would also incur the wroth of the faithful.
    Anyway, to me it bears many of the hallmarks of organized religion with the senior practitioners being like the Bishops, and the patients the equivalent of the congregation. It must be very difficult (as it is with religion), after years of time and self invested in it, to then disinvest oneself of it, however compelling the evidence for such an action. Homeopathy and religion also share much in their approaches to science.

  30. Dr Aust said,

    June 20, 2007 at 10:23 am

    I have likened homeopathy to a religion before, pv. The analogy is even better, because apart from the priesthood and the adherents, and working entirely through faith, it also has a sacred book which cannot be questioned (Hahnemann’s Organon), plus other sacred writings which give instructions(Hahnemann’s other writings and the works of one or two of his followers), plus (best of all, IMHO) the magical transformation of mundane materials into something special to be taken by the faithful, which happens through prescribed ritual carried out by the priest-figure.

  31. Pastafarianbabe said,

    June 20, 2007 at 5:30 pm

    projektleiterin – Having looked carefully at the numbers they give in their abstract and methodology and the numbers they quote in the results tables I’m not sure that all the mice in the off-site control group DID die. As far as I can make out they did five experiments each with 15 mice in the “healed and “non-healed” on-site groups and 25 mice in the “non-healed off site” group making a total of 55 mice per experiment for five experiments (275 mice in total), however if you look at the results table (which also includes a final tally which purports to be overall figures for ALL the experiments) they add up to results for far fewer mice than they should have used – for example they only report the results of 8 of the mice in the off site control group – all of them are reported to have died but that’s only 8 mice out of 125 – what happened to the other 117? even if they all ran for the hills or spontaneously combusted or something that should have been mentioned instead the fate of the overwhelming majority of the very mice most crucial to the experimenters case is not mentioned anywhere in the paper – you are just expected not to notice that they do not appear in the results.

    Now I don’t know about you but the words “selective reporting” are looming very large in my mind and I am forced to concur with pv – this study was either written by charlatans or Sokal style hoaxers (here’s hoping – I do always try to see the best in people).

  32. brainteaser said,

    July 31, 2007 at 9:13 pm

    Has anyone read actually the paper?

    Here is the kicker: the mice were injected with a form of cancer that normally kills all mice within 27 days. That’s 100% fatality within 27 days, folks. No mouse in the history of cancer research has ever lived with this cancer to the 28th day, no matter what form of treatment they received. Most of THESE mice recovered to live their full lifespans and developed immunity to the cancer. Does this compute? This research has succeeded where no other cancer research has to date: IT CURED THE MICE OF CANCER.

    There were 6 experiments in 3 different medical schools over a period of several years, with mice from different sources and the involvement of bona fide scientists throughout, including a medical school professor. So it would be difficult to impute errors in the handling or injecting of the mice or to say the cancer “did not take”. Judging from the extensive ulceration and from the presence of viable cancer cells at all stages until total cure, it clearly took.

    There were no crystals involved :)

    Please read the article before you leap to knee-jerk conclusions.

  33. The Master said,

    September 21, 2007 at 12:32 pm

    @brainteaser

    Sorry I am late to this and normally would not post.

    I have read the paper and see no reference to corroborate mouse survival in previous experiments, maybe I missed it where did you get your information.

    Also I see nothing in the paper that says in previous experiments the mice have never lived past 28 days

    can you enlighten me!!!

  34. diudiu said,

    December 21, 2009 at 6:39 am

    free shipping ugg
    free shipping ugg

  35. vladtn said,

    April 28, 2010 at 9:46 pm

    projektleiterin, Pastafarianbabe, brainteaser, The Master

    Am quite late on this too, the authors mention the 27 days in other sources, such as volume 2 of this journal (available to download):

    www.scientificexploration.org/edgescience/

    I quote:

    “One of the chair’s department members had been doing mice studies on a particular form of mammary adenocarcinoma that is 100 percent fatal within 27 days of injection. The model itself was so well understood that statistical studies of lifespan were routinely done, even as no mouse had ever lived past 27 days. If we
    could even get our mice to live closer to the 27 day mark, that would be strong evidence of a healing effect. If a mouse were to live to day 28, well, then we’d own the world record.”

    I agree with brainteaser that IF the the 27 days claim is verified it would be useful investigate, and that the blacebo effect discussion is not justified.

    does anyone know anything about this “particular form of mammary adenocarcinoma”?