Saturday December 3, 2005
People often ask me [pulls pensively on pipe] “what is science?” And I reply thusly: science is exactly what we do in this column. We take a claim, and we pull it apart to extract a clear scientific hypothesis, like “homeopathy makes people better faster than placebo” or “the Chemsol lab correctly identifies MRSA”; then we examine the experimental evidence for that hypothesis; and lastly, if there is no evidence, we devise new experiments. Science.
Back in December 2003, as part of our Bad Science Christmas Gift series, we discovered The Perfect Sommelier, a rather expensive wine conditioning device available in all good department stores (www.badscience.net/?p=192) In fact there are lots of devices like this for sale, including the ubiquitous Wine Magnet: “Let your wine ‘age’ several years in only 45 minutes! Place the bottle in the Wine Magnet! The Wine Magnet then creates a strong magnetic field that goes to the heart of your wine and naturally softens the bitter taste of tannins in ‘young’ wines.”
At the time, I mentioned how easy it would be to devise an experiment to test whether people could tell the difference between magnetised and untreated wine. I also noted how strange it was that none of the manufacturers seemed to have bothered, since it could be done in an evening with 50 people.
Over to Dr James Rubin et al of the Mobile Phones Research Unit at Kings College London, and their new article doing just that, in the esteemed Journal of Wine Research (full reference at badscience.net) They note the dearth of experimental research (quoting, rather chuffingly, the Bad Science column itself) and go on: “One retailer states, ‘we challenge you to try it yourself – you won’t believe the difference it can make’.” Unwise words. “A review of Medline, PsychInfo, Cinahl, Embase, Amed and the Web of Science using the search term ‘wine and magnet’ suggested that, as yet, no scientists have taken up this challenge.”
Now this study was, I have to say, an extremely professional operation. Before they even started, they did a power calculation: this is to decide how big your sample size needs to be, to be reasonably sure you don’t miss a true positive finding by not having enough subjects to detect a small difference. Since the manufacturers’ claims are dramatic, this came out at a mere 50 subjects.
Then they recruited their subjects, using wine. This wine had been magnetised, or not, by a third party, and the experimenters were blind to which wine was which. The subjects were also unaware of whether the wine they were tasting, which cost Â£2.99 a bottle, was magnetised or not. They received wine A or wine B, and it was a “crossover design” – some people got wine A first, and some people got wine B first, in case the order you got them in affected your palate and preferences.
There was no statistically significant difference in whether people expressed a preference for the magnetised wine or the non-magnetised wine. To translate back to the language of commercial claims: people couldn’t tell the difference between magnetised and non-magnetised wine. I realise that might not come as a huge surprise to you. But the real action is in the conclusions. “Practitioners of unconventional interventions often cite cost as a reason for not carrying out rigorous assessments of the effectiveness of their products. This double-blind randomised cross-over trial cost under Â£70 to conduct and took one week to design, run and analyse. Its simplicity is shown by the fact that it was run by two 16-year-old work experience students (EA and RI).”
“Unfortunately,” they continue, “our research leaves us no nearer to an understanding of how to improve the quality of cheap wine and more research into this area is now called for as a matter of urgency.”
“Drawn to drink: A double-blind randomised cross-over trial of the effects of magnets on the taste of cheap red wine” G. James Rubin, Gareth Hahn, Edward Allberry, Ross Innes And Simon Wessely. Journal of Wine Research Volume 16, Number 1 / April 2005 pp.65- 70
Â·Send your bad science to firstname.lastname@example.org