Saturday January 19 2008
In 1954 a man called Darrell Huff published a book called “How to lie with statistics“. Chapter one is called “the sample with built in bias” and it reads exactly like this column, which I’m about to write, on a Daily Telegraph story in 2008.
Huff sets up his headline: “The average Yaleman, Class of 1924, makes $25,111 a year!” said Time magazine, half a century ago. That figure sounded pretty high: Huff chases it, and points out the flaws. How did they find all these people they asked? Who did they miss? Losers tend to drop off the alma mater radar, whereas successful people are in Who’s Who and the College Record. Did this introduce “selection bias” into the sample? And how did they pose the question? Can that really be salary rather than investment income? Can you trust people when they self-declare their income? Is the figure spuriously precise? And so on.
In the intervening fifty years this book has sold one and a half million copies, it’s the greatest selling stats book of all time (tough market) and it remains in print, at just eight pounds ninety nine.
Meanwhile “Doctors say no to abortions in their surgeries” is the headline in the Daily Telegraph. “Family doctors are threatening a revolt against Government plans to allow them to perform abortions in their surgeries, The Daily Telegraph can disclose.” A revolt? “Four out of five GPs do not want to carry out terminations even though the idea is being tested in NHS pilot schemes, a survey has revealed.”
A survey. Channeling Huff through my fingers, in a trancelike state, I went in search of the figures. Is this a systematic survey of all GPs, with lots of chasing to catch the non-responders? Telephoning them at work? A postal survey, at least? No. It was an informal poll through an online chat site for doctors, producing this major news story about a profession threatening a revolt.
The question was this: “GPs should carry out abortions in their surgeries” You can “Strongly agree, agree, don’t know, disagree, strongly disagree.”
I might be slow, but I myself do not fully understand this question. Is that “should” as in “should”, as in, “ought to” as in “coerced”? And in what circumstances? With extra training, time, and money? With extra systems clearly in place for adverse outcomes? This is a chat website where doctors go to grumble, cynically, in good company. Are they saying “no” because of more work and low morale? Would you even click the “abortion” link in the chat pages index, if you didn’t already have an interest in abortion?
And stepping bravely beyond the second word “should”, what do they mean by “carry out abortions in their surgeries”? Looking at the comments in the chat forum – as I am right now – plenty of the doctors seemed to think the question referred to surgical abortions, not the relatively safe oral pill for termination of early pregnancy. Doctors aren’t all that bright, you see.
Here are some quotes from the doctors there. “This is a preposterous idea. How can GP’s ever carry out abortions in their own surgeries. What if there was a major complication like uterine and bowel perforation?” “The only way it would or rather should happen is if GP practices have a surgical day care facility as part of their premises which is staffed by appropriately trained staff, i.e. theatre staff, anaesthetist and gynaecologist… any surgical operation is not without its risks, and presumably [we] will undergo gynaecological surgical training in order to perform.”
“What are we all going on about? Let’s all carry out abortions in our Surgeries, living rooms, kitchens, garages, corner shops, you know, just like in the old days.” Oh, and my favourite: “I think that the question is poorly worded and I hope that DNUK do not release the results of this poll to the Daily Telegraph.”
Seriously, if you haven’t read it, the book only takes about two hours to get through, and it remains a ripper. Highlights include “Correlation does not imply causation” and “Using Random Sampling“, as well as the obvious critique of graphs which is always worth re-doing. What’s surprising is how most of it feels so obvious now, and yet the same old canards show up time and again in newspapers.
Statistical Science, 20 (3), 2005, 205–209.