The news you didn’t read

August 16th, 2008 by Ben Goldacre in bad science, publication bias | 25 Comments »

Ben Goldacre
The Guardian
August 16, 2008

There’s not exactly a whole bunch of news going on right now. According to the Mail we are witnessing the “Invasion of the killer jellyfish” (except Portuguese Men O’ War have been reported on British shores since at least 2003), the hunt for the Yeti continues, and there’s always room for another “equation for” story.
Somehow what doesn’t get into the papers is as interesting as what does. Right now I’m looking at a press release on a story which seems pretty important to me: people with serious mental illnesses are committing fewer murders than ever before, by a truly enormous margin. Homicides in this group increased from around 40 a year in the 1950s to 100 a year in the 1970s, in line with a similar increase in the general population. But while murders by people like you have continued to increase, and roughly trebled (0.6 per 100,000 of population in the 1950s, and almost 2 per 100,000 now), murders by people with serious mental illnesses, despite the hype and the fear, the public pronouncements and the headlines, have come down massively since the 1970s, to fewer than 20 a year today.

Alongside the silly season stories, this startling new analysis of several different databases worth of information was not considered newsworthy. It got coverage in New Scientist (ooh) and BBC Online only. Nobody else touched it. What a mystery.

Journalists are traditionally fascinated by mental illness after all. Celebrities with schizophrenia or depression can expect to have their hospital admissions (and embarrassing behaviour when unwell) diligently documented by the newspapers, and murders associated with mental illness receive blanket media coverage, with extensive campaigns both in the media and at grassroots level. When the “mental health czar” Louis Appleby called for more effort at reducing murders by people with serious mental health problems last year (a “bloke has opinion” story if ever I saw one) it was news to every single newspaper.

Journalists also love numbers – they use them for a spurious sense of precision, and for an air of scienciness – and this story had plenty, from a proper study, published in the British Journal of Psychiatry.

Journalists also routinely take stories handed to them on a plate by press releases, as a quick scan of any newspaper will rapidly reveal. According to research commissioned by Nick Davies by Cardiff University, about 80% of news coverage is rehashed from them).

There is a crowd pleasing answer here, of course, which is that society is very simply prejudiced. But there is also a more mundane explanation. In a generous mood, you wouldn’t say this was a very badly written press release, but it doesn’t give the story on a plate, at first glance, with a populist headline, a catchy narrative to hang it on, the offer of photos (with breasts) and the idea that “stigmas and deeprooted fears are misguided” flagged up in neon lights. You had to pay attention to find the news.

And it wasn’t widely disseminated. My friend Nadia Stone writes positive stories about people with mental health problems (it’s why we’re friends). There was one simple reason she didn’t run with it: she gets plenty of nonsense from PR companies on pills and cosmetics, but the academic journal didn’t send their press release to local papers, which often cover the very anecdotes that can wrongly callibrate peoples fears and prejudices. “It might actually have been useful considering two men with schizophrenia escaped from a mental hospital near Exeter on Tuesday night, and everyone’s been very scared.”

We are convinced by the media that people with serious mental illnesses make a significant contribution to murders, and we formulate our approach as a society to tens of thousands of people on the basis of the actions of about 20. Once again, the decisions we make, the attitudes we have, and the prejudices we express are all entirely rational, when analysed in terms of the flawed information we are fed, only half chewed, from the mouths of morons.

If you like what I do, and you want me to do more, you can: buy my books Bad Science and Bad Pharma, give them to your friends, put them on your reading list, employ me to do a talk, or tweet this article to your friends. Thanks! ++++++++++++++++++++++++++++++++++++++++++

25 Responses

  1. Dr* T said,

    August 16, 2008 at 8:56 am

    For some details on the KILLER JELLYFISH stories (showing this story comes up every year about this time) see here.

    Just for clarity, I think I’m right in saying (for comparison sakes) that the annnual rate of murders by people like us are 2 per 100,000 (your statistic above), and murders linked to people with mental disorders are 0.07 per 100,000 (from the press release.

  2. Toenex said,

    August 16, 2008 at 10:03 am

    I thought it always was a common misconception that people with diagnosed mental health problems were a greater danger to the general public than to themselves? Although perhaps this would be a revelation in the universe inhabited by Daily Mail readers and therefore headline material.

    I liked the ‘people like you’ bit. You think the people who read this blog are without mental issues? And you being such a proponent of evidence based reasoning Ben! 🙂

  3. doctormonkey said,

    August 16, 2008 at 1:17 pm

    It sounds as though the problem is also about generalisation of people with mental health problems – do you lump in everyone who gets a prescription for antidepressants EVER from their GP?

    or do you just look at the maximum security “wards” at Broadmoor?

    Do you include personality disorders?

    Do you include those with drug and alcohol problems?

    I suppose I could read the journal article but that would potentially make me sound as though I know what I am talking about!

  4. ChrisR said,

    August 16, 2008 at 1:56 pm

    According to Ministry of Justice figures there were 2241 murders in England and Wales in the three years up to and including 06/07 ( I make that a murder rate of about 1.4 per 100k per year, given a population of about 54m. Given the tiny number of murders (20 you claim) committed by those with mental illness, the figure of 1.4 is likely to be a reasonable average for those who are not mentally ill.

    It is also interesting to note another aspect of murder rates that is rarely, if ever, reported: race. In the same Ministry of Justice report it breaks out the race of the principal suspect(where the main suspect is known). The rate for whites is about 0.9 per 100k, and for blacks about 6.6 per 100k.

    Clearly the Ministry of Justice data that these figures are based on will be imperfect, with all sorts of measurement problems. Despite that it seems unlikely that such a massive difference – a factor of about seven – could arise through data errors or by chance.

    It is therefore puzzling why such a massive discrepancy has not received media attention, particularly in the light of the huge media focus on the disproportionality of stop-and-search between races.

  5. Toenex said,

    August 16, 2008 at 5:13 pm

    I think there is definitely an issue with the popular definition of ‘mental illness’ as a single category but also, as with so much of medicine, the limited understanding of ‘normality’ and what constitutes normal parameters. For instance is it possible that anyone who commits murder could be regarded as exhibiting some degree of mental illness?

  6. RS said,

    August 16, 2008 at 5:43 pm

    ChrisR – you’ve normalised for different age, gender, socioeconomic, urban/rural and other relevant demographic differences between white and black people of course.

  7. Robert Carnegie said,

    August 16, 2008 at 5:43 pm

    I think the headline to watch in this issue is the total number of murders by mentally ill people, because there are problems defining it as a proportion. For one thing, mental illness can indeed be reduced or as-good-as eliminated by modern effective treatment, although often with unpleasant side effects. But on the other hand, Alzheimer’s disease and other senile dementia is rising. Some people suffering these diseases can behave dangerously, but being elderly they usually aren’t strong enough to do a lot of harm, so, fewer murders.

  8. warumich said,

    August 16, 2008 at 8:34 pm

    Interesting – do you have a reference for Nick Davies’ work on press releases? Did this stuff get on AlphaGalileo or something? May be a good idea to ask the people involved what media strategy they had in mind.

    Incidently, I’m still trying to find the Bauer/Durant paper – are you sure it was them though? I’ll see Martin next week, I’ll ask him then.

  9. eponymous85 said,

    August 17, 2008 at 8:53 pm

    I like the authority with which ‘people like you’ and ‘people with serious mental illness’ are stated as two discrete categories.

  10. maninalift said,

    August 18, 2008 at 9:47 am

    A common maximam amongst mental health professionals is that the mentally ill are usually much more at risk to other other people than they are themselves a risk.

    Although S=said mental health professionals probably use a much more politically-correct term than “the mentally ill”.

  11. Andy said,

    August 18, 2008 at 5:28 pm

    Ben, I think you’re guilty of one of the things you complain about, giving numbers out of context.

    The murder rate by normal people has tripled to 2 in 100,000, over 1,000 murders in the UK per year. Only 20 murders in the last year were by people with serious mental issues.

    So yes the chances are that if you are going to be murdered it’s not going to be bu someone with a serious mental illness. Not really too much of a shock there.

    However surely the more important question is whether someone with serious mental problems poses a greater risk to the population than a “normal” person. There is no way of knowing that based on the numbers you gave.
    You never give the murder rate for people with mental issues only the total number, without that the numbers fairly meaningless.

    Also you don’t indicate if there is any control in those numbers to allow for changing definitions of “serious mental issues”. This is total speculation on my part but I’d guess that the definition has changed a little since the 50’s for both medical and political reasons. If the proportion of the population within the group changes then you would expect to see a change in the number of murders which doesn’t match the population growth.

  12. ChrisR said,

    August 18, 2008 at 8:44 pm

    RS – clearly not. Doing a normalisation might well reduce the difference, although given the size it would be surprising if it removed it altogether. But I agree with your implication that is an important piece of analysis to do. As it would be for analysing the difference in stop-and-search rates, and indeed all other reported differences in statistical measures, between races.

    However, my point was more that taking the numbers at face value the way that the media completely ignored a potentially interesting story (and I don’t believe that they have done so because they are waiting for the results of the normalisation analysis before drawing conclusions).

  13. heavens said,

    August 18, 2008 at 9:05 pm

    I wonder if the rate of murders is different among strangers? That is, what scares the public is the idea that they’ll be minding their own business, (say, riding the bus, to use a recent high-profile murder by a psychotic person) and be suddenly attacked by a psychotic stranger.

    Most murders don’t involve killing strangers. Most murders involve people you know: your girlfriend, the neighbor you’re always arguing with, the family member that did you wrong, the kid down the street who is looking for drug money. These may be higher risk situations, but they’re less worrisome to us, because we feel like we understand and control the situation: I won’t argue with him when he’s drunk, I’ll make sure other neighbors are around when I talk to the jerk, I’ll get the family to help me with Joe, I’ll call the kid’s parents and suggest they get him in treatment for his addiction.

    But the random psychotic person is a tough question. How was I (a person who just got off a bus) supposed to protect myself if the crazy guy next to me had attacked me?

    The response from the scientific community seems to be, “It’s not very likely to happen, so you just don’t need to worry about it.”

    Well, it’s true that it’s not very likely to happen in general. But would you have been happy giving that advice to Tim McLean? Could you have stood up at his funeral nine days ago and said, “Well, these things really don’t happen often enough for people to worry about them”?

  14. RS said,

    August 18, 2008 at 9:39 pm

    ChrisR – I’ve heard mention of differential offending rates by race a fair few times in the media. Often in the context of differential stop-and-search rates.

  15. RS said,

    August 18, 2008 at 9:42 pm

    heavens – but then the question is how frequent killing of non-acquaintances is amongst the psychotic. Certainly many murders by psychotic people are of friends and family rather than random strangers.

  16. heavens said,

    August 18, 2008 at 11:46 pm

    RS: Yes, that’s exactly my question. If you split the data according to whether the victim knew the murderer, would you get significantly different rates for the psychotic and non-psychotic murderers?

  17. healthydistrust said,

    August 20, 2008 at 3:16 pm

    This question has probably been asked before, but I’ll plough on regardless (and regardless of the fact that it’s not directly related to this particular post): why is it that The Guardian excises the hyperlinks in their online version of Bad Science articles? They do include links in other parts of the paper (sometimes), so why never for Bad Science? Is there a technical reason for it? (Something to do with the web publishing software they use for example). I just compared the two versions of this article, the one here and the one there, and none of the links have been included.

  18. Ben Goldacre said,

    August 20, 2008 at 3:19 pm

    i submit references, usually, but they never print them, which i guess is fair enough as a style issue. the links i generally do on friday evening as i’m posting the blog version (which is also longer and uncut by subs, if there have been harsh cuts). i can see that there are differences between a blog and a newspaper, but i can also see why you’d find it annoying.

  19. healthydistrust said,

    August 20, 2008 at 4:46 pm

    It’s not just the blogs, though, The New York Times has been linking to sources for a while now (check out this article as a random example), it’s curious The Guardian doesn’t have the same policy, given it has one of the better newspaper websites. You’d think it was high time newspapers reviewed their attitudes when it comes to granting to their readers access to the sources their journalists use: it makes for more transparency, which, arguably, might make for more accuracy.

  20. used to be jdc said,

    August 20, 2008 at 5:15 pm

    Following on from Healthy Distrust’s comments about refs:
    I’ve been moaning to the quality papers (and the Daily Mail) about their habit of publishing stories without refs. I understand TV & Radio might find it difficult to give proper refs, but when they publish these stories on the internet why can’t they add a link to the paper that’s being discussed? It would be so simple. I blog-whinged about it here: References Please!

    If they are going to refer to studies and not tell you which studies they are on about then it is going to make it very difficult to work out how accurate their coverage is. Some media outlets don’t even give you handy clues (e.g., author’s name), whereas the NYT (as HD says) give full refs. If they can do it…

  21. nekomatic said,

    August 21, 2008 at 9:09 am

    Blimey, there are some silly comments on here this week. “How was I (a person who just got off a bus) supposed to protect myself if the crazy guy next to me had attacked me?” How are you supposed to protect yourself if a non-crazy guy next to you attacks you? The question is whether the specific media focus on attacks by people with mental health problems is justified. I don’t know what your point about Tim McLean’s funeral is supposed to be. I expect you agree that people are very unlikely to die by being struck by lightning – could you stand up at someone’s funeral who had died thus and say ‘these things really don’t happen often enough for people to worry about them’? How would this be relevant?

    ChrisR, if you normalised and found that a difference still existed between murder rates among black and white people, and you went looking for causes, you might well come up with something like a more prevalent culture of gang membership and/or carrying weapons among young black men, something that has been extensively covered in the media I think you’ll find. Where’s the cover-up here? What conclusion are you driving at?

    Finally, am I the only one who thought that Ben’s use of ‘people like you’ was simply, er, amusing? You, the Guardian reader, couldn’t possibly be mad – but you might be a murderer…

  22. romdjoll said,

    August 21, 2008 at 11:23 am

    Another point, and a particular bugbear of mine, is that it is not just regular media outlets (press, TV news etc.) that are responsible for the demonisation of people with metal illnesses – a vast proportion of extremely lazy bestselling crime fiction writers do the same thing. Pick up a James Patterson book at random and I can lay odds that the killer turns out to be bipolar. Just about every time. Other books even go so far as to mark completely benign things such as synaesthesia as a mark of dangerous instability. There’s a constant confluence/confusion about psychosis vs psychopathy in just about all forms of mainstream media.
    Sad fact is that scare stories sell more papers than “Mad people less homicidal that you’d think” pieces ever could. As one of the “mad” types, I’m just glad of the work this site does in highlighting smile-making news I wouldn’t find anywhere else.
    Cheers Ben!

  23. SpiderJ said,

    August 21, 2008 at 12:25 pm

    Fascinating that the whole debate is about how the media don’t care – but the important thing I take away from this is that the press release itself isn’t very interesting.

    I accept that the underlying data and trends are important – but if you are a journalist bombarded with 100s of press releases every day, then the headline and first paragraph have to grab you by the balls and make you take notice. Instead, this comes across as a very dry and heavy statistical piece, and as such, will get added to the ‘get round to it at some point’ pile.

    It’s not that journalists are stupid or lazy (although, like in every profession, including medicine, science, etc. I’m sure there are a few), but that they have suffered from huge cutbacks in staffing levels, to the extent that if something isn’t given to them on a plate, then they just don’t have the luxury to read something like this in detail to properly understand what it is trying to convey.

    If you truly have something important to say then you need to invest a reasonable effort in working out how to best say it and to whom – its not enough to just whack it on an email and hope ‘the meeja’ will just run with it.

    Methinks the The Royal College of Psychiatrists need a better PR team.

  24. Robert Carnegie said,

    August 25, 2008 at 12:52 am

    …and also (I should keep a list of caveats) are “people with serious mental illness” only those who have been diagnosed? What about people who have gone mad without anyone noticing, such as MPs and business middle management? Surely they count as “normal people” because we only know about “people with serious mental illness” when we know about their serious mental illness, unless they can be examined after some hideous event has taken place.

    No one goes around questioning whether “normal people” are a danger to the public – perhaps they should.

  25. diohdan said,

    November 10, 2008 at 2:52 am

    A Well Constructed Sales Presentation

    Mass media received from the others, such as the Associated Press, or Reuters, to be released for publication. It really does not matter who may have composed such a press release, but in my opinion, they are overall nothing short of a well-constructed sales pitch by sponsors of media outlets who are instructed specifically how the press release will be presented to readers in an authoritarian manner. It is public relations for the sponsor of the embellished if not intentionally incomplete results reported that are designed to attract favorable media attention not only regarding this content, but towards the corporation who designed the release who implemented its release, who coerced outlets to present it to the public. Mass media receiving press releases are transformed into front groups to offer third party legitimacy for the content of the press release, which is biased by design, which allows the sponsor to escape liability for its content, perhaps, yet remains vicariously liable for the flaws associated with the presentation of the content. Flaws that reflect reckless disregard for the public. The readers of what likely is not objective and complete.
    An example is an anonymous press release posted on the Medical News Today website from March of 2006. The title: “Cymbalta Safely and Effectively Treats core anxiety symptoms associated with generalized anxiety disorder.” Clearly, this title included words associated with relief or elation, which are subjective and not objective elements which would clearly be more appropriate.
    The first paragraph repeats the results mentioned in the title of this article, but also states Cymbalta offers relief of painful symptoms associated with anxiety, as well as improved functional impairment, also claimed to be associated with anxiety.
    Cymbalta was not approved for anxiety or any of the symptoms associated with this condition. In fact, cymbalta was not filed to the FDA for this speculated new indication desired by Eli Lilly until May of 2006. By definition, this press release is clear off-label promotion and misbranding that was performed overtly.
    Shortly before the lightly stated disclaimers about Cymbalta were annotated in this press release, testimonials were intentionally created, I surmise, and were spoken about Cymbalta. The first testimonial was from the lead author, who expanded the claims made initially with various medical terms included, followed by his hope about the great potential of Cymbalta based on this study, which was planned to be shortly announced and discussed in greater detail soon after this press release at a national anxiety association meeting. The second testimonial was Eli Lilly’s Medical Advisor expressing his elation about what the lead author just stated, followed by how much he was encouraged by these results that will benefit so many others.
    What was not discussed in this press release was the devastating post-marketing adverse events correlated with Cymbalta- which include what is termed discontinuation syndrome, along with suicidal ideation as well as cases of suicide by those on Cymbalta. There are more, but these are the most concerning to others, yet not stated overtly in the press release.
    As with any healthcare journalism, objectivity has to be a necessary requirement with any publishing that is potentially exposed to so many others, more so with such medical issues in particular. Because these so many potential readers are in fact us- public citizens who deserve much more than half truths and fabrications from those whose purpose is supposed to be the sharing of complete and unbiased information, yet apparently and clearly is not the case.

    Dan Abshear

    Author’s note: What has been written has been based upon information and belief.