This is a response (as they say in Youtube) to a previous piece in the Times Higher by Bob Ward, which is pasted at the bottom. Hey, I’m in the THES. I am officially “old”.
Clinical cost of making headlines
Ben Goldacre
21 September 2007
Times Higher Education Supplement
Paul Broca was a French craniologist who measured brains. He was famous, and his name is given to Broca’s area, the part of the brain involved in generating speech, which is often damaged in strokes. But Broca had a problem: his German brain specimens were 100g heavier than his French ones, and by rights, the French should have been superior.
So Broca decided that other factors, such as overall body weight, must also be taken into account when measuring brain size. This, he said, clearly explained the larger German brains. For his prominent work showing that men had larger brains than women, Broca made no such adjustments.
Cesare Lombroso, the 19th-century pioneer of “biological criminology”, also fudged the interpretation of his results: he cited insensitivity to pain among criminals and “the lower races” as a sign of their primitive nature, but identified the same quality as positive evidence of courage and bravery in Europeans.
If Broca or Lombroso had failed to publish their methods and results, merely presenting their interpretations informally to the popular media, or through press releases, then the take-home messages would have been: “Science proves French people cleverer than Germans”, “Science proves men cleverer than women”, and “Science proves criminals are biologically inferior”. These would be seductive messages to science writers even today.
So I was surprised to see Bob Ward, former head of communications at the Royal Society, speaking out in these pages against the Ingelfinger rule (“We ought to get all findings out fast”, August 31). Taken at its toughest interpretation, this rule means academic journals will refuse to publish research that has been presented to the media, and with good reason.
Ward gives two explanations for why editors enforce it: first, the journals are venal and want the publicity to be attached to their own organs; second, he concedes, peer review may be an effective filter for dodgy research. But Ward called for the rule to be overturned, and for journalists to be permitted to write about unpublished data, speedily, and in the public interest.
First I have minor quibbles. Many papers are available online before print publication, rapidly after acceptance. Those with urgent clinical implications, specifically, are fast-tracked to paper publication, sometimes within a month. It is in the interest of journals to be quick, after all, to attract good contributors, and some compete very efficiently: the BMJ will reject a manuscript within a couple of weeks. And all sane editors would consider it fair to go to the media before publication if a paper contained an urgent health message.
But there is a far bigger issue at stake. It is not peer review that makes a full academic publication important: it is universal access to the full methods and results of the study. With scientific research, today as much as in the era of Broca and Lombroso, the devil is in the detail, and those details are not to be found on the pages of daily papers.
Were my 19th-century examples extreme, inflammatory and unfair? Ward’s example was not so hot, either. His cited case was the “grapefruit causes breast cancer” study from July, a piece of observational epidemiology that elicited blanket media coverage.
This grapefruit paper is exactly the kind of work that is routinely misunderstood and misrepresented in the media, which seems collectively incapable of basic critical appraisal of academic work even when it is published. It was a speculative piece of observational epidemiology, and the authors themselves have been clear that it is interesting, theoretical, but highly tenuous stuff, specifically advising caution in its interpretation and implementation and flagging up the need for replication.
This is a trivial example. The archives at badscience.net are overflowing with just a small sample of the media’s crimes: preposterous cherry-picking, outrageous overextrapolation, startling ignorance or whitewashing of known methodological flaws and, worst of all, reporting the authors’ speculative conclusions, from the discussion section of a paper, as if they were the experimental results themselves.
The sad reality is, from the extremes of the media’s MMR scaremongering to the grind of the Daily Mail‘s bizarre ongoing ontological project of dividing all the world’s inanimate objects into those that cause or cure cancer, the media commentariat has not earned privileged early access to scientific knowledge and information. And stories such as the media’s MMR hoax (as it will come to be known) have been perpetuated by the promotion of unpublished research as if it were gold-standard work.
With the Ingelfinger rule half-relegated, popular media debate on a piece of research often takes place in a strange “post-embargo pre-publication” twilight period, where the work is available only to the media commentariat, not to doctors or academics. By the time those in the latter group get to read the paper it is too late to point out how grotesquely its significance was caricatured.
There is also a wider, gentler issue: science is a features subject, which fits uncomfortably into the “news” template at best, because it moves ahead by the gradual emergence of new themes, each supported by a raft of evidence from a range of disciplines. New discoveries – newsworthy by virtue of being unexpected – are by their nature often overturned, which is to say, they are wrong, misleading or red herrings. Often new discoveries deserve less attention, not more, and reach prominence only because of the media’s obsession with unrepresentative “breakthroughs”.
With medical science, bad media coverage carries a genuine clinical cost. Patients arrive at surgeries brandishing clippings, and this puts doctors in an extremely difficult position. To take the clipping, nod sternly over half moon specs and explain that newspapers are full of “lies” is inappropriate, dismissive and injurious to the doctor-patient relationship.
But to practise medicine based on the content of newspapers is an equally absurd suggestion. Doctors need access to original research papers so they can critically appraise their methods, results and relevance to clinical practice. When the media’s coverage – especially the more mendacious and misleading material – is based on informal personal briefings, half-overheard conference presentations, industry press releases or pure fantasy, this is impossible.
If it is accurate coverage we are after, then the media should cover only research that has been published in full – in any format where the methods and results are fully available, regardless of “peer review” – and with a prominent link to the primary source. If journalists are sure a piece of research is relevant enough to fit into the template of an eyecatching headline, then they should be happy to see their artistic interpretations displayed alongside the original.
And of course it would be great if all journals moved quicker. But the media has not earned any privileged access to data; if anything it has abused it.
Ben Goldacre is a writer, broadcaster and medical doctor who works full-time for the National Health Service.
Note:
The subheader for this piece, not written by me I hasten to add, was, bafflingly, “Scientific papers should not be thrown open to the media, says Ben Goldacre, unless we want the public even more misled.” This is almost the polar opposite of what I believe. Papers should be thrown open to everyone, as I have argued on more occasions than I could possibly count, including at conferences dedicated to the question of open access academic journals: the point is that results should not be taken up by the media before a paper exists, for the reasons explained. Sigh. Gah. But if you demand to see the standfirst (that’s the poncy name for that bit) before the page goes then the editors and the subs all think you’re an arsehole. Life.
And that was in response to…
I’m posting this because otherwise you won’t know what I’m responding to, because Bob Ward is ex-head of communications at the Royal Society and his views deserve credit, and because I hate that thing where you can’t read the thing that some other thing is about.
We ought to get all findings out fast
Bob Ward
Published: 31 August 2007
www.thes.co.uk/search/story.aspx?story_id=2038066The public interest is not served by publishers sitting on the results of scientific research. It’s time for the practice to end, says Bob Ward
If you leafed through a newspaper or watched a TV news bulletin on July 16, the chances are that you learnt of research findings that linked the consumption of grapefruit with higher risks of breast cancer. It was typical of study results that are reported in scientific and medical journals on a weekly basis, the wide dissemination of which is clearly in the public interest. Yet why did the British Journal of Cancer wait almost a month after the paper was accepted for publication to release it to the media? The answer reflects the pervasive legacy of a marketing strategy devised by a journal editor who died 27 years ago.
Franz Ingelfinger was editor of The New England Journal of Medicine between 1967 and 1977. Two years into his editorship, he declared that he would carry only papers that had been “neither published nor submitted elsewhere (including news media and controlled-circulation publications)”. Although simply an articulation of practices already operated by some other journals, this statement soon gathered wide support among other editors and eventually acquired the name “the Ingelfinger rule”.
One recent study suggested that almost three quarters of major publishers now enforce the rule, and its central principle has been endorsed by bodies such as the International Committee of Medical Journal Editors. The two most prominent multidisciplinary scientific publications, Nature and Science, both incorporate the Ingelfinger rule in their instructions to authors, as do leading medical journals such as The Lancet.
In its most severe form, it means that a journal will refuse to consider a submitted paper if it has already received media coverage. At the very least, it means that an author is discouraged from talking to journalists about the paper before the journal’s own efforts to gain media coverage have begun.
Supporters of the Ingelfinger rule point out that it prevents the premature disclosure of research results before they have been scrutinised and approved for wider circulation by independent peer review. On the whole, this has its benefits for the public, even if peer review is not always effective at filtering out research that is poorly devised and executed, erroneously interpreted or even fabricated.
Whatever the advantages of preventing disclosure of results before they have undergone peer review, it is less clear what public benefit there is to delaying media coverage of a paper once it has been accepted for publication. Preparing a paper for appearance in a journal once it has cleared peer review depends on the volume of work and the resources available to the publisher. The length of delay also depends on the publisher’s schedule, which takes account of marketing priorities. Journals like to publish at regular intervals, with batches of papers of roughly the same size. An article can take longer to appear if it is stuck in a queue.
But another decisive factor is that most journals want to gain publicity for themselves through media coverage of the papers they publish. If an author talks to journalists about his or her paper once it has been accepted but before it has been published, there is a danger that the publication will not receive a credit in any resulting media coverage. And these days, many publishers believe that media coverage increases demand for their products and hence boosts their income.
As a result, there is often a gap of many weeks or months between the date on which a paper is accepted and the date on which the media are able to report its existence. In the case of research that might influence the behaviour of policy-makers, businesses or citizens, this delay might mean that decisions are made without crucial information that would be otherwise available if it were not for the marketing strategy of a journal. In such cases, enforcement of the Ingelfinger rule creates a conflict between the interests of the journal and the public.
It does not have to be this way. Members of the public do not need a paper to be in its final published form before they learn of its contents. Journal papers are often too technical and jargon-laden for a layperson and frequently omit a proper discussion of implications for the public. Journalists really need access only to the authors, and perhaps the manuscript, to prepare a full and accurate report. There must be a case for journals to set aside their marketing interests to promote the public interest by making manuscripts available as soon as they are approved for publication. This would finally loosen the steel grip that Franz Ingelfinger is still applying nearly three decades after his death.
Bob Ward is director of Global Science Networks at Risk Management Solutions. The views expressed are his own.
Nellie Dean said,
September 21, 2007 at 4:32 am
Bravo.
The dodgy Dr David Horrobin, the man who brought evening primrose oil to the world, was against peer review. Well he would be, wouldn’t he? He was also editor of a vanity publication called Medical Hypotheses (you paid through the nose to be published in it).
Re brain isze and gender: there’s a splendid book called The Human Brain in Figures and Tables by two Russian neuroscientists whose names I’ve forgotten, with this and other data in. It puts Paul Broca in his place.
Love, Nellie
Dr T. fortunei said,
September 21, 2007 at 8:29 am
To add to JohnD, (and to stay off topic for a while – sorry!)
Pay-for-publication is not the same as pay-to-avoid-peer-review. The open access movement is one that should be supported by all academics and adding the cost of publication in open access journals is increasingly an admissiable cost on grant applications to UK research councils as well as Charities like Wellcome. It’s a great thing, and should not be confused with vanity publishing.
www.plosjournals.org/perlserv/?request=index-html&ct=1
Andrew Clegg said,
September 21, 2007 at 8:51 am
‘To take the clipping, nod sternly over half moon specs and explain that newspapers are full of “lies” is inappropriate, dismissive and injurious to the doctor-patient relationship.’
I would actually respect a doctor much more if I brought up some half-understood piece of Media Science ™ in the surgery and he/she replied “no no, no offense but that’s actually bullshit”.
Andrew.
CommanderKeen said,
September 21, 2007 at 9:02 am
I’m reminded of this criticism of Bob Ward for demanding the censorship of (profoundly dodgy) science that he didn’t agree with.
commentisfree.guardian.co.uk/francis_sedgemore/2007/05/nobody_expects_the_scientific.html
Has he changed his mind over the rigour in which science should be presented to the public?
Steve Senior said,
September 21, 2007 at 10:56 am
Publication in the mass media before the peer-reviewed journals is a hallmark of crap science. The Pons & Fleischmann cold fusion debacle springs to mind. Also, once the media has whipped up a storm about something, it’s much harder to put it to bed – the cold fusion thing would have been over in a fifth of the time had it been fought out in academic journals. The trouble as I see it is that science is only really rational when considered as a whole. Much as we scientists like to think of ourselves as objective, dispassionate observers, it just doesn’t work like that, as anyone who’s ever undergone the process of having a manuscript reviewed knows. The whole thing isn’t helped by the big journals (you know who you are) screening manuscripts for ‘press releaseability’ before they even send them to peer review.
outeast said,
September 21, 2007 at 12:03 pm
Note that Ward does specify after acceptance for publication – so despite what some seem to be suggesting he is not advocating dissemination of findings in the media prior to peer review. That doesn’t undermine Ben’s points in the least, but let’s keep the criticisms of Ward on track eh? Where Ward seems to be going wrong is is giving too much credence to journalistic integrity and/or ability to grok science…
Ben Goldacre said,
September 21, 2007 at 12:06 pm
bob ward is sensible and sound, peer review is not the issue for the ingelfinger rule, it’s a question of whether the media should be allowed to publicise results before, or worst case instead of, the proper journal paper appearing.
speaking of which i should follow this up:
www.badscience.net/?p=159
i vaguely remember i promised to do it annually.
Steve Senior said,
September 21, 2007 at 12:29 pm
Outeast: Sorry, I wasn’t clear. I was referring to the actual emergence in print. Still, the delay can be very long, even to electronic publication. And as we know, a story can be picked up and repeated by any number of papers, news stations and blogs in a few hours.
manigen said,
September 21, 2007 at 3:11 pm
“…stories such as the media’s MMR hoax (as it will come to be known)…”
Certainly by me, at any rate.
Maybe there’s a good explanation for this that hasn’t occured to me, but why doesn’t medical science have an equivalent of arXiv? A comprehensive open access pre-pub archive would be helpful, right?
Mark Frank said,
September 21, 2007 at 3:30 pm
A couple of questions:
How much difference does availability of the full paper make to what the press write and the public hear? Look at what Melanie Philips wrote about the Cochrane report on MMR.
Do GPs really have the time to look up the original research when someone comes in waving a press clipping?
And a comment:
It is easy to blame the press but the scientific community needs to get its own house in order as well. It was Wakefield who decided to make misleading accouncements about MMR. It was the Uni of Utah plus Fleischman and Pons who did the same for cold fusion. The Ingelfinger rule didn’t stop either of them.
Ben Goldacre said,
September 21, 2007 at 3:38 pm
when mel p wrote ludicrous things about the cochrane report, because it was published, people could go and read it for themselves, double check, and establish that what she said was rot. that was only possibly because it was published and available. if she had written that before it was out, nobody could have called her out on it, not the public, not journalists, not academics, because the paper was not openly available.
re: blaming the “scientific community”. there will always be cranks and rightly so, sometimes they have excellent ideas, i would be horrified at any kind of central gagging system and i cannot imagine for the life of me what such a thing could possibly look like, or how it would work. it is no surprise there are odd cranks. what is wrong is that they are reported as if they represent a mainstream plausible opinion, as if their views represent a credible survey of the literature.
roueche said,
September 21, 2007 at 5:05 pm
Dr. Goldacre,
An interesting analysis, although as a journalist I don’t entirely agree that embargoes and the Inglefinger rule are good things.
But I have a question: If you believe that journalists should only report on peer-reviewed published papers that are available to everyone, would you then ban journalists from covering medical conferences?
Virtually none of the presentations at conferences are peer-reviewed, and except for societies that publish un-peer-reviewed brief abstracts, the papers presented are not widely accessible.
Ben Goldacre said,
September 21, 2007 at 5:11 pm
yes. i think covering unpublished uncompleted research from conferences is a very significant risk factor for reporting inaccurately, and on a turkey.
that bloke ioannidis’ think pieces and other research on this are v interesting, nice brief summary here, more in eg ioannidis’ plos medicine article:
online.wsj.com/article/SB118972683557627104.html
rob said,
September 21, 2007 at 6:28 pm
There must be a middle way in this, I’m sure. Many branches of science have ‘pre-print’ servers, on which papers are posted prior to being published (but often after peer review). If a peer-reviewed but awaiting-publication paper is posted on such a server (which would normally be free to access, unlike many journals) then surely nothing should stand in the way of press coverage.
roueche said,
September 21, 2007 at 6:32 pm
Newspapers have been described as “the first draft of history.” You wouldn’t prohibit journalists from covering political and economic developments because historians years hence may have different, perhaps more accurate descriptions of and interpretations of the events.
In that spirit, conference coverage could be described as “the first draft of science.” Indeed, that’s the motto of the International Medical News Group (a division of Elsevier), which publishes more than a dozen trade papers for specialist physicians (i.e. Family Practice News, Clinical Neurology News, etc.)
You could argue that these publications are targeted to highly educated physicians, who presumably understand the tentative nature of scientific findings. But why hide these stories from the general public, assuming that the articles are accurately written (a big assumption, I know) and that they don’t imply that the latest finding is the last word on the subject.
pv said,
September 21, 2007 at 10:59 pm
If journalists write about unpublished or uncompleted research why is it seemingly so difficult for them to actually point out that it is neither published nor completed and say what the ramifications of that are?
Do the public in general know the difference between published and unpublished research? Probably not, but aren’t journalists and editors actually relying on them not knowing the difference?
buffalo66 said,
September 22, 2007 at 9:13 am
@ roueche (#16): “But why hide these stories from the general public, assuming that the articles are accurately written (a big assumption, I know)”
But that’s just the problem – the articles are often NOT written accurately. Look at how the person writing the subheader for Ben’s THES article got the message entirely wrong. There are loads of examples on badscience of journalists getting the story completely wrong. If the full scientific paper is available at the time that the story appears in the media, then any misreporting can be detected and followed up immediately. This can’t happen if the paper slips out quietly a few months later, after everyone has stopped talking about it.
And what about cases where the story is reported accurately, but the science is dodgy? If the full scientific paper is available at the time that the story appears in the media, then the science can be subjected to full scrutiny at that time. The thing that really convinced me that it was safe for my daughter to have MMR was reading Andrew Wakefield’s original paper, and seeing how unconvincing his arguments were. If scientific papers are available at the time that a story appears in the media, it protects the public from bad reporting, AND bad science.
buffalo66 said,
September 22, 2007 at 1:45 pm
Even without the malign influence of some newspaper proprietors, newspapers and other media will inevitably be full of inaccuracies because they are usually only interested in something for a short period after the story breaks, before the full facts have emerged. With science reporting, it doesn’t have to be like that. If journalists could just wait until a paper is published, then all the facts would be sitting on a plate, ready to be consumed by anyone who’s interested.
roueche said,
September 22, 2007 at 7:26 pm
Newspapers report news.
When a scientific finding is announced publicly, at an open scientific meeting, a press conference, or publication in a peer-reviewed journal, that event is news.
I think it’s unreasonable to ask science journalists to wait to some time in the perhaps distant future, after the full paper is written, peer reviewed, rewritten, published, and fully available, to report the news.
Crime reporters aren’t asked to wait to report a crime until after the accused’s trial is completed and the transcript is available for full scrutiny.
Political reporters aren’t asked to wait to report on legislation until the final version of the law is passed and signed.
On the other hand, it’s not unreasonable to insist that journalists report scientific news as accurately as limited time and space allow. For example, they should quote impartial experts who can put the research into context and apply the necessary caveats.
buffalo66 said,
September 22, 2007 at 10:14 pm
I agree it’s difficult to expect journalists to wait before reporting things. When a story breaks, we all want to read about it, and journalists are just satisfying that demand. I’m not sure what the solution is, but I think we’d be better off if there was less of a rush to report the news. As it is, a large proportion of the news is just speculation about what may be happening, and about what decisions may be about to be made.
I wouldn’t say a crime reporter should wait until a trial is completed before reporting it, but it might be beneficial to wait until it’s started, because that’s when the evidence begins to receive scrutiny. Until then it’s just speculation. The results in a scientific conference presentation or press release can’t receive that level of scrutiny, because the details are not known until the paper comes out.
roueche said,
September 23, 2007 at 12:52 am
Buffalo66 (and others),
I think that if you argue that that conference presentations shouldn’t be reported until the paper comes out, you also have to argue that scientists should refrain from presenting at conferences until their paper is published.
After all, even if no reporters are present, a 15-minute conference presentation in advance of a published, peer-reviewed paper leaves even a professional audience without the full details they need to scrutinize the research adequately.
So it seems to me that to be consistent, you’d have to argue that about 99% of the presentations at about 99% of the scientific and medical conferences should never be delivered, or that all presentations should be lengthy recitations containing at least as much detail as in the published paper.
Do you really want to argue that?
buffalo66 said,
September 23, 2007 at 12:41 pm
I think it’s justified to make a brief conference presentation to a professional audience because they know they should reserve judgement until the paper comes out. Conference presentations serve a number of purposes:
1. They let scientists know what the current hot topics are.
2. They give socially challenged scientists things to talk to each other about at conferences, helping them get to know each other.
3. They provide an advert for the speaker’s work, like a movie trailer. Last time I presented something at a conference, several people expressed an interest, and I gave them a draft of the paper, containing most of the details that will be in the published version. This arrangement benefits everybody.
But look at what happens when stories appear in the media claiming that, for example, MMR is dangerous: if the work has not been published, then the story can’t be refuted, even if it later turns out to be rubbish. But, by then, the damage has been done. A few more parents fail to have their kinds vaccinated. The measles rate goes up.
Steve Senior said,
September 23, 2007 at 6:13 pm
On the subject of reporting on data shown at conferences, at the annual meeting of the society for neuroscience in Washington DC two years ago, I saw a poster called “Ihibitory effects of blueberry polyphenols on the production of proinflammatory mediators in activated microglial cells” and sponsored by the Wild Blueberry association of North America and the US highbrush blueberry council.
Since then I’ve found it hard to place too much faith in things reported at conferences. Not that conferences are a bad thing (expenses paid jollies are all the holidays some scientists get). But only a small proportion of posters and talks given at a conference are ever turned into published articles.
scherers said,
September 29, 2007 at 3:29 pm
As an interesting example about the dangers of reporting on research results that have not yet passed peer review, it seems that the New Scientist has an increasing problem of credability among professional physicists, see e.g.
golem.ph.utexas.edu/category/2006/09/a_plea_to_save_new_scientist.html
www.sunclipse.org/?p=296
golem.ph.utexas.edu/category/2007/09/the_virtues_of_american_scient.html
The reason seems to be that they often run feature stories on research reported about in the arxiv preprint server, which not always will make it, at the end of the day, through peer review.
diudiu said,
December 21, 2009 at 6:32 am
free shipping ugg
free shipping ugg