Ghostwriters in the sky

September 18th, 2010 by Ben Goldacre in bad science, big pharma, ghostwriters, regulating research | 42 Comments »

Ben Goldacre, The Guardian, Saturday 18 September 2010

If I tell you that Katie Price did not, necessarily, write her own book, this is not a revelation. From academics I have slightly higher expectations, but now the legal system has spat out another skip full of documents: this time, we get a new insight into the strange phenomenon of medical ghost-writing.

Attributed authorial assistance is one thing. This is different, and more cynical. A commercial medical writing company is employed by a drug company to produce a programme of academic papers that can be rolled out in academic journals to build a brand message. After copywriters produce the articles, in collaboration with the drug company, to their specifications, the ghostwriting company finds some academics who are willing to put their names to them, perhaps after a few modest changes.

The latest documents come from a court case brought against Wyeth by around 14,000 patients who developed breast cancer while taking their hormone replacement therapy, Prempro. The open access journal PLoS Medicine, acting with the New York Times, argued successfully in court that 1500 documents from the case which detailed the ghostwriting should be placed in the public domain, because they represent important information on a potential threat to public health. Now, PLoS has published the first academic analysis of these documents, which is free to access online.

HRT, we should remember, has had a rocky history. Initially the panacea to all ills, by 1998 the HERS trial showed it didn’t prevent cardiovascular events after all, and by 2002 the Womens Health Initiative trial showed it also increased the risk of breast cancer and stroke. We now know it increases the risk of dementia and incontinence. Survey data shows that even today, many gynaecologists continue to have beliefs about the efficacy of HRT that are in excess of the evidence. Reading how the literature was engineered, it’s not hard to see why.

The company DesignWrite boasts that over 12 years they have “planned, created, and/or managed hundreds of advisory boards, a thousand abstracts and posters, 500 clinical papers, over 10,000 speakers’ bureau programs, over 200 satellite symposia, 60 international programs, dozens of websites, and a broad array of ancillary printed and electronic materials”. They proposed a “planned publication program” to Wyeth, consisting of review articles, case reports, letters, editorials, commentaries and more, using the medical literature as a marketing tool.

DesignWrite wrote the first drafts, and sent them to Wyeth, who then advised on the creation of a second draft. Only then did the paper get sent to the academic who would appear as the “author”.  Review articles cost Wyeth $20,000. Abstracts are $4,000. The academics weren’t paid cash, but they did get an easy publication in an academic journal for their CV. And once the publication process was in train, the chap from Wyeth’s marketing department helpfully provided comments and suggestions for the authors to use in response to peer reviewers’ comments.

The PLoS documents show DesignWrite sold Wyeth more than 50 peer reviewed journal articles for HRT, and a similar number of conference posters, slide kits, symposia, and journal supplements. The analysis in PLoS (by an academic who appeared as a paid witness against the company in court) found that these publications variously promoted unproven and unlicensed benefits of Wyeth’s HRT drug, undermined its competitors, and downplayed its harms.

You might imagine there are rules against this. There are not: there are traditions, good faith, and leaky regulations. It’s illegal, for example, for a pharmaceutical company to promote its drug for “off label” use, which means selling it to treat a medical problem for which that drug has no license. In the case of Wyeth’s HRT drug, that meant they couldn’t market it for preventing Alzheimer’s, Parkinson’s, and wrinkles, to name but a few. The PLoS analysis finds that many articles produced by DesignWrite for Wyeth promoted the drug’s use for exactly these conditions: but academic journal publications are not regarded as promotional activity, so this was all legal.

Worst of all is the complicity of the academics, and in very large numbers. There is no possible way they could persuade themselves that what they were doing was correct. “Research shows high clinician reliance on journal articles for credible product information,” said DesignWrite, in their initial pitch. They’re right, and that’s for a reason: when you read an academic paper, you trust it was written by the person whose name is on it.

There are very simple solutions. If a commercial writer employed by a pharmaceutical company has the idea for a paper, and writes it, then their name and their company’s name should be on the paper. If the first, last, and corresponding authors on a paper didn’t write or lead on it, they should say so, loudly and clearly. Universities, which are currently inconsistent on ghost writing, could take a lead, but they don’t, and so these serious problems in academia will persist, because they are slightly complicated, and hidden from public scrutiny. That’s why you should read about them in PLoS, talk about them, crane your neck over, scratch your chin, and mutter in astonishment. Nobody in a regulatory role is interested. Our only hope is the power of shame.


++++++++++++++++++++++++++++++++++++++++++
If you like what I do, and you want me to do more, you can: buy my books Bad Science and Bad Pharma, give them to your friends, put them on your reading list, employ me to do a talk, or tweet this article to your friends. Thanks! ++++++++++++++++++++++++++++++++++++++++++

42 Responses



  1. pterid said,

    September 18, 2010 at 1:10 am

    Excellent article – also, hope to see you tomorrow :)

  2. Sqk said,

    September 18, 2010 at 1:28 am

    Well, I won’t be there tomorrow. Campaign for me!

    In the meantime, although it appeared recently, I feel a reminder is in order: www.newscientistjobs.com/jobs/job/creative-medical-writer-healthcare-advertising-london-london-1400883736.htm

    You can’t say they’re not at least blatant about it!

    And on a lighter note, this is the version I was raised on: www.youtube.com/watch?v=9CPqFuBiwV4

    …which is obviously the more cultured version as these were the guys who wrote ‘Flower of Scotland’.

  3. twaza said,

    September 18, 2010 at 8:14 am

    on the button — the article, and Peggy Lee!

  4. drunkenoaf said,

    September 18, 2010 at 8:56 am

    Naughty Wyeth, and there’s the similar story with AZ, ximelagatran, and a company called Rx Communications, which when the story broke (best part of a decade ago), ended up with GPP guidelines being introduced: www.gpp-guidelines.org/ — which should (and has, mostly) stop(ped) doctors’ names being added to the top of completed manuscripts.

    Added to journal rules — www.icmje.org– these days, you’d have to have a bunch of very naughty people all round to do that these days.

    Having said that, pharma companies still have publications plans and marketing messages… but at least it has to bear a passing resemblance to the data, else (rightly) a tonne of bricks will fall on them rapidly, or if not rapidly, eventually.

  5. Andrew Clegg said,

    September 18, 2010 at 10:30 am

    I think the peer review process at the journals in question must take at least *part* of the blame for failing to distinguish adverts from decent research.

    In an ideal imaginary world, it shouldn’t matter whose name is on the paper, the reviewers should have accepted it or rejected it based on its merits (or lack of) alone.

  6. DTL said,

    September 18, 2010 at 11:38 am

    Well, that’s nicely depressing. Another reason why this publish or perish thing in academia is nonsense – there’s so much pressure that agreeing to ridiculous things like this starts to seem a reasonable idea.

  7. jodyaberdein said,

    September 18, 2010 at 1:07 pm

    So for our journal club a few weeks ago I did the recent Lancet paper on bocapravir for hepatitis C (paywalled):

    www.thelancet.com/journals/lancet/article/PIIS0140-6736(10)60934-8/fulltext

    Which at the end says:

    ‘We thank the patients for their participation and support during this trial; and David McNeel and Holli Brousseau (medical writers, Sui Generis Health New York, NY, USA) for their assistance with report preparation (funded by Schering, a Division of Merck).’

    And after a bit of googling you can find a job advert

    www.mediabistro.com/publicityclub/joblistings/jobprint.asp?joid=97951&page=12

    Not saying anything at all, just piqued my interest.

  8. Dan Kimberg said,

    September 18, 2010 at 2:38 pm

    There may not be rules against what DesignRight and Wyeth do, but there are often rules against what the nominal authors of these articles have done. Misrepresenting authorship is identified as a mode of research misconduct at my institution, and I’d certainly be fired if I were discovered doing that (tenured faculty might fare better). It’s certainly acknowledged that our system for scientific reporting depends on the honesty of scientists, and everyone involved in this kind of nonsense is at best behaving unethically.

    It should be noted that peer review as a system has tremendous weaknesses, and has never been understood as a safeguard against fraud and dishonesty. It’s a safeguard against incompetence, incompleteness, accident, etc. If the scientific merit of the article, as written, were the only issue, then of course the review process would be partly to blame. But that’s not the issue at all. The issue is dishonesty, and the specific reason why Wyeth wants to hide its name is because it has conflicts of interest that often run into billions of dollars. So I don’t blame the review process. But if you want to, then it absolutely does matter to the review whose name is on the article. And I’m not going to fault a journal for failing to detect a fraudulent practice that’s backed by companies that probably spend more on their restrooms than the average journal’s annual budget.

    For other reasons why peer review is unhelpful, see this:

    physicsworld.com/cws/article/news/43691

  9. Dr Aust said,

    September 18, 2010 at 3:28 pm

    The example Jody Aberdein gives from the Lancet is probably telling us that the Lancet has some kind of formal checklist for authors requiring such “assistance” to be declared. Which is at least something – if ghost writers paid by a PharmaCo wrote the paper, then we should be told that that is what happened.

    In an interesting comment at the Guardian online version of Ben’s article, “DianthusMed” (a medical writer) notes that few journals or publishers do this, even though the medical writers themselves have suggested in print that it would be a way forward – s/he cites this article in PloS Medicine.

    One of the points of such checklists would be that the academics authoring articles would have to declare formally that they had taken ghost-writing assistance – or they would have to lie specifically by omitting to mention it. This would at least have the effect of meaning that, if some omissions came to light later, they could not possibly say

    “But I didn’t understand this was a problem/know there was a rule about declaring this”.

    This in turn would make it far harder for them to avoid disciplinary penalties (as well as loss of reputation) if “busted”. In this sense it would be slightly analogous to one of the things most UK Universities now do about student plagiarism, getting students to sign formal written undertakings that they are clear what plagiarism is, that they understand it is unacceptable and a serious offence, and that they won’t do it.

  10. Errodiel said,

    September 18, 2010 at 4:38 pm

    I’m reviewing a paper at work at the moment that openly admits to using a medical writing company to assist in production of the paper, and quite frankly even that much makes me edgy.

    And it also makes me wonder, because the paper’s not brilliant – are these people worth the cash? How bad would it have been otherwise?! It’s a clinical trial for a drug likely to be approved by the EMEA soon, which really makes me hope that they’re aware of these kinds of practices, and thorough with their evidence sifting.

  11. minsx02 said,

    September 18, 2010 at 5:19 pm

    I looked at the report breifly, but what I was hoping for wasn’t easy to find: a list of the “professional” scientists who published in this manner. It’s hard to put shame on persons you don’t know, but I’d be willing to write a letter to the dean and president of any college or university in my area who employs these individuals.

  12. HungryHobo said,

    September 18, 2010 at 10:48 pm

    Browsing the documents, I can’t find any definitive list of doctors who took part either.
    There’s publications here:
    dida.library.ucsf.edu/search?query=dt%3Apublication&ct=1
    but I’m not sure which are fraudulent and which are merely papers used by one side or the other in the court case.

    One really creepy thing I found:

    “List of doctors — Neutralize/discredit”
    dida.library.ucsf.edu/pdf/oxx10u10

    From my reading it’s a list of doctors critical of them and it suggests ways to “Neutralize” them by buying them off, influencing them or discrediting them.
    For each person it has lists of options like buying them dinner, inviting them to conferences, inviting them to speak, grants etc etc

  13. jimjim237 said,

    September 19, 2010 at 4:02 pm

    I notice that the Police and the Crown Prosecution Service (I suppose it’s not a private prosecution) have been moved to charge a County Cricketer with conspiracy to defraud for which the maximum sentence is ten years. His “crime” was apparently not playing with his best effort for a few minutes of a cricket match.[1]

    I find it strange that in other more egregious circumstances, and I think that the events described in Ben’s article can be so described, no prosecuting authority is interested.

    I wonder why such a charge could not be laid in the cases described here by Ben, or against large portions of the Complementary (oops I mean Alternative) (oops I mean Integrated) medicine industry?

    The profits of bookmakers seems to be held more important to a substantial degree than peoples health. I suppose I should not be surprised.

    [1] www.telegraph.co.uk/sport/cricket/news/8006208/County-cricketer-charged-with-fraud-for-playing-poorly-deliberately.html

  14. Lois said,

    September 19, 2010 at 5:29 pm

    Great article. Senator Grassley has been a busy fellow in the US.
    Grassley’s focus upon journals came on the tail end of writing the pharmaceutical giant, Wyeth, and DesignWrite, a medical education and communications company, regarding allegations Wyeth hired DesignWrite to draft articles (ghostwrite) promoting the company’s hormone products and seeking academic investigators to sign on as the primary authors.

    truthaboutheartdisease.org/sen-grassley-probes-medical-conflicts-of-interest/

  15. MedicLewis1 said,

    September 20, 2010 at 10:39 am

    I have an alarming anecdote to share on this subject. During a pharmacology problem based medicine session last year a fellow Newcastle MBBS student was bragging how her father was constantly being offered £20k for being such a ghost writer. The young lady in question thought her father a fool. Lets just hope that adequate legislation prevents this soon and that she was joking.

  16. AdamJacobs said,

    September 20, 2010 at 11:40 am

    Ghostwriting is undeniably a bad thing, and when it distorts the science in biomedical publications, it is a particularly bad thing.

    When faced with stories of it, those working in the world of clinical research seem to divide into 2 groups in the way they respond:

    1. Take practical steps towards making it harder to get away with unethical practices.

    2. Write articles full of righteous indignation about how terrible it all is, while not actually doing anything practical to stop it happening again (presumably so that it doesn’t spoil the fun of writing further righteously indignant articles in the future).

    I like to think of myself as being in the first category. I have been working towards discouraging ghostwriting. For examples see here and here. There is a limit to what one person can do, of course. No one person or organisation can put everything right overnight, but many small steps by different people can add up to a big effect. I hope some of the things I have done contribute to that.

    Now, Ben is a well respected and influential chap. I hope he also considers himself to be in group 1, or at least that even if he isn’t in group 1 at the moment, he’ll be willing to join us there. I’m sure he could make a difference.

    Ben, if you’re reading this, I’m sure there are many things you could do, but one thing immediately springs to mind. I assume you’re a member of the BMA? Do they have any policies prohibiting members signing their names to ghostwritten articles? If not, perhaps you could start discussions with them to see if such a policy could be forthcoming?

  17. AdamJacobs said,

    September 20, 2010 at 12:35 pm

    @Dr Aust

    Thanks for pointing to my comment on the Guardian website. I think you make my points far better than I did myself!

    Adam aka dianthusmed

  18. Jason Loxton said,

    September 20, 2010 at 6:06 pm

    While not questioning her credibility, I note that the sole author of the PLoS article in question is a paid expert witness in the lawsuit against Wyeth. This would certainly be worthy of comment if the conflict ran the other way. Somewhat odd that it gets no mention here.

    “Competing interests: Dr. Fugh-Berman was a paid expert witness on behalf of plaintiffs in the litigation referred to in this paper. She was not paid for any part of researching or writing this paper.”

  19. Ben Goldacre said,

    September 20, 2010 at 6:19 pm

    hi there, i said she was a witness, thinking i’d made that clear, but youre right i didn’t explicitly state she was a paid expert witness, happy to add that in above, done, thanks.

  20. Dr Spouse said,

    September 20, 2010 at 9:53 pm

    I have come across papers in non-medical sciences where the “author” didn’t write it – one springs to mind that had an acknowledgement reading “thanks to X for collecting the data and Y for preparing the manuscript.” I’m assuming X and Y were research assistants rather than commercial paid writers, but this was probably a 20-year-old study, if not more.

  21. AdamJacobs said,

    September 21, 2010 at 9:48 am

    If we’re going to look at Dr Fugh-Berman’s competing interests, then the fact she’s acted in lawsuits against Wyeth is only part of the story. She is also the director of an organisation called PharmedOut, which has an explicit anti-pharmaceutical-industry agenda.

    To be fair, she did disclose that in the paper, although it’s something Ben neglects to mention in his article above.

    BTW, Ben, any answer to the questions I asked at comment #16?

  22. MedsVsTherapy said,

    September 21, 2010 at 3:36 pm

    Thanks, Dr. G, for continuing to follow this and highlight this! -I believe I have commented here before on this issue, with my point of view that truly is a sticky wicket: I believe two parties should be free to enter into a contract together, and identify their product however they want. Consider a celebrity or designer product: Brittny Spears, I am sure, did not gather a bunch of botanicals on her own, then go find a perfume company to develop her perfume in bulk, and distribute through their network. No – the perfume company sought out Ms. Spears, gave her a few choices, and said, “pick one.”

    Now, all can declare that Ms. Spears developed her own perfume, and whoever is free to sell it.

    No one is really fooled.

    Same with ‘Famous Amos’ cookies. Amos did not experiment in his kitchen with BHA and BHT. Keebler basically bought his image and blessings.

    Two parties entering into contract together, that’s all.

    Ghost-writing is the same. A firm makes up a study, and gets an endorser to promote the study. No big deal.

    Intellectually, this might hail back into patent history.

    So, the challenge for those of us who believe that science should not work like this is to figure out a system of check-and-balances.

    This is ALL in the private world. Public policy has limimted efficacy. How can “science” put any punishment on a researcher or a journal?

    HRT is an awesome textbook example. Wyeth can market however they want (a friend was an HRT peddler up til 2002 – I have heard the inside story).

    On this side of the pond, a doc can get sued for malpractice. But with BOTH the articles AND practice patterns on the side of the doc, this would be a losing battle.

    “Class Action” suit has to put burden on Wyeth. But Wyeth, if they still existed, could duck this charge by pointing out that the researcher stuck their name on the study, regardless of financial arrangement. That sounds like enough for a team of lawyers to work with FTW.

    And “ghostwriting” of sorts continues in the bench sciences as post-docs slave away, only to see their boss appear as first author. Everyone knows this, but it is a part of the indentured servitude status quo, with little recourse.

    So, these things happen regularly, without mechanisms in place to provide any sort of accountability or check. This is all simply a state of affairs in the private, not public, world of science and scientific publishing, where we hope we are all on our best behavior, but there is no beat cop to lock us up for disorderly conduct.

    Private mechanisms have “advanced.” As years have gone by, I have had to sign ever-more strongly worded statements that I and other authors contributed meaningfully, and “vouch” for the results. Sure – great mechanism there: “I promise I earned authorship. No, I REALLY promise.”

  23. jonathanincheshire said,

    September 21, 2010 at 3:41 pm

    I’m a big fan of Ben but in this particular piece I think he should have made a few things clear:

    1) The activities described in Adriane Fugh-Berman’s paper occured between 1999 and 2004 and are not necessarily a reflection of current practice.

    2) This kind of behaviour is hardly new news (Aubrey Bluhmson has an excellent blog at scientific-misconduct.blogspot.com/ that has other examples. JAMA also published a similar article about Merck and Vioxx see:
    Guest authorship and ghostwriting in publications related to rofecoxib: a case study of industry documents from rofecoxib litigation.
    Ross JS, Hill KP, Egilman DS, Krumholz HM.
    JAMA. 2008 Apr 16;299(15):1800-12

    3) You will be hard pressed to find a pharma company that does not now follow GPP2. GPP2 is explicit in which the way in which professional medical writers should be acknowledged in scientific manuscripts. See:
    Research Methods & Reporting. Good publication practice for communicating company sponsored medical research: the GPP2 guidelines.
    Graf C, Battisti WP, Bridges D, Bruce-Winkler V, Conaty JM, Ellison JM, Field EA, Gurr JA, Marx ME, Patel M, Sanes-Miller C, Yarker YE; International Society for Medical Publication Professionals.
    BMJ. 2009 Nov 27;339:b4330. doi: 10.1136/bmj.b4330

    It is thanks to the work of the likes of Ben Goldacre and others that industry has been forced to change its behaviour for the better. Organisations such as TIPPA and ISMPP are now in place to develop guidelines and encourage good publication practice throughout the industry. A scan of industry sponsored articles published in recent years will show a degree of transparency and disclosure not seen in the years 1999-2004. Is it perfect? Probably not – but things are continually changing and improving. So, please Ben, keep up the pressure and the good work but at the same time show a little more balance in your writing or you might find yourself accused of the bias you so often accuse indutry of.

  24. MedsVsTherapy said,

    September 21, 2010 at 3:44 pm

    “If we’re going to look at Dr Fugh-Berman’s competing interests, then the fact she’s acted in lawsuits against Wyeth is only part of the story. She is also the director of an organisation called PharmedOut, which has an explicit anti-pharmaceutical-industry agenda.
    To be fair, she did disclose that in the paper, although it’s something Ben neglects to mention in his article above.”

    Sure, mention it. But skepticism is a hallmark of science. It doesn’t matter WHO the skeptic is.

    You declare your hypothesis, and you test operationalize it and test it. HRT relieves some unknown menopausal symptoms, and does not give you breast cancer – well, the burden is on you to give evidence, and when you agreed to jumped into this game, you agreed that fair criticism from ANYONE – paid or not, is the rules of the game.

    The COI/payola/deception is a big problem on the claimer side, not the critic side.

    (BTW; same with AGW: so what if the oil companies are paying me to criticise Michael Mann’s crappy 1998 principal components analysis: it is still a crappy principal components analysis.)

  25. Squigg said,

    September 21, 2010 at 7:28 pm

    Relax, it’s not as bad as you think!

    I’ve been a medical writer for 8 years and I think our work generally ensures the text of a manuscript is accurate, well-referenced and unambiguous. In addition, in a field where English is the common language, many doctors for whom English is not their first language appreciate some editorial assistance as it can help clarity and level the playing field.

    I believe that a company that has invested 100s of millions of pounds in a trial has the right to ensure the results are presented accurately in well-respected journals or conferences. The company’s short hand for this is ‘publication planning’. If it’s done correctly, this will lead to good sales of the product and more money going into clinical research. There’s no shame in this and is the only way in which doctors get their tools for treating disease.

    In general, the medical writer, together with the authors, develops the outline. The medical writer then drafts the text according to the authors’ instructions and then the authors make multiple reviews. The pharmaceutical company usually never sees it until the final draft.

    As other commentators have said, the GPP guidelines encourages full disclosure at the end of the article. Modern medical writing is not ‘ghost-writing’.

    A doctor may be a very good clinician but that doesn’t mean he or she is comfortable with writing and referencing a paper. Although most authors are meticulous and always mindful of evidence-based medicine, some clinicians will want to include personal observations, opinions or even very promotional statements (e.g. ‘the medication is safe and effective’) and it is our job to say ‘hang on, what’s the evidence?’.

    Moreover, it is a mistake to think that a medical writer working in an agency paid by a pharma company will definitely be more biased than an author whose research has the same source of funding and who will personally receive much greater sums of money.

    What if there’s the later data show a safety concern? Well, of course then it’s the pharma company’s responsibility to present those data clearly too. Now, there are many issues here with disclosure of data etc, but to blame the medical writer is to pick a scapegoat. Safety signals need to be detected through statistical analysis of long-term data, usually conducted by the pharma company. Providing investigators with no editorial assistance would not improve ability to identify concerns.

    I do think that there is a problem with the terminology used. The term ‘medical writer’ and ‘publication planning’ artificially inflate the role of the agency – like most sales speak, they’re not accurate. The medical writer is an editorial assistant – they don’t have substantial input. Similarly,‘publication planning’ is misleading in that it suggests that the pharma company has more control than it does in reality.

  26. AdamJacobs said,

    September 21, 2010 at 8:24 pm

    @MedsVsTherapy

    I agree with you that the competing interests are no big deal. It’s far better to look at what Fugh-Berman is saying than to assume that competing interests automatically invalidate it. But since others brought it up, I thought I’d mention that quite a significant one had been missed out.

    For the record, I don’t doubt the conclusion that some of the HRT papers published 10 years or so ago were dodgy. What I do object to, however, is the unwarranted generalisation from that to conclude that all medical writers are therefore evil. That’s simply not backed up by the evidence.

    If you go to the article on the PLoS Medicine website (Ben has helpfully provided a link) and look at the comments on the article, you’ll see some much more detailed thoughts from me, together with references to research studies to back up what I say.

  27. The Mentalist said,

    September 22, 2010 at 12:24 am

    Not remotely surprised at any of this. I wrote a long email to you about stuff connected with this issue, and I am glad to see that you are maintaining your gaze upon these unscrupulous quacks. I hope that your readers help others to joint the dots up – so much rests upon the general public thinking that they aren’t qualified to understand or question academic research papers on the sciences… How patronising to the average Joe – sorry mate; you have to take what we say as God’s honest truth. Because you are too thick, too busy or too superficial to create a counter argument. After all; you can trust us – we’re academics being paid by doctors! ‘Nuff said…

  28. brookster said,

    September 22, 2010 at 10:13 am

    HRT is a funny old topic: I’ve read as much literature saying that the benefits outweigh the risks, as the risks outweigh the benefits.

    But I’ve read a lot of criticism of the Womens Health Initiative trial, in terms of its study design and its emphasis on relative risks (rather than absolute risks) that were statistically weak.

  29. brookster said,

    September 22, 2010 at 12:04 pm

    Right, so the PLoS article says HRT “increased the risk of breast cancer”; a view repeated by Ben. It then cites two JAMA papers. So I find the original papers.

    The first one (Rossouw et al (2002)) says: “The 26% increase observed in the estrogen plus progestin group almost reached nominal statistical significance”. So, in other words, it wasn’t statistically significant.

    The second one (WHI Steering Committee (2004)) states in the abstract: “A possible reduction in breast cancer risk requires further investigation.”

    I haven’t worked in research for a while, but I’m scratching my head as to how one can conclude from this that HRT causes increased breast cancer.

  30. jodyaberdein said,

    September 22, 2010 at 8:17 pm

    You could look at the actual WHI estrogen plus progestin paper:

    Chlebowski RT, Hendrix SL, Langer RD, et al. Influence of Estrogen Plus Progestin on Breast Cancer and Mammography in Healthy Postmenopausal Women: The Women’s Health Initiative Randomized Trial. JAMA. 2003;289(24):3243-3253.

    Not firewalled, pdf availabe here:

    jama.ama-assn.org/cgi/citmgr?gca=jama;289/24/3243

    From the abstract 16608 women randomized, with 245 vs 185 total cases of breast cancer, p=0.003. I’ve not read the detail mind.

  31. brookster said,

    September 22, 2010 at 9:54 pm

    @jodyaberdein
    Thanks for the link; I’ll try and read that later.

    My point was that the papers cited in the report don’t support the conclusions ascribed to them.

  32. Guy said,

    September 23, 2010 at 8:31 am

    Jody, your link is to the citation,
    the download is
    jama.ama-assn.org/cgi/reprint/289/24/3243?maxtoshow=&hits=10&RESULTFORMAT=1&author1=+&andorexacttitle=and&andorexacttitleabs=and&andorexactfulltext=and&searchid=1&FIRSTINDEX=0&sortspec=relevance&volume=289&firstpage=3243&resourcetype=HWCIT

    Interesting- as the number needed to treat for a year to produce one invasive breast cancer is 1898. Hardly a horribly dangerous drug. 1898 women not getting hot flushes vs one invasive breast cancer is difficult to equate.

  33. brookster said,

    September 23, 2010 at 2:07 pm

    @jodyaberdein

    The paper you refer to is the follow-up to the 2002 study that the PLoS report references.

    In the follow-up paper, the statistically insignificant relative breast cancer risk of 1.26 falls to 1.24.

  34. jodyaberdein said,

    September 24, 2010 at 12:07 am

    OK, so the preliminary data were published in 2002:

    Writing Group for the Women’s Health Initiative Investigators. Risks and Benefits of Estrogen Plus Progestin in Healthy Postmenopausal Women: Principal Results From the Women’s Health Initiative Randomized Controlled Trial. JAMA. 2002;288(3):321-333.

    Not paywalled, available here

    jama.ama-assn.org/cgi/content/full/288/3/321

    This trial recruited through to 1998 and planned a mean 8.5 year follow up.

    It published in 2002 because it was stopped early due to increased breast cancer deaths on the weighted interim analysis.

    The above significance quote in context is:

    The 26% increase (38 vs 30 per 10 000 person-years) observed in the estrogen plus progestin group almost reached nominal statistical significance and, as noted herein, the weighted test statistic used for monitoring was highly significant.

  35. jodyaberdein said,

    September 24, 2010 at 12:08 am

    sorry not deaths, cases, my bad

  36. brookster said,

    September 24, 2010 at 2:41 pm

    @jodyaberdein

    I’m still none the wiser. Are they saying they took a statistically insignificant increased risk and did some more statistics to make it significant? Which sounds methodologically suspect.

  37. MedsVsTherapy said,

    September 24, 2010 at 3:03 pm

    Right, so the PLoS article says HRT “increased the risk of breast cancer”; a view repeated by Ben. It then cites two JAMA papers. So I find the original papers.
    Brookster said: “The first one (Rossouw et al (2002)) says: “The 26% increase observed in the estrogen plus progestin group almost reached nominal statistical significance”. So, in other words, it wasn’t statistically significant.
    The second one (WHI Steering Committee (2004)) states in the abstract: “A possible reduction in breast cancer risk requires further investigation.”
    I haven’t worked in research for a while, but I’m scratching my head as to how one can conclude from this that HRT causes increased breast cancer.”

    This blog post is awesoem, and the discussion is awesome. Sadly, in science, we don’t quite have an official format for these types of discussions.

    –Regarding breast cancer in WHI: These types of studies have their a priori hypotheses to be tested, but they also have a component of study design that monitors for adverse events – i.e., everyone believes HRT will lead to heart health, but let’s monitor other health events between treatment and control to see if there is some unanticipated harm.

    This is formally done by the “DSMB” data safety monitoring board. For WHI, this was contracted to a group including Janet Wittes and her firm
    www.statcollab.com/people/janet-wittes.html

    While in an RTC you are not allowed to peak at the data until it is all in, the DSMB does – to determine if harm is happening.

    The DSMB sets up, a priori, the kill switch. Criteria for pulling the plug on an intervention trial are different – philosophically entirely different – from the criteria for considering an outcome to be “statistically superior” to placebo or whatever the control is.

    When breast cancer incidence hit a certain level for the HRT vs control group, the DSMB blew the whistle and that specific aspect of WHI was halted, never to resume.

    This partly explains why the ‘outcomes’ for breast cancer for the WHI look different from the outcomes from a typical efficacy trial.

    I myself believe that, if analyzed a different way, the DSMB would have halted the HRT aspect of WHI more quickly.

    In retrospect, consider two things: 1. women were harmed by a purportedly helpful intervention. Med research should hold this possibility seriously. 2. there was a rgeat edal of observational evidence, “priors,” suggesting that HRT would reduce heart attacks (and other undesirable outcomes).

    So, the calculation in WHI was a tension between anticipated benefit and unexpected but monitored harm: reduced MI versus increased breast cancer or other monitored side effects/adverse events.

    Instead of setting the heart attack outcome hypothesis to be a null hypothesis, as if knowledge began de novo in 1990 as WHI was dreamed up, there should have been a prior probability to meet: MI reduction of some recognized amount, such as hailed by (pharma sponsored) Grady et al 1992 Ann Int Med ‘HRT to prevent disease…” and other (pharma sponsored) thought leader reviews.

    Then, as WHI progressed and the DSMB monitored both MI incidence and breast cancer incidence (and other adverse events), this benefit/harm ratio would have looked different: HRT would be failing to produce the cardiac protection it seemed to show from observational studies (in retrospect attrib to ‘healthy user effect’ as noted by Grodstein 2005), and the adverse evetns would be showing up as they did. The benefit/risk ratio would look different.

    Benchmarking MI outcomes to priors would have more quickly shown that HRT does not prevent heart attacks — the randomization had erased the bias of the self selection (for uptake of HRT by ‘healthy users) happening in the real world.

  38. Guy said,

    September 24, 2010 at 4:22 pm

    MVT, interesting post. You seem to be suggesting that this trial should have been stopped earlier. If you look at the diagnosis curves, early on there was less ca breast in the treatment group, so stopping it early might have produced the opposite result ie HRT prevents Ca Breast. In reality I still don’t see why a NNH of 1898 is enough to stop a trial. Trials sometimes seem to be stopped early and we don’t get a clear answer.

  39. brookster said,

    September 24, 2010 at 9:03 pm

    @MedsVsTherapy

    What you should bear in mind was that the WHI study was reanalysed in 2007. By then, the breast cancer risks had disappeared; as had most (but not all) of the cardiovascular risks

  40. AdamJacobs said,

    September 28, 2010 at 8:33 am

    Well, the comments on the PLoS Medicine article have got a bit more interesting. There’s now a robust response from Wyeth’s lawyers, pointing out that Fugh-Berman actually failed to produce evidence that any of the manuscripts were dodgy, and also that both Fugh-Berman and PLoS Medicine itself have been less than forthcoming with declaring their competing interests.

  41. Alasdair.Kay said,

    September 29, 2010 at 4:08 pm

    Perhaps if the University took more responsibility for the publications of their academics?

    Universities love to get publications from their staff, but if a University reputation was on the line for all the publications made by their academic staff then there would be more comback on the staff. Given that the university should be aware of what research the academic is actually involved in, it shouldn’t be too difficult for the University to state the veracity of the work they are claiming to have undertaken. And, as you say, these academics must have been aware of what they are doing, so perhaps they would have been more wary if they knew it needed University approval. It’d have to be efficiently performed, of course, so as not to delay genuine publications.

  42. AdamJacobs said,

    October 1, 2010 at 10:41 am

    In principle, Alasdair, I like your idea, but I’m not sure how well it could work in practice.

    You could imagine a system in which a university polices all the publications by its academic staff, and satisfies itself that those publications were produced in an ethical and transparent manner. Any found not to be would lead to disciplinary action against the staff member responsible.

    However, how exactly would that work in practice? Are universities qualified to do that task? Would it spawn a whole load of unmanageable bureaucracy? It’s a good few years since I last worked in a university so I’m really not sure how, or even if, you could go about doing something like that.

    Anyone in the academic world care to comment?

    Personally, I still think the most promising avenue is for medical journal editors to take the lead in enforcement. I think they could do a lot quite easily if they wanted to. It’s a shame that there seems to be very little appetite for this in the journal community.