Sarepta, eteplirsen: anecdote, data, surrogate outcomes, and the FDA

September 30th, 2016 by Ben Goldacre in bad science | 4 Comments »

The Duchenne’s treatment made by Sarepta (eteplirsen) has been in the news this week, as a troubling example of the FDA lowering its bar for approval of new medicines. The FDA expert advisory panel decided not to approve this treatment, because the evidence for any benefit is weak; but there was extensive lobbying from well-organised patients and, eventually, the FDA overturned the opinion of its own panel. There have been calls for paper retractions, and so on.

This is not the first time we’ve seen peculiar activity around the treatment. Below is a column I wrote in 2011 (which I forgot to archive here at the time). It shows how benefits on a “surrogate outcome” were somehow puffed into benefits on real-world outcomes, by the media, and by the press releases. What’s a surrogate outcome? Explaining that is the point of the piece below. Then I’ve pasted an odd piece of, er, “Bad Science ephemera”. As you will see, The Lancet’s press release on this trial was very sensible; but the press release from UCL Institute for Child Health / Great Ormond Street Hospital made misleading allusions to benefit on real-world outcomes which never appeared anywhere in the academic paper. After my column on this was published Andrew Copp, the Director of the ICH, published a response in the ICH/GOSH newsletter. It’s not just that he misses the central argument of the article; the tone is also pretty peculiar. I encourage you to click on the image, read it, and judge for yourself.

Here we go!

Anecdotes are great! If they convey data accurately…
Ben Goldacre
The Guardian
Saturday 30 July, 2011.

On Channel 4 News, scientists have found a new treatment for Duchenne’s muscular dystrophy. “A study in the Lancet today shows a drug injected weekly for three months appears to have reduced the symptoms” they say. “While it’s not a cure, it does appear to reduce the symptoms.”

Unfortunately, the study shows no such thing. The gene for making a muscle protein called dystrophin is damaged in patients with DMD. The Lancet paper shows a new treatment led to some restoration of dystrophin production in some children in a small unblinded study.

That’s not the same as symptoms improving. But Channel 4 reiterates its case, with the mother of two participants in the study. “I think for Jack … it maintained his mobility … with Tom, there’s definitely significant changes … more energy, he’s less fatigued.”

Where did these positive anecdotes come from? Disappointingly, they come from the Great Ormond Street Hospital press release (which was tracked down online by evidence-based policy wonk Evan Harris). It summarises the dystrophin results accurately, but then, once more, they present an anecdotal case study going way further: “Our whole family noticed a marked difference in their quality of life and mobility over that period. We feel it helped prolong Jack’s mobility and Tom has been considerably less fatigued.”

There are two issues here. Firstly, anecdotes are a great communication tool, but only when they accurately illustrate the data. The anecdotes here plainly go beyond that. Great Ormond Street deny this is problematic (though they have changed their press release online). I strongly disagree (and this is, of course, not the first time an academic press release has been suboptimal).

But this story is also a reminder that we should always be cautious with “surrogate” outcomes. The biological change measured was important, and good grounds for optimism, because it shows the treatment is doing what it should in the body. But things that work in theory do not always work in practice, and while a measurable biological indicator is a hint something is working, such outcomes can often be misleading.

Examples are easy to find, and from some of the biggest diseases in medicine. The Allhat (Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial) was vast, comparing various blood pressure drugs. One part compared 9,000 patients on doxazosin against 15,000 on chlorthalidone. Both were known to lower blood pressure, and people assumed they would also lower the risk of real-world outcomes, such as stroke and heart attack.

But patients on doxazosin turned outto have a higher risk of stroke, and cardiovascular problems, than patients on chlorthalidone – even though both lowered blood pressure – to such an extent that the trial was stopped early. Blood pressure, in this case, was not a reliable surrogate outcome for assessing the drug’s benefits on real-world outcomes.

This is not an isolated example. In the treatment of diabetes, HbA1c is often monitored, as it is an indicator of blood glucose levels over the preceding few weeks. Many drugs, such as rosiglitazone, have been licensed on the grounds they reduce your HbA1c level. But this, again, is just a surrogate outcome: what we really care about in diabetes are real-world outcomes such as heart attacks and death. And when these were finally measured, it turned out rosiglitazone, while lowering HbA1c levels effectively, massively increased your risk of heart attack. (It has now been suspended.)

So improvements on surrogate biological outcomes that can be measured in the body are a strong hint that something works – and I hope this new DMD treatment does turn out to be effective – but even in the most well-established surrogate measures, and drugs, these endpoints can turn out to be misleading. People writing press releases, and shepherding misleading patient anecdotes into our living rooms, might want to bear that in mind.

Screen Shot 2016-09-28 at 22.17.00 Screen Shot 2016-09-28 at 22.16.41

If you like what I do, and you want me to do more, you can: buy my books Bad Science and Bad Pharma, give them to your friends, put them on your reading list, employ me to do a talk, or tweet this article to your friends. Thanks! ++++++++++++++++++++++++++++++++++++++++++

4 Responses

  1. Michele said,

    September 30, 2016 at 11:15 am

    Other disturbing features of this approval that need more critical attention:

    1.) Co-opting of patient advocacy groups as pharma marketing representatives. Some of these groups are, in fact, astroturf organizations wholly funded by pharma and VCs. Health News Review touches on this topic here:

    2.) Complete up-ending of the accepted clinical trial process by allowing what is essentially phase III data to be collected after approval. Good luck with that. Once the drug is available, what family will agree to enroll their child in a study? Even if they can get the data, will the FDA actually pull an approved drug off the market simply because it is not effective? I am not aware of that ever happening, but could be wrong. It is more likely that the drug is here to stay (and for health systems to pay millions for) unless there is substantial evidence of actual danger from it after market.

    Once again, the FDA abdicates its role in order to appease pharma and in response to emotion, not science. Truly shameful.

  2. Penny said,

    September 30, 2016 at 12:16 pm

    I think there are some issues with the outcome measures used in trials of potential therapies for Duchenne. Prolonged mobility (e.g. as assessed using the 6 minute walk test) is just one hoped-for outcome, but other benefits such as improved cardiac and respiratory function are also very important and would need to be evaluated in longer-term studies. The question is clearly whether dystrophin production will be restored to the extent that people with Duchenne will live longer and have a decent quality of life when they do. Now that my son with Duchenne is 16, I can tell you that he has well and truly got his head around not being able to walk any more, but the prospect of losing hand function is more daunting – and he would obviously like his heart and lung function to be preserved! Even small increments are highly meaningful to patients, and in my view the outcome measures do not always reflect what is most important to them.

  3. Simon Coury said,

    September 30, 2016 at 6:20 pm

    Great to see you back here posting, Ben. Why is nobody commenting? Are comments disabled? Guess I’ll find out when I try submitting this.

  4. Ben Goldacre said,

    October 22, 2016 at 9:05 pm

    Apologies, there’s so much spam on blogs these days I don’t often have the chance to look at what’s in the queue, and people have lots of other places to post responses, I’ll put something on the page to point that out when I get a mo!