I did a new talk at TED about drug companies hiding the results of clinical trials, it went up today. This is a huge, ongoing problem, and it results in patients suffering and dying unnecessarily. So I’m really pleased that TED were able to give the story a platform. Video after the click:
That said, all I can think when I watch this jetlagged video is “fix your hair, take your hands out of your pockets, and for god’s sake tell some jokes”.
If you want to read more about this problem of “publication bias”, there’s lots on this site, here, and even more in Bad Pharma, a book I wrote about how to fix the problem of bad behaviour in the pharmaceutical industry (and among doctors, academics, regulators, patient associations…).
More on publication bias:
www.badscience.net/category/publication-bias/
A great blog from TED
blog.ted.com/2012/09/27/5-prescription-drugs-doctors-had-no-idea-could-hurt-their-patients/
The book:
The foreword:
www.badscience.net/2012/09/heres-the-intro-to-my-new-book/
An extract in the Guardian:
www.guardian.co.uk/business/2012/sep/21/drugs-industry-scandal-ben-goldacre
Previous TED talk:
mikey said,
September 28, 2012 at 12:17 pm
I can’t say I didn’t notice the hair, but the talk was really good, rapid pacing with good spacing and insiteful analogies that really helped keep things moving along. You really do seem to be one of the best at the TED talk format.
robb said,
September 28, 2012 at 8:25 pm
Hi Ben,
In my experience, papers with actual negative results are very rare, at least in medical science. When reviewers get over their surprise, they’re quite willing to accept them. There IS a bias against publishing *inconclusive* results – papers where the experiment failed to show a significant effect but where either the variance is simply too large to draw any meaningful conclusions or the authors have not attempted to calculate limits on the maximum effect.
It seems to me the general problem is not publication bias (although it is certainly true that drug companies would prefer only positive studies be published) but rather the poor statistical skills of many researchers. They lack the ability to produce papers reporting negative results.
Interpreting non-significant results as “negative” is dangerous. A pair of papers caused a stir a couple of months ago in my field, multiple sclerosis. One found that treatment with disease modifying therapies did have a beneficial effect on long term outcome. The other did not. An excited PhD student announced to our institution mailing list that we’d be having a special journal club to discuss these “opposite” results. He was quite confused when I commented that the two papers were completely consistent with each other.
The “negative” paper, on closer examination, HAD calculated confidence intervals on their result. The high side of the 95% confidence interval corresponded to treatment being the single most predictive factor of disease progression studied, more so than such obvious factors as how fast your disease had progressed in the past, or how long you had had it. The lower CI just brushed 0. Nestled somewhere in the middle was the estimate of the positive effect from the other paper. The authors went so far as to suggest in their discussion that we should reconsider how we treat patients.
This influential paper, from a good group and published in a good journal, had made one of the most basic statistical mistakes: assuming that a non-significant result meant “no effect.”
Your thoughts?
d10 said,
September 29, 2012 at 7:25 am
What was the specific paper you referenced as the systemic review of publication bias which was published in 2010? (referenced at 8:30 in the video)
0101011 said,
October 5, 2012 at 3:23 am
Mandating the publication of all medical studies is as desirable as it is unpractical. Many studies have probably been “lost”. Costly law suits will have to be fought. Whether we should or should not I know not. All I know is it is not, by any stretch of the imagination, “simple”.
In other words, you know a lot about science and little about law.
None the less, much of your work remains dear to me. A statistically significant portion in fact.
chemist.jones said,
October 7, 2012 at 3:56 pm
0101011, I don’t think you would necessarily have to retroactively publish results. How would you choose how far back to look? I think instead the best course of action is a fresh slate and a new requirement. This approach would be simple – or at least manageable.
Ben, as usual you killed this TED Talk. You’re a very engaging speaker. It seems that the TED format was designed for you.
rjmunro said,
October 7, 2012 at 10:37 pm
Shouldn’t NICE be able to fix this? If they have evidence that a drug has had trials that were not registered, they should not recommend it for use in the NHS, at least until the drug company pays for a large independent properly registered retrial.
Amanda Acevedo said,
October 11, 2012 at 6:00 pm
You did great! And can I just say THANK YOU!!! Thank you for the information, thank you for being brave, thank you for addressing the problem and thank you for your books.