The UK House of Commons Science and Technology Select Committee are currently looking at the problem of clinical trial results being withheld from doctors and patients (partly, the committee says, in response to Bad Pharma, which is heartening). A clear, thoughtful report and policy recommendations from this committee could be an important step towards fixing these problems.
I gave oral evidence this week on a panel with Roche, GSK, and the ABPI (who have previously tried to pretend that all the issues in Bad Pharma were “historic” and “long addressed”). I’ve posted the video below, and I’ve posted my written evidence underneath that. First is my submission addressing the specific questions posed by the Committee, and then my appendix, giving background on the problem of withheld trial results.
This video is also available here on the UK Parliament website, and all the documents associated with the Clinical Trials Inquiry are here.
Please do remember to support AllTrials, our campaign calling – very simply – for the release of all trial results, from all the trials that have been done, on all the treatments currently in use. We now have 50,000 signatures and support from over 100 patient groups, together with almost all the major faculties, societies, academies and Royal Colleges, as well as GSK, one of the biggest drug companies in the world (with more to follow).
Also in the pipeline: the National Audit Office are also currently looking at the problem of missing data on Tamiflu (with a view to a possible Public Accounts Committee inquiry); the European Union Clinical Trials Regulation is currently being updated; the European Medicines Agency is currently developing new guidelines on sharing individual patient data from clinical trials (more on this later); and the Health Select Committee have already given strong words on missing trial results.
Lastly, I’ll also be writing some follow up notes to the Committee, in response to the various oral evidence sessions, which I’ll post up here shortly.
Submission to Science and Technology Select Committee Inquiry into Clinical Trials and Disclosure of Data
Dr Ben Goldacre, 20th Feb 2013
I am a medical doctor, currently working as a Research Fellow in Epidemiology at London School of Hygiene and Tropical Medicine. For the past ten years I have written about problems in science for the Guardian, and in two books: Bad Science, and Bad Pharma. I am also a co-founder of alltrials.net, a widely supported non-profit campaign group seeking to improve access to clinical trial results.
Background:
Healthcare professionals and patients need the results of clinical trials to make informed choices about which treatment is best. Currently, drug companies and researchers are allowed to withhold the results of clinical trials, on treatments currently in use, from doctors and patients if they wish to. This means that we are misled about the benefits and risks of treatments. We can be misled into prescribing an expensive new drug, for example, when in reality an older cheaper one is more effective. As a consequence, patients are exposed to avoidable harm, and money is wasted unnecessarily.
Withheld results are a problem for both industry and academic trials. The best currently available evidence, from the most current systematic review, estimates that only half of all trials are published, and trials with positive results are twice as likely to be published. A systematic review is the most robust form of evidence, since it is an unbiased overview of the evidence. This systematic review is published by the NHS NIHR HTA programme.
www.hta.ac.uk/fullmono/mon1408.pdf
The ongoing problem of withheld trial results has not been adequately addressed by any of the initiatives in place today. The FDA Amendment Act 2007, for example, requires that results for a subset of trials (one research site in the US, studying a currently licensed drug, etc) are posted at clinicaltrials.gov within one year of completion. This legislation is widely cited as evidence that the problem of missing trials has been fixed. However there was no routine public audit of implementation, and when one was finally conducted, and published in the BMJ in 2012, it found that this law has been ignored by four trials out of five.
Prayle AP, Hurley MN, Smyth AR. Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study. BMJ. 2012;344:d7373.
www.bmj.com/content/344/bmj.d7373
In any case, this and other interventions would have done little to improve medicine today, even if they were effective. This problem cannot be addressed prospectively, by ensuring access to trials finishing after 2008, or after 2013: around 85% of medicines prescribed in the UK are “generic”, and came to the market a decade ago or more. It is the evidence from this era of clinical trials that we need most – 2003, 1999, 1993 – to ensure prescribing today is safe and effective. In almost all cases (although perhaps not for aspirin trials six decades ago) this information still exists. Doctors and patients should be given access to it to make informed decisions.
“What is the role of the Health Research Authority (HRA) in relation to clinical trials and how effective has it been to date?”
I am working with colleagues on low cost randomised controlled trials, seamlessly embedded in routine clinical care, using the General Practice Research Database, and have submitted a response with them on the separate issue of administrative barriers to doing clinical trials more efficiently in the UK.
So far ethics committees have not sought to address the issue of withheld trial results. This is problematic, as it is one of the key ethical problems in medical research. Patients participate in trials in the belief that they are helping to improve knowledge and treatments for future patients. Where trial results are withheld, those patients have been misled. I understand that Janet Wisely, the new head of the HRA, is keen to engage on this issue. In my view there are certain elements that should be laid down in the legislation for this body.
Research ethics committees should ensure that researchers do not have a previous track record of leaving trial results unpublished, before granting them permission to conduct further studies on trial participants. This can be done at almost no administrative cost, by simply requesting a signed statement from the lead medic or primary investigator that they are not witholding the results of any trials more than one year after completion. Similarly the HRA should insist on a commitment to publication, then publicly monitor and audit compliance. Again this would not require any significant administrative or investigative activity: an ethics committee can simply make a diary note, write to the primary investigator, and ask for a link to results publication, whether in an academic paper or on a results registry, one year after completion of the trial.
“What evidence is there that pharmaceutical companies withhold clinical trial data and what impact does this have on public health?”
I have addressed this in an attached memorandum, as suggested, because it is adapted from an earlier document which I initially drafted as a briefing note for Earl Howe, and then as a briefing note for alltrials.net.
How could the occurrence and results of clinical trials be made more open to scrutiny? Who should be responsible?
In my view there has been a widespread systemic failure by regulators, industry, universities, policy makers, and medical and academic professional bodies to take ownership of this problem. As a consequence we have seen an incomplete patchwork of interventions that have failed to address the core issue – we need doctors and patients to have access to all results of all trials on all currently used treatments. Instead we have engaged with peripheral details.
“Clinical trial registries” are a clear illustration of this problem. Registers are public lists that contain a small subset of all the trials that have been conducted on a medicine. They are incomplete by design. The European Clinical Trials Register is a list of trials conducted within Europe over the past few years. It is not a list of all the trials that have been conducted on all the medicines currently available in Europe. It should be, or it should at least strive to be. Clinicaltrials.gov, similiarly, is mostly trials conducted in the US, mostly from the past ten years, and with compulsory registration only since 2007 (though even here compliance is uncertain). These limitations reflect the early administrative origins of these registries. They are not what trial registries should be, or need to be, to inform evidence based clinical practice, and to achieve the clear goal of access to all results.
The European Clinical Trials Register should simply be a list of all the trials ever conducted, on all the medicines currently prescribed in Europe. It should include results. Where and when these trials were conducted is irrelevant. It is clear to me, from my experience of talking to the public, journalists, doctors, policy makers and academics about these problems, that many people believe a trials register to be just this: a complete list of all the trials that have been conducted. This is indeed what they should be.
There are many stories of how companies have refused to hand over information. These in turn have generated discussions about what levers are available, and how we can force companies (in particular) to hand over trial results. I am struck that we have never tried simply asking, in a systematic fashion. The European Medicines Agency could say: “You have a marketing authorisation to sell your medicine in Europe. We maintain a list of all trials ever conducted on all uses of all medicines currently prescribed, so that doctors and patients can make informed decisions. Here are the forms: please tell us about all the trials you hold, or are aware of.”
However there are many other stages where influence could be brought to bear. IQWiG, the German equivalent of NICE, has developed a reputation for demanding high standards of evidence before approving a drug for use, and also for requiring all trial results to be shared with them, and then making those public. It is through this mechanism that we have become aware of major problems with currently used medicines such as reboxetine.
www.bmj.com/content/341/bmj.c4737
We could use this more robust approach in the UK. We could also insist that a treatment is only available for prescription after all the trial results have been made publicly available.
Universities could also insist that all results are published, and that all collaborative contracts between academics and industry include the right to publish, and the right of access to data.
There is also the matter of culture. I am concerned that the impact of withheld results on patient care is currently a cultural blindspot in medicine and academia, even despite systematic review evidence showing that half of all trials do not go on to be published. It is a peculiar paradox that we spend so much money on each individual trial, investing huge effort to ensure that they are free from bias, carefully appraising their strengths and weaknesses, then allow so many of them to be simply deleted from the record. This non-publication reintroduces all the biases we spend so much time and money avoiding back into the evidence base.
We need wider recognition that this is a serious problem, and that it gravely undermines our attempts to practice evidence based medicine and make informed decisions. If a researcher selectively deletes the unflattering data points from one single trial, in order to massage the results and get the result they want, they are rightly regarded as being guilty of research misconduct. This is a serious business, since they are misleading doctors and harming patients. However, if a group of researchers delete whole trials from the overall research picture, then there are no reputational or professional consequences, even though we know that this will distort the apparent benefit of the treatment, just as surely as one researcher fraudulently manipulating the results of a single trial.
This could be addressed in part through medical and academic membership bodies. They could demonstrate leadership on this issue, state clearly that withholding the results of clinical trials is research misconduct, and impose sanctions or even ejection where appropriate.
I am concerned that currently most major UK medical and academic bodies have done the opposite. Many are currently signatories to a pair of documents produced by the “Ethical Standards in Health and Life Sciences Group” that give false reassurance around the issue of withheld trial results. The ESHLSG is co-chaired by the ABPI (the UK pharmaceutical industry body) and the Royal College of Physicians. Engaging with industry on ethical challenges is plainly a good thing. However the ESHLSG documents appear to make misleading statements about the problem of withheld results. For example they make extensive reassuring comments about current regulations, while failing to disclose the best currently available evidence, from fully published academic papers, in leading peer reviewed journals, which demonstrates that these regulations have been routinely ignored. I believe this issue may be covered in more detail in a submission by the “Bad Guidelines” group, but I am happy to give more details.
“Can lessons about transparency and disclosure of clinical data be learned from other countries?”
No. The European Clinical Trials Register is incomplete even by its own standards, with many trials still withheld from the public form of the register, and the EMA has failed outright to deliver on key objectives, such as their promise to carry results on their register by 2012. The US registry at clinicaltrials.gov is incomplete by design, as discussed above – it is not retrospective, and does not cover all trials on a treatment – and even for its limited remit the FDA Amendment Act 2007 has not been adequately implemented, with trial results still routinely withheld, as shown by Prayle et al 2012 referenced above. Even if similar legislation was perfectly implemented, and muscularly enforced, any beneficial impact of getting trial results from now onwards would only be felt in several decades’ time.
We need to ensure that doctors and patients have access to all results of all trials that have ever been conducted on all treatments currently in use, in order to make informed decisions about which treatment is best. We need serious working groups to discuss how to achieve this objective, urgently.
We need a credible public process to explore the details of the costs involved to industry and academia of retrospective disclosure, in either summary results form or Clinical Study Reports, where those exist. The EMA now discloses Clinical Study Reports on the small proportion of trials that they hold, and the European Ombudsman said that the administrative burden of doing so – and of removing some identifiable patient information where appropriate – is not significant. GSK have also committed to releasing Clinical Study Reports, in signing up to alltrials.net
I would also suggest a pilot of full disclosure, either for some commonly used drugs, or for some commonly used classes of drugs. This would allow us to identify any costs, the changes to the evidence base for current decisions, and therefore the public health benefits.
I believe Sir Iain Chalmers, co-founder of the Cochrane Collaboration, has made a submission on how he has been told this problem is being fixed for three decades now. There are many who use the reassuring language of “engagement” on missing trials, but act inconsistently. We should not lose momentum on this important public health issue.
Declaration of interests:
I am currently a Research Fellow in Epidemiology at London School of Hygiene and Tropical Medicine. I earn income as a doctor, academic, writer and broadcaster. In my work I discuss problems in science, including publication bias, which is a major theme in my book Bad Pharma. I am a co-founder of alltrials.net with the BMJ, Oxford University Centre for Evidence Based Medicine, the James Lind Library and Sense About Science. Alltrials.net is a non-profit campaign group to improve access to clinical trial results with extremely broad support.
Appendix: Missing trial data – briefing note.
Ben Goldacre (adapted from alltrials.net briefing note, grateful for assistance and support from alltrials.net team).
Healthcare professionals and patients need the results of clinical trials to make informed choices about which treatment is best. Currently, drug companies and researchers are allowed to withhold the results of clinical trials, on treatments currently in use, from doctors and patients if they wish to. This means that we are misled about the benefits and risks of treatments. We can be misled into prescribing an expensive new drug, for example, when in reality an older cheaper one is more effective. As a consequence, patients are exposed to avoidable harm, and money is wasted unnecessarily.
This problem is very well documented, and widely discussed within the professional academic literature. It has not been adequately engaged with by policy makers.
The scale of the problem
The current best estimate is that around half of all the clinical trials that have been conducted and completed have never been published in academic journals, and trials with positive results are twice as likely to be published as others. This figure comes from a systematic review conducted in 2010 by the NHS NIHR Health Technology Assessment programme. A systematic review is an unbiased overview, that covers all the research that has ever been done on a particular question in science. This is the most robust form of evidence, and it is risky to permit unsystematic “cherry picking” of other evidence.
www.hta.ac.uk/fullmono/mon1408.pdf
This problem occurs for industry and non-industry trials, internationally, at all stages of drug development, and for trials of all sizes.
The majority of drugs in current use were approved several years ago. Around 85% of prescriptions in the UK are for generic medications: these generally came on the market more than ten years ago. For this reason, it is unsatisfactory to fix the problem with greater access to the results of all trials starting from now. The rates of missing data over past few decades have the greatest detrimental impact on current clinical practice, and this missing data will continue to do harm for decades to come, as those drugs will continue to be used.
However, evidence collected since the NHS NIHR HTA review shows that the problem also persists at very high rates. Results from individual studies rather than systematic reviews should be interpreted with caution. With that caveat, here are three prominent recent studies (assembled by Dr Carl Heneghan for the alltrials.net briefing document):
Ross JS, Tse T, Zarin DA, Xu H, Zhou L, Krumholz HM. Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis. BMJ. 2012 Jan 3;344:d7292. doi: 10.1136/bmj.d7292.
“Among 635 clinical trials completed by 31 December 2008, 294 (46%) were published in a peer reviewed biomedical journal, indexed by Medline, within 30 months of trial completion. The median period of follow-up after trial completion was 51 months (25th-75th centiles 40-68 months), and 432 (68%) were published overall.”
“Despite recent improvement in timely publication, fewer than half of trials funded by NIH are published in a peer reviewed biomedical journal indexed by Medline within 30 months of trial completion. Moreover, after a median of 51 months after trial completion, a third of trials remained unpublished.”
Bourgeois FT, Murthy S, Mandl KD. Outcome reporting among drug trials registered in ClinicalTrials.gov. Ann Intern Med. 2010 Aug 3;153(3):158-66. doi: 10.1059/0003-4819-153-3-201008030-00006.
“Overall, 362 (66.3%) trials had published results. Industry-funded trials reported positive outcomes in 85.4% of publications, compared with 50.0% for government-funded trials and 71.9% for nonprofit or nonfederal organization–funded trials (P < 0.001). Rates of trial publication within 24 months of study completion ranged from 32.4% among industry-funded trials to 56.2% among nonprofit or nonfederal organization–funded trials without industry contributions (P = 0.005 across groups).”
Deborah A. Zarin, M.D., Tony Tse, Ph.D., Rebecca J. Williams, Pharm.D., M.P.H., Robert M. Califf, M.D., and Nicholas C. Ide, M.S. The ClinicalTrials.gov Results Database — Update and Key Issues. N Engl J Med 2011; 364:852-860March 3, 2011DOI: 10.1056/NEJMsa1012065
“We characterized the 79,413 registry and 2178 results of trial records available as of September 2010. From a sample cohort of results records, 78 of 150 (52%) had associated publications within 2 years after posting.”
“ClinicalTrials.gov provides access to study results not otherwise available to the public. Although the database allows examination of various aspects of ongoing and completed clinical trials, its ultimate usefulness depends on the research community to submit accurate, informative data.”
There have been no changes to legislation since the 2010 NHS NIHR HTA systematic review was conducted.
Failed initiatives
Two major failed initiatives are commonly cited as evidence that the problem of “missing trial results” no longer exists: journals requiring registration before publication, and FDA legislation requiring results to be posted on clinicaltrials.gov within one year of completion.
ICMJE and Journals requiring registration before publication
Trialists are encouraged to register the existence of their trials publicly, to ensure that there is a clear record of trials in progress. Although this does not guarantee reporting of results, it allows some public scrutiny of whether completed studies have been published.
In 2005, after concern that trial registration was not being used, the International Committee of Medical Journal Editors said they would only publish trials that had been registered at inception. The intention was to force trialists to register trials. There was no public audit of this promise, however, and in 2009 it was shown that half of all trials published in major medical journals after this requirement had been announced had not been properly registered, and a quarter had not been registered at all.
Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P. Comparison of Registered and Published Primary Outcomes in Randomized Controlled Trials. JAMA. 2009 Sep 2;302(9):977–84.
jama.jamanetwork.com/article.aspx?articleid=184503
This is perhaps not surprising, since academic journals have their own conflicts of interest around: publishing positive findings (positive results may get higher citations, and these will increase an academic journal’s “impact factor”, the widely used index of journal quality); and publishing industry findings (such as revenue from reprints and adverts).
Failure of the European Medicines Agency clinical trials register
The EMA was asked to produce a clinical trials register in 2001, and then in legislation in 2004. It is important to be clear that a register contains notice of the existence of a trial, and some aspects of the trial’s design, but not its results. For many years the contents of this ‘transparency tool’ were held by the EMA in secret. Since March 2011, after public criticism, the EMA has made a publicly accessible register. However, despite claims that this is complete, in reality, details of the existence of several thousand trials are still currently withheld from the public, while the EMA considers whether they should be disclosed. There should be a presumption that details about all trials on the register are to be disclosed.
Furthermore, unlike the US register at Clinicaltrials.gov, information about all Phase 1 trials are held in secret. This is despite the inquiry into the TGN1412 trial stating that Phase 1 trials should be made more widely available, to prevent harm to trial participants (pp86-87).
www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@dh/@en/documents/digitalasset/dh_073165.pdf
There is currently no facility on the EMA register to allow researchers to disclose the existence of their registered Phase 1 trials, even if they wish to do so.
The EMA has also stated on many occasions that it would produce a publicly accessible database of trial results in 2012. It is now 2013 and they have not done so.
Finally, it is perhaps worth noting that in my (BG) anecdotal experience of checking specific trials, the contents of the EMA register can be in error, or at least shown to be inconsistent with the contents of other registers, when trials have been posted on more than one register.
FDA legislation requiring results to be posted on clinicaltrials.gov within one year of completion
The US government FDA Amendment Act 2007 requires that, for all trials with at least one site in the US, researching a currently licensed drug, etc., all results must be posted on the website clinicaltrials.gov within a year of completion of a trial. This law is widely cited as evidence that the problem has been fixed. However there has been no official public audit of compliance, and no publicly accessible structured data on due dates for results.
An audit was conducted independently and published in the British Medical Journal in 2012. This is the best currently available evidence, and it shows that only one in five trials has met this reporting requirement (compliance was 40% for industry funded trials, and 10% for mixed industry and independent trials). Despite this very low compliance, no fine has ever been levied against any company or researcher for failing to post results. Even if a fine had been levied, it is $10,000 a day, or $3.65m a year: this is trivial for a large organisation.
Prayle AP, Hurley MN, Smyth AR. Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study. BMJ. 2012;344:d7373.
www.bmj.com/content/344/bmj.d7373
Even if this law had been implemented, it would not have fixed the problem. It would only require publication of trials completing after 2008, whereas prescribers mostly use medicines approved over several previous decades, and so we rely on evidence from trials done much earlier than 2008 to make decisions about which current treatment works best. It also misses many trials conducted outside the USA, which is increasingly common now that trials, including post-marketing trials, are commonly run by Contract Research Organisations in Brazil, Russia, India, China etc.
Tamiflu and access to Clinical Study Reports.
Tamiflu (oseltamivir) is a drug the UK government has spent £500m stockpiling. The manufacturer, Roche is currently withholding important information about trials on this drug from the Cochrane Collaboration, the large international non-profit academic collaboration that produces rigorous systematic reviews of reliable evidence on the effects of drugs and other forms of healthcare interventions for doctors and patients.
www.plosmedicine.org/article/info:doi%2F10.1371%2Fjournal.pmed.1001201
Tamiflu is not an isolated case, and the evidence above shows that this problem is widespread throughout the whole of medicine. Tamiflu is, however, the most thoroughly documented case, because of the extent of the research by the Cochrane Acute Respiratory Infections Group.
There is one specific issue here: for several Tamiflu trials, while there are brief summaries available in the public domain, these do not contain sufficient information to make an informed judgement about the reliability of the trials, and whether they were methodologically rigorous. This is why the Cochrane researchers have sought access to the full Clinical Study Reports. Roche have publicly promised in writing that they would share these documents, in December 2009, but have since refused to do so. Their recent suggestion that they would convene a committee to look at this issue addresses none of the relevant issues around transparency, and they continue to withhold data.
In 2010 the European Medicines Agency made a commitment to release more CSRs on request, after a change of personnel, and a finding of maladministration against the Agency from the EU Ombudsman over their withholding information from Cochrane researchers. However the EMA only holds documents for very recently approved treatments, and does not hold all the full CSRs on Tamiflu.
Researchers and industry are not the only ones at fault:
Universities have failed to ensure that contracts with companies sponsoring trials run by academics allow the academics to publish the results of trials, regardless of whether the company is happy with them.
Ethics committees that approve research projects have failed to protect patients, because they have not insisted that researchers publish results, and they do not check to see if researchers are withholding the results of previously approved trials. This may currently be being addressed by the Health Research Authority.
Medical membership bodies have failed to act on this issue, and have failed (with one exception) to state publicly that withholding the results of clinical trials is research misconduct.
These all represent opportunities to help address the problem of missing results for future trials.
It is also worth noting that non-publication of trial results presents a major ethical breach: patients participate in research, experiencing inconvenience and sometimes risk, in the belief that their participation will improve our understanding of which treatment works best. If the results of trials are withheld, then the participants have been misled.
Journals
Researchers and industry sometimes claim that medical journals will not publish negative trial results. This was a modest problem in another era of medicine, but was fixed a decade ago. The most current NHS NIHR HTA systematic review (cited above) found that overall journals were not the main barrier to publication. With the advent of open access journals where the business model is not dependent on the need to sell subscriptions with high profile “positive” papers, there are now several open-access academic journals – such as the open-access journal Trials, and journals from BioMedCentral and the Public Library of Science – that will publish trials regardless of whether the results are positive. All researchers may struggle to get their academic paper into their first choice of high end academic journal. But academic journals are no longer the key barrier to publishing trial results; they are no barrier to posting results on clinicaltrials.gov; and they are no barrier to sharing Clinical Study Reports.
Why do doctors need to see results as well as regulators?
A medicine does not simply “work” or “not work”. Some drugs work very well, some work less well than other drugs, but are still better than nothing. A medicines regulator decides if a drug should go on the market at all, and they have a low bar for approval. This is good: we need some less effective drugs to come on the market. For example, a patient may have idiosyncratic side effects from the best available drug for their condition, in which case it is useful to have a less effective drug to try next.
Doctors and patients need all the information about all the clinical trials that have been conducted on drugs (and other treatments) in order to make informed decisions. A clinical decision (“should this patient receive this drug?”) is very different to a regulator’s decision (“overall, is it in the interests of society that this drug should be on the market for use at all?”).
Furthermore, regulators can sometimes miss important problems with medicines. For example, as with Tamiflu, the problems with the drugs Vioxx and Rosiglitazone – both now effectively off the market – were spotted by academics and clinicians rather than regulators. This is not because regulators are incompetent: these are difficult problems, so it is good to have many eyes working on them.
It is also good for patients if the evidence behind regulators’ decisions can be independently assessed, to ensure regulators have made good decisions. Science is built on transparency: on people critically appraising the methods and results of scientific research to decide whether they agree with each others’ conclusions. This is the absolute core of science, it is how we have progressed as a society: the motto of the Royal Society is “nullius in verba”, “on the word of nobody”. Decisions about which treatment is best for a patient are some of the most important scientific decisions ever taken.
Data on individual patients who have participated in clinical trials
This is a separate issue but also important. Better systems for sharing clinical trial data about individual patients would create more transparency, permit more accurate assessments of drugs by doctors and patients, and enable more accurate identification of any subgroups of patients who might respond to a treatment, to a greater or lesser extent than average.
Sharing data is very valuable because it facilitates more accurate estimates of the effects of interventions. This is well demonstrated by the work of the Early Breast Cancer Trialists’ Collaborative Group, which has demonstrated leadership in this field by conducting highly accurate and informative systematic reviews and meta-analyses using individual patient data from a large pool of clinical trials to inform breast cancer treatment supported by research funding agencies around the world.
Effect of radiotherapy after breast-conserving surgery on 10-year recurrence and 15-year breast cancer death: meta-analysis of individual patient data for 10 801 women in 17 randomised trials. The Lancet. 2011 Nov;378(9804):1707–16.
www.thelancet.com/journals/lancet/article/PIIS0140-6736(11)61629-2/abstract
Analyses such as this present challenges in ensuring confidentiality for individual patients (although these can be overcome, and the work of the YODA data sharing project at Yale presents one interesting early proposal of how to manage these issues). This is a different issue, however, to claiming confidentiality about the existence and aggregate summary results of the trials themselves; or withholding information about their design, methods, and conduct.
Existing promises to share clinical trial data
GSK has indicated it will share data on Relenza, its influenza treatment, with the Cochrane Acute Respiratory Infections Group working on Tamiflu. This has not yet happened.
GSK has also offered to share more data from its earlier phases of drug development, since 2007, with named researchers, behind closed doors, after review by a GSK panel, to enhance collaboration and innovation. Again, while this is positive, it has not yet happened, and it is unclear what the processes will entail.
GSK has signed up as a supporter of alltrials.net, and has committed to share CSRs for medicines currently in use, going back to the founding of GSK by merger a decade ago. This information has not yet been shared, and will take time to produce. However, all of GSK’s commitments are extremely important positive steps.
The EMA has indicated that it intends to share individual patient data given to them as part of the licensing process for a drug (which is not all trials on that drug). It is presently unclear how this will work. At the EMA meeting to discuss options, industry representatives made clear that they felt industry should be allowed to control access to data and to decide who would be allowed to inspect the data. It was also suggested that this transparency should only be permissible for trials starting in 2014 or later.
Because drugs continue to be used for many decades, this will do nothing for the evidence base of currently used treatments, and will have little impact for a decade, if not longer. This is especially the case since many of the most widely used drugs have been on the market for some time, and good treatments continue to be used for as long as they are the best in their class.
What needs to change?
All involved parties need to work to ensure that all results of all clinical trials – past and future – on all treatments in current use are available to doctors and patients, so that they can make informed decisions about treatments. At present, although this problem is thoroughly documented and ongoing, many in industry seek to deny it exists. NICE, regulators, and medical membership bodies have all failed to accept ownership of the problem or show leadership in addressing it.
There is no UK legislation requiring the results of all trials on all drugs in current use to be made available to doctors and patients. There is legislation requiring disclosure of adverse events and other monitoring of clinical trials conducted within the UK, such as regulation for “good clinical practice” in the conduct of trials, by the MHRA. This legislation does not address biased under-reporting of clinical trials and should not be confused with the issue of missing results.
This problem can be addressed through many means.
In Germany, the equivalent of NICE (IQWiG) has developed a reputation for requiring high standards of evidence before recommending the use of drugs. It has also refused to allow drugs to be used until companies have disclosed all trial results to them, which IQWiG has then made public. It is through this mechanism that we have become aware of major problems with currently used drugs such as reboxetine.
www.bmj.com/content/341/bmj.c4737
This more robust approach could and should be replicated in the UK. One option is to insist that a treatment can only be used by the NHS if all information about all trials conducted on it are made publicly available to doctors and patients; or that a company’s treatments can only be used by the NHS if all information about all trials on all their drugs are made publicly available. This may be more powerful, or practical, if similar agreements can be sought with other countries in the EU or elsewhere.
The current form of the draft EU Clinical Trials Regulation is weak, and does not adequately address the problem of missing results for medicines currently in use. This must be addressed urgently as the Regulation is being considered by the European Parliament now.
Research ethics committees must address publication bias for all future trials by insisting on commitment to publication, and publicly monitoring and auditing compliance. Ethics committees could also insist on evidence of publication of researchers’ previous trials before giving permission for additional research, for example by requesting a signed assurance from the primary investigator or key medical personnel.
Finally, Universities should insist on the rights of access to data and the right to publish results in all collaborative contracts.
Pete said,
April 27, 2013 at 1:46 am
I think it’s horribly ironic that an article about openness in trial data should be almost immediately stumped by a requirement for the reader to install “Microsoft Silverlight”
I’m sure I could jump through hoops and get something to display it, but if you want data to be open, surely you need touse open standards?
jamesf said,
April 27, 2013 at 10:14 am
I get your point, but to be fair this is an embedded video from UK parliament. Clearly Ben’s star is still on the rise but I don’t think he is in a position to dictate the UK government’s choice of video streaming solution!
Good submission, looking forward to listening to you in Wellington
muscleman said,
May 6, 2013 at 10:36 pm
Having just watched, in batches, the whole session what struck me most was how the two industry bods hastened at the end to demonstrate all the advantages and how signed up they were after your last somewhat corruscating (but perfectly valid) comments.
You did a good job bringing some necessary reality to the proceedings and in front of the committee. With the professional funders/sponsors previously and the industry people with you the risk would have been of a fairly ‘steady as she goes but there will be shoals’. What you reality interjected was the reality of how the failures impacted on patients. Your point about having personally prescribed reboxitine in ignorance was powerful.
In short, I’m glad you were there. Thank you.
BiomedicalDude said,
May 10, 2013 at 1:28 am
People on the “Science and Tech Select Committee” should first have a working understanding of what “Science” and “Tech” mean…
JohanBenesch said,
June 3, 2013 at 5:52 pm
there is a proposal to change the current European legislation on clinical trials on medicines for human use.
the proposal includes requirement for proponents to submit a Clinical Study Report. see the draft regulation here:
www.europarl.europa.eu/sides/getDoc.do?pubRef=-//EP//NONSGML+COMPARL+PE-504.236+01+DOC+PDF+V0//EN&language=EN
turnip2 said,
June 3, 2013 at 7:43 pm
Thank you Ben for making this all public in a way that non experts like myself can understand! I first found you thru TED presentation. I now have a library copy of Bad Pharma on my desk. I intend to get a Kindle copy of it to refer and cite from. I am trying to spread the word. I had given up on FDA years ago. The conflict of interest is outrageous. I am older female and was much too compliant to bad pharma/medicine and paid the price dearly. I now do not find medicine credible for the most part based on my personal experiences. It is back firing in a big way. I haven’t been to a doctor in 3 yrs now and can’t bring myself to go to one since I can not believe anything they say based on so much biased research they base their recommendations on. I haven’t been sick with a cold or flu or anything for 3 yrs and doing much better on my own. We are forced to pay 800.00 a month for healthcare we never use anymore. I wish we could actually cancel with employer and keep over 10K a year! I have autoimmune conditions(S) and have improved my health dramatically through diet not pills or ointments. I think the reality is that we really don’t need all those specialists and medications. They have invented diseases and pills for them. Healthcare doesn’t deserve non profit tax status at all in the U.S. I was duped with Paxil and I believe the studies will never be done on how it impacts our guts which leads to autoimmunity. It either kills gut bacteria which increases leaky gut/intestinal permeability and/or disconnects the gut’s ability to signal properly just like it disconnects our ability to remember. This affects gut’s ability to retain tight junctions and now opens us up to autoimmunity in a big way. They lied to me and shame on them. (I wish I could afford to sue them.) But shame on me if I allow them to do that again! I am also the matriarch of our family and make sure family and friends and co workers don’t make the same mistakes. I have time to do the research most don’t have time to do. I will be long dead before they figure it out but in the meantime I would pay an annual fee to be able to access an independent data base which had a complete library of all research papers. It should also provide everyone the ability to review the papers so we all can see a robust discussion on the pros and cons of the paper. Research publications are too fragmented into too many journals and hidden from the public. So much of these independent studies are done with tax payer grants or professors who are state paid workers etc and should be easily available to see. These journal editors are the gate keeps to all of this with their membership fees and regular citizens should not have to pay twice for the results. If one were allowed to be a member of all these journals and pay all the fees it would be extremely expensive! I also think these pharma companies should be taxed heavily in order to pay for replicate studies by outside groups to prove their claims for their drugs. From several groups that have no monetary link to the drug companies. No drug should be allowed on the market until that is completed. They have proven over and over again bad faith in all of this. They can NOT be trusted. I wouldn’t waste so much time and effort to get politician to do the right thing since they are so corrupt in the U.S. I do think putting efforts in creating a better alternative we could use now might be a better way to invest efforts. Creating a better place for us aging baby boomers to share our concerns and find better alternative ideas and more balanced research would make these journals a sorry alternative for everyone. Baby boomers are sick in big numbers right now due to bad medicine et al and healthcare’s golden goose. Chronic illness is big pharmas bread and butter. If you watch the evening news here in the U.S. all you see are autoimmune suppressing commercials one right after another. Baby boomers are more likely to not be so gullible. More outspoken, who can support their positions on how bad pharma/medicine has harmed them so far. They are first hand witnesses to the harm these companies have done in the name of profit. Word of mouth is very powerful in this digital age 🙂