Archive piece – Medical research threatened by patient consent

June 6th, 2007 by Ben Goldacre in bad science, regulating research | 7 Comments »

I was reminded of this article during a conversation with Julian Peto here last night: it’s a golden gasser from yesteryear, a piece I wrote for the Guardian in 2001. It originally appeared with the wrong surname, so you’ll just have to trust that it really was me what wrote it.

At the bottom, I’ve pasted an article on a similar subject by Peto that he sent me this morning. If it’s a subject that interests you then it’s worth following the link to the BMJ page (assuming you can access) and following up the related stories and also the Rabid Responses, as they are, er, known on the street.

If you have ever seen a doctor, you are being tracked. Information about you is being stored and analysed as part of a vast pool of medical data. But now this crucial area of research is under threat, warns Ben Goldacre, because of moves to force medics to seek patients’ permission for every study

Thursday April 12, 2001

Every time you step inside a hospital, the chances are that someone, somewhere is watching you – though that someone is more benevolent than your average stalker, and it is your disease and your data that he or she is interested in, not you.

Epidemiologists, or medical statisticians, watch the figures for health across the country, spotting trends. They work by trawling through information from the records of millions of people. It is epidemiologists who assess the dangers of the contraceptive pill, whether ethnic minorities have equal access to health care and whether electricity pylons cause leukaemia. It is they who work out which treatment is best for breast cancer – and whether your GP might be quietly murdering old ladies in their homes.

Because they work with whole populations, epidemiologists can see what is dangerous and what isn’t, whether particular illnesses are becoming more or less common, what treatment works and what doesn’t, and – in the case of surgeons – who is dangerous and who isn’t. Epidemiology has helped save more lives than any of the other more glamorous branches of medicine.

Every time a doctor advises you about your health or decides how best to treat you, he or she is using the evidence provided by epidemiologists. Universal clean water and sewage facilities, and advising people not to smoke, were among their most famous ideas.

But all of this may have to come to an end. The General Medical Council (GMC) has decided that this kind of purely observational work is an infringement of patients’ rights – even though it reveals nobody’s identity and does nothing to alter or withold anyone’s treatment.

The GMC has declared that each study should go ahead only with every patients’ expressed and informed consent, and that doctors should not provide data about their patients to any other doctors without that consent. It’s a nice idea: patient autonomy and patient confidentiality are crucial, and it would be wrong to dismiss them out of hand. But because such consent is often unworkable – for a host of practical and scientific reasons – the change might stop epidemiology and clinical audit in their tracks and severely impede further advances in medicine.

Ironically, epidemiologists and other doctors who use records purely for research and statistical purposes are the people in medicine who have taken confidentiality and data encryption most seriously. Anyone worried about the confidentiality of material relating to their health should be more concerned about the underpaid temporary staff working on the front desk of their local GP surgery than about a group of medical statisticians cooing over a meta-analysis of 50,000 cases of cancer. There are no scandals in epidemiology.

The problem is that the data being shared must, at some stage, include the identity of the patient. Looking at how our world-renowned national net work of cancer registries works, you can see why. For decades, 11 cancer research centres around the country have collected morsels of data on patients afflicted with cancer in their region. From the type of cancer to how they were treated, from the details of their laboratory investigations to their death certificate, all the information is collated, stored and analysed in order to improve the care of the next generation of patients.

There is a good reason that it is not possible for this information to be filed entirely anonymously, without identification of the patients attached. The cancer registry system collates data from innumerable diverse and unrelated bodies: the registrar of deaths in the town hall, the GP, the hospital pathologist. And the patient’s identity is the only thing that unifies these disparate pieces of information, allowing the registry to match up tissue diagnosis with symptoms with death certificate and trace the life of one person’s cancer.

But it is not as if the information is being broadcast on the nine o’clock news. This is the confidential transfer of information between doctors, which is then held with obsessive security, with the doctors being accountable to the GMC if they treat the information irresponsibly.

It is the most comprehensive national cancer database ever constructed and is the envy of the world. Every doctor in the country is expected to file data to the cancer registries on their patients. But the GMC has given a deadline of October 2001 for the cancer registries to put mechanisms in place for seeking and recording the consent of everyone on their books. Without this consent, doctors will have to stop the flow of information to them. Because this is impractical, cancer registries may have to close.

But if the GMC wants us to, why can’t we just ask people for consent? Can it really be so difficult? In some circumstances it may be, and not just for practical reasons. Insisting on consent can systematically skew the data, and give answers that are weaker, or simply wrong.

If you ask people to agree to having their data used in an observational study, some will refuse. That, of course, is the reason you are asking them in the first place. And when they refuse, particularly if certain types of people are consistently more likely to refuse, such as the demented, the psychotic, the homeless, or those embarrassed by their health problems, that skews your data.

This is what medical statisticians call “consent bias”. It is important because it can turn your data on its head, render it useless and give the wrong conclusions. In medicine, that costs lives.

There are unscalable practical barriers to seeking consent from the whole population. It remains a Herculean task even if, instead of considering specific cases with patients, we assume that we only want to ask everyone in the country the rather uninformative and vacuous question: “Do you consent to your data being shared, on a confidential basis, for anything we can think of?” There are 60 milion of us, after all.

How quickly can you get informed consent, and explain why you want it? Will three minutes tacked on to the first consultation with a GP be enough? A tick-box? Who will keep a central record? What about the patient with Alzheimer’s who can’t give consent? What if you change your mind? What if you switch GPs? What if you never go to a GP, but just pitch up in A&E one day? It is a far bigger undertaking than the census, and it is hard to imagine it ever happening.

It would be far easier to legislate to protect the work of cancer registries and other organisations concerned with epidemiology and health services research, to ensure that the confidential data they need could never be used for non-medical purposes. That would protect the public and promote new medical knowledge.

As a doctor, I don’t expect patients just to tolerate the kind of work that the cancer registries and epidemiologists do: I believe they would be astonished if it weren’t done. Epidemiology is the foundation on which all preventative medicine is based. Without it, much would be just anecdote and small drug trials. This move to tie its legs together can only be motivated by some unthinking adherence to an abstract philosophical doctrine about privacy. The point of the principle of confidentiality was to protect the patient from untoward disclosure of information which could be used to his or her detriment. It should not be a moral absolute, at so great a cost.

Is our adherence to principles about privacy so important that we would sagely shave months or years off our life expectancies? I’d rather have another birthday, thanks.



Data protection, informed consent, and research
BMJ 2004;328:1029-1030 (1 May), doi:10.1136/bmj.328.7447.1029
Julian Peto, Olivia Fletcher, Clare Gilham.

Medical research suffers because of pointless obstacles

Cancer Research UK and other medical research charities have warned the government that the human tissue bill will cause damaging confusion among doctors and hamper medical research unless crucial sections are clarified.1 The Data Protection Act, another well intentioned but loosely drafted law, has also been in the news. The Bichard inquiry has been investigating the decision by Humber-side police to erase the records of Ian Huntley’s sexual offences involving children because he had not been convicted, and Richard Thomas, who as information commissioner is responsible for interpreting the Data Protection Act, has announced a public information campaign to prevent such embarrassing “misinterpretations” of the act.2 He says that “data protection principles are largely a matter of common sense,” and told the Bichard inquiry that the decision to erase Huntley’s records was “astonishing.” This decision, like the General Medical Council’s instruction to doctors that they might face litigation under the Data Protection Act if they notified their patients to cancer registries without obtaining fully informed consent, stemmed from the legal muddle that the Data Protection Act has engendered. To blame the muddled majority is to miss the point. It is the law, not police or medical training, that must be amended. Access to personal records should not require informed consent in certain circumstances, and these should be specifically exempted. The criterion of an overriding public interest has proved to be too ambiguous to be useful.

The deaths that will occur because of the effects of data protection law on British medical research attract less publicity than child murders; but the pointless obstacles that bona fide researchers, particularly epidemiologists, face when they seek access to individual medical records are now causing serious damage. An important recommendation of the new Wanless report is that the forthcoming white paper on public health “should address the possible threat to public health research which arises from the difficulty of obtaining access to data because of the need to strike a balance between individual confidentiality and public health research requirements.”3 Lord Falconer says: “Data can be used for any medical research purpose under the [Data Protection] Act, without the need for the consent of individuals. So Professor Julian Peto is simply wrong when he states that the Data Protection Act is preventing data from being passed to medical researchers.”4 That those who enact and interpret radical social legislation should be so ignorant of its actual effects is alarming. Custodians of medical records are fearful of litigation if they allow any access for research or even audit without each patient’s informed consent. The effects on British epidemiology are illustrated by two of our current studies.

The NHS cancer screening programme commissioned us to abstract the screening records of women who had died of cervical cancer and an anonymised 1% random sample of all British women. We had to correspond with the director of public health or the Caldicott guardian in almost 100 former health authorities for permission to obtain these data. These negotiations occupied much of a senior researcher’s time for two years, and the data have still not been released in a few areas.

The Health and Safety Executive commissioned us to conduct a national case-control study of patients with mesothelioma. We asked the Department of Health to provide the names of clinicians with newly diagnosed eligible patients (not the patients’ names) from the hospital inquiry system, but this was refused for over two years. The information commissioner’s office finally advised the hospitals that it is not illegal to put medical researchers in touch with doctors. To obtain random population controls from general practitioners’ lists again entailed protracted correspondence with NHS data custodians in each region before the project could begin, and several areas, including the whole of Scotland, still refuse to participate.

At a public meeting in November 2002, organised by the Parliamentary Group on Cancer and opened by Alan Milburn, then secretary of state for health, the audience were provided with an electronic voting facility. After a discussion of the restrictions on access to medical records that British epidemiologists now face and their effects on our work, the audience were invited to vote for or against the following proposed law: “Consent is not required for access to medical records for non-commercial medical research that has no effect on the individuals being studied and has been approved by an accredited research ethics committee.”

The vote in favour was 93%. The audience included members of the general public, patients’ support groups and cancer charities, doctors, nurses, and public health workers. The widespread belief among politicians and senior civil servants that the public would no longer tolerate access to their records by bona fide medical researchers is assiduously promulgated by many medical ethicists and lawyers, but it is not true. Even the tiny minority of patients and controls who express any concern that we have identified and contacted them through their medical records are almost always satisfied when we explain our work to them.

The only substantive difference between the proposed exemption for medical research that this audience supported so overwhelmingly and the relevant parts of the 1998 Data Protection Act and the 2001 Health and Social Care Act is the requirement that permission for medical researchers to access individual records without informed consent must be granted by the Patient Information Advisory Group. Multicentre research ethics committees typically include general practitioners, hospital specialists, nurses, ethicists, lay members, and patients or their representatives. The government has not explained why the Health and Social Care Act usurped the authority of this effective national network of highly professional committees by creating the Patient Information Advisory Group.

None of the hundreds of data custodians in hospitals, cancer registries, and primary health care trusts with whom we have corresponded has ever indicated that our access to their records might raise ethical concerns. These are resolved in discussion with the multicentre research ethics committee before a research protocol is approved. Their concern relates solely to their legal liability, and it can be allayed only by clarifying the law. The Data Protection Act was intended to accommodate medical research, as Lord Falconer claims it does. In practice, however, data custodians are increasingly cautious, and the current government seems unlikely to grant the explicit exemption for noncommercial medical research that would resolve their fears.

Lord Falconer further asserts that it is the common law, not the Data Protection Act, that demands informed consent for medical researchers to access named patients’ records “unless there is an overriding public interest.”4 This suggests a practical solution. Medical researchers have been allowed confidential access to medical records throughout the ages, and until recently such access without informed consent was explicitly allowed under the Medical Research Council’s guidelines on research. To reverse this ancient tradition in the name of the common law seems absurd, and a high court judge could presumably restore the status quo by simply stating in a test case that the novel demand for informed consent to access records for non-commercial medical research has no basis either in law or in established common practice. He or she would become a minor icon of 21st century medical research, for many lives and a great deal of public money would be saved.

But no such remedy can rescue British medical research from the ambiguities of the Human Tissue Bill. Doctors who store any human tissue that may be used for future research without obtaining informed consent could face punitive criminal sanctions, and many will simply discard it.

Julian Peto, Cancer Research UK chair of epidemiology


Olivia Fletcher, lecturer

London School of Hygiene and Tropical Medicine, London WC1E 7HT

Clare Gilham, medical statistician

Institute of Cancer Research, Sutton, Surrey SM2 5NG

Competing interests: None declared.


1. Cancer Research UK. Call for clarity on human tissue bill. Press release, 14 January 2004. (accessed 18 Feb 2004).
2. Information Commissioner. Data protection—helping organisations to get it right. Press release, 14 January 2004. (accessed 2 Mar 2004).
3. Wanless D. Securing good health for the whole population. Final report, 25 February 2004. (accessed 31 Mar 2004).
4. Lord Falconer of Thoroton. Privacy law and medical research [letter]. Times 2001 May 17: 21.

Related Articles

National survey of British public’s views on use of identifiable medical data by the National Cancer Registry
Geraldine Barrett, Jackie A Cassell, Janet L Peacock, and Michel P Coleman
BMJ 2006 332: 1068-1072. [Abstract] [Full Text]

Shouldn’t patients decide who should access their records?
Joan Higgins
BMJ 2004 329: 573. [Extract] [Full Text]

Data protection, informed consent, and research: Data Protection Act does not bar medical research
Stewart Room
BMJ 2004 328: 1437. [Extract] [Full Text]

Data protection, informed consent, and research: Interpretation of legislation should reflect patients’ views
Jonathan Fletcher, Joanna Marriott, and David Phillips
BMJ 2004 328: 1437. [Extract] [Full Text]

NHS national programme for information technology: Programme erodes confidentiality of medical records
Ted A Willis
BMJ 2004 328: 1200. [Extract] [Full Text]

This article has been cited by other articles:
(Search Google Scholar for Other Citing Articles)

* Califf, R. M (2006). Clinical trials bureaucracy: unintended consequences of well-intentioned policy. Clin Trials 3: 496-502 [Abstract]
* Trevena, L, Irwig, L, Barratt, A (2006). Impact of privacy legislation on the number and characteristics of people who are recruited for research: a randomised controlled trial.. J. Med. Ethics 32: 473-477 [Abstract] [Full text]
* Barrett, G., Cassell, J. A, Peacock, J. L, Coleman, M. P (2006). National survey of British public’s views on use of identifiable medical data by the National Cancer Registry. BMJ 332: 1068-1072 [Abstract] [Full text]
* Ford, H. L (2006). The effect of consent guidelines on a multiple sclerosis register. Mult Scler 12: 104-107 [Abstract]
* Shaw, S., Barrett, G. (2006). Research governance: regulating risk and reducing harm?. J. R. Soc. Med. 99: 14-19 [Abstract] [Full text]
* Hotopf, M., Wessely, S. (2005). Can epidemiology clear the fog of war? Lessons from the 1990-91 Gulf War. Int J Epidemiol 34: 791-800 [Abstract] [Full text]
* Yusuf, S. (2004). Randomized clinical trials: Slow death by a thousand unnecessary policies?. CMAJ 171: 889-892 [Full text]
* Higgins, J. (2004). Shouldn’t patients decide who should access their records?. BMJ 329: 573-573 [Full text]
* Pirmohamed, M., Darbyshire, J. (2004). Collecting and sharing information about harms. BMJ 329: 6-7 [Full text]
* Room, S. (2004). Data protection, informed consent, and research: Data Protection Act does not bar medical research. BMJ 328: 1437-1437 [Full text]
* Fletcher, J., Marriott, J., Phillips, D. (2004). Data protection, informed consent, and research: Interpretation of legislation should reflect patients’ views. BMJ 328: 1437-1437 [Full text]
* Willis, T. A (2004). NHS national programme for information technology: Programme erodes confidentiality of medical records. BMJ 328: 1200-1200 [Full text]

If you like what I do, and you want me to do more, you can: buy my books Bad Science and Bad Pharma, give them to your friends, put them on your reading list, employ me to do a talk, or tweet this article to your friends. Thanks! ++++++++++++++++++++++++++++++++++++++++++

7 Responses

  1. Gimpy said,

    June 6, 2007 at 8:00 pm

    Ben (or anyone),

    Do you know what the position is on the release of patient data to biotech companies?

  2. RS said,

    June 6, 2007 at 8:15 pm

    Obviously that should have said “opt-out organ donation.

    I think it is odd that in a nation so pro-fluffy animals and anti-animal research there is so little commitment to research or even life-saving donation involving post-mortem tissue. We’re a funny, selfish lot.

  3. Observer said,

    June 6, 2007 at 11:13 pm

    For those of us on the other side of the pond who have to pay for medical insurance, there is another side to this question. For example see the BBC article:

    titled “Serious diseases genes revealed”
    Imagine trying to get health insurance if you were known to have one of the suspect genes.

  4. RS said,

    June 7, 2007 at 10:24 am

    The insurance industry made a very clever mover regarding susceptibility genes – it imposed a moratorium (that I think may well have run out now) which essentially allowed it to side-step the issue.

    In principle there is nothing wrong with insurance companies knowing about large genetic risks – they already do all they can to find out about other kinds of risks and duly penalise you for them (as any gay man or cancer patient could tell you). The problem is the population wide effect – the only solution is state legislated risk sharing (such as happens in Ireland). Even that has downsides in somewhere like the UK where insurance cover is rare – because then the patient has an asymmetrical knowledge advantage over the insurance company.

    Is it gattaca where it is illegal to do tests on people but the technology is so developed that they just do it without your knowledge? A bit like the compulsion now to get a health form filled in by your doctor and sent to a potential employer, they can’t legally discriminate against you if you have a disability (althoug funnily enough, if it isn’t serious enough to be considered a disability, e.g. mild depression in the past, they can), but who’s to know?

  5. terryhamblin said,

    June 9, 2007 at 11:05 am

    My research, which defined which types of chronic lymphocytic leukaemia were lethal and which were benign, and which won international acclaim and recognition would have been illegal under the Human Tissues Act.

    My advice to epidemiologists and their employers is to ignore the law. I wouild welcome being prosecuted under it because it would then be exposed as unworkable. As someone who always drives at 70 mph on motorways I can tell you that bad laws are always ignored and the law enforcement agencies turn a blind eye.

  6. Goozelle said,

    June 13, 2007 at 3:11 pm

    Misplaced concern about privacy and the overplaying of issues to do with the security of medical records is hampering the NHS National Programme for IT which is developing a “data spine” so that healthcare professionals can access medical records wherever they are in the country. There is a campaign to opt-out of having your medical information on the spine (I think the Guardian was prominent in this). I hope the people who have been persuaded to opt out do not find themselves in a position where emergency medical staff cannot access vital information about their condition that may save their life.

  7. diudiu said,

    December 21, 2009 at 6:35 am

    free shipping ugg
    free shipping ugg