I’m very much looking forward to this important press release, of a study in 4 children, making massive international news. The experiment is part of the promotional activity for another omega-3 pill called VegEPA, and a Channel Five documentary on children and diet to be broadcast later this week: it is unpublished, and this time the study, amazingly, was funded by the TV production company Endemol. They love these stories so much, they’ve started paying for the research. This represents a really interesting new development in the interplay between commercial companies making seductive claims about pills solving complex social problems, and the media who love them. They’ll be giving out their own degrees next.

For more on the dangers of making great leaps of faith on the real world abilities of a treatment using a theoretical surrogate outcome, I’d always recommend reading the excellent Trisha Greenhalgh, here on “Evidence and Marketing”:

www.bmj.com/archive/7106/7106ed.htm

This is taken from her book “How to Read a Paper: The Basics of Evidence-Based Medicine”, a highly readable, industry standard, medical student text. As you are aware, the whole of badscience.net is really just a covert excuse to bring the joys of evidence based medicine and primary academic literature to the masses.

The fact that medical students are taught this kind of stuff – the critical appraisal of the evidence behind marketing claims – while nutritionists play actively into the hands of the people trying to sell them stuff with “science” is, to my mind, the key difference between “nutritionism” and real science. I honestly cannot recommend highly enough that you read the Greenhalgh chapter. Go on. Click the picture. Buy the book. Seriously. It’s got jokes about drug reps and everything. Much funnier than the dry online version. And evidence based medicine is the most significant development in the history of western thought for a generation [I’ve cheekily pasted the whole of the relevant chapter in the comments below now, sorry Prof..]

Diet pill ‘made children smart’

Press Association
Monday March 12, 2007 6:43 AM

Four children whose brains and mental abilities were suddenly transformed after they took a simple dietary supplement have astonished scientists.

Scans showed their brains underwent three years’ of development in just three months.

At the same time they displayed remarkable improvements in tests of reading, concentration, problem-solving and memory.

One boy who previously scorned books and was hooked on TV developed a love of reading and declared he was “bored” of television.

The three boys and one girl, aged between eight and 13, were taking part in a pilot study looking at the effects of diet on developing young brains.

Scientists believe the results are powerful evidence of the harm “junk food” is doing to Britain’s children.

The children were given a supplement called VegEPA, which contains a combination of omega-3 and omega-6 essential fatty acids, and encouraged to cut down on fatty snacks and fizzy drinks and be more active. At the end, they underwent brain scans at St Georges Hospital, London, in a machine that can detect a key biochemical indicator of brain development called NAA (N-Acetylaspartate).

Higher levels of NAA correspond to more nerve fibres growing in the brain..

Professor Basant Puri, from Imperial College London, who led the study, said: “The results were astonishing. In three months you might expect to see a small NAA increase. But we saw as much growth as you would normally see in three years.

“It was as if these were the brains of children three years older. It means you have more connections and greater density of nerve cells, in the same way that a tree grows more branches.”

© Copyright Press Association Ltd 2007, All Rights Reserved.

Sarah Lovell < ###@suzannemartin.biz> |
| Subject: Re: Children Improve by Three Years in Three month Study |
>——————————————————————————————————————-|

Professor Puri is available for Interview please call Suzanne Martin on
0207 ######## or 07950 #####

MIND THE FAT
A study reveals astonishing findings

We know that a bad diet can affect a child’s physical health, but could it
be bad for their minds as well? All is revealed in the FIRST EVER study of
its kind with dramatic findings on channel five, 15th March 2007 in a
documentary entitled ‘Mind The Fat: Does Fast Food = Slow Kids?

In this fascinating documentary, four clinically overweight children who
are struggling at school take part in a groundbreaking three-month study.

The four children taking part are eight-year old Zach, who weighs eight
stone; George and Rochel, who are both 11 and weigh 11 stone each; and
13-year old Gareth, who weighs 12 stone.

Leading neuroscientist Professor Basant Puri conducted the study whereby he
carried out a series of tests with the children, involving a number of
challenges including handwriting, concentration, problem-solving ability
and memory. One test in particular the Stroop test, where the child has to
say out loud the colour of each word printed (the words all describe the
names of colours, but they are not printed in the actual colour that they
describe) was also conducted.

Prof Puri also scanned the brains of each child using state-of-the-art MRI
techniques, at a major London teaching hospital to assess whether the size
and makeup of their brains will alter. The exact tests were carried out
again after the three-months.

Professor Puri is also prescribed the children a dietary supplement called
VegEPA – a unique patented formulation containing a natural source of
omega-3 and omega-6 essential fatty acids consisting of EPA from marine
fish oil and virgin cold-pressed evening primrose oil. Most distinctly,
VegEPA does not contain DHA- and each child had to take two capsules per
day during the three-month study as well as keeping to their healthy eating
and fitness regime.

While Puri’s existing research shows a strong link between mental
performance and EPA, he was hoping that combining these supplements with a
change in diet would produce even more remarkable results.

Puri explains “Junk food contains bad fats, and those bad fats insinuate
themselves into the brain. They replace the good fats and make brain cell
membranes dry up so that signals don’t pass well between one brain cell and
another.”

After a challenging three months during which all the participants
attempted to eat more healthily, exercise more and take daily the VegEPA
supplements, a weigh-in reveals that three out of four have lost weight.
Of course, this was not the primary purpose of the programme. More
significant is the fact that all the children now perform much better than
before in the four challenges. In fact, they all perform at least one and a
half years above the expected results for their age group.

Professor Puri comments, “The results of this study were astonishing. After
taking VegEPA daily for just three months, the children showed an increase
in reading age of well over a year. Also, during three months we would
expect their brains to have grown by three months’ worth. In fact, they
showed an increase in their brain growth by three years’ worth. This was
reflected in their improved arithmetical and constructional skills and in
their memory and concentration. Their teachers and parents noticed the
improvements, such as better concentration and attention in class, better,
neater and more accurate handwriting, and even, in the case of one of the
boys, developing a preference for reading over watching television (which
was in marked contrast to his view of reading just three months’ earlier).”

www.vegepa.com

Orderline 0845 ###

-ends-

For further information, to interview Professor puri or for samples of
VegEpa please contact Suzanne Martin or Tina Barratt on 0207 #### or
email at @suzannemartin.biz

There now follows a subliminal message.

This has now hit a phenomenal number of media outlets. The media love these stories so much, now they’re funding them for themselves!

Diet pill 'made children smart'
ic Wales, UK – 7 hours ago
Four children whose brains and mental abilities were suddenly transformed after they took a simple dietary supplement have astonished scientists. …

Kids' brains boosted by pills ITN, UK – 4 hours ago Four children whose mental abilities were suddenly improved after they took a simple dietary supplement have astonished scientists. …

Supplement 'boosts' brain power BBC News, UK – 3 hours ago Brain scans showed three years worth of development in just three months in the children, says Imperial College researcher Professor Basant Puri. …

And now, over to the Prof…

www.bmj.com/archive/7106/7106ed.htm

“Evidence” and marketing
If you prescribe drugs, the pharmaceutical industry is interested in you and is investing a staggering sum of money trying to influence you. The most effective way of changing the prescribing habits of a clinician is through personal representatives (known in Britain as “drug reps” and in North America as “detailers”), who travel round with a briefcase full of “evidence” in support of their wares.(1)

Pharmaceutical “reps” do not tell nearly as many lies as they used to (drug marketing has become an altogether more sophisticated science), but they have been known to cultivate a shocking ignorance of basic epidemiology and clinical trial design when it suits them.(2) It often helps their case, for example, to present the results of uncontrolled trials and express them in terms of before and after differences in a particular outcome measure.(3) The recent correspondence in the Lancet and BMJ on placebo effects should remind you why uncontrolled before and after studies are the stuff of teenage magazines, not hard science.(4-12)

Summary points
Pharmaceutical “reps” are now much more informative than they used to be, but they may show ignorance of basic epidemiology and clinical trial design
The value of a drug should be expressed in terms of safety, tolerability, efficacy, and price
The efficacy of a drug should ideally be measured in terms of clinical end points that are relevant to patients; if surrogate end points are used they should be valid
Promotional literature of low scientific validity (such as uncontrolled before and after trials) should not be allowed to influence practice

Making decisions about treatment

Sackett and colleagues have argued that before giving a drug to a patient the doctor should:

identify, for this patient, the ultimate objective of treatment (cure, prevention of recurrence, limitation of functional disability, prevention of later complications, reassurance, palliation, relief of symptoms, etc);

select the most appropriate treatment, using all available evidence (this includes considering whether the patient needs to take any drug at all); and

specify the treatment target (to know when to stop treatment, change its intensity, or switch to some other treatment).(13)

For example, in treating high blood pressure, the doctor might decide that:

the ultimate objective of treatment is to prevent (further) target organ damage to brain, eye, heart, kidney, etc (and thereby prevent death);

the choice of specific treatment is between the various classes of antihypertensive drug selected on the basis of randomised, placebo controlled and comparative trials – as well as non-drug treatments such as salt restriction; and

the treatment target might be a phase V diastolic blood pressure (right arm, sitting) of less than 90 mm Hg, or as close to that as tolerable in the face of drug side effects.

If these three steps are not followed (as is often the case – for example in terminal care), therapeutic chaos can result.
Surrogate end points

A surrogate end point may be defined as a variable which is relatively easily measured and which predicts a rare or distant outcome of either a toxic stimulus (such as a pollutant) or a therapeutic intervention (a drug, surgical procedure, piece of advice, etc) but which is not itself a direct measure of either harm or clinical benefit. The growing interest in surrogate end points in medical research, and particularly by the pharmaceutical industry, reflects two important features of their use:

they can considerably reduce the sample size, duration, and, therefore, cost, of clinical trials; and

they can allow treatments to be assessed in situations where the use of primary outcomes would be excessively invasive or unethical.

In the evaluation of pharmaceutical products, commonly used surrogate end points include:

pharmacokinetic measurements (for example, concentration-time curves of a drug or its active metabolite in the bloodstream);

in vitro (laboratory) measures such as the mean inhibitory concentration of an antimicrobial against a bacterial culture on agar;

macroscopic appearance of tissues (for example, gastric erosion seen at endoscopy)

change in levels of (alleged) serum markers of disease (for example, prostate specific antigen(14) );

radiological appearance (for example, shadowing on a chest x ray film).

But surrogate end points have some drawbacks. Firstly, a change in the surrogate end point does not itself answer the essential preliminary questions: “what is the objective of treatment in this patient?” and “what, according to valid and reliable research studies, is the best available treatment for this condition?” Secondly, the surrogate end point may not closely reflect the treatment target – in other words, it may not be valid or reliable. Thirdly, overreliance on a single surrogate end point as a measure of therapeutic success usually reflects a narrow clinical perspective. Finally, surrogate end points are often developed in animal models of disease, since changes in a specific variable can be measured under controlled conditions in a well defined population. However, extrapolation of these findings to human disease is likely to be invalid.(15-17)

Features of the ideal surrogate end point

The surrogate end point should be reliable, reproducible, clinically available, easily quantifiable, affordable, and show a “dose-response” effect (the higher the level of the surrogate end point, the greater the probability of disease)

It should be a true predictor of disease (or risk of disease) and not merely express exposure to a covariable. The relation between the surrogate end point and the disease should have a biologically plausible explanation

It should be sensitive – a “positive” result in the surrogate end point should pick up all or most patients at increased risk of adverse outcome

It should be specific – a “negative” result should exclude all or most of those without increased risk of adverse outcome

There should be a precise cut off between normal and abnormal values

It should have an acceptable positive predictive value – a “positive” result should always or usually mean that the patient thus identified is at increased risk of adverse outcome

It should have an acceptable negative predictive value – a “negative” result should always or usually mean that the patient thus identified is not at increased risk of adverse outcome

It should be amenable to quality control monitoring

Changes in the surrogate end point should rapidly and accurately reflect the response to treatment. In particular, levels should normalise in states of remission or cure

The features of an ideal surrogate end point are shown in the box. If the “rep” who is trying to persuade you of the value of the drug cannot justify the end points used, you should challenge him or her to produce additional evidence.

One important example of the invalid use of a surrogate end point is the CD4 cell count in monitoring progression to AIDS in HIV positive subjects. The CONCORDE trial was a randomised controlled trial comparing early and late start of treatment with zidovudine in patients who were HIV positive but clinically asymptomatic.(18) Previous studies had shown that starting treatment early led to a slower decline in the CD4 cell count (a variable which had been shown to fall with the progression of AIDS), and it was assumed that a higher CD4 cell count would reflect improved chances of survival.

However, the CONCORDE trial showed that, although CD4 cell counts fell more slowly in the treatment group, the three year survival rates were identical in the two groups. This experience confirmed a warning that was issued earlier by authors suspicious of the validity of this end point.(19) Subsequent research in this field has attempted to identify a surrogate end point that correlates with real therapeutic benefit – that is, delayed progression of asymptomatic HIV infection to clinical AIDS, and longer survival time after the onset of AIDS.(20-21) Using multiple regression analysis, investigators in the USA found that a combination of markers (percentage of CD4:C29 cells, degree of fatigue, age, and haemoglobin concentration) was the best predictor of progression.(20)

Other examples of surrogate end points which have seriously misled researchers include ventricular premature beats as a predictor of death from serious cardiac arrhythmias,(22-23) blood concentrations of antibiotics as a predictor of clinical cure of infection,(24) and plaques seen on magnetic resonance imaging in monitoring the progression of multiple sclerosis.(25)

Before surrogate end points can be used in the marketing of pharmaceuticals, those in the industry must justify the utility of these measures by showing a plausible and consistent link between the end point and the development or progression of disease. It would be wrong to suggest that the pharmaceutical industry develops surrogate end points with the deliberate intention to mislead the licensing authorities and health professionals. However, the industry does, theoretically, have a vested interest in overstating its case on the significance of these end points. Given that much of the data relating to the validation of surrogate end points are not currently presented in published clinical papers, and that the development of such markers is often a lengthy and expensive process, one author has suggested setting up a data archive that would pool data across studies.(26)
How to get evidence out of a drug rep

Any doctor who has ever given an audience to a “rep” who is selling a non-steroidal anti-inflammatory drug will recognise the argument that “this NSAID reduces the incidence of gastric erosion in comparison to its competitors.” The question to ask the rep is not “what is the incidence of endoscopic signs of gastric erosion in volunteers who take this drug?” but “what is the incidence in clinical practice of potentially life threatening gastric bleeding in patients who take this drug?” Other questions, collated from recommendations in Drug and Therapeutics Bulletin(27) and other sources,(1)(3) are listed below.

See representatives only by appointment. Choose to see only those whose product interests you, and confine the interview to that product

Take charge of the interview. Do not hear out a rehearsed sales routine but ask directly for the information below

Request independent published evidence from reputable, peer reviewed journals

Do not look at promotional brochures, which may contain unpublished material, misleading graphs, and selective quotations

Ignore anecdotal “evidence,” such as the fact that a medical celebrity is prescribing the product

Using the STEP acronym, ask for evidence in four specific areas:
Safety – the likelihood of long term or serious side effects caused by the drug (remember that rare but serious adverse reactions to new drugs may be poorly documented)
Tolerability – best measured by comparing the pooled withdrawal rates between the drug and its most significant competitor
Efficacy – the most relevant dimension is how the product compares with your current favourite
Price – should take into account indirect as well as direct costs

Evaluate the evidence stringently, paying particular attention to the power (sample size) and methodological quality of clinical trials, and the use of surrogate end points. Do not accept theoretical arguments in the drug’s favour (|mKlonger half life,” for example) without direct evidence that this translates into clinical benefit

Do not accept the newness of a product as an argument for changing to it. Indeed, there are good scientific arguments for doing the opposite(28)

Decline to try the product via starter packs or by participating in small scale, uncontrolled “research” studies

Record in writing the content of the interview and return to these notes if the “rep” requests another audience

Checklist for evaluating information provided by a drug company

Does this material cover a subject which interests me and is clinically important in my practice?

Has this material been published in independent peer reviewed journals? Has any significant evidence been omitted from this presentation or withheld from publication?

Does the material include high-level evidence such as systematic reviews, meta-analyses, or double-blind randomised controlled trials against the drug’s closest competitor given at optimal dosage?

Have the trials or reviews addressed a clearly focused, important and answerable clinical question which reflects a problem of relevance to patients? Do they provide evidence on safety, tolerability, efficacy and price?

Has each trial or meta-analysis defined the condition to be treated, the patients to be included, the interventions to be compared and the outcomes to be examined?

Does the material provide direct evidence that the drug will help my patients live a longer, healthier, more productive, and symptom-free life?

If a surrogate outcome measure has been used, what is the evidence that it is reliable, reproducible, sensitive, specific, a true predictor of disease, and rapidly reflects the response to therapy?

Do trial results indicate whether (and how) the effectiveness of the treatments differed and whether there was a difference in the type or frequency of adverse reactions? Are the results expressed in terms of numbers needed to treat, and are they clinically as well as statistically significant?

If large amounts of material have been provided by the representative, which three papers provide the strongest evidence for the company’s claims?

In conclusion, it is often more difficult than you are being led to believe to weigh the potential benefits of a drug against its risks to the patient and cost to the taxpayer.(29) The difference between the science of critical appraisal and the pharmaceutical industry’s well rehearsed tactics of marketing and persuasion should be borne in mind when you are considering “evidence” presented by those with a commercial conflict of interest.

Incidentally, if anyone’s able to capture the TV footage on this that would be great. My favourite way of guerilla archiving TV stories at the moment is the video feature on cameraphone or digital camera. Hardly much worse than youtube. Can give you a way to email big files too. Bravo!