Sources I just looked at do seem to say no, it would put blood pressure up, so it’s quite a bad example.

Recently we published a paper in the BMJ (www.bmj.com/cgi/content/full/bmj.b2981) where a couple of methods were presented to tackle these reporting biases. Of course, these methods are not panacea but represent an improvement with respect to what was available until then.

Yes, they can. Basically, a big study is likely to get published whatever the results. It’s the small studies that don’t show up. If the small studies show an effect, but the big studies don’t, then that’s evidence that small, negative studies aren’t being published.

Good luck trying to extract any information like that from the Mathieu paper. As I’ve been trying to say, there’s absolutely no information on drug, Pharma, phase of trial, nature of endpoint or anything that would give basic context to the issues they are trying to highlight.

My friend’s daughter’s school have their HPV jabs today. The day after Natalie Morton died, the school made sure all the girls knew about it and told them to make sure their parents knew about the situation in case it would affect their decisions. Unbelievable.

“Blaming the girl, not the vaccine
Today, the mainstream media is reporting an obviously-fabricated explanation for her death. A pathologist is declaring that Natalie died from a “malignant chest tumor” that just coincidentally and suddenly killed her within hours after she received the cervical cancer vaccine.

This explanation is obviously a cover story to protect the vaccine industry; and it’s not even a convincing cover story at that. Natalie Morton had never been diagnosed with a chest tumor before, and she showed absolutely no symptoms of a cancer tumor. Chest tumors don’t just “lash out” and attack their hosts all of a sudden, without warning. A typical death from a cancer tumor is more often a slow, painful wasting away that can take months or years. Natalie Morton was killed in hours, and the description of her symptoms exactly matches what might be expected from an inflammatory reaction to a chemical vaccine.”

Thanks for replying but that’s a really terrible misrepresentation of what I’m questioning. Obviously you “can’t imagine how I hold my opinion” because you’ve not read what I’ve actually said.

My posts clearly show that I think witheld data from registration or pivotal phase III trials would have that effect and be a very bad thing for patient safety. That’s becuase those trials uniquely are the ones that would alter prescription practice. It’s stated clearly in my first post. How have you missed that?

What I’ve said, as have others, is that altered primary endpoints on phase I or even phase II trials would have no influence whatsoever on patient safety because none of those trials are designed to, or should, influence what drugs a patient population gets, regardless of outcome. They simply provide evidence to justify the large phase III trials that are powered to give results strong enough to influence patient prescriptions.

I thought your catchphrase was “I’m afraid it’s more complex than that” but you’ve repeatedly ignored the complexity that’s important here.

So my question to you is this: from the JAMA paper you rely on, can you tell me which trials have ambiguous or altered endpoints? Are all trials impacted equally or is it the case that the trials with altered or badly reported primary endpoints are the early stage ones?

To me if those trials are early stage trials, where endpoints are going to be more exploratory by definition, there’s little problem. If however the endpoints on phase III trials are being modified after the fact, your concerns are crucial. It’s a missed opportunity for the JAMA authors not to have focused the analysis on pivotal and regulatory-approval studies or at least provided a breakdown of findings against trial type.

jcmacc: i don’t say that the FDA hold back safety data, i explain that they hold back some efficacy data which drug companies choose not to publish. this is so commonplace that when I debated the head of wyeth at the oxford union last week, me and fiona godlee were amazed to find that he thought this was perfectly reasonable. from your posts, you seem not to believe that witheld data leads to irrational prescribing decisions and so increased suffering and death at a population level. that’s your position, but i don’t think it’s a common one and i cannot empathise with it, or even imagine how you manage to hold it. distorted data leads to inferior treatment decisions. that, in medicine, is a “really bad thing”.

He didn’t suggest it, he stated it. The statements I objected to are these:

“Bizarrely, regulatory bodies like the FDA get to see this negative data, but often enough doctors do not.This is a familiar problem, and a murderous one, because overall the results of all 8 trials combined might show that the treatment is ineffective…etc”

This clearly states the FDA hold back safety data and using the word “murderous” actually implies an intent to kill in the system given the definition of “murderous”. Unlike the Simon Singh “bogus” argument about definitions, Ben has not proposed any other meaning of the word that would mitigate the meaning by context. It’s far too emotive to use that language especially as the JAMA paper can not support any of the concerns given it’s weaknesses in lumping all types of trials as one uniform body.

Also:

“The story may be less emotive than one dead teenager, but it costs many more lives, and you should struggle to be angry about it, because the boring regulators we trust to monitor boring problems have repeatedly failed us on this one.”

I’m not sure what meaning you get from this quote that I’m missing other than that regulators are directly responsible for deaths.

And to repeat : I’m not trying to make light of the core issue, but we’re dealing with evidence-based claims and the JAMA paper does not give any information to support what Ben over-emotively concludes. Ben would rightly tear apart a woo-meister making grand claims about deaths using a scientific paper that wasn’t connected or wrongly interpreted, so I think he should be held to the standards he himself applies to others.

The best examples I know of this are all CAM meta-analyses. Given initial all-inclusive meta-analysis may be ambiguous or show a dubious positive (like just about every kind of CAM bias out there), then once you get rid of the flawed studies from your exhaustive list, the decent & worthwhile studies are left and hopefully are less susceptible to these biases. Of course, publication bias is probably the most difficult one to counter in this way…

‘What evidence is there that the FDA and other regulatory authorities are deliberately keeping quiet about trial results they are aware of i.e. what new drugs are causing deaths or what patients are being given ineffective new drugs over effective old ones?’

I’m just not sure Ben suggested this. I think he said that regulators had failed to follow up promises they made about trial results. But I’m really not sure that Ben suggested this.

Calm down next time!

The problem with this otherwise accurate statement is that it is not supported by the JAMA paper in question. Most clinical trials do not aim to alter medical practice, they aim to justify later pivotal studies or eliminate those new drugs that aren’t good enough to test in those types of study.

The JAMA paper is an intersting exercise but is a selective reporting of clinical trials only from journals they consider important (i.e. “high impact”) in very selective therapy areas – and thinking of “costing lives” and the importance of accurate reporting etc, areas such as oncology where efficacy to toxicity margins are crucial are not part of the analysis.

Crucially the JAMA paper has no useful breakdown of accuracy of reporting against trial stage thus lumps in phase I trials with phase III and even phase IV as if the importance of primary endpoint is equally valid in all contexts, it just isn’t. The authors of the paper, like Ben, have assumed all trials influence the ultimate prescribing patterns of drugs to patients which, if you understand trials can’t be true, otherwise there would only be a single phase of testing and no such concept as phase III or “registration trials”.

If the JAMA paper was about pivotal trials only, instead of all trials regardless of stage in selected journals, the findings would have real implications, and Ben’s conclusions would be correct. However, the JAMA paper describes trial databases being uploaded with 220 new trials per week all of which it holds to equal standards. There’s no way on earth that 220 phase III registration trials, which will influence prescribing and thus patient safety, begin each and every week.

The full reviews of any FDA approved drug are available on the fda.gov website thanks to US Freedom of Information law (at least for anything approved within recent memory). If you’re mildly obsessive about evidence-based medicine, it’s worth your while to pick a drug of interest and start reading.

FDA statisticians consider all the available data, including that from any unpublished, negative trials. Furthermore, trials typically need to registered at protocol stage, so burying negative results is difficult. For any given indication, the FDA publishes guidelines (also freely available) about the type and level of evidence required to demonstrate efficacy.

The FDA makes its mistakes but I see no sign of a conspiracy to hoodwink the public. Guidelines continue to improve, and the bar for safety and efficacy seems to be getting higher in most therapeutic areas. If you dig into the comments of the FDA reviewers, you’ll usually find conclusions that are more sober and carefully considered than those published in the NEJM, JAMA, Lancet etc.

There is a big difference between an abstract which says,”Drug X resulted in a 30% reduction in BP but this was not associated with any reduction in heart attacks” as opposed to, “Drug X caused 30% reduction in BP OMFG!!!”