You’re right, it goes beyond them; I was just using them and the pro- abstinence partners as an example. But informed pedantry is something we all do well…

A bit of pedantry… but the pro-male circumcision lobby for HIV prevention goes beyond PEPFAR; it’s also a key policy of UNAIDS (www.unaids.org/en/PolicyAndPractice/Prevention/MaleCircumcision/) and of the World Health Organisation (www.who.int/hiv/topics/malecircumcision/en/index.html).

At first the evidence looked very good, so much so that the randomised control trials had to be stopped because it was considered unethical to withhold male circumcision from the control group. The results (I can’t remember them exactly, but it was something like a 40-60% risk reduction) were obviously pretty seductive to policy makers who have been desperately trying, for decades, to find ways of making inroads in HIV prevention.

But, yes there were potential biases, and of course it’s not really possible to blind a trial of such an intervention… Now, as you suggest, there are concerns about risk compensation among circumcised men (i.e. having more unprotected sex because they feel safe) and therefore about how effective the strategy will be in the longer term.

Anyway, enough of the pedantry. You are absolutely right about questioning the evidence base for the policies of different donors. But the most insidious part of PEPFAR has been about “abstinence only” programming; tragically it is only in the last year or so that we’ve seen clearer evidence that abstinence-only programmes are a waste of time, and maybe counterproductive.

One quirk of dealing with developing countries: People towards the top of organizations in these societies tend to flaunt their powerful social status in ways that offend most Western science geeks. You may be the most important *person* in the world, but your impressive social position and enormous power doesn’t change the *data*, which is what I want to talk about. I don’t need a big build-up about how important you are; if I know anything about the subject, I’ll eventually answer your question (and also tell you how confident I am in my ability to answer it). I’m just as likely to answer questions about “my sister has breast cancer” or “I think I have an autoimmune disease” as I am from some high muckety-muck in government that needs to make a big public health decision.

One thing to encourage people to check is who is funding initiatives that appear scientific and question what their motives might be.

For example, PEPFAR is a US partnership promoting and heavily funding circumcision over condoms as HIV/AIDS prevention in Africa.

When you dig into who their partners are, a lot of them are religious organisations who are anti-condom and pro-abstinence. When you look at the research for the effectiveness of circumcision and the problems in carrying it out, the initiative starts to look pretty dodgy.

So do look a gift horse in the mouth.

Sorry to blow my own not very scientific trumpet but I’ve done a bit of digging and blogged the details here: tessera2009.blogspot.com

Putting a load of graphs and charts up on an overhead can still bedazzle a room full of supposedly educated people with an interest in an area like autism. When in reality the presentation owes more to PT Barnham, than BF Skinner (insert preferred scientist)

There is also a rather wealthy television presenter across the pond who could do with a little accurate think tank data analysis too. (or not, perhaps her science is that of Mr Barnham and his populist approach to getting insanely rich)

xx

I worked mainly on biomass energy. At the 1991 UN Energy Conference, in Nairobi, my Kenyan colleagues arranged a march of women carrying wood on their heads. The developed country visitors thought the world’s energy problems were about oil and electricity. But at that time, the main energy source for 95% of the world’s population was wood. It was only the rich in the developed worlds and some of the big cities in underdeveloped countries that used large amounts of fossil fuels. So everyone pushing large dams or nuclear power plants on developing countries as a solution to that energy crisis were providing a solution to the wrong problem: the oil crisis, not the woodfuel one.

My colleagues were the snake oil salesmen at the time, trying to persuade governments to improve the energy efficiency of wood and charcoal stoves, to give people enough time to plant more trees. It was based on good science (heat transfer, laws of thermodynamics, kinetics of combustion), but it took a lot of work to turn that into the Kenya Ceramic Jiko, and get artisans to turn out 125000 stoves a year, saving 1/3rd of the charcoal previously used.

How did we make sure that the better stoves were taken up, and some silly ideas (like gasifying stoves) were not? By lots of testing, and getting all the people involved to talk to each other – the civil servants, aid agencies, NGOs, university departments, small companies, so that everyone understood the links between design, testing, and energy saving in the home.

So you should get the civil servants to make a list of all the scientifically aware people or organisations in their countries who could help them assess proposals from outside. The big risk is in turnkey development, be it just buying in a factory, or buying in drugs, without understanding it well enough to change things when the inevitable problems arise. But get together a lot of sceptical scientists, and it is possible to pick out the best and worst of the proposals.

This sounds really interesting.

In your introduction you emphasise procurement; the issue of evidence (as others have pointed out) is also important for prioritisation of public policy, and design of public health programmes more generally, including those funded by external donors of course. Take the example of AIDS programmes.

Firstly, there needs to be an understanding of who is most affected, and who is most at risk, not only because it is important to support those who are disproportionately affected but also because if programmes can target these people then they will probably have the most impact at a national level. So here we need to understand relative risk and population attributable risk.

Often with AIDS, the people most affected are often disadvantaged for other reasons – it may be possible to help them to access the services that everyone else has access to, but more often it can be necessary to design specific programmes that help deal with the specific issues they face – for instance sex workers often face violence and other forms of discrimination; homophobia also poses a major challenge for AIDS programmes. In these circumstances, handing out leaflets and condoms and telling them to get tested isn’t enough.

Just to complicate matters, it is these “specific factors” that mean a) that there is a lack of evidence for relative risk/attributable risk in these groups (because no-one is doing the research); b) that even where there is evidence, donors and civil servants don’t particularly care about these groups or want to go there because it is politically or morally controversial to do so (if you’ve got a big pot of money for AIDS, it’s a lot easier and sexier to spend it on “deserving” school kids than on “morally corrupt” sex workers, drug users or men who have sex with men); c) that even if the data on attributable risk are there, the proposed interventions are more likely to be determined by the moral attitudes of the society, government or donors in question than by evidence; d) that even if governments and donors agree to do the right thing, the local health care providers or non-governmental organisations that get the funding to provide services may spend the money on something else anyway.

You’ve talked a bit about c) before when discussing the US government’s policy on AIDS programmes for sex workers and on abstinence-only programming. (By the way, without wanting to let the US government off the hook for this it’s worth recognizing that these policies only confirmed pre-existing attitudes in many countries; it’s not as if they were all empowering sex workers and promoting comprehensive sex education in schools before Bush arrived).

So: while the influence of foreign donors and businesses can be pernicious, it’s important not to assume that they are the only factors that compromise the design of public programmes… and while it’s crucial for national level decisions on programme strategy and procurement are based on evidence, it’s also crucial to strengthen the evidence base and to ensure that this knowledge filters down to those who are actually delivering services.

Excessive precision. 34.62%, 23,405 jobs will be created.

Lack of uncertainties, inappropriate uncertainties (wrong precision), or if given no indication of what the uncertainty is (std err, sd etc etc). 34.62 +/- 5.25

Poor referencing. A study showed 23,405 jobs would be created. (Man, I want to author a “study” one day that for 24 hours is on every news report).

What I call “colloquial” language. An actual quote from a submission to a Select Committee: “Fascist New World Order”. Usually though milder but uses words like “silly”.

Any discussion of motivations for alternative views. (Cuts both ways so not always a useful criterion).

Multiple but not related lines of “evidence”. E.g. “I don’t accept climate change because it is a hoax with no evidence and anyhow it is all caused by the sun.”

Forcing on plots. The text cites a strong relationship. The graph shows an amorphous blob of 97 scattered data at low x and low y of the axes with 3 points at high x high y.

Absurd extrapolations. Improbable Research used to give these. Since no one ever throws away National Geographic magazines it was possible to calculate when the accumulated mass would cause the US to sink into the sea.

Junk references. Michael Crichton was many things but he was not a reliable source of evidence to cite about climate change.

Re 79

I think you misunderstand what Ben is suggesting – not lecturing people on climate change (perhaps avoid it completely if it is likely to be counterproductive) but more generally explaining how industry or idealogical groups (from all sides of the political spectrum and beyond) can misrepresent science and mislead people.

see denialism on scienceblogs

“Denialism is the employment of rhetorical tactics to give the appearance of argument or legitimate debate, when in actuality there is none. These false arguments are used when one has few or no facts to support one’s viewpoint against a scientific consensus or against overwhelming evidence to the contrary. They are effective in distracting from actual useful debate using emotionally appealing, but ultimately empty and illogical assertions.

Examples of common topics in which denialists employ their tactics include: Creationism/Intelligent Design, Global Warming denialism, Holocaust denial, HIV/AIDS denialism, 9/11 conspiracies, Tobacco Carcinogenecity denialism (the first organized corporate campaign), anti-vaccination/mercury autism denialism and anti-animal testing/animal rights extremist denialism. Denialism spans the ideological spectrum, and is about tactics rather than politics or partisanship. Chris will be covering denialism of industry groups, such as astroturfing, and the use of a standard and almost sequential set of denialist arguments that he discusses in his Denialist Deck of Cards.

5 general tactics are used by denialists to sow confusion. They are conspiracy, selectivity (cherry-picking), fake experts, impossible expectations (also known as moving goalposts), and general fallacies of logic.”

www.scidev.net/en/science-communication/practical-guides/spotting-fraudulent-claims-in-science.html

There also an editorial that David Dickson, director and editor of the website scidev.net wrote last year called “Building a web we can trust”. The link is here:

www.scidev.net/en/editorials/how-to-build-a-web-we-can-trust.html

A good case history — which acts as a bit of a counter example– is the treatment of critics scientists who protested at the Gambian president’s claim that he could cure HIV/AIDS.

See: www.scidev.net/en/news/who-dampens-gambian-presidents-hivaids-claim.html

Also the fact that the Nigerian Academy of Sciences was sued when it issue a critical report about a local doctor who claimed similar powers.

See: www.scidev.net/en/news/hiv-cure-doctor-sues-science-academy.html

I think you misunderstand what Ben is suggesting – not lecturing people on climate change (perhaps avoid it completely if it is likely to be counterproductive) but more generally explaining how industry or idealogical groups (from all sides of the political spectrum and beyond) can misrepresent science and mislead people.

see scienceblogs.com/denialism/about.php

“Denialism is the employment of rhetorical tactics to give the appearance of argument or legitimate debate, when in actuality there is none. These false arguments are used when one has few or no facts to support one’s viewpoint against a scientific consensus or against overwhelming evidence to the contrary. They are effective in distracting from actual useful debate using emotionally appealing, but ultimately empty and illogical assertions.

Examples of common topics in which denialists employ their tactics include: Creationism/Intelligent Design, Global Warming denialism, Holocaust denial, HIV/AIDS denialism, 9/11 conspiracies, Tobacco Carcinogenecity denialism (the first organized corporate campaign), anti-vaccination/mercury autism denialism and anti-animal testing/animal rights extremist denialism. Denialism spans the ideological spectrum, and is about tactics rather than politics or partisanship. Chris will be covering denialism of industry groups, such as astroturfing, and the use of a standard and almost sequential set of denialist arguments that he discusses in his Denialist Deck of Cards.

5 general tactics are used by denialists to sow confusion. They are conspiracy, selectivity (cherry-picking), fake experts, impossible expectations (also known as moving goalposts), and general fallacies of logic.”