I’ve been asked to facilitate a couple of sessions with some civil servant types from various countries in the developing world who advise their governments on science, and particularly on the science informing policy and purchasing decisions. The idea is to focus on how people might try and mislead you with science, and the range of scientific background and understanding in this group will be pretty wide, as it always is with civil servants. Since I’ve noticed a recurring theme for readers of this blog to be a bit cleverer than me (albeit less dogged and obsessive) I was hoping you might have some ideas about the kinds of areas to cover, the themes that are relevant, and the kind of structure to use.
I’ve been told that there are lots of people who go out to Africa, for example, with glossy brochures, claims of scientific evidence, and one bloke with a dodgy PhD who endorses the product, and then try to hustle and sell their systems and wares which are either ineffective, or at stupidly inflated prices. Most of these scammers fail, but it would be nice to increase the fail rate.
Subject areas that immediately spring to mind (and I’m hoping you can think of more) include:
- HIV and Aids denialism
- Climate change denialists
- Dodgy miracle cures, pharma and quack.
- Get rich quick scams
- Ropey industry claims
Now there is obviously no way you can teach the entire process of how to check evidence in every field, I wouldn’t be the person to do that, and I’m not suggesting that due diligence on govt contracts is non-existent. I think the idea is that there are some ideas and recurring themes from the kind of work we do on this blog, and others, that might be relevant to these problems.
I suppose at base we might be able to get together a rough set of nostrums around:
- checking evidence for quality and completeness (cf misrepresentation and cherry picking)
- checking background
- what constitutes evidence
- how to seek independent expert opinion
But there is also a wealth of material on stuff like:
- cognitive biases
- classic recurring themes in sciencey blags
- the themes of “denialists” generally, from the denialism blog
- and so on.
The other background issue here is that obviously I am very keen not to feel like some white dude telling people how to live and think. But I guess that’s a problem for me.
So, I could really do with some ideas here.
samarkeolog said,
May 19, 2009 at 9:09 pm
I like theonlyrick and QOF’s idea of getting them to create their own scam, because it’s engaging and self-teaching, which is empowering; but I guess it depends how much time you have.
It’s also dependent upon the audience: if you’re trying to teach non-scientists, they may not have enough base knowledge to imagine how to create a scientific scam; then you wouldn’t get good examples to laud, and they would lose confidence in themselves.
nigel edwards said,
May 19, 2009 at 9:58 pm
There is a lot of wisdom here and the sort of check list that you could compile from your own knowledge and the suggestions above would certainly be helpful. But, this is not likely to anchor the learning in a way they will remember. For influencing behaviour of managers and policy makers experience and reflection seems to work best – better still if there is some theory to hold it together.
I think you need to get them to work through some case studies that use the checklist. My experience is that participatory approaches with this sort of audience is tough, but worth a try. They do the work, help each other, you prod and suggest. This changes the dynamic about being there to tell them.
Secondly,You might ask them to reverse the process and identify what are the areas where they have seen scams, nearly got scammed, etc. You could add your stories to this. The output of this would be to spot the weaknesses in their systems and what to do about this in terms of expert advice, procurement process, keeping reps away from ministers, handling the media etc.
Thirdly, you can’t prepare them for everything so can you challenge them about how they would set up a network of advice between themselves and with their own expats in academia elsewhere using the web and other cheap technology.
The economics and cost effectiveness points mentioned are key – they are probably the ones your audience is best acquainted with. The difference between cost effectiveness and return on investment and the best bet for reducing the overall disability / disease burden is an important distinction.
VaughanB said,
May 20, 2009 at 7:30 am
Some reflections on doing science in the developing world.
It’s easy to say ‘look at the evidence’ but most of the evidence is in expensive scientific journals that are inaccessible. Furthermore, although there are many English speakers, not all scientists or practitioners in the developing world speak English.
This means that practice can be more easily influenced at the point of information dissemination. Talks, training courses, distribution of journal articles, textbooks and so on.
Often, this gap is filled by Big Pharma, and they have an influence precisely because almost every clinician and researcher I might was intelligent and motivated to use all possible opportunities to improve their field and its applications. Given a choice between some evidence and virtually no evidence, most people would make a similar choice.
Perhaps because of this, I found virtually no critical engagement with the pharmaceutical industry.
Furthermore, research has a much lower status in many developing countries. Universities are less concerned about getting good researchers because there is no big pot of government money to be won by research. However, there is money to be gained from Big Pharma drug research, adding to their influence.
Additionally, most medical professors I met couldn’t survive on university wages alone, so they do private practice in addition. This means they have little time for research.
Poor resources also mean that developing world institutions may be taken advantage of by developed world institutions. For example, the department I worked in collaborated with a large Ivy league institution doing an impressive study which involved extensive and time consuming recruitment, structural imaging and genetics. The department got their names on the first paper, but because they didn’t have the skills (or weren’t given the training) to do further analysis the many subsequent papers were attributed to the US institution. This was an ideal opportunity to disseminate skills in a rare well-funded project, and yet it left the department in the less developed country feeling like they did the donkey work while someone else got the credit.
Some countries may have only been awarding PhDs themselves for a few years. In many departments, only few of staff will have research experience.
Saying this, I found several brilliant centres of research that blew me away, these are important hubs but because scientific communication is often limited, they perhaps seem a little encapsulated by our standards. For example, there are much fewer foreign researchers passing through.
In TV advertising unsupportable health claims may be more common and unregulated (I saw lots of TV ads for Omega-3s claiming they treated cancer, mental illness and the like). Hence, I suspect people are more used to seeing strong medical claims for commercial products as normal, where we would see them as dramatic. Often these are supported by ‘doctors’, ‘scientists’ and the like.
Like many things, science can be very politicised and it is much more common for public debate to include claims that science is being used as a tool of some particular political interest.
It is important to note that I found almost no evidence of poor reasoning, lack of knowledge of scientific principles or ignorance of experimental principles. The differences were largely in communication (access to unbiased sources of information and other scientists) and motivations for doing research.
NeilHoskins said,
May 20, 2009 at 8:29 am
Although your own field is medicine and public health, the way technology and particularly IT is being introduced to the developing world is my own interest. Personally, I wonder about the usefulness (or otherwise) of the “one laptop per child” device that’s being shoved in their faces, despite the fact that it’s not entirely clear whether they asked for it.
Then there’s the ongoing battle between Microsoft and open source. “Software Libre” would intuitively seem to be the obvious choice, and Microsoft are coming under fire for negotiating binding deals, but Microsoft counter (with some evidence to back them up, frankly) that the total cost of ownership of their stuff can be less.
And it turns out that yer average Ugandan farmer’s main IT requirement was simply to know what the prices are at his local market, without having to get on a matatu to go and see for himself. This kind of thing can be (and is being) handled perfectly well with mobile phones.
It’s a fascinating field. There’s more at
www.kiwanja.net/ if you’re interested.
Dudley said,
May 20, 2009 at 10:04 am
Other people are answering the content question better than I could, but here’s a piece of advice regarding delivery: DON’T JOKE. Irony, witticism, satire – African civil servants, particularly ones who’re motivated enough to take this presentation seriously, in many cases don’t think that a funny presentation can also have serious points to make. It’s almost a reflex for English speakers, and incredibly difficult to repress, but you’ll do much better if you’re serious and respectful throughout.
Synchronium said,
May 20, 2009 at 10:07 am
I think science education might be an interesting topic that deserves your input.
Teach a man to fish ‘n’ all that.
Dudley said,
May 20, 2009 at 10:17 am
“English speakers” = “speakers at an event who come from England”
jonathon tomlinson said,
May 20, 2009 at 12:56 pm
I’ve worked as a doctor in India, Nepal and Afghanistan. If the money spent on drugs in the various clinics I worked in was spent on improving access to potable water, agriculture and nutrition, education, esp. safe childbirth, breast-feeding and contraception, clean fuel etc. I’m sure far more lives would have been saved. The history at home in the UK is that the improvement in mortality that has come about because of nutrition and sanitation vastly ecceeds that of modern medicine. Even now in the UK, with sanitation and nutrition ‘mostly’ solved, if we actually applied the evidence that we have to our clinical practice (and made less mistakes) and stopped spending money on new drugs or other interventions, we could make enormous improvements to the health of the nation.
What all nations have to do is apply the knowledge we already have, rather than be distracted by ‘new’ technological fixes.
cat said,
May 20, 2009 at 2:29 pm
Simple. Basic.
Tell people that the moment anyone suggests that their method/treatment/drug isn’t susceptible to being tested by the gold standard scientific method, the controlled double-blind trial…give them the two-fingered salute. Things work and can be shown to work; or they don’t and can’t.
cat said,
May 20, 2009 at 2:34 pm
Oh, and with drugs, to ask for the name of the active ingredient. Source the generic rather than buy the expensive ex-patent stuff.
chinaphil said,
May 20, 2009 at 2:51 pm
Most important: remind them that real science is basically free these days. If they’re being asked to pay for knowledge, then something’s gone wrong. Point to PLOS.
Second, remind them that real academics are generally more than happy to talk. If you’re not sure about a particular claim, just email someone well-known in the field and ask them.
Like you say, you can’t teach these people to distinguish bad science – particularly not if it’s clever bad science cooked up by a big company. So it’s much more important to teach them to find people who do have that ability.
And with the internet, finding those people is now free and easy.
Nik said,
May 20, 2009 at 6:45 pm
Never be afraid of asking questions! If a salesman or representative makes a claim, investigate it thoroughly, check the facts and do the arithmetic yourself. When something isn’t clear, keep asking until the answer either makes sense, or you are sure it is fluff. Often, salesmen will try and make you feel stupid for asking perfectly reasonable questions that might result in your not buying their products. Take your time and never accept the science at face value. Vast amounts of money are invested in R&D, in the hopes that vaster amounts of money will be made from selling the product. Something that’s worth having will still be there tomorrow, but if you are being sold a pup they will want to reach a binding agreement and get you before you can find out the reality. And don’t believe that money alone can buy happiness.
On specific subjects – effective and safe use of pesticides and herbicides springs to mind.
JustOneThing said,
May 20, 2009 at 10:39 pm
I’d take a little humility and listen before I advised people in the ‘Developing World’ (whatever that is), same as I would speaking anywhere to assure myself that I was starting from where the audience was at (to paraphrase Piaget).
SVC said,
May 21, 2009 at 8:35 am
I deal extensively with people from Africa, especially west and central Africa.
I would say that you should be careful to treat them with respect, which I’m sure you would. There is often a tendency for people to assume that those from “developing” countries are poorly educated or do not have ready access to the information we take for granted – for civil servants and businessman from these places you are likely to find that they are very well educated, often in Europe or the States.
If, indeed, any of these people are from Africa you should also be aware of two key components which seem to be common.
Firstly is post-colonialism; while most understand the need for foreign technical help and other support there can be a great deal of resentment entangled with accepting it. Be cautious to present your views without too much passion. Reasoned arguments rather than statements or instructions are likely to get you more mileage.
What I absolutely would not do is spend any time finger pointing – yes we all know there any many examples of snake-oil sellers duping African nations, but believe me you wont get very far with that line. Examples are best left for their own research. Even if they actually agree with you, a lot of Africans will side with another African nation against accusations of any kind from a “developed” nation. They are very good neighbours.
Denialism fits in here. It would be very easy for you to go on at length with the many examples you could bring up. I wouldn’t. There are also some cultural issues here, surrounding traditional belief systems and practices. It’s a big, touchy subject and I think you risk putting yourself on the back foot by bringing it up regardless of how important it is.
Secondly is something called “Africa time”. You really actually have to work with Africans on an equal basis to understand this. Things have a different, much more gentle pace. This can be a bit of a pain for us rushing westerners, but can be useful to you – following a line advising cautious fact checking, taking the time to make choices and listening to advise from many people. That will sound like excellent sense to Africans. Excellent sense for everyone, in fact.
In fact, I would heavily weight your talk towards the ideas of seeking and reviewing independent advise, and on how to critically assess those who give it. Choice, self determination, is extremely important for developing nations.
Gareth said,
May 21, 2009 at 11:26 am
This topic got me thinking about what I’d say to my non-scientific housemates if, for some reason, I had to teach them how to interpret science and defend against pseudoscientific reasoning (and could hold their attention on such a topic for more than 30 seconds).
I think, basically, a crash course in critical thinking would be in order. With a good sceptical attitude and reliable sources of information, it’s possible for a reasonably intelligent person to come to a pretty reasonable conclusion on a topic, even if it’s not within their area of expertise or education.
It’s like the old “Give a man a fish and you feed him for a day. Teach a man to fish and you feed him for a lifetime.” thing. You could give specific examples of why certain things are pseudoscientific and a non-scientist would know why those topics aren’t science. But teach that person how to think like a scientist and they have one of the tools required to make their own reasoned conclusions. Back this up with a few good information sources to get them started and some examples of these tools in action and they’ll better far equipped to deal with such issues than your average lay person.
Tigger0039 said,
May 21, 2009 at 1:25 pm
Re: Pharma – people need a source to use to determine whether the drugs being pushed are standard and inexpensive. Coming from the US, where health insurance is private, with all the copays and whatnot, insurance companies have tiered formularies that can serve as good proxies for what’s considered to be the cheapest form of xyz drug. Undoubtedly, these generics can be sourced even more cheaply from India or Brazil or S. Africa, but the generic formulary (or even Wal-Mart’s $4 formulary) would quickly give a civil servant an idea of whether there isn’t a cheaper alternative to whatever pharmaceutical is being pushed. And asking the question of the salesperson, “why is your drug better than what Wal-Mart would retail for $4, and that I could get wholesale from India for $0.25?” seems to me to be a good start. If the salesperson squirms, the Civil Servant should know it’s a crock.
Tigger0039 said,
May 21, 2009 at 1:35 pm
Rather than “teach” people how to think scientifically, I would ask them to think about all the various forms of nonsense there is out there. How many times and ways have they been buffaloed. Every culture has its own colorful words for charlatans and different kinds of fakery, so the question has to be “who is a reliable source for information”. To determine that, one needs to have an idea of how the source benefits by giving the information that it does and does it give that same information to people with other interests. In short, I would advise the civil servants to “Follow the Money”. And if the money is being offered to go into your pockets, that says that the seller has no faith in the quality of his product to stand on its own.
igb said,
May 21, 2009 at 2:19 pm
The problem in some parts of the developing world is that they want to be deceived. AIDS denialism wasn’t foisted on South Africa by evil white deceivers (a view which which, by denying agency to the government, is itself rather racist). Mbecki wanted to prove that Africa could solve a disease that the rest of the world couldn’t, and that African solutions were the best way to do that. By taking that position, he was making himself an open target for every charlatan that came along: people who knew the science couldn’t deliver the `too good to be true’ that he wanted, but that laid him open to people who promised things that were too good to be true…
Robert Carnegie said,
May 22, 2009 at 12:09 am
Civil servant types who aren’t science literate themselves will serve that need well by having one or more people on the team who do have appropriate expertise – not necessarily always the relevant speciality of the day, but they can either have some understanding of fields besides their own or else get in touch with more people who have better knowledge of an area. Britain’s own top government science guy was just interviewed somewhere, I think he had the same job title as Doctor Who…
A few problems with current issues like AIDS or climate change in this context: they are unsolvable, they are cases where the developing world does feel that richer countries are talking down to them and dictating what to do, and, in fact, “we” have been and are doing that. Meanwhile, people die, the climate does change, and rich countries demand biofuel products to replace oil. So this is the controversial stuff. Instead, look at industrialised agriculture – done wrong – or at Lysenko – if the verified facts live up to the legend.
kim0 said,
May 22, 2009 at 6:00 am
I think the main problem is misunderstanding science. The concept itself is simple, but is typically confused, misrepresented, overcomplicated, etc.
So my solution is to express science as simply as possible, and this is my take on it:
(Note that it contains falsification, Ockhams razor, and the theory concept, using plain language words.)
Science is the making of knowledge.
Knowledge is made by finding better explanations.
Explanations are better when they are shorter or explain more.
To know how much an explanation explains, it must be tested.
Kim Øyhus (c)
Alan Bird said,
May 22, 2009 at 6:23 am
In the longer term I don’t think that you really need to rewrite the message – that’s already been done in your book and dozens of other excellent examples (see above for some). What you need is to get that information to a new audience in other countries and other languages. Are there any plans to translate your Bad Science book into other languages? (I dearly hope it comes out out in Chinese for example.) I note that there are companies/systems in place to produce cheap legal local reprints (often in English but sometimes translations in the local languages) of textbooks in other countries. China eg has dozens of titles in linguistics, maths, medicine, all sorts of disciplines (I assume intellectual property rights etc are sorted out). If they could widen the field of text books to include selected popular science books – badscience, Bryson, how to lie with statistics etc etc (I’m sure you could produce a shortlist) – you would ensure simple, clear information can be available on tap to the educated people, and perhaps the general public, in those countries.
Do the Royal Society already do this kind of work? Do they see it as part of their role? If so I imagine they have oodles of resources. Maybe Professor Colqhoun could advise.
One resource often overlooked is the local Standards body. A national standard isn’t a popular science book but it’s the correct place to define an approved method of eg sterilising water supplies or sinking a well, especially if there are local considerations to be taken into account. If you don’t know whether the countries your delegates are from actually have one, you could check with BSI (British Standards Institute) about what bodies exist in other countries. In fact you could ask BSI about your problem. They not only have access to a vast body of national & international standards but they have the knowhow on setting up a standards writing and approval process.
JoanCrawford said,
May 22, 2009 at 9:12 am
At the risk of teaching you to suck eggs, the actual mechanics of the meeting itself need to be considered.
We need to understand what you’re working with here. What do these civil servants actually do? They might surprise you. Which countries specifically are involved? South American issues might well be different from, say, Far Eastern ones. How long do you have? 2 hours? 2 days? Given the timescale, what are the stated objectives of the meeting? What do the various delegates a) bring with them b) take away?
As a facilitator (assuming that your role has been correctly defined) then with quite a lot of people in the room (it sounds like), you will need to be pretty hard-arsed about keeping people on topic, to time, and making sure that the contributions are constructive. I only say that because I know that your speaking style is a bit digressionary, and wasn’t sure if you allowed that in others when leading a discussion.
Put simply, how will you know that the meeting has been a success?
rob said,
May 22, 2009 at 3:10 pm
Completely OT, but definitely of interest to readers of this forum: The perfect formula for a talk” – a blog entry on Ben’s talk at the Bristol Festival of Ideas.
ceec said,
May 23, 2009 at 9:12 am
Re. delivery – to echo a couple of points made above, my experience of this type of teaching (in the UK rather than in situ) is that you have to be very careful not to underestimate the audience.
I find it less easy to be wrong-footed if I assume I know much less than the audience about just about everything relevant to their lives and work – then when that turns out to be true, as it invariably does, I look like less of an arse. Good preparation also helps a lot.
Also, again from my teaching experience, I’d say you need to be careful with language – use simple constructs and vocab and speak slowly or you may well lose people even if their English is excellent – British people have very weird accents and we swallow a lot of crucial syllables.
“Lose the jokes” suggestion is a good one too, although you will probably find you do that anyway once you are in the situation because it will probably seem inappropriate.
Bon voyage!
ceec said,
May 23, 2009 at 9:13 am
Sorry – should have said “teaching this type of audience”
PeteB99 said,
May 24, 2009 at 8:11 pm
Yes – maybe use Western examples : this is how we got fooled, make sure you don’t
I think that is a pretty good list
Maybe link The Tobacco Papers/ Passive smoking denial / Climate Change Denial ( many of the same people/organisations in all three) and the use of Astroturfing
stinkychemist said,
May 24, 2009 at 8:56 pm
I’d stay off climate change, myself, the people of Africa, with the exception of a few cities, don’t produce the sort of CO2 output per head that “Northerners” do. Unless they actually want to know about non-carbon energy resources.
Sorry, I’m just an intellectual lightweight in this company, but it may come up.
martin-clmb said,
May 25, 2009 at 12:02 pm
Reading through these posts, there are some good ideas. I think that SVC’s suggests are about the best. Combine that with examples from the North and careful use of language and then that will help. In terms of what subject focus – well it obviously depends on the audience. I think that the audience you’re dealing with would benefit from:
– the importance of evidence in policy making AND implementation
– tips and hints on getting evidence reliably
– suggestions for getting assistence (careful of the word “help”) when it is needed in interpreting evidence
– the dangers of bad evidence and perhaps examples and discussion of common sources and causes of bad evidence
I’d try and focus examples on cases in the North and, like any talk aimed at a non-specialist audience, pitch them at an easy level to understand. I’d personally not recommend getting into too much detail (as in most of your book), as what you’re trying to achieve is an understanding (or agreement) in the need for _evidence_.
Of course you may also need to walk the tightrope of evidence informed policy vs. politics informed policy (US domestic drugs policy anyone?) and that could be considered a bit of a neo-colonial issue – the development sector is keen on evidence informed policy making but how good are they at implementing that in their own countries? Perhaps best just not to raise this as an issue.
The kinds of civil servants you’ll be dealing with will be amongst the elite in their setting. Some of the advice above about education and intellectual levels is worth noting. Overall intellectual ability will be high (perhaps higher than in a similar civil service gathering in the UK) but what will differ is that the levels of scientific training and experience may well be lower, possibly much lower (unless you are dealing with a very specialist group, which I suspect is not the case). So that should be taken into account.
Also, unless something like this is billed as “training session” then people all over the world tend to think that they do not need training (especially in something so semingly obvious as the importance of evidence). So pitching things at a “management seminar” type level is often good advice. Afterall, managers and other elites (including scientists all over the world) often feel that they don’t need training. But being “briefed” at a seminar is another thing!
Good luck.
jsymes said,
May 25, 2009 at 8:32 pm
Dr Goldacre: Exactly what can you bring to the table regarding “Climate change denialism”? You are a doctor of medicine. Do you also have a degree in climatology or paleo-climatology you’ve kept hidden under a bushel? Or do you intend merely to browbeat and hector third world countries with the western media consensus about the causes of climate change just because you once saw a movie by Al Gore and feel sorry for those poor little polar bears? Stick to medicine, related policy and its implementation. The arguments about climate change and its causes are best left to those who know what they are talking about.
PeteB99 said,
May 26, 2009 at 10:53 am
Re 79
I think you misunderstand what Ben is suggesting – not lecturing people on climate change (perhaps avoid it completely if it is likely to be counterproductive) but more generally explaining how industry or idealogical groups (from all sides of the political spectrum and beyond) can misrepresent science and mislead people.
see scienceblogs.com/denialism/about.php
“Denialism is the employment of rhetorical tactics to give the appearance of argument or legitimate debate, when in actuality there is none. These false arguments are used when one has few or no facts to support one’s viewpoint against a scientific consensus or against overwhelming evidence to the contrary. They are effective in distracting from actual useful debate using emotionally appealing, but ultimately empty and illogical assertions.
Examples of common topics in which denialists employ their tactics include: Creationism/Intelligent Design, Global Warming denialism, Holocaust denial, HIV/AIDS denialism, 9/11 conspiracies, Tobacco Carcinogenecity denialism (the first organized corporate campaign), anti-vaccination/mercury autism denialism and anti-animal testing/animal rights extremist denialism. Denialism spans the ideological spectrum, and is about tactics rather than politics or partisanship. Chris will be covering denialism of industry groups, such as astroturfing, and the use of a standard and almost sequential set of denialist arguments that he discusses in his Denialist Deck of Cards.
5 general tactics are used by denialists to sow confusion. They are conspiracy, selectivity (cherry-picking), fake experts, impossible expectations (also known as moving goalposts), and general fallacies of logic.”
Clair G-S said,
May 26, 2009 at 3:58 pm
Good point. At the very least someone should draw Goldacre’s attention to this article by Julie Clayton entitled “Spotting Fraudulent Claims in Science”.
www.scidev.net/en/science-communication/practical-guides/spotting-fraudulent-claims-in-science.html
There also an editorial that David Dickson, director and editor of the website scidev.net wrote last year called “Building a web we can trust”. The link is here:
www.scidev.net/en/editorials/how-to-build-a-web-we-can-trust.html
A good case history — which acts as a bit of a counter example– is the treatment of critics scientists who protested at the Gambian president’s claim that he could cure HIV/AIDS.
See: www.scidev.net/en/news/who-dampens-gambian-presidents-hivaids-claim.html
Also the fact that the Nigerian Academy of Sciences was sued when it issue a critical report about a local doctor who claimed similar powers.
See: www.scidev.net/en/news/hiv-cure-doctor-sues-science-academy.html
elspeth2009 said,
May 26, 2009 at 10:08 pm
Depending on the level of scientific knowledge, I would emphasize something from basic statistics, which I find always catches me out, which is statistical significance (I think that’s the term). So you might have a difference in two outcomes or whatever, but is it statistically significant? Look at some simple examples of things that look significant but aren’t. And speak slowly is really good advice. I would be tempted do a set of principles, then practise them by doing a few (maybe real life) case studies, would you buy this product or not, if so, why, if not why not. If there are two sessions the second could focus on the second set of ideas, cognitive biases, recurring themes, etc etc, either principles then application, or looking at some cases (maybe from North) and deciding what the problems are. This should have good results if people involved are receptive to this type of learning. Having said this I would really worry about doing this kind of stuff. I would make sure I knew exactly who had asked me to do it and why, and also maybe try and make sure that you take something away from it, so there is two way learning. I definately wouldn’t stay off climate change personally because I think that Africa will be affected by it even if they don’t contribute to it, ie in droughts etc. Good luck!
elspeth2009 said,
May 26, 2009 at 10:31 pm
Oh, and as far as structure goes, and not being a white dude telling people what to think, it would obviously be useful to spend a lot of time at the beginning finding out what the actual day to day decisions and problems that these civil servants are dealing with are. For instance, you may be assuming that it’s all HIV in Africa, and climate change in South Asia, when in fact it’s death in childbirth in both areas. I’m sure you know this already though.
PeteB99 said,
May 27, 2009 at 3:36 pm
I think this got caught by the spam filter so I’ll repost it without the link
Re 79
I think you misunderstand what Ben is suggesting – not lecturing people on climate change (perhaps avoid it completely if it is likely to be counterproductive) but more generally explaining how industry or idealogical groups (from all sides of the political spectrum and beyond) can misrepresent science and mislead people.
see denialism on scienceblogs
“Denialism is the employment of rhetorical tactics to give the appearance of argument or legitimate debate, when in actuality there is none. These false arguments are used when one has few or no facts to support one’s viewpoint against a scientific consensus or against overwhelming evidence to the contrary. They are effective in distracting from actual useful debate using emotionally appealing, but ultimately empty and illogical assertions.
Examples of common topics in which denialists employ their tactics include: Creationism/Intelligent Design, Global Warming denialism, Holocaust denial, HIV/AIDS denialism, 9/11 conspiracies, Tobacco Carcinogenecity denialism (the first organized corporate campaign), anti-vaccination/mercury autism denialism and anti-animal testing/animal rights extremist denialism. Denialism spans the ideological spectrum, and is about tactics rather than politics or partisanship. Chris will be covering denialism of industry groups, such as astroturfing, and the use of a standard and almost sequential set of denialist arguments that he discusses in his Denialist Deck of Cards.
5 general tactics are used by denialists to sow confusion. They are conspiracy, selectivity (cherry-picking), fake experts, impossible expectations (also known as moving goalposts), and general fallacies of logic.”
Doug Mackie said,
May 28, 2009 at 1:00 am
Things I look for:
Excessive precision. 34.62%, 23,405 jobs will be created.
Lack of uncertainties, inappropriate uncertainties (wrong precision), or if given no indication of what the uncertainty is (std err, sd etc etc). 34.62 +/- 5.25
Poor referencing. A study showed 23,405 jobs would be created. (Man, I want to author a “study” one day that for 24 hours is on every news report).
What I call “colloquial” language. An actual quote from a submission to a Select Committee: “Fascist New World Order”. Usually though milder but uses words like “silly”.
Any discussion of motivations for alternative views. (Cuts both ways so not always a useful criterion).
Multiple but not related lines of “evidence”. E.g. “I don’t accept climate change because it is a hoax with no evidence and anyhow it is all caused by the sun.”
Forcing on plots. The text cites a strong relationship. The graph shows an amorphous blob of 97 scattered data at low x and low y of the axes with 3 points at high x high y.
Absurd extrapolations. Improbable Research used to give these. Since no one ever throws away National Geographic magazines it was possible to calculate when the accumulated mass would cause the US to sink into the sea.
Junk references. Michael Crichton was many things but he was not a reliable source of evidence to cite about climate change.
srd said,
May 28, 2009 at 7:08 am
Why assume that people in developing countries don’t use/know science? But more seriously, there are tonnes of academics and practitioners doing research on these things – in science studies, environmental history, veterinary history, agrarian studies there’s a huge wealth of expertise on science and policy in Africa and Asia, much of it harnessed by DFID, the Africa Union etc. I think it is great if you can add to this, but no need to reinvent the wheel. Talk to the people who do this professionally and then see what you can add!
nongovernmentalindividual said,
May 30, 2009 at 7:16 am
Ben
This sounds really interesting.
In your introduction you emphasise procurement; the issue of evidence (as others have pointed out) is also important for prioritisation of public policy, and design of public health programmes more generally, including those funded by external donors of course. Take the example of AIDS programmes.
Firstly, there needs to be an understanding of who is most affected, and who is most at risk, not only because it is important to support those who are disproportionately affected but also because if programmes can target these people then they will probably have the most impact at a national level. So here we need to understand relative risk and population attributable risk.
Often with AIDS, the people most affected are often disadvantaged for other reasons – it may be possible to help them to access the services that everyone else has access to, but more often it can be necessary to design specific programmes that help deal with the specific issues they face – for instance sex workers often face violence and other forms of discrimination; homophobia also poses a major challenge for AIDS programmes. In these circumstances, handing out leaflets and condoms and telling them to get tested isn’t enough.
Just to complicate matters, it is these “specific factors” that mean a) that there is a lack of evidence for relative risk/attributable risk in these groups (because no-one is doing the research); b) that even where there is evidence, donors and civil servants don’t particularly care about these groups or want to go there because it is politically or morally controversial to do so (if you’ve got a big pot of money for AIDS, it’s a lot easier and sexier to spend it on “deserving” school kids than on “morally corrupt” sex workers, drug users or men who have sex with men); c) that even if the data on attributable risk are there, the proposed interventions are more likely to be determined by the moral attitudes of the society, government or donors in question than by evidence; d) that even if governments and donors agree to do the right thing, the local health care providers or non-governmental organisations that get the funding to provide services may spend the money on something else anyway.
You’ve talked a bit about c) before when discussing the US government’s policy on AIDS programmes for sex workers and on abstinence-only programming. (By the way, without wanting to let the US government off the hook for this it’s worth recognizing that these policies only confirmed pre-existing attitudes in many countries; it’s not as if they were all empowering sex workers and promoting comprehensive sex education in schools before Bush arrived).
So: while the influence of foreign donors and businesses can be pernicious, it’s important not to assume that they are the only factors that compromise the design of public programmes… and while it’s crucial for national level decisions on programme strategy and procurement are based on evidence, it’s also crucial to strengthen the evidence base and to ensure that this knowledge filters down to those who are actually delivering services.
daven said,
May 31, 2009 at 1:07 am
I worked for 2 years in Mozambique and 4 years in the Kenyatta University Appropriate Technology Centre. Working with people from villages, hundreds of NGOs and different government agencies I came across against many examples of outsiders pushing particular technologies that, although they worked, were not the answer to the country’s problems. So I would add to any session some time considering relevance and, especially, appropriateness.
I worked mainly on biomass energy. At the 1991 UN Energy Conference, in Nairobi, my Kenyan colleagues arranged a march of women carrying wood on their heads. The developed country visitors thought the world’s energy problems were about oil and electricity. But at that time, the main energy source for 95% of the world’s population was wood. It was only the rich in the developed worlds and some of the big cities in underdeveloped countries that used large amounts of fossil fuels. So everyone pushing large dams or nuclear power plants on developing countries as a solution to that energy crisis were providing a solution to the wrong problem: the oil crisis, not the woodfuel one.
My colleagues were the snake oil salesmen at the time, trying to persuade governments to improve the energy efficiency of wood and charcoal stoves, to give people enough time to plant more trees. It was based on good science (heat transfer, laws of thermodynamics, kinetics of combustion), but it took a lot of work to turn that into the Kenya Ceramic Jiko, and get artisans to turn out 125000 stoves a year, saving 1/3rd of the charcoal previously used.
How did we make sure that the better stoves were taken up, and some silly ideas (like gasifying stoves) were not? By lots of testing, and getting all the people involved to talk to each other – the civil servants, aid agencies, NGOs, university departments, small companies, so that everyone understood the links between design, testing, and energy saving in the home.
So you should get the civil servants to make a list of all the scientifically aware people or organisations in their countries who could help them assess proposals from outside. The big risk is in turnkey development, be it just buying in a factory, or buying in drugs, without understanding it well enough to change things when the inevitable problems arise. But get together a lot of sceptical scientists, and it is possible to pick out the best and worst of the proposals.
lisadom said,
June 1, 2009 at 9:14 pm
I can think of plenty of people in the so-called Ist world who could benefit from a little bit of genuine analyis and realistic interpretation of data in their decision making.
Putting a load of graphs and charts up on an overhead can still bedazzle a room full of supposedly educated people with an interest in an area like autism. When in reality the presentation owes more to PT Barnham, than BF Skinner (insert preferred scientist)
There is also a rather wealthy television presenter across the pond who could do with a little accurate think tank data analysis too. (or not, perhaps her science is that of Mr Barnham and his populist approach to getting insanely rich)
xx
lisadom said,
June 1, 2009 at 9:17 pm
Doug Mackie – I have copied your comment. Just ticking a lot of boxes for me!!
xx
timboson said,
June 3, 2009 at 8:46 am
I’d say its the same in the developed world as it is in the developing world :o)
Tessa K said,
June 3, 2009 at 11:39 am
Sorry I don’t have time to read all the responses so someone may have suggested this already.
One thing to encourage people to check is who is funding initiatives that appear scientific and question what their motives might be.
For example, PEPFAR is a US partnership promoting and heavily funding circumcision over condoms as HIV/AIDS prevention in Africa.
When you dig into who their partners are, a lot of them are religious organisations who are anti-condom and pro-abstinence. When you look at the research for the effectiveness of circumcision and the problems in carrying it out, the initiative starts to look pretty dodgy.
So do look a gift horse in the mouth.
Sorry to blow my own not very scientific trumpet but I’ve done a bit of digging and blogged the details here: tessera2009.blogspot.com
heavens said,
June 4, 2009 at 4:48 pm
I want to second Fontwell and others that suggest using informal contacts to their advantage. I’ve occasionally gotten “random” e-mail from people with science-related questions, and I try to answer them all, even if all I can do is to send them on to someone else. Most of us science geeks are really quite happy to share what we know, and a lot of good questions get handled this way. For example, I’ve asked researchers to re-crunch their numbers in a slightly different way, or to control for a different factor, to make sure that it didn’t change the outcome. Sometimes it sets us on an interesting question; sometimes (usually, I think) it confirms the outcome.
One quirk of dealing with developing countries: People towards the top of organizations in these societies tend to flaunt their powerful social status in ways that offend most Western science geeks. You may be the most important *person* in the world, but your impressive social position and enormous power doesn’t change the *data*, which is what I want to talk about. I don’t need a big build-up about how important you are; if I know anything about the subject, I’ll eventually answer your question (and also tell you how confident I am in my ability to answer it). I’m just as likely to answer questions about “my sister has breast cancer” or “I think I have an autoimmune disease” as I am from some high muckety-muck in government that needs to make a big public health decision.
nongovernmentalindividual said,
June 4, 2009 at 8:04 pm
Tessa K,
A bit of pedantry… but the pro-male circumcision lobby for HIV prevention goes beyond PEPFAR; it’s also a key policy of UNAIDS (www.unaids.org/en/PolicyAndPractice/Prevention/MaleCircumcision/) and of the World Health Organisation (www.who.int/hiv/topics/malecircumcision/en/index.html).
At first the evidence looked very good, so much so that the randomised control trials had to be stopped because it was considered unethical to withhold male circumcision from the control group. The results (I can’t remember them exactly, but it was something like a 40-60% risk reduction) were obviously pretty seductive to policy makers who have been desperately trying, for decades, to find ways of making inroads in HIV prevention.
But, yes there were potential biases, and of course it’s not really possible to blind a trial of such an intervention… Now, as you suggest, there are concerns about risk compensation among circumcised men (i.e. having more unprotected sex because they feel safe) and therefore about how effective the strategy will be in the longer term.
Anyway, enough of the pedantry. You are absolutely right about questioning the evidence base for the policies of different donors. But the most insidious part of PEPFAR has been about “abstinence only” programming; tragically it is only in the last year or so that we’ve seen clearer evidence that abstinence-only programmes are a waste of time, and maybe counterproductive.
Tessa K said,
June 4, 2009 at 9:50 pm
nongov (if I may be familiar)
You’re right, it goes beyond them; I was just using them and the pro- abstinence partners as an example. But informed pedantry is something we all do well…
bob_calder said,
June 6, 2009 at 3:18 am
I didn’t read all of the posts, but I see it as a search authority problem. The thing is that you can’t do it without knowing a fair amount about the subject. The best idea is to call someone who markets multiple solutions for the problem. Period. They need to build a network of reliable advisors.
worloc said,
June 14, 2009 at 8:41 pm
Ben, should say from the start that I’m from a qualitative background and maybe a more participative approach may work. In groups, ask your civil servants about instances of ‘bad science as a con’, that they can think of. Get them to vote (against criteria e.g. most cost, least effective) for the worst case. Work through the ways that ‘evidence’ in this case is/was/could be used as a convincer. Finally ask them to provide ‘operational’ solutions and round-up with feedback and summary session. The main benefit is that you would be ‘handing-over-the-stick’ and consequently, not be seen as the patronising white-man.
Jo_C said,
August 25, 2009 at 7:01 pm
How do the civil servants want to use research? If they are from the sexed up dossier school then training in spotting misleading research is a bit wide of the mark, and not meeting their needs. What pressures are put on them from above to find the right evidence? But perhaps the workshop has been and gone.