What would you say to people from the developing world who use science to make decisions, but don’t necessarily always have a lot of time, or know a lot about it?

May 16th, 2009 by Ben Goldacre in africa, aids, bad science | 98 Comments »

I’ve been asked to facilitate a couple of sessions with some civil servant types from various countries in the developing world who advise their governments on science, and particularly on the science informing policy and purchasing decisions. The idea is to focus on how people might try and mislead you with science, and the range of scientific background and understanding in this group will be pretty wide, as it always is with civil servants. Since I’ve noticed a recurring theme for readers of this blog to be a bit cleverer than me (albeit less dogged and obsessive) I was hoping you might have some ideas about the kinds of areas to cover, the themes that are relevant, and the kind of structure to use.

I’ve been told that there are lots of people who go out to Africa, for example, with glossy brochures, claims of scientific evidence, and one bloke with a dodgy PhD who endorses the product, and then try to hustle and sell their systems and wares which are either ineffective, or at stupidly inflated prices. Most of these scammers fail, but it would be nice to increase the fail rate.

Subject areas that immediately spring to mind (and I’m hoping you can think of more) include:

  • HIV and Aids denialism
  • Climate change denialists
  • Dodgy miracle cures, pharma and quack.
  • Get rich quick scams
  • Ropey industry claims

Now there is obviously no way you can teach the entire process of how to check evidence in every field, I wouldn’t be the person to do that, and I’m not suggesting that due diligence on govt contracts is non-existent. I think the idea is that there are some ideas and recurring themes from the kind of work we do on this blog, and others, that might be relevant to these problems.

I suppose at base we might be able to get together a rough set of nostrums around:

  • checking evidence for quality and completeness (cf misrepresentation and cherry picking)
  • checking background
  • what constitutes evidence
  • how to seek independent expert opinion

But there is also a wealth of material on stuff like:

  • cognitive biases
  • classic recurring themes in sciencey blags
  • the themes of “denialists” generally, from the denialism blog
  • and so on.

The other background issue here is that obviously I am very keen not to feel like some white dude telling people how to live and think. But I guess that’s a problem for me.

So, I could really do with some ideas here.

If you like what I do, and you want me to do more, you can: buy my books Bad Science and Bad Pharma, give them to your friends, put them on your reading list, employ me to do a talk, or tweet this article to your friends. Thanks! ++++++++++++++++++++++++++++++++++++++++++

98 Responses

  1. Nocturno said,

    May 18, 2009 at 10:56 pm

    Take into account that Third World leaders tend to be more authoritarian and self-deluded than even European leaders, so the task is monumental (as a Mexican journalist I’ve been there, failed there). I would go for examples from the countries themselves (Rath and the late Naomi Dziba in the case of AIDS denialists) and making it clear how being critical empowers the leaders rather than disempowers them.

    Hope it helps.

    Mauricio-José Schwarz

  2. munkeatlooi said,

    May 18, 2009 at 11:01 pm

    Hi Ben,

    You might find the resources over at SciDev.Net (www.scidev.net) useful. They’re a news organisation specialising in communicating science for the good of development and with a good reputation in the field.


  3. kateweb said,

    May 18, 2009 at 11:01 pm

    As a non-scientist (which I suspect many of these civil servants will be) and thinking about what I found particularly helpful in “Bad Science”, is there anything in a “This is how I’d try to scam you” sort of approach (thinking of some of the stuff on how to skew a study).
    As you say, it’s impossible to go through everything in terms of checking evidence, but what you could do is look at some of the common threads of how science scams/scares/fake products work, from what sort of issues they target (generally where there’s public concern and/or funding) to some of the techniques they use to make non-evidence look like evidence. Just a thought, but I suspect some of the big obvious things you could come up putting yourself into a scammer’s shoes aren’t a million miles away from schemes these people are approached about.

  4. pentcheff said,

    May 18, 2009 at 11:32 pm

    A recurring theme for cranks is the “I, the solitary genius, have disproven generally-held scientific opinion”. Of course, there’s a very small possibility the proprietor of that opinion is correct. But it’s a very bad bet for a developing economy’s government to be the first to invest in said self-proclaimed genius’s scheme.

    Let wealthier outfits make the initial investments on such proposals to see if they will work out in the long run. Instead, invest in outfits with a proven track record in that type of initiative.

  5. ellieban said,

    May 18, 2009 at 11:39 pm

    I would spend a few minutes showing them the best ways to check up on the internet. For example, a quick Google of the institution someone claims to be from, followed by popping their name and a paper title into pubmed will go a long way towards spotting the worst scammers and doesn’t require any specialist knowledge at all.

    I would focus on how to spot dodgy evidence before you even spend time reading it, rather than how to critically evaluate poor science writing.

  6. treeofpain said,

    May 18, 2009 at 11:48 pm

    Was there a natural structure to the book, is there the arc of a story there, a natural falling in to place of the individual components. Which ones are the critical ‘scenes’ (regression to the mean for me).
    It perhaps doesn’t have to be perfect, but at least it is a path that you are familiar with.
    It all rather depends on ‘who’ you are facilitating for.

  7. AnneArchibald said,

    May 19, 2009 at 2:26 am

    There’s always the excellent (and short!) How to Lie With Statistics. Not actually a how-to book, it’s a very nice introduction to critical reading of statistics.

  8. peterd102 said,

    May 19, 2009 at 2:27 am

    Basic Idea id go with is explaining simple principles for evaluating claims. Hope that they can use them. Or even simple places to check, people to ask about claims.

    I think “Its a bit more complicated than that” is a good theme to run with. The principles of over-interpreting data. What data actually means, especially when it relates so a social aspect where there are innumerable factors that could be involved.

    Another area could be the quick fixes that scammers provide – Introduce a fake problem or excessive/unnecessary guilt and then sell you and amazing quick fix – one payment to make all the fears dissapear.

    Theres so much you could talk about – all of it interesting (fi your a nerd) – but to explain it to civil servants? Tricky

  9. speedweasel said,

    May 19, 2009 at 3:12 am

    I say start with Atomic Theory and Thermodynamics. Some of Richard Feynman’s writings (Six Easy Pieces, for example) could form part of an excellent introduction to the physical sciences.

    People need to have a high-level understanding of what is, and what is not possible in nature. They’ll surely be much less likely to set up a homeopathic hospital or invest in the development of a perpetual motion machine if they have even a rudimentary understanding of chemistry or physics.

  10. phayes said,

    May 19, 2009 at 6:28 am

    “The other background issue here is that obviously I am very keen not to feel like some white dude telling people how to live and think.”

    Invite some UK civil servants. If they accept, you’ll just be talking to civil servants. If they decline, you’ll be talking to those civil servants who want to do better for their countries than fund and/or otherwise encourage quackery, pseudoscience degrees and EMdrives.

  11. fontwell said,

    May 19, 2009 at 6:41 am

    I would suggest having informal contacts in various reputable universities around the world and asking them to briefly comment on whatever or whoever.

    While some countries and universities often get sucked into particular pseudo science issues it appears to me that en masse they are usually quite rational.

  12. SimonW said,

    May 19, 2009 at 7:33 am

    Big pharma may be as dangerous to these folks as quacks.

    Remember the big pharma companies want to test their stuff on your citizenry because they feel they compensation will be smaller if it goes wrong, and possibly the regulations will be less stringent. One might reasonably compete on the second category within limits, but the first should be non-negotiable.

    Similar comments apply to waste disposal and natural resources, don’t sell out cheap now, employee an economist to price it for you.

  13. QuietKnoll said,

    May 19, 2009 at 7:36 am

    You might like to have a look at the Radical Statistics web site and their archive of past journals, one of which was a report of an annual conference a few years ago that looked at global poverty. www.radstats.org.uk/no089/index.htm

    This included a number of presentations on aspects of the Milennium goals and measuring of progress. One of these pointed out the evidence that improving access to potable water was a better “investment” than world class teaching hospitals in capital cities and far better than continuing to listen to well intentioned Europeans and Americans selling their consultancy contracts as the answer.

  14. msjhaffey said,

    May 19, 2009 at 7:37 am

    1. The area will for many be a difficult one to understand at first. So use little of no jargon. These may be words you are familiar with and they aren’t.

    Examples “memes”, “denialists”, “pharma”, “quack”. Use simple English instead.

    2. Give them a simple list of resources they can easily use. Your blog may be one. Pointers to responsible science reporting would be another. I particularly like the Sense About Science website, which typically has documents that are well and simply written.

    3. Show them a process to use to investigate and analyse claims without prejudice. Don’t assume they know the steps, or the sources. I like what you’ve written in Bad Science, but you as a science know the sources to check: I don’t. So document these, with a brief explanation as to what each source can be used for and why you chose it.

  15. samarkeolog said,

    May 19, 2009 at 7:41 am

    Maybe you should base it around HIV/AIDS because it’s recognised as an immediate concern (unlike the “normal” criminal get rich quick scams). Plus, although it is sciencey, the crisis means it has a base level of public awareness that makes it accessible.

    It involves denialists and dodgy miracle cures (which are frequently get rich quick scams), and those two groups’ claims are stylistically similar to the various dodgy industry claims; so, you could explore a few examples in depth, but they can apply the principles really widely.

  16. conejo said,

    May 19, 2009 at 7:47 am

    Definitely need to mention dealing with risk. After you recommended Gerd Gigerenzer’s book, Ben, I find myself using the ideas in lots of other contexts. It’s one of those ‘learn once, use many times’ concepts. The examples are also easy to grasp and very convincing demonstrations that numbers can be used to mislead or confuse. If nothing else, you need to get across the message “Don’t take numbers at face value, especially if someone is trying to sell you something”.

  17. thepoisongarden said,

    May 19, 2009 at 7:51 am

    Everyone knows that they are too clever to get caught out and your problem of being patronising is to suggest that they are not as clever as you.

    If it were me, I’d use a version of the ‘Yes, Minister’ salami slicing to show how the snake oil salesmen get clever people to buy into their propositions.

    Perhaps use some real examples from ‘developed’ economies to show that you should really be talking in Washington or London but people there aren’t open to learning new things about how to do their jobs better.

  18. dbhb said,

    May 19, 2009 at 8:14 am

    One wellspring of bad science (in this or any other country) that you’ve never really touched on, Ben, is the midwifery/antenatal-class world. It’s not all bad by any means, but this does seem to be a network where pseudoscience, quackery and the distrust of science seem to spread like gossipy wildfire amongst a vulnerable and anxious audience, pretty much unchecked and totally under the radar. Check it out for yourself and I think you’ll agree.

  19. smowton said,

    May 19, 2009 at 8:20 am

    I’d suggest that you ignore the climate change business, since it’s by far the least damaging of the enumerated items to ignore. Obviously this depends on the economic situation of the particular country in question, but I can’t imagine a country which struggles to feed and educate its populace gives two or more monkeys about clean energy, and quite rightly so. All the now-developed countries built an economy first and then worked on cleaning it up later, because they too cared more about a decent standard of living than the potential spectre of global warming.

  20. naomimc said,

    May 19, 2009 at 8:22 am

    HIV/AIDs is obviously a good theme but I’d also focus on the interventions that do not involve often expensive drugs; malaria nets, maternity interventions etc.

    I don’t know whether you want to get into drug rationing, generics and regulation, but personally I think this is one of the most important parts of developing world medicine. See stuff on WHO, DFID etc

  21. north5 said,

    May 19, 2009 at 8:43 am

    Consider who has an “agenda” and who doesn’t; and what they are likely to be.

    Good western universities and academics are (fairly!) independent from government and business, and care not a jot for “colonialism” or profit.

    “Entrepreneurs” and businesses are intrinsically after more cash (not always bad but bear it in mind); religious groups often come with a biased moral viewpoint which can be very damaging (*cough*popeinacondom*cough*). Etc etc.

  22. mauve said,

    May 19, 2009 at 8:43 am

    I would recommend they read the excellent book “Bad Science” by Ben Goldacre. It’s pretty much packed full of this kind of advice.

    Honestly, I think it is a lot more effective to give examples as to how scientific wisdom has been perverted and misrepresented, as you do in your book, than to extract dry general principles. I think it changes people’s mentality more to hear about people that have been hoodwinked than listing do’s and don’ts in the hope that they will be followed. Stories are very effective – just ask religion.

    You could tell the parable of Mattias Rath and then preach a sermon about how you could spot a Rath (sort the wheat from the Rath?)

    Who are your audience? If they are science advisers they may already be on board with good intent if not the knowledge and skills. Or might they be following some procedure to assess claims but not really get why that procedure is in place?

    Also how much time do you have with these people?

  23. Deen said,

    May 19, 2009 at 8:48 am

    Focus on the importance of double-blinding to prevent bias.

    For instance, teach them to be wary of devices like the Sniffex, which basically is a fancy dowsing-rod that’s supposed to locate explosives, that only works when the operators know where to look. These devices are always given a demonstration. Prospective buyers should be able to understand to what level a demonstration was blinded and understand how that affects the reliability of the demonstration.

    Maybe a hands-on demonstration of the ideomotor effect might be illustrative? Something like what was described here on quackwatch, in the chapter “A Video Demonstration” (which I just happen to have read in the past weekend).

    Just a few thoughts.

  24. rhebus said,

    May 19, 2009 at 8:54 am

    In case you missed it, there was an excellent Mind Hacks post about the psychology of being scammed:
    This was based on an OFT report into scammers, how they succeed, and what distinguishes victims and near-victims from non-victims.

    One interesting thing here was that a little extra specialist knowledge can make you more likely to be subject to a scam related to that area: for example, bingo players are more likely to fall for a fake lottery scam, or casual investors to fall for a Ponzi scheme. It means you have to be very careful about what knowledge will help to prevent scams, since some knowledge may actually do more harm than good.

    Two relevant quotes from the report (from paragraphs 12.5 onwards):

    Not all potential errors of judgement are equally involved in falling for a
    scam, and for efficiency, public education campaigns need to focus on
    those that seem to be ubiquitous. These are appeals to trust and
    authority, and the clouding of sensible decision-making by the offer of
    huge rewards. Publicity could usefully emphasise just how easy it is to
    produce an official looking document, and (though this is already the
    topic of much publicity) just how implausible it is that anyone would be
    offering huge sums in return for no effort.

    The fact that scam victims and near-victims report processing scam
    communications more thoroughly than non-victims suggests that efforts
    to persuade people to read scam materials more thoroughly or carefully
    are unnecessary and could even be counter-productive. Of course, if
    people get as far as the small print, they may well recognise a scam as a
    ‘bad buy’. But that is not what non-victims do: Rather, they discard
    scams virtually unread. The most effective education strategy may be to
    help potential victims recognise scams at a glance.

  25. theonlyrick said,

    May 19, 2009 at 8:57 am

    I like the idea of following, as much as you can, one concrete example (eg AIDS (& Rath) and branching off at places to make other points.

    Part of this should be to get them create a scam: What would they say? How would they say it? What would they claim to be? Would they get a qualification like our Dr McKeith?

    Important to show that just because an organisation has a weighty and impressive-sounding name, it’s no guarantee of independence. It’s important to find out who’s funding it. (EG tobacco industry suppressed evidence re lung cancer). Use Wiki, use google. (Although not perfect, they’re a very good start in a time-limited world.)

    Mention good news that Obama is putting science at the forefront of US policy making.

    I also want to second what pentcheff (#4) said about being slightly skeptical/conservative about things that are too good to be true.

    Perhaps build up a list of words that are ‘good’: peer-reviewed; independent; modest; the opposite of ‘name-dropping’; open to criticism; unlitigious

    If they have a science-y understanding, show excerpts from the (hard-to-find) Feedback page from NewScientist.com. They always have small funny miracle cures that they gently refute: (www.newscientist.com/topic/feedback)

    So much to do, so little time… Good luck!

  26. emen said,

    May 19, 2009 at 9:08 am

    Where are these civil servants from, Ben?

    “Various countries in the developing world”, what countries? – there is no use speculating about what they need before you find out a little about which areas their bad science is about.
    Different countries will have different obsessions: food, vitamins, HIV, medicine phobia, vaccinations, homeopathy, holistic miracle cures etc.

    Everything that you mention in the post is fine, but do ask them about it first, otherwise you might fall into the trap of “some white dude telling people how to live and think.”

  27. nswetenham said,

    May 19, 2009 at 9:24 am

    I would broaden ‘HIV/AIDS denialism’ to ‘Infectious disease denialism’ which can be anything from flu to TB thru Malaria, with a particular focus on HIV/AIDS as an example.

  28. DocSnipe said,

    May 19, 2009 at 9:33 am

    Always a useful checklist – Sagan’s Baloney Detection Kit: users.tpg.com.au/users/tps-seti/baloney.html

  29. CDavis said,

    May 19, 2009 at 9:49 am

    Dude – just give the the email address of Goldacre Consulting, and charge them £1,000/day to produce a full report of the trustworthiness or otherwise of the alleged science in question.

    And let me say that’s it’s a disservice to these people NOT to offer them such an option: where else could they receive such informed, disinterested, honest, well-researched opinion?

    Ultimately they may wind up doing something like this anyway, at a higher cost and for less rigour.

    You can always ask us fans if you get stuck…

  30. warhelmet said,

    May 19, 2009 at 9:50 am

    dbhb – you are right. I have a friend who is a scientist, skeptic and atheist. He was so disgusted by the stuff that the NCT spouted that he forbade his wife from going to their sessions.

  31. sarah_rubyred said,

    May 19, 2009 at 9:51 am

    I know it’s probably a little time-consuming to teach statistics and research methods in a single session, but is there any way you could encourage them to do their own studies – especially simple audits, comparisons etc? Maybe this way they could get the autonomy they need, and can teach others this important skill.

  32. yarrah.goffincher said,

    May 19, 2009 at 10:35 am

    Make sure the information being presented is as recent as possible! Have been working in the Liverpool School of Tropical medicine, and there’s problems with people applying tourniquets to snake bites on the recommendation of antivenom pamphlets which are over 20 years old…

  33. Confounded kids said,

    May 19, 2009 at 10:47 am

    Chamber of horrors approach to the following: how does bad science get done and disseminated, how does it inform bad decisions (gurus and anecdotes), what are the frequency and magnitude of the unwanted consequences. Pegs to hang it off… I’d plum for:

    1. Cognitive and motivational determinants of questionable beliefs (summarised by Thomas Gilovich How we know…)

    2. How to talk back to a statistic (Darrell Huff, How to Lie…)

    3. ‘cui bono?’ It’s worth reminding the innocent that corporations are encouraged (Select Committee Report on the Influence…) or even legally obliged (Joel Bakan) by many governments to put private wealth over public health. Regulatory capture.

  34. thekumquat said,

    May 19, 2009 at 10:48 am

    As a civil servant who’s had to buy stuff I know little about (I have a MSc and PhD in biology research but that doesn’t help much when buying IT consultancy), it’s been really useful to have procurement processes in place to protect us.

    For example, you have to get at least three quotes for any purchase; expenditure over a few thousand quid requires a detailed tendering process where the suppliers must meet certain accounting and management criteria, and expenditure must be budgeted for in advance – you need to decide first what you actually want to buy, so even if the moon on a stick was offered, you couldn’t buy it that financial year.

    Being able to say ‘system says no’ to people is very useful, and it ensures you can’t take anyone at face value and have to investigate everyone. We may not get complete quacks trying to sell stuff to UK government (only a few get through my spam filter), but oh the masses of consultants offering to produce us a shiny report containing The Answer…

    I often see job ads asking for qualified procurers to go to a developing country to help them create systems – perhaps someone who’s done that could help you?

    *warhelmet* – wondering what guff you were hearing from which bit of the NCT? My teacher was very good and cited real papers and discussed risk a lot, but there seems to be great variation. The central NCT publications are quite good, and then there’s the local newsletters that publish anything they’re sent and advertise anyone who pays, which results in half a magazine of woo-ads…

  35. sophiedb said,

    May 19, 2009 at 11:08 am

    Basic stats? The number of people how don’t understand what means and standard deviations, uh, mean is bound to include civil servants. The 50% line isn’t a target, damn it: it’s an AVERAGE! *grrr* What a decent sample size is, whether evidence has been peer-reviewed and all the other usual criteria for taking someone’s results seriously, too.

    Seconding thekumquatsaid re NCT – our antenatal group included 3 scientific professional types (at least one of whom – me – is atheist, though I’m not sure what that has to do with it), and we couldn’t fault the info given. Whether we followed it was our issue though, obviously, and there is a wealth of conflicting parenting info out there. I agree with dbhb in the sense that it’s easy to lose yourself in ante/postnatal advice. Even HCPs don’t give out a consistent message! (speaking from experience)

  36. misterjohn said,

    May 19, 2009 at 11:17 am

    Some of the examples in “Voodoo Science” would be useful, as they demonstrate that even in the “First World” there’s one born every minute, even in the Government. Carts that run on water, perpetual motion machines, etc.

  37. CDavis said,

    May 19, 2009 at 11:20 am

    @TheKumquat – interesting to hear things from your side of the fence. My miserable experience from the PoV of trying to sell good quality IT to the Public Sector has been that excellence, robustness and fitness for purpose go to the wall in favour of a low bottom line.

    The result is that the large IT companies have learned to tender any old crap as long as the price is kept low, and then treat anything not specifically written down as a chargeable extra – things like phone calls, meetings, tests, etc.

    The end of this mess is the IT horror stories we know so well: Tax Office, NHS, Police, MoD and the rest. The systems don’t work, and the final budgets are massively overrun anyway.

  38. QOF said,

    May 19, 2009 at 12:25 pm

    I’ve got a suggestion around format if you are interested. I sometimes run workshops around crime prevention/criminology. It depends how long you have with these people – you would need a few hours – but one approach that works for me is to let them be the bad guys.

    Give them a scenario, the bottom line of which is that they need to sell product to make money, no matter what. It generally doesn’t take people long to put on a “naughty hat” and start coming up with devious suggestions. Depending on the amount of time, you could put in more or less detail about research studies, cognitive biases, etc. or send them out for reading in advance. You have to do a bit of planning to make sure it goes the way you want, but if you are facilitating it’s easy to feed things in as the scenario develops.

    About half an hour from the end you stop and review what you have come up with – in this case, no doubt a horror list of bad science. Then you smile evilly and say well done, that is exactly how the criminals/woo-merchants do it, and give them a couple of specific examples to prove it. The point generally doesn’t need hammering home any more than that, and the feedback I’ve had is that they still remember it some time later.

    Hope this gives you some ideas!

  39. Gillian Wakefield said,

    May 19, 2009 at 12:45 pm

    How about doing the same thing in the ‘developed world’?

  40. tregenza said,

    May 19, 2009 at 12:54 pm

    Rather than focus on specifics, try and teach some rules of thumb that should be applied to anyone selling something.

    My suggestions?

    1) Is it too good to be true? Then it almost certainly is.
    Anything promising a quick, easy, solution to a problem is almost certainly bogus.

    2) The more they claim, the less it does.
    Whether it is a pill that treats anything from Baldness to HIV or ID cards that will stop terrorism, fraud, immigration, and crime; the simple truth is that specific problems need specific solutions.

    3) Anecdotes are not science (or cause & effect can be hard to distinguish).

    4) Science is hard and the human mind easily fooled.
    Most science is hard to understand and it is often counter intuitive. Plus the human brain is very easily fooled. Together that means that we all operate on false beliefs about the way the world works and we all get it wrong at times. The scientific method is the best way of preventing disasters such as thalidomide and o-ring failure.

    5) Even the best solutions have a downside – Anyone telling you any different should not be trusted.
    All medicines have side effects, all technology suffers breakdown, all projects take longer and cost more than expected.

    6) Fail-safe does not mean that something never fails, it means that *when* it fails, no one gets hurt.
    Politicians are especially bad at misusing this phrase.

    7) Scientific fact is just our best guess at the moment. We will always discover more and that will often invalidate what we already know. Spotting the difference between a real breakthrough and a press release is a skill everyone needs to learn.

  41. Ghoti said,

    May 19, 2009 at 1:43 pm

    Hi Ben,

    here’s another article to add to the pile, might give you some good ideas if you haven’t seen it already:
    How to spot a bad scientific argument

  42. andrewwyld said,

    May 19, 2009 at 1:47 pm

    I’ve been thinking about this a lot since reading the Matthias Rath chapter and I realized that what the developing world needs is its own really good scientists. If Western science is rightly repudiated for its tendency to use the developing world as a testbed, and then wrongly repudiated for everything else, locally-grown scientists are more likely to be listened to and trusted. Such people undoubtedly do exist, and perhaps the most useful advice you can give the civil servants is to employ one or two proper local scientists to help them make sensible choices on a day-to-day basis. You can help them identify the good scientists as a bootstrap if you like.

    Some countries will also have the resources to make better investments in educating its young people, and some won’t, of course, but I realized that the whole reason we believe science works is that you can do it yourself and get the same answer (assuming we’re not talking about the Large Hadron Collider, which for medical science we aren’t).

    So … yeah. Encourage local science and encourage the inquisitive spirit in local scientists.

  43. twaza said,

    May 19, 2009 at 2:04 pm


    Before you get to involved with the developing world you MUST read “Tropical Gangsters: One Man’s Experience with Development and Decadence in Deepest Africa” by Robert Klitgaard.

    The book may be a bit older than you — well I am guessing your age — but the nature of people and the problems of Africa haven’t changed one iota.

    Robert Klitgard is an American, but don’t let that put you off either. He is also an economist, surfer, and an entertaining writer when he isn’t shocking you.

  44. Dr Spouse said,

    May 19, 2009 at 2:07 pm

    I’ve been involved in doing science in developing countries for some time now (and, in response to the commenter above, including capacity-building, which is hampered by a lack of university graduates locally, the poor training of those graduates, and once they’ve been trained, the sucking in of all their time by local universities, and lack of funding for any of them to do any further research post-PhD).

    I would say that one salient point to make is that not everything that *looks* like it would be humanitarian/good/make sense will actually be worth the (very limited) funds. There is a tendency to assume a lower standard of proof is needed for e.g. interventions with children. Also medical and social sciences are probably a lot more relevant than physical sciences to these sorts of problems.

    Although it’s not of course a guarantee of any real scientific nous, I do tend to find that medics with a modicum of scientific education who end up in policy-making positions in developing countries are often a lot more sympathetic to the scientific method than people who have come by another route.

  45. murgatroyd said,

    May 19, 2009 at 2:53 pm

    Some of those in your workshop will be advising on technology investment and innovation policy – so their interest is in assessing the likely impact of investment (R&D + commercialization funds) in a science/ technology area which is intended to lead to jurisdictional advantage. What they need to know is how to assess the claims of science/technology to have “the next big thing” economically and to do so in the context of global competitive science.

    For example, nanotechnology (there is a nice major issue I would add to your first list which is assessing the health and safety risks of developments in science – genomics, metabolomics, nanotechnology for example) – should the government of India invest in specific nanotechnologies in the expectation of gaining an economic advantage over time..

    There is a lot in this space of science/ evaluation and innovation policy which you should at least acknowledge and not ignore.

    Good luck with this – shout if I can help stephen.murgatroyd@shaw.ca

    From one blogger to another – good luck!

    Stephen Murgatroyd, PhD FRSA

  46. skyesteve said,

    May 19, 2009 at 3:51 pm

    Ben – one slight quibble with your “subject list” – there’s a lot of crap science on both sides of the climate debate – both the denial lobby and the catastrophe brigade. I’m a born sceptic and I think that with climate change, as with many such “heated” debates (no pun intended), the truth probably lies somewhere in the middle. That’s not excusing poor environmental practices such as high polluting industries but at the same time we must avoid forcing developing nations into adopting measures which are unrealistically expensive and may not have the environmental benefits that some claim. There are genuine issues around Kyoto and how (or even if) it would benefit the developing nations but yet to raise such concerns (let alone discuss them)seems tantamount to heresy in some parts.
    The bottom line is that for minimal investment we could work wonders for ordinary folk in developing nations if we focused on clean drinking water, adequate sewage treatment/recycling, basic health needs, affordable housing, ensuring adequate foods supplies for all, encouraging democracy, etc. None of these involve clever science; none are prohibitively expensive; yet all could save or enhance the lives of billions of our fellow citizens.
    Mini rant over

  47. TheLady said,

    May 19, 2009 at 4:12 pm

    The other background issue here is that obviously I am very keen not to feel like some white dude telling people how to live and think.

    Speak slowly. You may think you are being enthusiastic and energetic – they may think you are trying to get rid of them as quickly as possible.

    Avoid drowning them in facts, anecdotes and examples. No audience is ever going to retain the amount of information that’s inside your head on any of this – that’s why you’re the expert. All they will come away with is a sense of your superiority, which would be disempowering for them and may lead them to reject your message.

    Be just a guy. You’ve got a tricky line to tread between representing the establishment – which is viewed with suspicion in many ex-colonial countries – and coming off as the iconoclastic maverick archtype that so many pill pushers make use of. I would say talking slowly, keeping your body language calm and trying to not get overexcited should help you there.

    I saw you speak last night at Bristol and enjoyed it very much. Seeing how deeply you felt about Matthias Rath’s murderous scam affected me deeply. Just please try not to tell a bunch of third world officials that he’s “our fault” or is “one of us”. The last thing they’re going to appreciate is a reminder of how pseudo-colonialist scumbags can still hoodwink them, even now.

  48. Diversity said,

    May 19, 2009 at 4:14 pm

    From my own experience talking to developing world civil servants about other topics, I would suggest:

    – Talk to them from what they know. They know people try to fake official approval The material in Joe Wilson’s report on supposed sales of uranium from West Africa to Saddam’s Iraq supplies an example. (Might make the sort of scenario Katewb andQOF have in mind?) People trying to fake scientific approval use similar techniques of supplying genuine looking papers.
    – Talk to them from what they fear. Who will be trying to fool them with fake science? You could go on about that all day.

    – Talk to them about where to find scientific help easily and quickly. Just how you Google scientific topics could probably help a lot today (elieben said that first).

    _ Get them interested in seeing what is fishy. I often used Darrell Huff’s ‘How to Lie with Statistics’ for this (great minds – see above); you may have something equivalent and more clearly about science (Like ‘Bad Science’?)

  49. j said,

    May 19, 2009 at 6:41 pm

    Definitely something about risk, relative risk, etc. Important in policy-making, spending priorities, and so forth.

    Hand out copies of this, too? www.apa.org/journals/features/psp7761121.pdf An invaluable caution when for assessing one’s own, and others’, competence to advise on particular areas…

  50. julie oakley said,

    May 19, 2009 at 7:53 pm

    Warhelmet and dbhb – in defence of the NCT, the one thing they do really, really well in my experience is to arm prospective mothers with all the information they need to breastfeed successfully. The guff I heard from NHS midwives (who I believe only get about an hour’s teaching about breastfeeding in their training) – I’m not surprised so many mothers think they can’t breastfeed, or that their babies need bottled milk because they’re starving etc. So with my NCT group – not a single ‘natural’ birth but we all breastfed all of our babies. Course, all you need to do is train midwives properly.

  51. samarkeolog said,

    May 19, 2009 at 9:09 pm

    I like theonlyrick and QOF’s idea of getting them to create their own scam, because it’s engaging and self-teaching, which is empowering; but I guess it depends how much time you have.

    It’s also dependent upon the audience: if you’re trying to teach non-scientists, they may not have enough base knowledge to imagine how to create a scientific scam; then you wouldn’t get good examples to laud, and they would lose confidence in themselves.

  52. nigel edwards said,

    May 19, 2009 at 9:58 pm

    There is a lot of wisdom here and the sort of check list that you could compile from your own knowledge and the suggestions above would certainly be helpful. But, this is not likely to anchor the learning in a way they will remember. For influencing behaviour of managers and policy makers experience and reflection seems to work best – better still if there is some theory to hold it together.

    I think you need to get them to work through some case studies that use the checklist. My experience is that participatory approaches with this sort of audience is tough, but worth a try. They do the work, help each other, you prod and suggest. This changes the dynamic about being there to tell them.

    Secondly,You might ask them to reverse the process and identify what are the areas where they have seen scams, nearly got scammed, etc. You could add your stories to this. The output of this would be to spot the weaknesses in their systems and what to do about this in terms of expert advice, procurement process, keeping reps away from ministers, handling the media etc.

    Thirdly, you can’t prepare them for everything so can you challenge them about how they would set up a network of advice between themselves and with their own expats in academia elsewhere using the web and other cheap technology.

    The economics and cost effectiveness points mentioned are key – they are probably the ones your audience is best acquainted with. The difference between cost effectiveness and return on investment and the best bet for reducing the overall disability / disease burden is an important distinction.

  53. VaughanB said,

    May 20, 2009 at 7:30 am

    Some reflections on doing science in the developing world.

    It’s easy to say ‘look at the evidence’ but most of the evidence is in expensive scientific journals that are inaccessible. Furthermore, although there are many English speakers, not all scientists or practitioners in the developing world speak English.

    This means that practice can be more easily influenced at the point of information dissemination. Talks, training courses, distribution of journal articles, textbooks and so on.

    Often, this gap is filled by Big Pharma, and they have an influence precisely because almost every clinician and researcher I might was intelligent and motivated to use all possible opportunities to improve their field and its applications. Given a choice between some evidence and virtually no evidence, most people would make a similar choice.

    Perhaps because of this, I found virtually no critical engagement with the pharmaceutical industry.

    Furthermore, research has a much lower status in many developing countries. Universities are less concerned about getting good researchers because there is no big pot of government money to be won by research. However, there is money to be gained from Big Pharma drug research, adding to their influence.

    Additionally, most medical professors I met couldn’t survive on university wages alone, so they do private practice in addition. This means they have little time for research.

    Poor resources also mean that developing world institutions may be taken advantage of by developed world institutions. For example, the department I worked in collaborated with a large Ivy league institution doing an impressive study which involved extensive and time consuming recruitment, structural imaging and genetics. The department got their names on the first paper, but because they didn’t have the skills (or weren’t given the training) to do further analysis the many subsequent papers were attributed to the US institution. This was an ideal opportunity to disseminate skills in a rare well-funded project, and yet it left the department in the less developed country feeling like they did the donkey work while someone else got the credit.

    Some countries may have only been awarding PhDs themselves for a few years. In many departments, only few of staff will have research experience.

    Saying this, I found several brilliant centres of research that blew me away, these are important hubs but because scientific communication is often limited, they perhaps seem a little encapsulated by our standards. For example, there are much fewer foreign researchers passing through.

    In TV advertising unsupportable health claims may be more common and unregulated (I saw lots of TV ads for Omega-3s claiming they treated cancer, mental illness and the like). Hence, I suspect people are more used to seeing strong medical claims for commercial products as normal, where we would see them as dramatic. Often these are supported by ‘doctors’, ‘scientists’ and the like.

    Like many things, science can be very politicised and it is much more common for public debate to include claims that science is being used as a tool of some particular political interest.

    It is important to note that I found almost no evidence of poor reasoning, lack of knowledge of scientific principles or ignorance of experimental principles. The differences were largely in communication (access to unbiased sources of information and other scientists) and motivations for doing research.

  54. NeilHoskins said,

    May 20, 2009 at 8:29 am

    Although your own field is medicine and public health, the way technology and particularly IT is being introduced to the developing world is my own interest. Personally, I wonder about the usefulness (or otherwise) of the “one laptop per child” device that’s being shoved in their faces, despite the fact that it’s not entirely clear whether they asked for it.

    Then there’s the ongoing battle between Microsoft and open source. “Software Libre” would intuitively seem to be the obvious choice, and Microsoft are coming under fire for negotiating binding deals, but Microsoft counter (with some evidence to back them up, frankly) that the total cost of ownership of their stuff can be less.

    And it turns out that yer average Ugandan farmer’s main IT requirement was simply to know what the prices are at his local market, without having to get on a matatu to go and see for himself. This kind of thing can be (and is being) handled perfectly well with mobile phones.

    It’s a fascinating field. There’s more at
    www.kiwanja.net/ if you’re interested.

  55. Dudley said,

    May 20, 2009 at 10:04 am

    Other people are answering the content question better than I could, but here’s a piece of advice regarding delivery: DON’T JOKE. Irony, witticism, satire – African civil servants, particularly ones who’re motivated enough to take this presentation seriously, in many cases don’t think that a funny presentation can also have serious points to make. It’s almost a reflex for English speakers, and incredibly difficult to repress, but you’ll do much better if you’re serious and respectful throughout.

  56. Synchronium said,

    May 20, 2009 at 10:07 am

    I think science education might be an interesting topic that deserves your input.

    Teach a man to fish ‘n’ all that.

  57. Dudley said,

    May 20, 2009 at 10:17 am

    “English speakers” = “speakers at an event who come from England”

  58. jonathon tomlinson said,

    May 20, 2009 at 12:56 pm

    I’ve worked as a doctor in India, Nepal and Afghanistan. If the money spent on drugs in the various clinics I worked in was spent on improving access to potable water, agriculture and nutrition, education, esp. safe childbirth, breast-feeding and contraception, clean fuel etc. I’m sure far more lives would have been saved. The history at home in the UK is that the improvement in mortality that has come about because of nutrition and sanitation vastly ecceeds that of modern medicine. Even now in the UK, with sanitation and nutrition ‘mostly’ solved, if we actually applied the evidence that we have to our clinical practice (and made less mistakes) and stopped spending money on new drugs or other interventions, we could make enormous improvements to the health of the nation.
    What all nations have to do is apply the knowledge we already have, rather than be distracted by ‘new’ technological fixes.

  59. cat said,

    May 20, 2009 at 2:29 pm

    Simple. Basic.
    Tell people that the moment anyone suggests that their method/treatment/drug isn’t susceptible to being tested by the gold standard scientific method, the controlled double-blind trial…give them the two-fingered salute. Things work and can be shown to work; or they don’t and can’t.

  60. cat said,

    May 20, 2009 at 2:34 pm

    Oh, and with drugs, to ask for the name of the active ingredient. Source the generic rather than buy the expensive ex-patent stuff.

  61. chinaphil said,

    May 20, 2009 at 2:51 pm

    Most important: remind them that real science is basically free these days. If they’re being asked to pay for knowledge, then something’s gone wrong. Point to PLOS.
    Second, remind them that real academics are generally more than happy to talk. If you’re not sure about a particular claim, just email someone well-known in the field and ask them.
    Like you say, you can’t teach these people to distinguish bad science – particularly not if it’s clever bad science cooked up by a big company. So it’s much more important to teach them to find people who do have that ability.
    And with the internet, finding those people is now free and easy.

  62. Nik said,

    May 20, 2009 at 6:45 pm

    Never be afraid of asking questions! If a salesman or representative makes a claim, investigate it thoroughly, check the facts and do the arithmetic yourself. When something isn’t clear, keep asking until the answer either makes sense, or you are sure it is fluff. Often, salesmen will try and make you feel stupid for asking perfectly reasonable questions that might result in your not buying their products. Take your time and never accept the science at face value. Vast amounts of money are invested in R&D, in the hopes that vaster amounts of money will be made from selling the product. Something that’s worth having will still be there tomorrow, but if you are being sold a pup they will want to reach a binding agreement and get you before you can find out the reality. And don’t believe that money alone can buy happiness.

    On specific subjects – effective and safe use of pesticides and herbicides springs to mind.

  63. JustOneThing said,

    May 20, 2009 at 10:39 pm

    I’d take a little humility and listen before I advised people in the ‘Developing World’ (whatever that is), same as I would speaking anywhere to assure myself that I was starting from where the audience was at (to paraphrase Piaget).

  64. SVC said,

    May 21, 2009 at 8:35 am

    I deal extensively with people from Africa, especially west and central Africa.

    I would say that you should be careful to treat them with respect, which I’m sure you would. There is often a tendency for people to assume that those from “developing” countries are poorly educated or do not have ready access to the information we take for granted – for civil servants and businessman from these places you are likely to find that they are very well educated, often in Europe or the States.

    If, indeed, any of these people are from Africa you should also be aware of two key components which seem to be common.

    Firstly is post-colonialism; while most understand the need for foreign technical help and other support there can be a great deal of resentment entangled with accepting it. Be cautious to present your views without too much passion. Reasoned arguments rather than statements or instructions are likely to get you more mileage.

    What I absolutely would not do is spend any time finger pointing – yes we all know there any many examples of snake-oil sellers duping African nations, but believe me you wont get very far with that line. Examples are best left for their own research. Even if they actually agree with you, a lot of Africans will side with another African nation against accusations of any kind from a “developed” nation. They are very good neighbours.

    Denialism fits in here. It would be very easy for you to go on at length with the many examples you could bring up. I wouldn’t. There are also some cultural issues here, surrounding traditional belief systems and practices. It’s a big, touchy subject and I think you risk putting yourself on the back foot by bringing it up regardless of how important it is.

    Secondly is something called “Africa time”. You really actually have to work with Africans on an equal basis to understand this. Things have a different, much more gentle pace. This can be a bit of a pain for us rushing westerners, but can be useful to you – following a line advising cautious fact checking, taking the time to make choices and listening to advise from many people. That will sound like excellent sense to Africans. Excellent sense for everyone, in fact.

    In fact, I would heavily weight your talk towards the ideas of seeking and reviewing independent advise, and on how to critically assess those who give it. Choice, self determination, is extremely important for developing nations.

  65. Gareth said,

    May 21, 2009 at 11:26 am

    This topic got me thinking about what I’d say to my non-scientific housemates if, for some reason, I had to teach them how to interpret science and defend against pseudoscientific reasoning (and could hold their attention on such a topic for more than 30 seconds).

    I think, basically, a crash course in critical thinking would be in order. With a good sceptical attitude and reliable sources of information, it’s possible for a reasonably intelligent person to come to a pretty reasonable conclusion on a topic, even if it’s not within their area of expertise or education.

    It’s like the old “Give a man a fish and you feed him for a day. Teach a man to fish and you feed him for a lifetime.” thing. You could give specific examples of why certain things are pseudoscientific and a non-scientist would know why those topics aren’t science. But teach that person how to think like a scientist and they have one of the tools required to make their own reasoned conclusions. Back this up with a few good information sources to get them started and some examples of these tools in action and they’ll better far equipped to deal with such issues than your average lay person.

  66. Tigger0039 said,

    May 21, 2009 at 1:25 pm

    Re: Pharma – people need a source to use to determine whether the drugs being pushed are standard and inexpensive. Coming from the US, where health insurance is private, with all the copays and whatnot, insurance companies have tiered formularies that can serve as good proxies for what’s considered to be the cheapest form of xyz drug. Undoubtedly, these generics can be sourced even more cheaply from India or Brazil or S. Africa, but the generic formulary (or even Wal-Mart’s $4 formulary) would quickly give a civil servant an idea of whether there isn’t a cheaper alternative to whatever pharmaceutical is being pushed. And asking the question of the salesperson, “why is your drug better than what Wal-Mart would retail for $4, and that I could get wholesale from India for $0.25?” seems to me to be a good start. If the salesperson squirms, the Civil Servant should know it’s a crock.

  67. Tigger0039 said,

    May 21, 2009 at 1:35 pm

    Rather than “teach” people how to think scientifically, I would ask them to think about all the various forms of nonsense there is out there. How many times and ways have they been buffaloed. Every culture has its own colorful words for charlatans and different kinds of fakery, so the question has to be “who is a reliable source for information”. To determine that, one needs to have an idea of how the source benefits by giving the information that it does and does it give that same information to people with other interests. In short, I would advise the civil servants to “Follow the Money”. And if the money is being offered to go into your pockets, that says that the seller has no faith in the quality of his product to stand on its own.

  68. igb said,

    May 21, 2009 at 2:19 pm

    The problem in some parts of the developing world is that they want to be deceived. AIDS denialism wasn’t foisted on South Africa by evil white deceivers (a view which which, by denying agency to the government, is itself rather racist). Mbecki wanted to prove that Africa could solve a disease that the rest of the world couldn’t, and that African solutions were the best way to do that. By taking that position, he was making himself an open target for every charlatan that came along: people who knew the science couldn’t deliver the `too good to be true’ that he wanted, but that laid him open to people who promised things that were too good to be true…

  69. Robert Carnegie said,

    May 22, 2009 at 12:09 am

    Civil servant types who aren’t science literate themselves will serve that need well by having one or more people on the team who do have appropriate expertise – not necessarily always the relevant speciality of the day, but they can either have some understanding of fields besides their own or else get in touch with more people who have better knowledge of an area. Britain’s own top government science guy was just interviewed somewhere, I think he had the same job title as Doctor Who…

    A few problems with current issues like AIDS or climate change in this context: they are unsolvable, they are cases where the developing world does feel that richer countries are talking down to them and dictating what to do, and, in fact, “we” have been and are doing that. Meanwhile, people die, the climate does change, and rich countries demand biofuel products to replace oil. So this is the controversial stuff. Instead, look at industrialised agriculture – done wrong – or at Lysenko – if the verified facts live up to the legend.

  70. kim0 said,

    May 22, 2009 at 6:00 am

    I think the main problem is misunderstanding science. The concept itself is simple, but is typically confused, misrepresented, overcomplicated, etc.

    So my solution is to express science as simply as possible, and this is my take on it:
    (Note that it contains falsification, Ockhams razor, and the theory concept, using plain language words.)

    Science is the making of knowledge.
    Knowledge is made by finding better explanations.
    Explanations are better when they are shorter or explain more.
    To know how much an explanation explains, it must be tested.

    Kim Øyhus (c)

  71. Alan Bird said,

    May 22, 2009 at 6:23 am

    In the longer term I don’t think that you really need to rewrite the message – that’s already been done in your book and dozens of other excellent examples (see above for some). What you need is to get that information to a new audience in other countries and other languages. Are there any plans to translate your Bad Science book into other languages? (I dearly hope it comes out out in Chinese for example.) I note that there are companies/systems in place to produce cheap legal local reprints (often in English but sometimes translations in the local languages) of textbooks in other countries. China eg has dozens of titles in linguistics, maths, medicine, all sorts of disciplines (I assume intellectual property rights etc are sorted out). If they could widen the field of text books to include selected popular science books – badscience, Bryson, how to lie with statistics etc etc (I’m sure you could produce a shortlist) – you would ensure simple, clear information can be available on tap to the educated people, and perhaps the general public, in those countries.

    Do the Royal Society already do this kind of work? Do they see it as part of their role? If so I imagine they have oodles of resources. Maybe Professor Colqhoun could advise.

    One resource often overlooked is the local Standards body. A national standard isn’t a popular science book but it’s the correct place to define an approved method of eg sterilising water supplies or sinking a well, especially if there are local considerations to be taken into account. If you don’t know whether the countries your delegates are from actually have one, you could check with BSI (British Standards Institute) about what bodies exist in other countries. In fact you could ask BSI about your problem. They not only have access to a vast body of national & international standards but they have the knowhow on setting up a standards writing and approval process.

  72. JoanCrawford said,

    May 22, 2009 at 9:12 am

    At the risk of teaching you to suck eggs, the actual mechanics of the meeting itself need to be considered.

    We need to understand what you’re working with here. What do these civil servants actually do? They might surprise you. Which countries specifically are involved? South American issues might well be different from, say, Far Eastern ones. How long do you have? 2 hours? 2 days? Given the timescale, what are the stated objectives of the meeting? What do the various delegates a) bring with them b) take away?

    As a facilitator (assuming that your role has been correctly defined) then with quite a lot of people in the room (it sounds like), you will need to be pretty hard-arsed about keeping people on topic, to time, and making sure that the contributions are constructive. I only say that because I know that your speaking style is a bit digressionary, and wasn’t sure if you allowed that in others when leading a discussion.

    Put simply, how will you know that the meeting has been a success?

  73. rob said,

    May 22, 2009 at 3:10 pm

    Completely OT, but definitely of interest to readers of this forum: The perfect formula for a talk” – a blog entry on Ben’s talk at the Bristol Festival of Ideas.

  74. ceec said,

    May 23, 2009 at 9:12 am

    Re. delivery – to echo a couple of points made above, my experience of this type of teaching (in the UK rather than in situ) is that you have to be very careful not to underestimate the audience.

    I find it less easy to be wrong-footed if I assume I know much less than the audience about just about everything relevant to their lives and work – then when that turns out to be true, as it invariably does, I look like less of an arse. Good preparation also helps a lot.

    Also, again from my teaching experience, I’d say you need to be careful with language – use simple constructs and vocab and speak slowly or you may well lose people even if their English is excellent – British people have very weird accents and we swallow a lot of crucial syllables.

    “Lose the jokes” suggestion is a good one too, although you will probably find you do that anyway once you are in the situation because it will probably seem inappropriate.

    Bon voyage!

  75. ceec said,

    May 23, 2009 at 9:13 am

    Sorry – should have said “teaching this type of audience”

  76. PeteB99 said,

    May 24, 2009 at 8:11 pm

    Yes – maybe use Western examples : this is how we got fooled, make sure you don’t

    I think that is a pretty good list

    Maybe link The Tobacco Papers/ Passive smoking denial / Climate Change Denial ( many of the same people/organisations in all three) and the use of Astroturfing

  77. stinkychemist said,

    May 24, 2009 at 8:56 pm

    I’d stay off climate change, myself, the people of Africa, with the exception of a few cities, don’t produce the sort of CO2 output per head that “Northerners” do. Unless they actually want to know about non-carbon energy resources.

    Sorry, I’m just an intellectual lightweight in this company, but it may come up.

  78. martin-clmb said,

    May 25, 2009 at 12:02 pm

    Reading through these posts, there are some good ideas. I think that SVC’s suggests are about the best. Combine that with examples from the North and careful use of language and then that will help. In terms of what subject focus – well it obviously depends on the audience. I think that the audience you’re dealing with would benefit from:

    – the importance of evidence in policy making AND implementation
    – tips and hints on getting evidence reliably
    – suggestions for getting assistence (careful of the word “help”) when it is needed in interpreting evidence
    – the dangers of bad evidence and perhaps examples and discussion of common sources and causes of bad evidence

    I’d try and focus examples on cases in the North and, like any talk aimed at a non-specialist audience, pitch them at an easy level to understand. I’d personally not recommend getting into too much detail (as in most of your book), as what you’re trying to achieve is an understanding (or agreement) in the need for _evidence_.

    Of course you may also need to walk the tightrope of evidence informed policy vs. politics informed policy (US domestic drugs policy anyone?) and that could be considered a bit of a neo-colonial issue – the development sector is keen on evidence informed policy making but how good are they at implementing that in their own countries? Perhaps best just not to raise this as an issue.

    The kinds of civil servants you’ll be dealing with will be amongst the elite in their setting. Some of the advice above about education and intellectual levels is worth noting. Overall intellectual ability will be high (perhaps higher than in a similar civil service gathering in the UK) but what will differ is that the levels of scientific training and experience may well be lower, possibly much lower (unless you are dealing with a very specialist group, which I suspect is not the case). So that should be taken into account.

    Also, unless something like this is billed as “training session” then people all over the world tend to think that they do not need training (especially in something so semingly obvious as the importance of evidence). So pitching things at a “management seminar” type level is often good advice. Afterall, managers and other elites (including scientists all over the world) often feel that they don’t need training. But being “briefed” at a seminar is another thing!

    Good luck.

  79. jsymes said,

    May 25, 2009 at 8:32 pm

    Dr Goldacre: Exactly what can you bring to the table regarding “Climate change denialism”? You are a doctor of medicine. Do you also have a degree in climatology or paleo-climatology you’ve kept hidden under a bushel? Or do you intend merely to browbeat and hector third world countries with the western media consensus about the causes of climate change just because you once saw a movie by Al Gore and feel sorry for those poor little polar bears? Stick to medicine, related policy and its implementation. The arguments about climate change and its causes are best left to those who know what they are talking about.

  80. PeteB99 said,

    May 26, 2009 at 10:53 am

    Re 79

    I think you misunderstand what Ben is suggesting – not lecturing people on climate change (perhaps avoid it completely if it is likely to be counterproductive) but more generally explaining how industry or idealogical groups (from all sides of the political spectrum and beyond) can misrepresent science and mislead people.

    see scienceblogs.com/denialism/about.php

    “Denialism is the employment of rhetorical tactics to give the appearance of argument or legitimate debate, when in actuality there is none. These false arguments are used when one has few or no facts to support one’s viewpoint against a scientific consensus or against overwhelming evidence to the contrary. They are effective in distracting from actual useful debate using emotionally appealing, but ultimately empty and illogical assertions.

    Examples of common topics in which denialists employ their tactics include: Creationism/Intelligent Design, Global Warming denialism, Holocaust denial, HIV/AIDS denialism, 9/11 conspiracies, Tobacco Carcinogenecity denialism (the first organized corporate campaign), anti-vaccination/mercury autism denialism and anti-animal testing/animal rights extremist denialism. Denialism spans the ideological spectrum, and is about tactics rather than politics or partisanship. Chris will be covering denialism of industry groups, such as astroturfing, and the use of a standard and almost sequential set of denialist arguments that he discusses in his Denialist Deck of Cards.

    5 general tactics are used by denialists to sow confusion. They are conspiracy, selectivity (cherry-picking), fake experts, impossible expectations (also known as moving goalposts), and general fallacies of logic.”

  81. Clair G-S said,

    May 26, 2009 at 3:58 pm

    Good point. At the very least someone should draw Goldacre’s attention to this article by Julie Clayton entitled “Spotting Fraudulent Claims in Science”.


    There also an editorial that David Dickson, director and editor of the website scidev.net wrote last year called “Building a web we can trust”. The link is here:


    A good case history — which acts as a bit of a counter example– is the treatment of critics scientists who protested at the Gambian president’s claim that he could cure HIV/AIDS.

    See: www.scidev.net/en/news/who-dampens-gambian-presidents-hivaids-claim.html

    Also the fact that the Nigerian Academy of Sciences was sued when it issue a critical report about a local doctor who claimed similar powers.

    See: www.scidev.net/en/news/hiv-cure-doctor-sues-science-academy.html

  82. elspeth2009 said,

    May 26, 2009 at 10:08 pm

    Depending on the level of scientific knowledge, I would emphasize something from basic statistics, which I find always catches me out, which is statistical significance (I think that’s the term). So you might have a difference in two outcomes or whatever, but is it statistically significant? Look at some simple examples of things that look significant but aren’t. And speak slowly is really good advice. I would be tempted do a set of principles, then practise them by doing a few (maybe real life) case studies, would you buy this product or not, if so, why, if not why not. If there are two sessions the second could focus on the second set of ideas, cognitive biases, recurring themes, etc etc, either principles then application, or looking at some cases (maybe from North) and deciding what the problems are. This should have good results if people involved are receptive to this type of learning. Having said this I would really worry about doing this kind of stuff. I would make sure I knew exactly who had asked me to do it and why, and also maybe try and make sure that you take something away from it, so there is two way learning. I definately wouldn’t stay off climate change personally because I think that Africa will be affected by it even if they don’t contribute to it, ie in droughts etc. Good luck!

  83. elspeth2009 said,

    May 26, 2009 at 10:31 pm

    Oh, and as far as structure goes, and not being a white dude telling people what to think, it would obviously be useful to spend a lot of time at the beginning finding out what the actual day to day decisions and problems that these civil servants are dealing with are. For instance, you may be assuming that it’s all HIV in Africa, and climate change in South Asia, when in fact it’s death in childbirth in both areas. I’m sure you know this already though.

  84. PeteB99 said,

    May 27, 2009 at 3:36 pm

    I think this got caught by the spam filter so I’ll repost it without the link

    Re 79

    I think you misunderstand what Ben is suggesting – not lecturing people on climate change (perhaps avoid it completely if it is likely to be counterproductive) but more generally explaining how industry or idealogical groups (from all sides of the political spectrum and beyond) can misrepresent science and mislead people.

    see denialism on scienceblogs

    “Denialism is the employment of rhetorical tactics to give the appearance of argument or legitimate debate, when in actuality there is none. These false arguments are used when one has few or no facts to support one’s viewpoint against a scientific consensus or against overwhelming evidence to the contrary. They are effective in distracting from actual useful debate using emotionally appealing, but ultimately empty and illogical assertions.

    Examples of common topics in which denialists employ their tactics include: Creationism/Intelligent Design, Global Warming denialism, Holocaust denial, HIV/AIDS denialism, 9/11 conspiracies, Tobacco Carcinogenecity denialism (the first organized corporate campaign), anti-vaccination/mercury autism denialism and anti-animal testing/animal rights extremist denialism. Denialism spans the ideological spectrum, and is about tactics rather than politics or partisanship. Chris will be covering denialism of industry groups, such as astroturfing, and the use of a standard and almost sequential set of denialist arguments that he discusses in his Denialist Deck of Cards.

    5 general tactics are used by denialists to sow confusion. They are conspiracy, selectivity (cherry-picking), fake experts, impossible expectations (also known as moving goalposts), and general fallacies of logic.”

  85. Doug Mackie said,

    May 28, 2009 at 1:00 am

    Things I look for:

    Excessive precision. 34.62%, 23,405 jobs will be created.

    Lack of uncertainties, inappropriate uncertainties (wrong precision), or if given no indication of what the uncertainty is (std err, sd etc etc). 34.62 +/- 5.25

    Poor referencing. A study showed 23,405 jobs would be created. (Man, I want to author a “study” one day that for 24 hours is on every news report).

    What I call “colloquial” language. An actual quote from a submission to a Select Committee: “Fascist New World Order”. Usually though milder but uses words like “silly”.

    Any discussion of motivations for alternative views. (Cuts both ways so not always a useful criterion).

    Multiple but not related lines of “evidence”. E.g. “I don’t accept climate change because it is a hoax with no evidence and anyhow it is all caused by the sun.”

    Forcing on plots. The text cites a strong relationship. The graph shows an amorphous blob of 97 scattered data at low x and low y of the axes with 3 points at high x high y.

    Absurd extrapolations. Improbable Research used to give these. Since no one ever throws away National Geographic magazines it was possible to calculate when the accumulated mass would cause the US to sink into the sea.

    Junk references. Michael Crichton was many things but he was not a reliable source of evidence to cite about climate change.

  86. srd said,

    May 28, 2009 at 7:08 am

    Why assume that people in developing countries don’t use/know science? But more seriously, there are tonnes of academics and practitioners doing research on these things – in science studies, environmental history, veterinary history, agrarian studies there’s a huge wealth of expertise on science and policy in Africa and Asia, much of it harnessed by DFID, the Africa Union etc. I think it is great if you can add to this, but no need to reinvent the wheel. Talk to the people who do this professionally and then see what you can add!

  87. nongovernmentalindividual said,

    May 30, 2009 at 7:16 am


    This sounds really interesting.

    In your introduction you emphasise procurement; the issue of evidence (as others have pointed out) is also important for prioritisation of public policy, and design of public health programmes more generally, including those funded by external donors of course. Take the example of AIDS programmes.

    Firstly, there needs to be an understanding of who is most affected, and who is most at risk, not only because it is important to support those who are disproportionately affected but also because if programmes can target these people then they will probably have the most impact at a national level. So here we need to understand relative risk and population attributable risk.

    Often with AIDS, the people most affected are often disadvantaged for other reasons – it may be possible to help them to access the services that everyone else has access to, but more often it can be necessary to design specific programmes that help deal with the specific issues they face – for instance sex workers often face violence and other forms of discrimination; homophobia also poses a major challenge for AIDS programmes. In these circumstances, handing out leaflets and condoms and telling them to get tested isn’t enough.

    Just to complicate matters, it is these “specific factors” that mean a) that there is a lack of evidence for relative risk/attributable risk in these groups (because no-one is doing the research); b) that even where there is evidence, donors and civil servants don’t particularly care about these groups or want to go there because it is politically or morally controversial to do so (if you’ve got a big pot of money for AIDS, it’s a lot easier and sexier to spend it on “deserving” school kids than on “morally corrupt” sex workers, drug users or men who have sex with men); c) that even if the data on attributable risk are there, the proposed interventions are more likely to be determined by the moral attitudes of the society, government or donors in question than by evidence; d) that even if governments and donors agree to do the right thing, the local health care providers or non-governmental organisations that get the funding to provide services may spend the money on something else anyway.

    You’ve talked a bit about c) before when discussing the US government’s policy on AIDS programmes for sex workers and on abstinence-only programming. (By the way, without wanting to let the US government off the hook for this it’s worth recognizing that these policies only confirmed pre-existing attitudes in many countries; it’s not as if they were all empowering sex workers and promoting comprehensive sex education in schools before Bush arrived).

    So: while the influence of foreign donors and businesses can be pernicious, it’s important not to assume that they are the only factors that compromise the design of public programmes… and while it’s crucial for national level decisions on programme strategy and procurement are based on evidence, it’s also crucial to strengthen the evidence base and to ensure that this knowledge filters down to those who are actually delivering services.

  88. daven said,

    May 31, 2009 at 1:07 am

    I worked for 2 years in Mozambique and 4 years in the Kenyatta University Appropriate Technology Centre. Working with people from villages, hundreds of NGOs and different government agencies I came across against many examples of outsiders pushing particular technologies that, although they worked, were not the answer to the country’s problems. So I would add to any session some time considering relevance and, especially, appropriateness.

    I worked mainly on biomass energy. At the 1991 UN Energy Conference, in Nairobi, my Kenyan colleagues arranged a march of women carrying wood on their heads. The developed country visitors thought the world’s energy problems were about oil and electricity. But at that time, the main energy source for 95% of the world’s population was wood. It was only the rich in the developed worlds and some of the big cities in underdeveloped countries that used large amounts of fossil fuels. So everyone pushing large dams or nuclear power plants on developing countries as a solution to that energy crisis were providing a solution to the wrong problem: the oil crisis, not the woodfuel one.

    My colleagues were the snake oil salesmen at the time, trying to persuade governments to improve the energy efficiency of wood and charcoal stoves, to give people enough time to plant more trees. It was based on good science (heat transfer, laws of thermodynamics, kinetics of combustion), but it took a lot of work to turn that into the Kenya Ceramic Jiko, and get artisans to turn out 125000 stoves a year, saving 1/3rd of the charcoal previously used.

    How did we make sure that the better stoves were taken up, and some silly ideas (like gasifying stoves) were not? By lots of testing, and getting all the people involved to talk to each other – the civil servants, aid agencies, NGOs, university departments, small companies, so that everyone understood the links between design, testing, and energy saving in the home.

    So you should get the civil servants to make a list of all the scientifically aware people or organisations in their countries who could help them assess proposals from outside. The big risk is in turnkey development, be it just buying in a factory, or buying in drugs, without understanding it well enough to change things when the inevitable problems arise. But get together a lot of sceptical scientists, and it is possible to pick out the best and worst of the proposals.

  89. lisadom said,

    June 1, 2009 at 9:14 pm

    I can think of plenty of people in the so-called Ist world who could benefit from a little bit of genuine analyis and realistic interpretation of data in their decision making.

    Putting a load of graphs and charts up on an overhead can still bedazzle a room full of supposedly educated people with an interest in an area like autism. When in reality the presentation owes more to PT Barnham, than BF Skinner (insert preferred scientist)

    There is also a rather wealthy television presenter across the pond who could do with a little accurate think tank data analysis too. (or not, perhaps her science is that of Mr Barnham and his populist approach to getting insanely rich)


  90. lisadom said,

    June 1, 2009 at 9:17 pm

    Doug Mackie – I have copied your comment. Just ticking a lot of boxes for me!!

  91. timboson said,

    June 3, 2009 at 8:46 am

    I’d say its the same in the developed world as it is in the developing world :o)

  92. Tessa K said,

    June 3, 2009 at 11:39 am

    Sorry I don’t have time to read all the responses so someone may have suggested this already.

    One thing to encourage people to check is who is funding initiatives that appear scientific and question what their motives might be.

    For example, PEPFAR is a US partnership promoting and heavily funding circumcision over condoms as HIV/AIDS prevention in Africa.

    When you dig into who their partners are, a lot of them are religious organisations who are anti-condom and pro-abstinence. When you look at the research for the effectiveness of circumcision and the problems in carrying it out, the initiative starts to look pretty dodgy.

    So do look a gift horse in the mouth.

    Sorry to blow my own not very scientific trumpet but I’ve done a bit of digging and blogged the details here: tessera2009.blogspot.com

  93. heavens said,

    June 4, 2009 at 4:48 pm

    I want to second Fontwell and others that suggest using informal contacts to their advantage. I’ve occasionally gotten “random” e-mail from people with science-related questions, and I try to answer them all, even if all I can do is to send them on to someone else. Most of us science geeks are really quite happy to share what we know, and a lot of good questions get handled this way. For example, I’ve asked researchers to re-crunch their numbers in a slightly different way, or to control for a different factor, to make sure that it didn’t change the outcome. Sometimes it sets us on an interesting question; sometimes (usually, I think) it confirms the outcome.

    One quirk of dealing with developing countries: People towards the top of organizations in these societies tend to flaunt their powerful social status in ways that offend most Western science geeks. You may be the most important *person* in the world, but your impressive social position and enormous power doesn’t change the *data*, which is what I want to talk about. I don’t need a big build-up about how important you are; if I know anything about the subject, I’ll eventually answer your question (and also tell you how confident I am in my ability to answer it). I’m just as likely to answer questions about “my sister has breast cancer” or “I think I have an autoimmune disease” as I am from some high muckety-muck in government that needs to make a big public health decision.

  94. nongovernmentalindividual said,

    June 4, 2009 at 8:04 pm

    Tessa K,

    A bit of pedantry… but the pro-male circumcision lobby for HIV prevention goes beyond PEPFAR; it’s also a key policy of UNAIDS (www.unaids.org/en/PolicyAndPractice/Prevention/MaleCircumcision/) and of the World Health Organisation (www.who.int/hiv/topics/malecircumcision/en/index.html).

    At first the evidence looked very good, so much so that the randomised control trials had to be stopped because it was considered unethical to withhold male circumcision from the control group. The results (I can’t remember them exactly, but it was something like a 40-60% risk reduction) were obviously pretty seductive to policy makers who have been desperately trying, for decades, to find ways of making inroads in HIV prevention.

    But, yes there were potential biases, and of course it’s not really possible to blind a trial of such an intervention… Now, as you suggest, there are concerns about risk compensation among circumcised men (i.e. having more unprotected sex because they feel safe) and therefore about how effective the strategy will be in the longer term.

    Anyway, enough of the pedantry. You are absolutely right about questioning the evidence base for the policies of different donors. But the most insidious part of PEPFAR has been about “abstinence only” programming; tragically it is only in the last year or so that we’ve seen clearer evidence that abstinence-only programmes are a waste of time, and maybe counterproductive.

  95. Tessa K said,

    June 4, 2009 at 9:50 pm

    nongov (if I may be familiar)

    You’re right, it goes beyond them; I was just using them and the pro- abstinence partners as an example. But informed pedantry is something we all do well…

  96. bob_calder said,

    June 6, 2009 at 3:18 am

    I didn’t read all of the posts, but I see it as a search authority problem. The thing is that you can’t do it without knowing a fair amount about the subject. The best idea is to call someone who markets multiple solutions for the problem. Period. They need to build a network of reliable advisors.

  97. worloc said,

    June 14, 2009 at 8:41 pm

    Ben, should say from the start that I’m from a qualitative background and maybe a more participative approach may work. In groups, ask your civil servants about instances of ‘bad science as a con’, that they can think of. Get them to vote (against criteria e.g. most cost, least effective) for the worst case. Work through the ways that ‘evidence’ in this case is/was/could be used as a convincer. Finally ask them to provide ‘operational’ solutions and round-up with feedback and summary session. The main benefit is that you would be ‘handing-over-the-stick’ and consequently, not be seen as the patronising white-man.

  98. Jo_C said,

    August 25, 2009 at 7:01 pm

    How do the civil servants want to use research? If they are from the sexed up dossier school then training in spotting misleading research is a bit wide of the mark, and not meeting their needs. What pressures are put on them from above to find the right evidence? But perhaps the workshop has been and gone.