However, their use in this field is limited by ethical and practical concerns (How do you justify allocating an income maximisation intervention to only some of those who need it? How do you double or in many cases even single blind a social policy? It can be difficult to agree what the outcomes of interest are, and then to develop means of measuring these effectively etc etc) In many cases, researchers have agreed to adopt a next best approach, by using before and after designs of a single cohort (ie: just the intervention group.

Anyway, don’t have access to any relevant links just now, but I know that the Campbell Collaboration have been involed in RCTs in a number of areas of soical policy (crime and education for example).

ann.sagepub.com/content/vol589/issue1/

I’ve gotta admit I study politics and work as a research assistant for an international economist and I’ve never come across a RCT in my day to day studies / work. There’s also a general attitude amongst academics that they’re “impossible” or nearly so in political science, and woe betide the academic or student who drops the word “experiment” into a paper. Political scientists don’t experiment – they compare different policy options.

As for “evidence based policy making” this usually seems to consist of commissioning a thinktank or academic to do a quick and dirty secondary literature review over the course of a couple of weeks, then releasing it with a misleading press statement if it conforms vaguely to the decision you’ve already taken. If it doesn’t it might surface in some form in a institutes working paper collection.

Ah Politics!

Using Random Allocation to Evaluate Social Interventions: Three Recent U.K. Examples

I’ve sent you a PDF.

However there is still a need to do better research. In particular we need to get better at setting out and measuring outcomes from social policies.

The real ‘problem’ is not just of ‘ da meeja’, but of democracy and the need for public policy decisions to be legitimate and accountable. Criminal justice is a good case, as mentioned above, where there there are conflicting objectives, other than maximising the Quality-Adjusted Life Years of a clinical trial.

Public policy theorists sometimes try to frame these conflicts using public value theory.

To turn the tables on medical research, are there lessons that you could learn from social research? For instance, do you take economic and social costs into account when considering the risk-benefit ratio for new interventions? Do you think about what patients and the public value in medicine?

Anyway, here is the Savings Gateway pilot mentioned at ippr that used random digit dialling to recruit participants. I haven’t read it but the authors Ipsos Mori and Institute for Fiscal Studies have good reputations in the public policy field.

www.hm-treasury.gov.uk/media/7/0/savings_gateway_evaluation_report.pdf

www.guardian.co.uk/science/story/0,,2130777,00.html

Now, I have to admit it’s not exactly clear to me what’s happening to the Science and technology committee, but there’s surely not enough evidence in policy making as it is.

And surely there’s the rub, at least in part.

In medicine, we tend to agree what outcomes are important – death, is pretty important and we’ve developed some ways of measuring thing like quality of life and disability adjusted life years. We mostly know to criticise drug companies for choosing outcomes they know they can reach (which of you are interested in you LDL-C at 6 weeks after starting your statin, when you could have one that actually stops you dying as early as you might?)

The Daily Mail would presumably like an outcome of “whatever would punish those crims the most” inviting the prospect of a fun questionnaire survey (“How would you rate the enjoyment of you time in jail? 1. Hugely enjoyable – would love to come back to 5. Hated it – let me out of here”). Most of us reading this, it seems, would like an outcome of “not going back to prison”, “getting a job” or something.

I’m not saying that getting at these outcomes is impossible, but that it probably needs a lot of discussion and thinking about.

I’m afraid Stats is not my strong point (as if anything is), but would you mind expanding on the last digit telephone number snippet from #21 – why such a school boy bogus method of randomisation?

Brave New World, etc etc.

Yes. They will fight over where they want to go. Only the best way to get there would be decided by research.

testament to the culture gap is this: someone at the IPPR yesterday remembered there had been an RCT on something, but after chatting more it turned out it used last telephone digit to randomise subjects into one group or another. 20,000 people in the study apparently (cant remember what it was on) but then they mess up by using a schoolboy bogus method of randomisation.

too weird.

Just for my 2p, there has been a lot of controversial research into the efficacy of restorative justice in the USA and Australia for different types of offender. The obvious problem with all the research is that it leads to a very publicly unpopular conclusion that different punishments may be more effective and cheaper for different inmates, i.e. male white inmates would get a custodial sentence, while female white inmates get restorative justice measures for the same offence – This rubs up against the public’s view of fairness and punishment, despite the potential increase in efficacy/harm reduction. Look up “Lawrence W. Sherman” and the “Jerry Lee Program in Randomized Controlled Trials of Restorative Justice” on Google if interested.