Politicians can divine which policy works best by using their special magic politician beam

May 22nd, 2010 by Ben Goldacre in bad science, crime, drurrrgs, politics | 38 Comments »

Ben Goldacre, The Guardian, Saturday 22 May 2010

So all good citizens this week are poring over the “Programme For Government”, and it’s true to say that there is much to be pleased with. Labour wasn’t all about unbridled credit and fun public sector spending sprees: they kept all your emails, kept records of the websites you visited, used “anti-terrorism” legislation on people who plainly weren’t terrorists, and so on.

But most interesting are the noises now being made about crime and evidence. “We will conduct a full review of sentencing policy” they say: “to ensure that it is effective in deterring crime, protecting the public, punishing offenders and cutting reoffending. In particular, we will ensure that sentencing for drug use helps offenders come off drugs.”

These are grand promises. Compulsory addiction rehabilitation with “Drug Testing and Treatment Orders” was introduced precisely 10 years ago as an alternative to custodial sentences or simple probation for those who have committed drug related crimes. Their implementation without adequate analysis represents just one particularly graphic example of our tragic ineptitude at running simple trials of social policy.

A judge making a decision on a criminal’s sentence is in the exact same position as a doctor making a decision on a patient’s treatment: they are choosing an intervention for an individual in front of them, with the intention of producing a particular set of positive outcomes (reduced crime, and reduced drug use); they get through a large number of individuals in a month; and in many important situations they don’t yet know what works.

If you randomly assign a fairly large number of criminals, or patients, to one of two interventions, in situations where you don’t know which intervention would be best, and measure how well they’re doing a year or so later, you instantly discover which intervention is best. Add in the cost and you know which is most cost effective. The basic principles behind this idea were first described in Daniel 1.12 of the old testament, and can hardly be considered new.

Before being rolled out nationally in October 2000 DTTO’s were extensively piloted in 3 cities by the Criminal Policy Research Unit of South Bank university, at considerable cost. What insights did this generate? There was no randomisation, and no “control” group of identical criminals given traditional sentences for comparison, so the only new knowledge generated by these pilots was the revelation that it is possible to set up a DTTO service and run it in some buildings in some cities.

As it happens, when they did follow up the people who had passed through the service, they hadn’t done particularly well, but this finding wasn’t published until after the service had been rolled out, and in any case, the chosen study design means we have no idea how these participants would have turned out if given a traditional custodial sentence, so there’s no sense in giving these results even a moment’s thought.

This is a tragedy, and not just because drug use is estimated – with the usual caveats on estimating nebulous notions – to cause 85% of shoplifting, 80% of domestic burglaries, over half of all robberies, and so on. This is a tragedy because it speaks to motives that will never go away.

It takes a brave and modern politician to say “well, I want to introduce a new policy, but I honestly don’t know if it will work”, to try it out on half of a group of people, and then measure their outcomes a few years later, perhaps long after the politician has moved on. This would revolutionise social policy, in far wider domains than sentencing policy, and if Cameron and Clegg were really young and visionary, they would step up to the plate. But politicians – we must share the blame – will always go further when they reason in terms of rhetoric and absolutes.


++++++++++++++++++++++++++++++++++++++++++
If you like what I do, and you want me to do more, you can: buy my books Bad Science and Bad Pharma, give them to your friends, put them on your reading list, employ me to do a talk, or tweet this article to your friends. Thanks! ++++++++++++++++++++++++++++++++++++++++++

38 Responses



  1. ellieban said,

    May 22, 2010 at 2:29 am

    Well said Ben. I noticed a large number of people complaining that the Lib Dems have previous for frequently flip-flopping policy ideas in the run up to this election and, political preferences aside, I always just wanted to shake them and yell “that’s a good thing you idiot!”.

    I mean seriously, it is time we stopped pillorying our politicians for realising a policy is not going to work and changing it and encouraged them to flip-flop when the evidence suggests they should instead. We can’t really blame them for sticking with terrible ideas when changing their minds sees us hound them out of office. Gah!

  2. Bob Kirk said,

    May 22, 2010 at 2:49 am

    This is indeed an old problem. When I was doing my MSc course back in 1966/7 we had a talk by someone from the Home Office OR department. He said “don’t quote me, but the worst thing you can do to first offenders is send them to jail. First offenders are the easiest to catch because they are new to the game. Put them in jail and they learn tricks from the old lags and learn resentment of the system because of the way they are treated. When they get out they reoffend are less likely to get caught”.

    While I agree that we all should be prepared to change our ideas when they facts show they are wrong, I mainly blame the politicians. They find it easier to play to the electorates prejudices rather than educate them. Indeed, this is hinted at by the “don’t quote me” bit above; raises the question of why does (or did) the Home Office have an OR department ?

  3. oldandrew said,

    May 22, 2010 at 5:26 am

    “A judge making a decision on a criminal’s sentence is in the exact same position as a doctor making a decision on a patient’s treatment: they are choosing an intervention for an individual in front of them, with the intention of producing a particular set of positive outcomes (reduced crime, and reduced drug use)”

    According to whom?

    A judge should be ensuring that criminals get what they deserve. No more, no less.

    This is a pretty basic philosophical point.

    Think about it.

    Would you let a murderer and rapist off of their punishment because they were unlikely to reoffend?

    Would you lock somebody up indefinitely for a petty crime because they were likely to reoffend?

    Would you pay criminals (with money and sex) to stop offending if that turned out to be an effective method of reducing crime?

    Would you sentence an innocent person to a severe sentence if you knew it would act as a deterrant to others?

    “Scientific” sentencing, which simply tries to engineer a social outcome, rather than to achieve justice, is a recipe for taking away people’s rights. Punishments have to be proportional to the crime. Being too lenient, or too harsh, would be wrong even if it were effective at reducing crime.

  4. alisonp said,

    May 22, 2010 at 6:59 am

    I do voluntary work with people with addictions, and I was out with a community drugs worker on Wednesday night. She voted Lib Dem in the election, and is quite pro. She spoke to me about the new drugs policy, which you quote above: ‘ensure that sentencing for drug use helps offenders come off drugs.’ Her view – and I do not know how informed she is, others then said the same – was that this means that they will drop the methadone maintenance programme, and require measurable ‘progress’ from those engaged with the system. That is, people will not be stabilised, but will need to move on to abstinence – as you say, a measurable result will be required.

    The view of the drugs workers was that 10-20% of addicts are able to benefit from this approach. They rapidly move from addiction to being clean – good result. However, the remainder are not ready for this, and therefore the approach which mandates progress fails these people.

    Most of the people I work with are prostitutes, because of their addictions. Giving them long term methadone, at the public expense, sometimes for years, is not a popular choice. On paper, in terms of ‘people removed from addiction’, it may seem difficult to measure progress. But methadone maintenance gets them out of prostitution (or at least reduces the financial pressure) and it reduces the damage to themselves and the community. If someone never makes progress, but is a medicalised addict until they die – but is no longer part of the violent smack culture – then something good has happened in my view. You are familiar with the phrase – ‘Sometimes the best is the enemy of the good’.

    So. I think it is misleading to laud the new approach as results-led. I think it takes a very limited view of what ‘results’ are. Heaven knows I would prefer people to cease to be addicts, but as addicts exist, sometimes damage limitation is the better option, although it doesn’t look as good on paper.

  5. fontwell said,

    May 22, 2010 at 7:34 am

    The thing is, are politicians really trying to achieve that set of outcomes? It’s possible that their main objective is to appease the staff and readers of the Daily Mail, in which case none of that tedious controlled study business is really necessary.

  6. fontwell said,

    May 22, 2010 at 7:37 am

    OK, now I feel like the guy who pointed out that Alan Partridge is a spoof.

  7. Mark Frank said,

    May 22, 2010 at 10:53 am

    I think you malign the pilots and the Criminal Policy Research Unit.

    1)Did the CPRU conduct the pilots? It seems most unlikely that an academic institution would have that responsibility. Surely they were just asked to assess the results.

    2)Pilots can have many objectives. Do you know that these pilots were intended to assess the effectiveness of DTTOs? They might for example be intended to learn out to run a DTTO service. Note that there were considerable teething problems which were gradually ironed out. This is not a trivial outcome.

    3)Even if they were intended to assess the effectiveness of DTTOs – is it realistic to allocate offenders at random to one policy or another? Imagine the judge who is instructed to sentence someone to either a prison sentence or a DTTO at the toss of a coin. Imagine the offender who finds themselves either in jail because the coin came down heads. It is a legal nightmare.

    4)There was some basis for comparison of reconviction rates. The write up includes:

    OGRS expected rate of reconviction

    Change in trend in conviction rates following DTTO

    Comparison with group “treated” with 1A(6)

    Comparison with offenders who were assessed as suitable for DTTO but not given orders for various reasons

    Comparison between those who completed their DTTO and those how had it revoked.

    5)As you (and the study) point out the conclusion was that assigning people to DTTO made little overall difference to conviction rates. This seems to be a fairly sound conclusion. The interesting result was the difference between those who completed the order and those who had it revoked – but of course that may well be because the less criminally inclined completed their orders (an RCT would have had exactly the same problem). These results seem to be well worth a second thought.

    6)It is not the fault of the research that the that the policy was implemented in any case.

    I don’t think it is as easy as you describe to apply the techniques of medical statistics to social policy. The idea that you randomly allocate criminals to one treatment or another and two years later out pops the answer seems naive. It is all much messier and its practitioners have to live with that.

  8. Nile said,

    May 22, 2010 at 12:45 pm

    OldAndrew’s been reading the Daily Mail again, and I can only hope and pray that someone does a rigorous study on the damaging effects that this can have on socialisation, sanity, and damage to the wider society.

    Magistrates *never* consider a sentence from a standpoint of considering what a criminal ‘deserves’ but, by training and by common decency, they look to achieve whatever is best for the individual and for society. Yes, they really do consider the convicted criminal as an individual, a person with real interests, and the one and only reason they apply a punishment – community service, a fine, imprisonment – is measured against the question: ‘What will put a stop to their crimes?’.

    Judges have far less leeway than magistrates: in criminal cases, they are dealing with more serious offences and the law (and Home Office sentencing guidelines) are far more prescriptive.

    But there is one area where both Magistrates and Judges are explicitly obliged by law to consife what is best for the convicted criminal: sentencing young offenders. The welfare and best interests of a child or adolescent come first, by law; obviously, the overriding concern is that the young offender stops committing crime and punishment is a valuable tool in achieving this – but it is not always essential and it is not always used; and the judiciary will always look to use anything that works to get offenders back on the ‘straight and narrow’.

    It is a source of continuing frustration to the judiciary that our politicians provide them with such poor tools – punishments that are generally ineffective and alternatives that are rarely available (and frequently withdrawn on a political whim if they prove effective) and all-too-often underfunded and haphazardly administered.

    The one clear case of evidence of effectiveness being rigorously applied in sentencing that springs to mind is the ‘ Short Sharp Shock’ for young offenders… But before OldAndrew jumps up and down with delight, I will point out that the evidence was effective in showing that the policy was not: follow-up studies and statistical analysis showed that the reoffending raye was higher under ‘Short Sharp Shock’ and the centres were closed or returned to their original purpose of remedial education, rehabilitation and training.

  9. Nile said,

    May 22, 2010 at 12:48 pm

    OldAndrew’s been reading the Daily Mail again, and I can only hope and pray that someone does a rigorous study on the damaging effects that this can have on socialisation, sanity, and damage to the wider society.

    Magistrates *never* consider a sentence from a standpoint of considering what a criminal ‘deserves’ but, by training and by common decency, they look to achieve whatever is best for the individual and for society. Yes, they really do consider the convicted criminal as an individual, a person with real interests, and the one and only reason they apply a punishment – community service, a fine, imprisonment – is measured against the question: ‘What will put a stop to their crimes?’.

    Judges have far less leeway than magistrates: in criminal cases, they are dealing with more serious offences and the law (and Home Office sentencing guidelines) are far more prescriptive.

    But there is one area where both Magistrates and Judges are explicitly obliged by law to consife what is best for the convicted criminal: sentencing young offenders. The welfare and best interests of a child or adolescent come first, by law; obviously, the overriding concern is that the young offender stops committing crime and punishment is a valuable tool in achieving this – but it is not always essential and it is not always used; and the judiciary will always look to use anything that works to get offenders back on the ‘straight and narrow’.

    It is a source of continuing frustration to the judiciary that our politicians provide them with such poor tools – punishments that are generally ineffective and alternatives that are rarely available and all-too-often underfunded and haphazardly administered; and frequently withdrawn on a political whim even (or especially!) if they prove effective.

    The one clear case of evidence of effectiveness being rigorously applied in sentencing that springs to mind is the ‘Short Sharp Shock’ for young offenders… But before OldAndrew jumps up and down with delight, I will point out that the evidence was effective in showing that the policy was not: follow-up studies and statistical analysis showed that the reoffending raye was higher under ‘Short Sharp Shock’ and the centres were closed or returned to their original purpose of remedial education, rehabilitation and training.

  10. Daibhid C said,

    May 22, 2010 at 1:39 pm

    Oldandrew: So why have judges at all? All you need is a table of how serious a given crime is, and what sentence is appropriate, right? The jury finds someone guilty, the table is consulted, and you know exactly what punishment is just. Then you lock them away for that amount of time, and throw them back on the streets afterwards. Repeat.

  11. Jon d said,

    May 22, 2010 at 2:23 pm

    Further to what alisonp said about methadone it also sounds like the end for heroin maintenance schemes which were I believe being piloted in a few areas with promising results.
    This talk of 20% success in alison’s post makes me wonder if there’s any work been done on finding out if there are any predictors of which people that sort of programme works for.. And what it would be right do with that information if you had it.

  12. James T said,

    May 22, 2010 at 4:26 pm

    Sounds a bit like a social policy fishing expedition, isn’t it? By the time you’ve subjected twenty arbitrary “interventions” to randomized trials… you know the rest… Or I’m missing something?

  13. A Pedant said,

    May 22, 2010 at 6:10 pm

    @ Jon d, # 11.

    I think it would be justified, not to mention exceptionally useful, to allow those data and relevant psychometric testing of the offender to be used by probation officers compiling pre-sentance reporting. That way judges would have an evidence based assessment of what would work not just in general but for the offender in front of them.

  14. Jon d said,

    May 22, 2010 at 6:20 pm

    Oh and yeah I do think that society wants to see people ‘getting what they deserve’ from the state.
    Seems to me that it might be quite important for society to think people are ‘getting what they deserve’, this doesn’t make me a raving daily nail enthusiast, as a card carrying Guardian reader I remember there certainly were a lot of Guardian types who were outraged that Gen. Pinochet didn’t ‘get what he deserved’ from the British system. (In the face of considerable medical evidence iirc)

  15. oldandrew said,

    May 23, 2010 at 12:32 am

    Sigh.

    Do you two want to read what I wrote again?

    I complained that basing punishments on some kind of clinical calculation rather than the ethical consideration of desert could lead to excessively harsh punishments, and I get a response accusing me of being a Daily Mail reader supporting severe punishments, and a response suggesting that I might support automatic sentencing rather than sentencing based on an ethical judgement.

    I can only assume that you have become so used to attributing these strawmen to anyone who doesn’t agree with you about criminal justice that you have missed that they are the exact opposite of what I was arguing for.

  16. Bishop Gillian Wakefield said,

    May 23, 2010 at 1:25 am

    If only Ben had some editorial control over the rest of the sciency stuff that goes into the Grauniad.
    Does anyone have any actual data to help explain why people get knocked off bicycles?
    [sigh]

  17. howfar said,

    May 23, 2010 at 3:02 am

    @oldandrew

    While I can understand how irksome it is to have your political views misrepresented, your representation of your opinion as some form of basic philosophical “fact” irritates the ethicist in me. It really is surprising how widespread the notion of retributive justice is, given how hard it is to ethically support. Can I mention that your response didn’t deal with Nile’s point that outcome based sentencing (even as currently practised) takes into account the outcome for the criminal as well as the rest of society?

    If you oppose evidence derived, outcome based sentencing on a philosophical level, there are a number of difficulties.

    Firstly, what determines the severity of a crime? Is it the revulsion it causes, or its overall effect on quality of life? Your argument suggests that we are able to intuitively know how wrong a crime is, and ascribe sentencing based upon this.

    Secondly, from where does society derive the right to retributive justice? Have we always possessed it, or did it arise somewhere in the history of the development of the criminal justice system (which did not spring, fully formed, in order to serve as an instrument of right and wrong)?

    Thirdly, as Daibhid C asks, what is the role of judges in sentencing? Does your model require them to determine merely the severity of the crime? Or are extenuating circumstances allowed? In which case, we’re caught up in an emphatically non-’basic’ philosophical debate about the nature of free-will, moral culpability, determinism and all the rest. Perhaps your concern is that evidence derived, outcome based sentencing would take power from judges. I see no reason to believe that this would be the case. Surely the principle of extenuating circumstances is that the criminal would not have committed the offence had circumstances been different? Hence we do not expect them to re-offend. This seems to fit hand in glove with outcome based sentencing.

    Fourthly, do you believe that the right to freedom from unjustified harm by the law comes before the right to freedom from unjustified harm by criminals? Do you believe it would be moral to ignore evidence which suggested that harsher sentences for burglars would lead to the disappearance of burglary? Equally, what if it were shown that more lenient sentences would have the same effect? Do you answer both questions in the same way? If not, why not?

    I think you got jumped on because one tends to associate simplistic claims about philosophy with unthinking authoritarianism. The problem here seems to be that you associate evidence based sentencing with a mechanistic slavery to a caricature of evidence. This seems like a variant on the old saw that scientific rationalism is incapable of imagination. All that is being argued here is that we should know the effects of our interventions and be honest about them, and that judges should be able and required to take this information into account when sentencing.

    As I’ve asked you so many questions, I will do you the courtesy of quickly answering yours.

    Q: “Would you let a murderer and rapist off of their punishment because they were unlikely to reoffend?”
    A: Punishment has a deterrent, as well as a retributive effect. However, parole boards, to a limited extent, do exactly what you describe.

    Q: “Would you lock somebody up indefinitely for a petty crime because they were likely to reoffend?”
    A: One would have to balance the costs and benefits to individuals and society against each other. However, there is nothing to stop you approaching evidence from the liberal assumption that you have to show compelling reason to remove someone’s liberty. The situation you describe is possible, but I think highly unlikely.

    Q: “Would you pay criminals (with money and sex) to stop offending if that turned out to be an effective method of reducing crime?”
    A: (I’m not sure where sex comes into this, except to make things more emotive) Yes, I would. But how likely is that? Remember the deterrent effect.

    Q: “Would you sentence an innocent person to a severe sentence if you knew it would act as a deterrant to others?”
    A: No. We’re discussing evidence-based sentencing, here, not the abolition of a functioning criminal justice system. If they have been convicted they are guilty in the eyes of the law. A broken justice system would not produce good outcomes for society.

    Excuse any typos, it’s been a long day. All the best.

  18. oldandrew said,

    May 23, 2010 at 2:54 pm

    “While I can understand how irksome it is to have your political views misrepresented, your representation of your opinion as some form of basic philosophical “fact” irritates the ethicist in me.”

    It’s not that it is “a fact”. It is that it is pretty hard to imagine a working system of ethics with no concept of desert in it (the best I can imagine is the ethics of some totalitarian regimes). Yet we have people not only acting as if they have such a system (but not describing or justifying it) but using strawmen and ad hominems against me for not sharing such a system.

    “It really is surprising how widespread the notion of retributive justice is, given how hard it is to ethically support.”

    It’s not hard to support at all, as long as you don’t start by assuming some kind of rationalist consequentialism as axiomatically the correct way to do ethics.

    “Can I mention that your response didn’t deal with Nile’s point that outcome based sentencing (even as currently practised) takes into account the outcome for the criminal as well as the rest of society?”

    What is to be dealt with? I am not defending our current system. I think it already leans too far in the wrong direction.

    “Firstly, what determines the severity of a crime? Is it the revulsion it causes, or its overall effect on quality of life?”

    The harm caused and the presence of definite intention to cause that harm.

    “Your argument suggests that we are able to intuitively know how wrong a crime is, and ascribe sentencing based upon this.”

    I haven’t made any claim at all as to how we ascertain the wrongness of things. I have assumed, as a given, that we can do this to some degree, on the grounds that it would be pretty pointless entering an ethical discussion like this one if one were convinced that we could not make such judgements. Do you disagree? Would it be more sensible to assume that we cannot distinguish between, say, the seriousness of littering and the seriousness of torturing a stranger to death for fun?

    “Secondly, from where does society derive the right to retributive justice?”

    Again, there is little point to entering a discussing on criminal justice if you don’t believe that society has a right to bring about justice.

    “… are extenuating circumstances allowed?”

    Yes.

    “In which case, we’re caught up in an emphatically non-’basic’ philosophical debate about the nature of free-will, moral culpability, determinism and all the rest.”

    Again, there is no point entering a discussion like this if you do not believe we have free will. This is a discussion about which type of criminal justice system should be chosen, it assumes, therefore, that we can make choices.

    “Perhaps your concern is that evidence derived, outcome based sentencing would take power from judges.”

    No. My concern is that it is not just.

    “I see no reason to believe that this would be the case. Surely the principle of extenuating circumstances is that the criminal would not have committed the offence had circumstances been different? Hence we do not expect them to re-offend.”

    Of course not. Extenuating circumstances limit responsibility, not the chance of reoffending. “I won’t do it again” is no excuse.

    “Fourthly, do you believe that the right to freedom from unjustified harm by the law comes before the right to freedom from unjustified harm by criminals?”

    What has this got to do with anything?

    “Do you believe it would be moral to ignore evidence which suggested that harsher sentences for burglars would lead to the disappearance of burglary?”

    I never suggested ignoring evidence.

    “I think you got jumped on because one tends to associate simplistic claims about philosophy with unthinking authoritarianism.”

    An association that apparently frees one from the obligation to consider other points of view.

    “All that is being argued here is that we should know the effects of our interventions and be honest about them, and that judges should be able and required to take this information into account when sentencing.”

    I’m sorry, but the original claim went beyond that. It suggest that the effects, rather than justice, were what we were concerned with.

  19. oldandrew said,

    May 23, 2010 at 3:08 pm

    By the way, you appear to have missed the point of my questions. They are thought experiments which suggest that desert-free consequentialist justice could have obviously unjust results. I want you to show whether you can give reasonable answers which fit your philosophy and are defensible, not to just declare that the situation is “unlikely” and, therefore, we can gloss over the problems. Let’s face it, even if you only once had to appease a rapist with sexual favours in order to stop them offending, it would be once too many. An ethical approach that suggests it is the right thing to do, is obviously flawed.

  20. howfar said,

    May 23, 2010 at 10:43 pm

    @oldandrew

    “there is little point to entering a discussing on criminal justice if you don’t believe that society has a right to bring about justice.”

    This where we differ. I tend toward the belief that “justice” is an act of self-protective coercion on behalf of society, and that it does derive its always dubious and debatable justification from consequentialist ethics, rather than a rights-based ethics. Hence I do not believe that a criminal justice system exists by right, but rather because it can be a way of promoting good outcomes and avoiding bad ones. You disagree with consequentialism, which is fair enough. It seems that we each understand and disagree with the other’s position.

    So yes, you’re correct, there’s very little point in having this discussion. It would inevitably lead us a thousand miles off-topic, and probably nowhere interesting into the bargain.

    (As to your second post, I envy you your moral certainty, but it seems based upon intuition, and I do not share it. Perhaps one day!)

    Thanks for your replies, best wishes.

  21. LRod said,

    May 24, 2010 at 12:55 am

    It seems when it comes to social engineering and addiction, politics aside, bad science is the norm. I read a great book on this Ben … can you say “Cherry Picking”. There seems to be much confusion about what are the symptoms and what are root causes. The only effective science appears to be that applied to creating new substances from which all involved may profit except the addicted and those they steal from etc.The call for more studies is denial of the obvious. It is not just the study data that is corrupted but whole systems and therefore any sound basis for the studies.As I said “I read a great book on this…”

  22. elvisionary said,

    May 24, 2010 at 12:15 pm

    @oldandrew,

    I think you were right to point out that some sense of justice/retribution inevitably is a major component of any approach to sentencing that is likely to be acceptable in the real world, whatever holes philosophers might be able to pick in the concept of retribution.

    Where I think you were wrong is in claiming that sentencing should be about what the perpetrator deserves, “no more, no less”. This leads you down two blind alleys. First, it rules out the ability of sentencers to take account of secondary objectives: if a judge has a choice between a range of penalties, all of which meet the test of being appropriate to the crime, then we should expect them to consider secondary benefits (e.g. likelihood of reoffending, ability to protect the community in the meantime, etc.). And second, it inevitably leads to the sterile debate you’ve just been having with howfar: if it is only about retribution, that implies that there is a clear answer to the question of what punishment retribution demands in any given situation. Unless you subscribe to brutalist old testament doctrines, there is no clear answer to that question, meaning that a range of other factors can also be taken into account.

    And if a judge doesn’t take into account of whether society needs to be protected from a particular individual when choosing a custodial sentence, I don’t think they’re doing their job.

  23. Allen Esterson said,

    May 24, 2010 at 3:56 pm

    Ben writes:
    “drug use is estimated – with the usual caveats on estimating nebulous notions – to cause 85% of shoplifting, 80% of domestic burglaries, over half of all robberies, and so on.”

    Usual caveats? You can say that again! Here is how the linked article at the Transform Drug Policy Foundation website describes it:

    “It is difficult to come to firm conclusions from any of this data. ‘Drug related crime’ is a nebulous concept at the best of times, and the data that has been collected is not methodologically strong and open to interpretation. We can really only make fairly broad statements that need to be caveated as inference/generalisations from a relatively poor research base.”

    I reckon Ben would rip into such estimates in other circumstances.

  24. paledave said,

    May 25, 2010 at 5:05 pm

    Here’s a nice little comic that does a nice simple and clear summary

    tallguywrites.livejournal.com/148012.html

  25. botogol said,

    May 27, 2010 at 1:24 pm

    you can say the same thing about big corporations: why don’t they have trials of new policies. I have never seen it.

  26. oldandrew said,

    May 27, 2010 at 8:50 pm

    “This where we differ. I tend toward the belief that “justice” is an act of self-protective coercion on behalf of society, and that it does derive its always dubious and debatable justification from consequentialist ethics, rather than a rights-based ethics.”

    The questions of “what action is the correct one for the state to take?” and “by what right does the state act?” are two completely different questions.

    Your answer to the first, no more answers the second than mine does, and this just seems to be one of many efforts on your part to change the topic to a broader philosophical question rather than defend your position on the point in question.

    “I envy you your moral certainty”

    I don’t envy yours.

  27. oldandrew said,

    May 27, 2010 at 8:54 pm

    “Where I think you were wrong is in claiming that sentencing should be about what the perpetrator deserves, “no more, no less”.”

    Hmmmm.

    This misunderstanding is probably my own fault.

    I meant that the sentence should be no more or less than the sentenced individual deserves.

    I didn’t mean that the whole criminal justice system should consider nothing more than desert.

  28. dcmtr said,

    June 1, 2010 at 1:00 am

    How does this rights versus consequentialism thing actually effect things?

    As far as I can see, consequentialism could lead to retribution (e.g. because retributive justice serves as a deterrent, and it makes people happy to see that it takes place) and deontology suggests a right to freedom (perhaps even if you are a criminal) more obviously than it suggests a need for punishment.

    Of course, these different schools of ethics could lead to the opposite conclusion, depending on how you understand them.

  29. robfuller said,

    June 2, 2010 at 1:59 am

    Calling for evidence-based policymaking isn’t completely pie -in-the-sky. The French government is working with the Poverty Action Lab from MIT on randomised controlled trials of various social policies: www.povertyactionlab.org/search/apachesolr_search/france?filters=type:evaluation

    If they can do this, then so can the British government!

  30. Susan P said,

    June 3, 2010 at 8:49 am

    I came across this interesting example of “evidence-based” education policy: www.education.gov.uk/~/media/Files/lacuna/academiesbillimpactassessment.ashx
    (It’s not difficult to find on the new Department of Education website as they seem to have consigned most of the content from the old DCSF to some sort of internet graveyard).
    Anyway, the link is to the impact assessment for the new policy on allowing outstanding schools to become academies. Impact assessments are rather technical bureaucratic documents which in theory set out the costs and benefits to various stakeholders of a proposed policy and so should summarise the evidence for the policy. This particular assessment stands out for the number of typos, the number of unfilled sections – eg about many costs and about future review of the policy – and a certain ambivalence about the name of the department which produced it. More importantly, the “evidence” seems to be based on a political commitment to allow schools to become academies by September this year, plus some dubious extrapolations from existing academies which are quite different from the proposed ones – the old academies were failing schools given lots of help whereas the new ones are already “outstanding” and will be encouraged to go it alone.
    Until last year I worked in a fairly senior post in another Whitehall department, and in the past I’ve worked as a government OR analyst. I know that evidence-based policy making is possible, though you need both sound data and analysis and the ear of the politicians – neither of which is necessarily easy (regardless of Ben’s comments above, controlled experiments aren’t always straightforward in social science and sometimes aren’t even ethical). However it is fairly clear that Michael Gove (who signed the impact assessment) was at the best selective in his use of evidence to support his new policy and at the worst had no evidence at all.
    Personally I have an open mind about academies and would genuinely like to know whether they are likely to achieve better outcomes than “normal” schools do.

  31. Tony Munton said,

    June 4, 2010 at 2:36 pm

    The Centre for British Teachers has published a new report on the paucity of evidence based policy in education. In a Guardian article (citing this column), a former director of the Institute of Education suggests education researchers should not be contrained by ‘orthodox’ views of science. Shurely shome mistake? details at www.matrix-evidence.com/blog.

  32. oldandrew said,

    June 6, 2010 at 10:25 am

    “The Centre for British Teachers has published a new report on the paucity of evidence based policy in education”

    It would have a lot more credibility if it didn’t come from an organisation that was involved in promoting the pseudo-science of Neuro-Linguistic Programming.

  33. Asian prokaryote said,

    June 8, 2010 at 9:25 pm

    If all that matters was whether the criminal reoffends, there’s a simple solution…

    Death row for any crime!

  34. Daibhid C said,

    June 12, 2010 at 11:34 pm

    oldandrew:
    “This misunderstanding is probably my own fault.
    I meant that the sentence should be no more or less than the sentenced individual deserves.
    I didn’t mean that the whole criminal justice system should consider nothing more than desert.”

    Fair enough; all I was saying was that it should.

  35. AndrewCurrall said,

    July 8, 2010 at 5:52 pm

    oldandrew:

    Justice is not, never has been, and most emphatically should not be about revenge. Revenge itself is entirely morally reprehensible and in no way justifiable: it consists of taking actions purely to cause pain and unhappiness in others. To resort to adage: two wrongs do not make a right. If the sole purpose of criminal justice were to punish the criminal, it would simply be evil (more evil than most crimes, in fact, which are generally committed to benefit the criminal, not purely to cause suffering). If nothing else, the sustained disagreement with your viewpoint here should convince you that your opinion can in no way be regarded as “fact”.

    Criminal justice has two purposes: rehabilitation of criminals, and redressing the balance of costs and benefits to ensure that crime does not usually benefit people (thus hopefully discouraging people from engaging in it). The latter purpose is easily confused with the concept of revenge, but is in fact quite different.

  36. oldandrew said,

    July 17, 2010 at 8:48 am

    AndrewCurral,

    The point is that punishment is not revenge. It is desert.

    I might add that I have never claimed that my opinions are “fact”, and I have already answered this strawman above.

    I did point out that anybody considering criminal justice without reference to desert is missing a very central (in my view the most central) philosophical point. For the reasons explained above, the purpose of criminal justice cannot simply be about rehabilitation or a cost benefit analysis. A sentence could be unduly harsh or unduly lenient even if it met those conditions perfectly.

  37. bitterkomix said,

    January 16, 2013 at 7:38 am

    Quick point: implementing RCTs will not necessarily provide you with good evidence on the impacts of prisons versus restorative justice programmes. This is because the political economy of nation-wide programmes may differ substantially from localised programmes. That is, RCTs may lack external validity in the same way that lab experiments often do. As an African, I can attest to this, since we are plagued by all manner of spurious RCTs.

    Below is a nice example of what I mean. Please read it carefully before claiming that bad evidence is better than no evidence at all.

    docs.google.com/viewer?a=v&q=cache:HSasgMxfrDsJ:www.cgdev.org/doc/kenya_rct_webdraft.pdf+&hl=en&gl=za&pid=bl&srcid=ADGEESiB0FzFHn51T69PrbIju18Cr-6DEnluGuZuodE_JXbgfqPE5U5PaAhcQVtRGmUN2uqueSYrXxqQXEvuFuRTvE3PFJy-eFMKTxHIzmqeHQFBVhCrH86HQ_rCJsaeGODgB2rnubgg&sig=AHIEtbQMRkKFncHNzM45s1Graa780hFaaw

  38. bitterkomix said,

    January 16, 2013 at 7:43 am

    That said, there is a wealth of RCTs on restorative justice programmes versus prison sentences. Here is an overview:

    www.restorativejustice.org/articlesdb/articles/7893

You must be logged in to post a comment.