Ben Goldacre, The Guardian, Saturday 7 August 2010
According to the Home Office this week, Sarah’s law – where any parent can find out if any adult in contact with their child has a record of violent or sexual crimes – has “already protected more than 60 children from abuse during its pilot“. This fact was widely reported and was the headline finding. As the Sun said: “More than 60 sickening offences were halted by Sarah’s Law during its trial”.
The number of sickening offences prevented by an intervention is a difficult thing to calculate: nobody explained where the number came from, so for my own interest I called the Home Office.
“It’s not that difficult to work out is it?” This is the Home Office telling me I’m stupid (I’m okay with that). “It’s the number of disclosures issued, how many were of sex offenders, and how many children would those offenders have had contact with.” Fair enough. This means telling a parent that someone in contact with their child had a history of abuse equated to preventing an act of abuse? Yes, they said: “Protecting that child means ensuring that offender did not have a way of having contact with that child. Therefore that child is being protected.”
Fair enough. This assumes that any such contact is itself abusive, or would definitely result in abuse. That might be correct, I don’t know.
Then I asked where the number 60 came from. I was sent to an excellent report assessing the programme written by a team of academics. Neither the number 60 nor the word sixty appear in that document.
So I contacted the lead author, Prof Kemshall, who said: “You are correct that reference to sixty children is not made in the report. As I understand it the Home Office have drawn on police data sources to quote this figure and therefore I cannot assist you further. As you will see from the report we were careful to state the limits of the methodology.”
I contacted them Home Office again. “The figure is over 60 and it comes from the number of disclosures made where there was a conviction of a sexual offence with a minor or violence against a minor. In total 21 disclosures were made specifically about registered sex offenders (RSO), a further 11 disclosures were made, for example relating to convictions for violent offending. These people had access to over 60 children.”
I’m not sure this is self-evident. The academics who wrote the report couldn’t work out where the number 60 came from, and at least two pieces have appeared trying to unpick it, both coming up with different answers to me and the Home Office. The excellently-named Conrad Quilty-Harper in the Telegraph and a promising new website called FullFact both – very reasonably – tried adding various categories of numbers from the academic report, including a figure on social worker activity which seemed to make up the numbers.
I’m not saying the figure 60 is wrong. While what it represents was probably overstated – by the Home Office and the press reports – the number itself isn’t absurd. But it does seem odd that just finding out where it came from involved so much mucking about, and it seems even odder to ignore the robust figures in a long academic report that you’ve commissioned (the scheme wasn’t cheap compared to, say, social worker salaries), and instead build your press activity around one opaque figure constructed ever so slightly on the back of an envelope.
Update 7/7/10 1pm:
I’ve just had an email from Christopher Cook, FT Education Correspondent.
He was given another completely different story by the Home Office about how they got to the number 60: “60 comes from the number of disclosures made during the pilots (21) and the number of safeguarding actions that were taken in relation to a risk of abuse (43) – this can be found on the first page under the key findings heading rds.homeoffice.gov.uk/rds/pdfs10/horr32c.pdf“.
I know it’s not the most important thing in the world but this really is a dismal game of silly buggers from the Home Office on a headline figure.
CC also pulled out this interesting paper on recidivism: “(old, relies on police records, but a starting point) called “Sexual offenders – measuring reconviction, reoffending and recidivism“. That suggests recidivism rates at about 16pc. So the ‘we sent a letter which stopped child abuse’ claim seems rather weak.”
Kimpatsu said,
August 7, 2010 at 4:35 am
It’s not just the dodgy statistics that bother me; it’s the civil liberties issues of havign parents commission investigations of me without my knowing. So, when they come up empty on the background checks, can I then sue them for breaching my privacy?
Daniel Rutter said,
August 7, 2010 at 5:59 am
The assumption appears to be that these sex offenders and child-beaters are such horrible people that they’re certain to molest or beat up any child with whom they “have contact” – yet, apparently, there are no grounds for just keeping these unredeemable monsters in jail.
It’s like the No-Fly List, that catalogues all the people the US government think are so likely to be terrorists that they shouldn’t be allowed on planes… but against whom the government have insufficient evidence to even make an arrest.
ellieban said,
August 7, 2010 at 6:29 am
Just because a law is named for some poor dead child tragically murdered, doesn’t mean it’s right.
I campaign for prison reform, restorative justice and better rehabilitation for all criminals, including sex offenders. I don’t do this from some huggy hippy sentimentalist view that prisoners deserve more love – I do it because it has been proven to be the best way to reduce the numbers of future victims of these horrible crimes.
Thank you for posting the full report, I confess I haven’t read every line, but I cannot find anywhere in it and analysis of whether this law actually does anything to protect children at all. It isn’t just that the figure of “60” isn’t there, answering the child protection question the question isn’t even one of the aims of the report.
One wonders how much value a report on a pilot scheme like this (however well written) is if it doesn’t even consider the issue the scheme was originally set up to tackle.
There is a very effective way of reducing re-offending amongst sex offenders, it is called a “circle of support and accountability” and works by making the offender the centre of a dedicated group of local people to whom they can go to at any time and tell them anything. What that means is that they confess their urges and can be supported through resisting them rather than suppressing them until they burst out uncontrolled. I cannot express enough the respect I have for the people who serve on circles.
This law does the opposite, it drives the offender further underground and increases their fear and alienation from their community. These are all things that are increase the likelihood of them committing future offences.
I am a scientist, if this report had provided me with solid data that showed the above analysis was mistaken and that this law really did prevent harm being caused to children, I would embrace it. But it doesn’t, that question isn’t even in its remit.
The report does show some benefit to the community from increased information sharing and hand-holding of worried parents, and that is worth bearing in mind. But it far from clear that the benefit was worth the financial cost of the scheme and I think there ought to be a way of providing reassurance to a community that doesn’t invade the privacy of innocent people or risk driving offenders further underground.
It may be distasteful to have to treat sex offenders with compassion and kindness, but it reduces their likelihood of re-offending and so it is better for society in the long run if we put our need for punishment and revenge to one side.
JohnMoore said,
August 7, 2010 at 7:29 am
This raises many issues.
Firstly it presumes risk is only generated by individuals with convictions – so under this scheme information about potential abusers who do not have convictions is misleading and may in fact therefore increase risk.
Secondly it presumes those with convictions all represent risks.
Thirdly it continues the myth that protecting children is an adult task. In fact the most powerful protection comes from empowering children; particularly in ways that enable them to resist intra-family abuse.
Yogzotot said,
August 7, 2010 at 9:22 am
Nice investigation, Ben, the headline angried me as well. It was clear that they equate potential contacts with actual abuse, which is sensational in itself, but that they even got their numbers… weird… is priceless.
I completely agree with JohnMoore: The conjecture is stupid, and the whole thing creates a false sense of security (again – airport frisking, anyone?).
Firstly, most abuse incidents are committed by close family members or friends, not the “weird neighbour”.
Secondly, it looks like sex offenders on average have potentially a lower propensity to re-commit than the rate is for “normal” offences.
Filias Cupio said,
August 7, 2010 at 9:30 am
There is a sensible sanity check.
How many children were ‘beneficiaries’ of the enquiries? (I.e. had contact with an adult who was enquired about, or were prevented from having contact with an adult as a result of enquiry.)
What were the exposure times of these children? (I.e. from earliest time the child did or would have had contact with an enquired-about adult until the end of the trial.)
Add up all the exposure times for a total of child-years. Multiply by the average rate at which child abuse occurs (from the number of adult-child pairings.) (There will be a fair bit of uncertainty in this number.)
If the result is much less than 60, then the result is clearly wrong – they’re claiming to have prevent much more abuse than could have occured even if they were 100% effective at protecting those children. If the result is on the order of 60 or greater, they may be right.
I suspect that the calculation assumes that everyone on the register will abuse every single child they come in contact with. If so, the calculation will be too high by a very large factor.
Kapitano said,
August 7, 2010 at 10:29 am
The figure of “60 children saved” is meaningless – it’s just provided to give a spurious justification to the law, for those who think a precise number coupled with a vague term constitutes proof of…something or other.
But that does suggest the question: If the highest figure the home office could come up with is 60, just how effective is this new law at achieving anything at all?
If a widely praised national inoculation program gave vaccinations to 60 children “at risk” from disease X, you might ask why only 60. If a traffic safety campaign proudly claimed to have saved 60 children from car accidents – based on “number of children crossing the road * traffic density” – you might ask “Why such a small program?”
ffutures said,
August 7, 2010 at 10:54 am
Re [1], I’ve worked in education for many years, and the checks have been a fact of life for many years – it’s something you implicitly agree to when you choose to work in education. There’s a small loss of privacy, but it’s presumably a lot less invasive than if e.g. you apply for a job where you need a security clearance. And the checks DO occasionally reveal problems – I suspect that the frequency is over-hyped, but it does happen.
Michael Gray said,
August 7, 2010 at 12:27 pm
Goaling ¼ of a Catholic priest would reliably account for 60 less molestations of infants.
Jon d said,
August 7, 2010 at 2:02 pm
It was a pilot in a limited area so it’d would be wrong to compare it to a national road safety scheme. 60 sounds like a good number for a small pilot scheme… If it’s a number representing something that’s actually happened.
Wrt the update ben shouldn’t you consider the recidivism rates for sex assaults on children alone. The proponents of sarah’s law iirc claim the patterns of reoffending are different between general sex offenders and sex offenders who offend against children.
ayupmeduck said,
August 7, 2010 at 3:06 pm
Slightly OT perhaps…
That 16% number for peodophile recidivists was a surprise, it’s by far the lowest I’ve ever seen, and reading the document I have big doubts about it, the whole thing seems to be based on too many assumptions and in any case it seems the 16% refers to all types of sexual offence.
I had to work on such data some years ago and found that the data gave rates were much higher than this – Prentky, Lee, Knight, and Cerce (1997) for example found that rates were above 50% and peodophiles had pretty much the worst recidivist rates of any crime.
I hope I’m wrong, it would great to find that treatment had come on in leaps and bounds in the last few years and rates have really dropped because 16% means that you can have a very different policy on this offence than with 50%. At over 50% you’ve almost gotta agree that the Daily-Mail-type idea that it makes sense to “lock ’em up and let ’em rot”
ellieban said,
August 7, 2010 at 3:11 pm
Jon d,
The proponents of Sarah’s law “claim” a lot of things, not least that this trial kept 60 children from harm. If they play as fast and loose with the recidivism figures as they have with these numbers you’ll forgive me if I don’t set much store by them.
Out of interest, why do you 60 sounds a good number?
cybergibbons said,
August 7, 2010 at 5:20 pm
Won’t somebody please think of the children?
Sorry.
reprehensible said,
August 7, 2010 at 9:20 pm
continuing on that line, though that number does seem like nonce sense, perhaps they stopped a paedophile dressed as a school
www.youtube.com/watch?v=IlEcGHxfltE
(Again sorry, couldn’t resist – brass eye was ace! Obviously the other question is was he teaching phonics).
Jon d said,
August 7, 2010 at 10:07 pm
ellieban,
Well it’s 60 children over the 12 month pilot over 4 police forces.
15 ‘children saved’ per year per police force – if it represented something in the real world it seems that it would qualify as ‘good’ and worth making a bit of a song and dance over in my subjective estimation.
gonzo99 said,
August 7, 2010 at 11:04 pm
Sadly ‘protecting’ or ‘safeguarding’ children is the favourite new statistic of many such organisations. CEOP love to trot out how many children they have ‘safeguarded’ which in this sense means exactly the same – protected from some possible future abuse that may or may not have happened had they not spent lots of money investigating.
Its the figure that keeps on giving. The Government can pretend they are doing something, the tabloids can froth in disgust at the offences they think might have been committed, and the sheeple can get a warm feeling thinking that all those nasty people are being hounded.
The very fact that they never give actual conviction rates alongside these figures should give rise to great suspicion. You know, convictions where they have to actually prove something at those pesky places called courts.
Either it means that none of the people were actually committing abuse at the time of the ‘intervention’ (in which case the assumption that future abuse is certain is very dodgy) or it means they intervened in EVERY case before anything happened, which suggests a level of competence and excellence that I simply don’t believe any Government department could achieve!
Meanwhile the vast majority of abuse will continue, within families by parents/guardians with no previous convictions and no sign of horns or dirty macs.
pv said,
August 8, 2010 at 12:55 am
Jon d, it hasn’t saved anyone, anywhere from anything.
And since most child abuse seems to happen within the family or close acquaintances, the only reason to “make a song and dance” is to give the impression that “something” is being done.
Marcus Hill said,
August 8, 2010 at 9:58 am
A better estimate for the number of cases of abuse prevented would surely be to look at all the child abuse cases in a similar population and timeframe and see how many were committed by people with previous convictions, then multiply by a factor based on the reporting rates. I’d be willing to bet such an analysis would fall significantly short of 60.
There is a serious danger in this scheme of giving parents a false sense of security, since it does nothing to prevent the first time offender.
I also have concerns about the privacy implications and the potential for lynch mobs treating someone who has a conviction for a (non-sexual) assault on children for, say, attempting to restrain a bunch of teenagers who have been damaging his property in the same way as they would someone who has groomed and sexually assaulted a child. Now, I support neither violence against unruly teenagers nor the lynching of pederasts, but I suspect even the Daily Mail would be against petrol bombing the guy who was overzealous in protecting his property.
Jon d said,
August 8, 2010 at 1:03 pm
PV I was replying to the person who said 60 in a small trial over 4 police forces should be compared with a national road safety figure and elleban who asked why I thought 60 was a good number.
I’m not making a personal endorsement of the method by which the HO arrived at the number 60 which clearly looks dodgy but if some tens of children had been saved in a small pilot that would be IMO a good thing.
See the word if in there? see it in my earlier post too.
gonzo: if for instance more plane crashes were caused by pilot error than the wings dropping off it’d still be worth spending some money at the plane factory making sure wing attachments were up to the job rather than shrugging and saying most crashes are caused by pilot error. doesn’t mean you stop looking at reducing pilot error.
just cos most child abuse happens within families by parents/guardians with no prior convictions doesn’t automatically make it a bad idea to do something about the exceptions as well.
wrt reconviction rates of general sex offenders vs sex offenders offending against children www.publications.parliament.uk/pa/cm200607/cmhansrd/cm070522/text/70522w0017.htm
——
The re-offending rates for men who sexually abuse children are not known, but we do have information on re-conviction rates for this group of offenders.
The most recent figures we have are for 2004.
Out of the adult offenders who were convicted of a sexual offence against a child, and were released from custody or who started a community penalty in the first quarter of 2004, 12.3 per cent. were reconvicted of a new offence of any kind.
These rates of re-conviction were over a period of two years.
For all adult male offenders convicted of a sexual offence (that is including those who offended against a child and those who offended against adults) and who were released from custody in 2001, 3 per cent. were reconvicted for a further sexual offence within two years.
——
HungryHobo said,
August 9, 2010 at 1:49 pm
the figure is just so absurd (number of children in contact with those with convictions) and seem to assume that every convicted offender will abuse every child they come in contact with.
Lets say joe blogs , a convicted burgler lives in a city.
Lets say the recidivism rates for convicted burglers is 20%.
There’s 1000 houses within walking distance of where this guy lives.
So the police send letters to those 1000 houses telling them that joe blogs is living nearby and is a convicted burgler.
How many burgleries have they just stopped?
By the logic of the Home Office 1000.
By the logic of multiplying by the recidivism rate 200.
Reality? Possibly 1 or 2, likely none.
People are far more likely to take no effective measures.
He might even just walk further down the road.
He might not have robbed anyone at all anyway.
You might cause a muder as some local resident decides to clean up the neighbourhood.
“just cos most child abuse happens within families by parents/guardians with no prior convictions doesn’t automatically make it a bad idea to do something about the exceptions as well.”
It does if you spend vastly more money on a small risk of wings falling off vs a large risk of pilot error when the money would save more lives spent more wisely.
Strangers are unlikely to rape your child, uncle bob is far more likely to.
If you spend all your time worrying about the wierd guy down the road and leave your child with uncle bob while you go lynch the wierdo then you’re very likely increasing the danger to your child.
but that isn’t how people think.
the outsider is the danger, the stranger, the odd person.
Not good old uncle bob.
irishaxeman said,
August 9, 2010 at 10:46 pm
I worked as a social worker in the late 70s and 80s mostly in first line investigation (then called non-accidental injuries).
None of this stuff is worth a mouse-fart. There are people still in local government offices responsible for far worse simply by imposing their stereotypes on investigations, and telling coal-face investigators who they should implicate. All men are rapists/paedophiles etc so you don’t investigate the women. Surprising then that women turn out to have deviant sexual appetites and violent tendencies far more often than comfortable.
This ’60 offences’ is another of these ‘beliefoids’, where no-one can actually say where the original evidence came from, along with Satanic abuse stats and so on.
And yes, it will be nice Uncle Bob or trusted Aunty Lesley (or both) who are kiddy-fiddling, not some weird stranger.
CoralBloom said,
August 9, 2010 at 11:25 pm
Lots of common sense here.
Parents need to be realistic, not terrified.
A few years back a neighbour of mine came across a two-year-old who’d managed to escape home! And she’d somehow got herself across a road, at a bend where fast traffic often appears ‘out of the blue’. He was terrified she’d go back across the road, and terrified of what he’d be accused of if he went anywhere near her to get her away from the road. He just stood there, frozen to the spot, praying an old lady would arrive before he had to run and grab the baby.
As a woman, I’m not sure how safe I’d feel if I was there and picked that baby up to get her away from the road. I’m not so these days. Safer than him, but still…
PS.
Old lady arrived and the neighbour left the scene, happy to be on his way, well away from it.
iamian said,
August 10, 2010 at 1:46 am
I found this paper from the Center for Sex Offender Management very interesting on recidivism:
www.csom.org/pubs/recidsexof.html
A good review of the factors involved.
Xobbo said,
August 10, 2010 at 5:14 am
Surely even if someone with a record was checked once they would just try again until they found someone who didn’t check them?
So in fact, even if 60 acts of abuse were averted, they would simply occur elsewhere, with no net reduction.
UNLESS of course, those with records were simply killed by vigilantes as soon as their checks came up positive…
cobas said,
August 10, 2010 at 3:56 pm
Aren’t children who are abused more likely to be abusers as adults? They’ve already ‘prevented’ countless future abuses stretching into the future…
Jon d said,
August 11, 2010 at 7:23 am
I found this paper from the Center for Sex Offender Management very interesting on recidivism
Yeah thanks… Padophillia associated with greater recidivism by several sources. I wish Ben had stopped at debunking the bogus number 60 instead of rolling onto to do a half cocked drive-by at the principle of disclosure like he has done here without researching it properly. Anyone else notice Increasingly he’s been doing for policy journalism what he used to take the piss out of others for doing when they were writing about medicine?
Jus sayin.
SimonW said,
August 13, 2010 at 7:20 am
Reminds me of report into investigations in to Child Abuse imagery, and one into “human trafficking”. Both press releases were upbeat, when you looked at the glossy published reports they were very upbeat, the politicians forwards were all about how successful it was, when you checked the details closely you discovered they were hugely expensive exercises and in terms of protecting the vulnerable the money would have been many orders of magnitude better spent on recruitment of social workers.
Does the Home Office has some sort of textbook on how to beef up poorly performing studies?
The depressing bit is that the top levels behave as if the glossy press releases are true, rather than acting on the actual numbers.
pv said,
August 13, 2010 at 11:47 am
No!
_anthony_ said,
August 14, 2010 at 10:39 am
If we stick to Ben’s excellent concept of identifying misleading claims:
1) “Protected”. This is a case of the ridiculous mis-use and mis-reporting of risk. Does my fire alarm “protect” me? Of course it does. Has it prevented a fire, or prevented me being burnt alive in my bed? No, because we have never had a fire. But it has protected me, because it has reduced by a small amount – at least I think it has! – the risk of me being burnt alive. So the question is: does the law reduce the risk by any material amount of a child becoming the victim of abuse? Well, it probably does, but that is not really the policy problem. The problem is, by what amount and at what costs? Maybe the Outed offenders will get a grudge and go off and do something nasty as a result of the new law? Who knows?
2) More than 60? Well, quite honestly, if you buy into the stupid concept of Protecting, then who cares? When will the BBC stop telling us that millions of people are “at risk from this, that or the other”?
What is troubling is that civil servants are actually paid to mislead us with a claim like this.
BobHarvey said,
August 15, 2010 at 7:55 pm
I’m a bit concerned by the various announcements last year that we need all the anti-terror legislation because it allowed the authorities to stop “more than 35 major plots”. Only one of which appears to have ended up at the old Bailey.
This use by Government of random numbers is just pedalling fear. We used to be better than this.
I recall the pride with which my Grandfather told me he had employed a couple of old lags to work in his timber yard. “See, lad, in Britain once you leave prison we treat you the same as everyone else. It’s a sort of contract: server yer time and the slate is wiped clean”. That noble principle has been thoroughly routed in the case of ‘sex offenders’ and there was talk some years ago about creating a ‘violent offenders register’ too. If people aren’t safe to live among us, they should not be living among us, surely? If they’ve served the time, then they are back in the state of normal grace, surely?
Snag is, we are no longer in the realm of medicine nor quackery, but something altogether muddier. It’s politics, of a sort. I’d pass this one off onto Tim Harford, he’s good at this sort of thing.
ajh1980 said,
August 16, 2010 at 12:01 pm
Interesting article from Switzerland regarding re-offending amongst Internet only offenders:
www.biomedcentral.com/content/pdf/1471-244X-9-43.pdf
Using the broadest definition of recidivism (including convictions, charges and ongoing investigations) only 6.0% of internet offenders recidivated with any offence over a 6-year follow up period and NONE were convicted of a ‘hands-on’ sexual offence during this period.
Bearing in mind a large number of registered sexual offenders which the ’60 children saved from abuse’ story refers to will be Internet only offenders, what have these children actually been saved from?
Jered said,
August 17, 2010 at 6:19 am
From the perspective of the Home Office, it’s easy to see why they used the number “60” and presented it the way they did. They obviously have a vested interest in selling the American public that the program is working… even if that means stretching some facts to sensationalize the story.
Bonsai said,
September 4, 2010 at 12:35 pm
Patrick Casement (a retired social worker, analyst and author of: Learning from Life: Becoming a Pyschoanalyst) talked at length about his relationship to a small girl aged 6 with whom he conducted psycho-analytic ‘therapy’. He called her ‘Joy’ because he enjoyed working with her so much. He used to hold her down in the sessions when she was angry or agressive:
‘I used to pin her wrists down and prevent her from moving, waiting for her to stop struggling, when she did I would let her go’ at a public address: 2009 (M, c70, white, Cambridge Graduate).
I was impressed by the lack of protection this little girl encountered in relation to this older very tall person convinced of his own ‘good-ness’. She, like he did, originated in the context of a very difficult family dynamic. I wondered at the way he spoke about this with pride. It made me cry. What to do? (Anecdotal?)