Transforming Rehabilitation: How to measure its effectiveness

5 Sep 2014

‘What can we learn from the Peterborough and Doncaster PBR pilots?’ asks Rod Clark, PET Chief Executive. He writes:

“As recess draws to a close and the substantial changes to probation under the Government’s Transforming Rehabilitation (TR) strategy are fast approaching, now is the time to reflect on some of the big news stories affecting prisons and criminal justice over the summer.

Inspection report after inspection report has highlighted the dangerous atmosphere in prisons across the country which many have dubbed a ‘crisis’. As reported in the Guardian, the education provider A4E will stop working in London’s prisons due to challenges impacting on learners and presumably its own targets, as the company said it decided to give notice “in order to not continue to deliver the contract at a loss”. On the other hand, as much of the probation service’s remit is being replaced by new Community Rehabilitation Companies under a payments-by-results (PBR) system, the Government has drawn much encouragement from their assessments of the two TR pilots at Doncaster and Peterborough (Payment by results pilots on track for success – MoJ press release).

On 7 August the Ministry of Justice (MoJ) published its final assessment of how the first cohort of ex-prisoners had fared against the expectations for reducing re-conviction.

The statistical papers make for some dense reading but they are important, showing that excellent work by the prisons, partners and voluntary organisations including the St Giles Trust in Peterborough and Catch 22 in Doncaster has led to a reduction in reoffending.

These pilots have focused on providing intensive post-release support for people who had served short prison sentences, a group that had previously been neglected. All the organisations that have worked together to achieve this are to be congratulated on the difference they have made to the lives of these people and their communities. The TR reforms are based on the assumption that support for this group would work and it would be truly tragic if such major changes turned out not to have a basis in the evidence.

So far so good; however the models as operated in these pilots do not provide as transformational an impact as some had hoped. The Peterborough pilot did not achieve the target for cohort 1 to reduce the rate of re-convictions by the full 10% hoped; although it did exceed the 7.5% target for both cohorts taken together. Expressed in terms of proven re-offending rates, the effect was significant and valuable, but not massive: a reduction of 2.3 percentage points from 55.7% to 53.4%. Doncaster’s target was to reduce the level of re-offending by 5 percentage points and the pilot exceeded this by 0.7 percent.

There is also an issue in the way the effectiveness of the two pilots was measured. In Peterborough the results are comparisons with a matched control group. For Doncaster, the method chosen for the contract was a comparison with a past baseline of the rate of re-offending of people released in 2009. The risk of using a historic benchmark in this way is that the nature of the prisoner population in 2009 may not have been comparable with the cohort included in the pilot released in 2011-12. There is at least some evidence in the statistical report (Offender Group Reconviction Scale Scores) that the 2009 group were more likely to re-offend and therefore in a sense the 2011-12 cohort should have had a lower rate of offending whether there was a pilot or not. It would make no difference to the payments under the PBR contract; but from the point of view of assessing the policy, it would be interesting to see how the Doncaster results would look under a comparison with a matched control group using the approach adopted for Peterborough.

If August started brightly for Serco’s HMP Doncaster it hit a veritable storm with the publication in the middle of the month of a damning HMIP report. While the resettlement services and approach to working with partners received praise, the prison was severely criticised on levels of safety. It is a reminder of the perennial challenge for prison managers to provide a comprehensive service addressing all the needs of people in custody.

What conclusions should we draw from the summer of mixed news? First, good support for resettlement really can and does make a difference to people’s lives; but it is not a panacea and it will not perform miracles on its own.

Prisons must work hard to remain places of safety, and they should recognise the continuing importance of other interventions. There is abundant powerful evidence of the difference that education and learning in custody can make both to managing behaviour in the short-term and desistance from crime long-term, by giving people a positive focus that ultimately, transforms their lives.”