By Megan Golden and Joe Waters
Last week, the Vera Institute for Justice released a fact sheet on its evaluation of the first Pay for Success project in the United States. The project, which sought to reduce recidivism among teenagers incarcerated at New York City’s Rikers Island, did not meet its reduction targets and will be discontinued in August. Because it did not meet its targets, the City of New York does not need to repay investors. So was this project a success?
It is important to understand the goals and the process of the project, as profiled in the ICS fact sheet on U.S. Pay for Success projects. In response to the fact that 50 percent of youth return to jail within one year in New York City, the project implemented ABLE (Adolescent Behavioral Learning Experience), which uses Moral Reconation Therapy (a type of cognitive behavioral therapy). ABLE, delivered by Osborne Association and Friends of Island Academy, cost about $800 per year per participant. The target population for the intervention was all 16-18 year-olds entering the NYC jail with a length of stay of more than 4 days (later increased to 7 or more days). The jail is on an island called Rikers Island and thus is often referred to as “Rikers.” The program, which started in February 2013 after a pilot perio, sought to reduce recidivism by 10 percent. The outcome metric that determined whether payment would be made was “recidivism bed-days”: the number of days adolescents spend in jail in the 12 months after they are initially released. A quasi-experimental study compared the recidivism bed-days for a cohort of 16-18-year-olds who entered custody while the ABLE program was operating with a matched cohort that entered before the implementation of the ABLE program.
The short version of what happened is that the PFS financing mechanism worked, but the intervention didn’t. Why might that have happened?
While Moral Reconation Therapy has a strong track record, it faced several challenges in the Pay for Success deal. The environment for adolescents on Rikers Island was particularly toxic during the time the program was in effect; a federal investigation found a “culture of violence” against teenagers there. It is hard to imagine any therapeutic program working in that context.
In addition, the vast majority of teens at Rikers are awaiting trial so their length of stay is uncertain and often ends up being very short. The service provider did an exceptionally good job of reaching the adolescents who entered the jail, reaching 87% of those who stayed for 7 days or more. But most of those teens did not stay long enough to complete even one of the program’s milestones, let alone the entire program. And, because payments were based on projected savings in the jail budget, there were not enough resources to ensure that those who did not complete the treatment in jail would continue and complete it in the community.
And yet, this first U.S. Pay for Success deal should not be considered a failure. At its most basic level, this deal helped embed the “outcome-based” culture necessary to PFS: it tracked and analyzed the impact on recidivism, and made program decisions based on the findings, a feat rarely accomplished in criminal justice or in other fields for that matter. First Deputy Mayor Tony Shorris underscored that the project “…allowed the city to test a notion that did not prove successful within the climate we inherited on Rikers. We will continue to use innovative tools on Rikers and elsewhere,” reinforcing the current administration’s interest in providing supportive services at the jail. Beyond this, as laid out in the piece in Huffington Post by the investors, Goldman Sachs and Bloomberg Philanthropies, the SIB worked as planned, and showed that it could be done. The fact that the City did not have to pay for the program may even bring comfort to some governments that are concerned that they will have to pay anyway. The big question is whether investors will be scared off since they were left bearing the full cost. Goldman Sachs seems not to be deterred; we hope others also show this fortitude. As noted in the HuffPo piece, this first PFS project provided an important learning experience for the future of the field:
Government got to try a promising approach to a thorny problem. Taxpayers spent no money and avoided paying for a program that fell short of its intended goals. We learned more about the challenges of implementing evidence-based approaches. And we can now say that this unique financing model made it all happen — and stands ready to help public sector leaders finance more efforts to better serve people who need support and better results.
This is not bad news for the PFS field. Jurisdictions continue to move forward to explore innovative means to fund overlooked programs, new parties take interest every day, and ICS continues its work to help jurisdictions gauge the feasibility of PFS designs for early childhood interventions. Overall, we remain cautiously optimistic about Pay for Success financing’s prospects. And we remain convinced that, regardless of what the future holds, bringing new attention to and accountability for outcomes will benefit children, families, government, and communities.