Using Evaluation to Improve System Responses to the Behavioral Health Needs of Justice-Involved Youth

In 2010, the Colorado Division of Criminal Justice (DCJ) began an in-depth effort to evaluate the Statewide Juvenile Diversion Program. The Program provides funding to multiple community-based programs and District Attorneys’ Office-based programs across the state that are focused on preventing further justice system involvement for youth with first-time, low level offenses. Youth who participate in juvenile diversion receive a range of services targeted at risk and protective factors with the aim of increasing accountability and reducing likelihood of future delinquency. 

OMNI Institute was engaged to conduct the statewide evaluation, working closely with DCJ and its grantees both to understand aggregate impacts, and provide individualized support and data for each of the funded juvenile diversion programs. Over the past decade, this collaborative effort has yielded critical insights that have been applied to continually refine programming and enhance services for youth served, leading to improved outcomes over time. 

 Throughout this 10-year project, it was evident the important role data can play in positively impacting lives. By tracking the right data and gaining insights through continued evaluation and learning, programs and agencies can make evidence-based decisions with results to back up their actions.

 Data Insights: Mental Health and Substance Use in the Youth Population

In particular, early data captured through the evaluation revealed a clear need to better identify and address youths’ mental health and substance use needs in order to improve success within diversion and reduce recidivism. Indeed, the prevalence of mental health and substance use disorders among justice-involved youth has been found to be over three times higher than the general population[1].

In response to these initial findings, DCJ began working with all the funded programs to integrate universal screening for mental health and substance use into program intake processes.  In 2020, over 95% of 1065 youth were systematically screened for substance use and mental health needs."

Getting to that 95% required extensive effort on the part of DCJ and the programs funded across the state and included work to identify appropriate validated screening tools for youth, integrate the screens into intake procedures, and train staff to administer the instruments. The programs also worked to develop relationships with local treatment providers to ensure youth received recommended services, and to execute data sharing agreements that allowed them to track youth engagement.

 Ongoing Monitoring: Building an Evaluation Dashboard

To support continuous quality improvement, OMNI developed a customized evaluation dashboard that provided individualized snapshots of data for each program to monitor their efforts.  Each program had ongoing access to their own rates and results of youth screening, assessment, and referrals to treatment.  DCJ was able to use these data in the aggregate to track achievement of benchmarks, document the overall prevalence of behavioral health needs among youth served, and identify where to prioritize resources to close remaining gaps.

 A key gap identified was sufficient access to appropriate substance use treatment services for all youth.  In 2016, DCJ used the evaluation findings to demonstrate need for additional state funds made available through the Marijuana Tax Cash Fund. The awarded funds have been used since then to enhance local capacity for addressing substance use, including staff training, testing, intervention, treatment, and use of incentives. 

Strong collaboration between the funder, the evaluator, and the funded programs ensured responsiveness to the issues presented by the youth and provided opportunities to increase the juvenile diversion programs’ ability to positively impact youth while in diversion as well as have a lasting impact of keeping the youth out of the juvenile justice system.

[1] Whitney DG, Peterson MD. US National and State-Level Prevalence of Mental Health Disorders and Disparities of Mental Health Care Use in Children. JAMA Pediatr. Published online February 11, 2019173(4):389–391. doi:10.1001/jamapediatrics.2018.5399