The Office of Inspector General’s (OIG) Public Safety Section has issued an advisory on the Chicago Police Department’s (CPD or the “Department”) risk models known as the Strategic Subject List (SSL) and Crime and Victimization Risk Model (CVRM). CPD received $3.8 million in federal grants and spent ten years developing these sequential models for predicting the likelihood that an individual would become a “party to violence” (PTV), i.e. the victim or offender in a shooting. The general areas of concern in the PTV risk model program include: the unreliability of risk scores and tiers; improperly trained sworn personnel; a lack of controls for internal and external access; interventions influenced by PTV risk models which may have attached negative consequences to arrests that did not result in convictions; and a lack of a long-term plan to sustain the PTV models. In August 2019, CPD informed OIG that it intended to decommission its PTV risk model program since grant funding was due to run out by September 30, 2019. Therefore, OIG provided this advisory to ensure proper decommissioning and inform any future implementation of risk models.
Every individual arrested by CPD at least once between August 2012 and June 2018—regardless of whether they had a history of violence or whether they were ultimately convicted—received a risk score or risk tier. As of July 2018, 399,412 individuals had an SSL risk score and as of March 2019, 313,513 individuals had a CVRM risk tier. Individuals who were victims of crimes, but never arrested, did not receive a risk score or tier. Over the course of the program, CPD utilized its PTV risk models in several ways: to identify at-risk individuals and connect them to social support services (Custom Notification Program); to identify repeat offenders with a high propensity toward violent, gang-related crime and enhance prosecution to detain, convict, and incarcerate those offenders (Targeted Repeat-Offender Apprehension and Prosecution [T.R.A.P]); and to target gang members and their associates through information gathering, analysis, and social network mapping (Gang Violence Reduction Strategy).
OIG issued a number of recommendations to ensure that the PTV risk model program was properly decommissioned and any references were removed or no longer accessible. The Department decommissioned the program on November 1, 2019, and agreed with our recommendations, but has yet to formally update all directives mentioning SSL or CVRM, and is still working on removing risk scores and tiers from all documentation (due to discovery laws and preservation orders, some records containing this data may not be altered).
OIG also issued best practice recommendations, should the Department develop programs in the future which use data to help predict PTV individuals:
- Clean and vet data and develop protocols to regularly update PTV-related information
- Conduct training in a timely manner with up-to-date material
- Create a policy dictating the intended purpose and allowed uses of PTV information and monitor the use of such information
- Consider all relevant data to predict and intervene those at risk of becoming PTV
- Continuously evaluate the accuracy and efficacy of predictive policing programs
- Develop plans and secure resources to ensure sustainability of such programs
CPD concurred with our recommendation for future risk models to execute policy with a clear purpose, but the Department did not speak to other programmatic concerns including data quality, training, external access, ongoing evaluation, and sustainability. “While these particular risk models have now been decommissioned, there are critically important and widely applicable lessons to learn here about the importance of careful data handling and thoughtful, purpose-driven policies and training, in ongoing and future efforts to harness data that predicts and ultimately prevents violent crime,” said Inspector General Joe Ferguson. “Therefore, we reiterate our hope that the Department will apply these lessons going forward so that it may effectively and responsibly marshal data to protect and support Chicago’s vulnerable citizens.”
The full report can be found online: bit.ly/CPDRiskModels.
Follow @ChicagoOIG on Twitter for the latest information on how OIG continues to fight waste, fraud, abuse, and inefficiency in Chicago government.
The mission of the independent and non-partisan City of Chicago Office of Inspector General (OIG) is to promote economy, effectiveness, efficiency, and integrity by identifying corruption, waste, and mismanagement in City government. OIG is a watchdog for the taxpayers of the City and has jurisdiction to conduct investigations and audits into most aspects of City government. If you see corruption, fraud, or waste of any kind, we need to hear from you. For more information, visit our website at: www.igchicago.org.