The Office of Inspector General (OIG) conducted a second audit of the Chicago Fire Department’s (CFD) fire and emergency medical response times. The first audit was published in 2013.
The objectives of the audit were to determine if,
- CFD has goals for fire and emergency medical services (EMS) response times consistent with state and national standards; and
- CFD response times meet state and national standards.
OIG concluded that CFD has not implemented performance management strategies that would allow it to evaluate fire and EMS response times in alignment with best practices. Nor has the Department remedied data issues identified in 2013.
OIG found that CFD has not implemented best practices for measuring response times. CFD does not produce annual department-wide reports that would allow it to evaluate response times, and it does not measure turnout and travel as separate components of response time or use industry-standard percentile measures. Notably, we recommended in 2013 that CFD correct these issues.
OIG further found that CFD has not documented response time performance goals outside of its state-required EMS plan. CFD documented its overall EMS response time goal as required by state law. However, the Department has not documented fire response time goals. Contrary to best practices, CFD has not set goals for turnout or travel time at the industry-standard 90th percentile. Here, again, we recommended in 2013 that CFD set and document such goals.
OIG also found that CFD’s data is not adequate to allow reliable measurement of response time. We analyzed records for emergency events from January 1, 2018 to November 30, 2020, finding that only 705,061 of 937,446 (75.2%) included data for all categories necessary to calculate turnout and travel times for the first arriving unit. CFD acknowledges that it has been aware of data reliability issues since at least 2013, but has not remedied them.
OIG recommends that CFD management acknowledge the importance of department-wide quantitative performance measures and begin public annual reporting on its response time performance. CFD management should establish and document department-wide turnout, travel, and total response time goals at the 90th percentile for both fire and EMS. If CFD management believe NFPA recommended turnout and travel times are unachievable in Chicago, they should conduct a systematic evaluation of local factors affecting response times and set reasonable goals for turnout, travel, and total response times accordingly. CFD should also identify, monitor, and remedy the cause of gaps in its data, and should consider hiring an internal data specialist to improve data quality. Finally, CFD should ensure that any external partners it engages to analyze departmental data conduct a full assessment of that data’s completeness and reliability.
In response to our audit findings and recommendations, CFD stated that it “acknowledges the importance of department-wide quantitative performance measures” and will implement OIG’s recommendations. CFD stated that it has “engaged Urban Labs at the University of Chicago, in part, to aid the department in analyzing its response time performance” and would work with OBM and DHR to hire additional data analytics staff. CFD stated that it would analyze its data to identify “causative factors and or trends” and “perform a complete and reliable measure of response time by each component piece and in total, reported as a percentile measure.” CFD stated it would determine a reasonable percentile goal “as the completeness of data elements improves.” Finally, CFD agreed to work with OEMC to improve and monitor data in the existing and new CAD systems.