The City of Chicago Office of Inspector General (OIG) has completed a second audit of the Chicago Fire Department’s (CFD) fire and emergency medical service (EMS) response times. The first audit was published in 2013. Similar to the 2013 audit’s objectives, OIG wanted to determine if CFD has goals and performance measures for fire and EMS responses, and if the Department’s response times meet state and national standards such as the National Fire Protection Association (NFPA) Standard 1710 and the Commission on Fire Accreditation International (CFAI). In its 2021 audit, OIG concluded that CFD has not implemented performance management strategies that would allow it to evaluate fire and EMS response times in alignment with best practices, nor has CFD corrected data issues identified eight years ago.
In 2013, OIG found that CFD did not meet the NFPA standard and that CFD’s internal reports lacked the elements necessary to accurately assess whether the Department was, in fact, meeting or exceeding the national standards as it claimed publicly. OIG provided various recommendations that CFD should: formally document its response time goals; update its policies and procedures; monitor the number of blank and inaccurate time fields in CFD’s dispatch system; conduct a data-based analysis to identify causes that are preventing CFD from meeting NFPA response time goals; and create and implement an action plan that will ensure verifiable compliance with national standards. In our 2015 follow-up report, OIG concluded that CFD did not intend to implement any corrective actions related to the original findings, which were based on NFPA and CFAI standards.
With our second audit, OIG determined that CFD still needs to correct issues with goals, performance measurements, and operations concerning response times. Specifically, OIG found that CFD:
- has not implemented performance management practices that would allow the Department to evaluate its fire and EMS response times;
- has not documented response time performance goals outside of its state-required EMS plan; and
- does not have adequate data to reliably measure key components of response time.
Similar to our previous suggestions, OIG recommended that CFD management acknowledge the importance of department-wide quantitative performance measures through its operations. Specifically, CFD should:
- begin public annual reporting on response time performance including geographic analyses that identify areas of the city where CFD is not meeting its response time goals;
- establish and document department-wide turnout, travel, and total response time goals for both fire and EMS at the 90th percentile; and, if CFD management believe NFPA recommended turnout and travel times are unachievable in Chicago, conduct a systematic evaluation of factors affecting response times and set reasonable goals;
- work with the Office of Emergency Management and Communication (OEMC) to assess the root causes of data gaps and address these issues moving forward; and
- ensure that its data analysis partners conduct a full assessment of its data completeness and reliability, then use the results of this assessment to address any operational errors that led to missing data.
In response, CFD agreed with our findings and “acknowledges the importance of department-wide quantitative performance measures.” Specifically, CFD stated that it will better analyze its data to identify “causative factors and or trends” and “perform a complete and reliable measure of response time by each component piece and in total, reported as a percentile measure.” CFD stated it has engaged with Urban Labs at the University of Chicago to aid the department in analyzing its response time performance. Additionally, CFD stated that it would determine reasonable percentile goals, and has agreed to work with OEMC to improve and monitor data in the existing and new computer-aided dispatch systems.
“It is unfortunate that the issues OIG identified in 2013 and 2015––CFD’s lack of performance management practices to evaluate fire and EMS response times––still pose the very same concerns in 2021. Had OIG’s previous recommendations been considered and the necessary operational changes put into place years ago, CFD and the City would be in a better place today, meeting state and national standards and following best practices,” said Inspector General Joe Ferguson. “Without performance management strategies in place, CFD has no way to determine if there is an increase in productivity or a decrease in errors. The Department is unable to determine what’s being done well or what needs to be improved. Performance and operational evaluations are important in any City agency, but most especially one tasked with responding to those in distress. However, with a new Commissioner and new administrators at the helm, OIG is encouraged that CFD will take the issues laid out in our second audit more seriously, and implement corrective actions to evaluate response times, evaluate factors that affect these times, and determine ways to measure goals to meet specific response times. We anticipate CFD applying these crucial recommendations with more urgency this time around.”
The full report can be found on OIG’s website.
Alongside this audit, OIG’s Center for Technology and Analytics has also released a set of dashboards displaying OEMC events for CFD response and an interactive map that shows where events occurred by location of service. The interactive dashboards display emergency service events from January 1, 2018 through present day, with the ability to filter by emergency event types and ward/zip code/community area. Some significant data findings include:
- Over 80% of requests for CFD response were EMS related, while the remainder of requests were for Fire Suppression and Rescue.
- The most common types of events––accounting for 23% of all events––were “person down from an unknown cause” and “difficulty breathing.”
- The greatest number of events recorded and the highest rate of events per 100,000 people by ward is in the 28th ward and by zip code is 60602, 60603, 60604 (all located downtown); the highest rates of overdose and gunshot related events by community area are found on the West Side, with West and South Side community areas showing the highest rates of gunshot related events per 100,000 people.
See the dashboards and analyze the data on OIG’s Information Portal: bit.ly/CFDEMSData
The mission of the independent and non-partisan City of Chicago Office of Inspector General (OIG) is to promote economy, effectiveness, efficiency, and integrity by identifying corruption, waste, and mismanagement in City government. OIG is a watchdog for the taxpayers of the City and has jurisdiction to conduct investigations and audits into most aspects of City government. If you see corruption, fraud, or waste of any kind, we need to hear from you. For more information, visit our website at: www.igchicago.org.