Independent Analysis of the MacArthur Justice Center Study on ShotSpotter in Chicago

Executive Summary

ShotSpotter commissioned Edgeworth Analytics (“Edgeworth”) to review a study by the MacArthur Justice Center (“MJC”) published May 2021 and provide an independent evaluation of the claims contained in it. Based on our analysis, Edgeworth concludes that the MJC study fails to provide a rigorous, balanced, and objective assessment of the use of ShotSpotter in Chicago and is, at best, misleading because of the inappropriate data source used for the study, the selective choice of data and a fundamental lack of understanding as to where ShotSpotter was deployed relative to the highest homicide rate areas of Chicago.

Specifically, we conclude the following:

  1. The OEMC data that was the primary data source used to support the MJC study’s conclusions regarding “unfounded” CPD deployments is an inappropriate source on its own to determine the ultimate outcome of an individual incident and, therefore, is not a reliable measure of ShotSpotter’s efficacy. The MJC study’s interpretation is misleading because the data obtained from the OEMC is not designed to capture and account for any subsequent police action resulting from an initial ShotSpotter alert. The conclusion that the lack of a police report is a measure of ShotSpotter’s accuracy is baseless and misleading.

  2. The MJC study mischaracterizes the placement of ShotSpotter technology as unduly burdening Black and Latinx communities. Specifically, it omits important context – that the placement is based upon areas of need across Chicago as measured by incidents of homicide and gun crime.

In addition to this analysis, Edgeworth has conducted an independent review of ShotSpotter’s claims regarding accuracy in gunshot reporting and false positives—sending an alert of gunfire when none occurred. Specifically, Edgeworth examined ShotSpotter’s representation that its system has an aggregated 97 percent accuracy rate that includes a 0.5 percent false positive rate across all customers over the last two years. Our review confirmed that (1) these claims are valid and based on actual customer feedback from a broad range of ShotSpotter customers and (2) despite substantial variation in the intensity of reporting potential errors across clients, ShotSpotter’s accuracy rate does not appear to be sensitive to differences in clients’ propensity to report potential errors. The details of this analysis are provided in a separate report.

MacArthur Justice Center Report

The MacArthur Justice Center (“MJC”) obtained Office of Emergency Management and Communications (“OEMC”) data on Chicago Police Department (“CPD”) deployments between July 1, 2019 and April 14, 2021 and prepared a study of calls for service (“CFS”) initiated by ShotSpotter alerts and 9-1-1 calls based on these data.[1] The study’s findings were posted on an MJC-created website and included in an amicus brief filed on May 3, 2021 in Cook County Circuit Court (the “Amicus Brief”). The study’s primary conclusions were that: (1) ShotSpotter-initiated alerts resulted in CPD finding no evidence of a gun-related crime or any crime the majority of the time during the period of study; (2) there were more than 40,000 “unfounded” deployments of CPD; and (3) these “unfounded” deployments were disproportionately in Black and Latinx neighborhoods where ShotSpotter is deployed.

Edgeworth Analytics Review

ShotSpotter commissioned Edgeworth Analytics to review the MJC study and provide an independent evaluation of the analysis contained in it.[2] For our analysis, we reviewed: (1) the MJC study and an Amicus Brief that describes it in detail; (2) the same publicly-available OEMC data MJC used to draw its conclusions, which was provided to ShotSpotter by the CPD, (3) the academic literature; (4) publicly available CPD data; and (5) analyses conducted by ShotSpotter.


What is ShotSpotter?

According to a report from the Brookings Institution, 88 percent of gunshot incidents go unreported to police.[3] ShotSpotter intends to help solve that issue. According to ShotSpotter, the company offers law enforcement agencies an acoustic gunshot detection service that detects, locates, and alerts police to gunfire enabling a precise and rapid response to incidents that likely would have gone unreported to police. The system uses wireless sensors throughout a coverage area to capture loud, impulsive sounds that may be gunfire. The data are transmitted to a central cloud service that classifies each incident with a gunfire probability percentage along with a location determined by triangulation enabled by multiple sensors. Then, specially-trained ShotSpotter employees called “reviewers” located across two ShotSpotter Incident Review Centers listen to the recorded pulses from the sensors that detected the incident audio with playback tools, visually analyze the audio waveforms to see if they match the typical pattern of gunfire, assess the grouping of sensors that participated, and either publish the incident as gunfire or dismiss it as non-gunfire. ShotSpotter said the entire process typically occurs in less than 60 seconds from the time of the gunfire to the time law enforcement is alerted to allow for a timely law enforcement response. The gunfire alerts that are sent to ShotSpotter customers, including the CPD, have three recorded audio snippets that patrol officers can listen to before they arrive on the scene.

Below are examples of gunshot and non-gunshot audio provided by ShotSpotter that were captured by ShotSpotter sensors from various locations nationwide. Each example of gunshots includes the date of the event, the rounds fired, the audio that was shared with the local police department, and a redacted Investigative Lead Summary (ILS) report for the event. For non-gunshot events, each example includes the date of the event, the type of event, and the audio that was shared with the local police department (ILS reports are not generated for non-gunshot events).

Example Audio of Gunshots Captured by ShotSpotter Sensors

Date: July 13, 2021

Rounds fired: 13

Investigative Lead Summary

Date: July 20, 2021

Rounds fired: 15

Investigative Lead Summary


Date: July 14, 2021

Rounds fired: 10

Investigative Lead Summary



Example Audio of Non-Gunshots Captured by ShotSpotter Sensors

Date: July 18, 2020

Type of Incident: Fireworks

Date: April 30, 2020

Type of Incident: Helicopter


Date: July 21, 2020

Type of Incident: Car Backfiring



Experts

Practice Areas

Jump to Page

This website uses cookies to improve functionality and performance. By continuing to use this website, you agree to the use of cookies in accordance with our Privacy Policy.  If you are a California resident, read our California Information Practices.