Patent Grant
11133021
Not Applicable
Not Applicable
The present invention generally relates to a system that reliably and autonomously classifies the sound of a gunshot and minimizes gunshot detection false positives and false negatives. In this gunshot sound context, a gunshot detection false positive is an event that was identified as being a gunshot, but was not actually a gunshot. A gunshot detection false negative is an event where a gunshot actually did occur, but the gunshot event was not detected.
“Shots Fired” reports in urban areas, schools, churches, offices, and businesses often trigger a significant Law Enforcement and First Responder response. Nearby First Responders often drop whatever they are doing to rush to the scene. Perimeter cordons are set up around schools, hospitals, offices, and residential areas. The area is usually locked down and evacuated. Normal community activities—schools, shopping malls, businesses, offices, residential areas, churches, and street traffic—stop for extended periods of time while the reported gunshot is investigated. Overall there is a significant disruption of normal community activity. A high rate of gunshot false positives can cause First Responder resources to triage or ignore some or all gunshot detection alerts unless there is a corroborating event. In the case of an actual gunshot, response delays have negative results to life and property. For these reasons, eliminating gunshot false positives and improving the timeliness of correct classification is very important to the community seeking such an alerting system.
Police Officers responding to a gunshot detection alert, especially in the heat of making split-second decisions with incomplete and imperfect information, risk mistakenly identifying an innocent bystander as a possible shooter; “friendly fire” mistakes are seemingly inevitable. Similarly, citizens in the vicinity of a possible gunshot event, particularly at night, may not be able to distinguish between First Responders and home invaders, drug dealers, gang members, or other assailants. As a result, such citizen or citizens may fire a weapon at First Responders in a good faith belief they are defending life and property or acting in self-defense. A false positive gunshot detection event could inadvertently turn into a “friendly fire” situation with tragic results for all involved.
Gunshot false positive detection reports may also cause so-called “Red Flag” events, where Police may believe that gunshots have repeatedly occurred at a location. In some states, a Red Flag alert warrants and/or authorizes seizing weapons from persons who are claimed on some basis, including unlawful weapons discharge, to be a threat to the community. Red Flag SWAT team assaults with no advance notice have been and may be launched into homes or businesses where law-abiding citizens with a legal right to possess a firearm work or reside. A citizen and/or a First Responder may inadvertently get killed or injured if First Responders do not identify themselves adequately, or if the citizen is deaf or does not speak English, or there is some other communications breakdown factor. For example, a Citizen might believe a Red Flag assault team is not the Police, but instead, a hostile or criminal invaders who is falsely claiming to be a police officer. In such a circumstance, the citizen might initiate or return fire in a good faith but mistaken case of identity. Red Flag street address mistakes have been made. Life or death decisions are made quickly and events happen with split-second timing. Risks to both First Responders and citizens are exacerbated if a high number of gunshot false positives are reported in an area.
Given the history of mass shooting events, almost any gunshot detection response is likely to increase the public's overall anxiety level. A full Police response to a gunshot false positive event will likely cause fear, uncertainty, and doubt amongst the public, including school children, teachers, parents, office workers, worshippers, shoppers, residents, visitors, et al, even if there is no actual gunshot event or shooter. Education classes and religious services are frequently interrupted; commerce is curtailed; parents pull children from school, adding to the general chaos of an event. Businesses suffer loss of revenue if people leave an area, or become nervous and reluctant to visit a store, office, bank, restaurant, gym, and other commercial location. Overall citizen confidence in Public Safety can decline if gunshot events are falsely reported.
Similarly, a high level of “false alarms” can cause a cumulative increase in public distrust and skepticism about gunshot detection technology overall. Like false positive fire and smoke alarm alerts, and/or car horn panic button alerts, a high rate of “false positives” will cause future gunshot alerts to be more likely to be ignored. One or two “Cry Wolf” experiences might cause delays in First Responders reacting to future actual gunshot incidents, and/or cause inadequate resources to initially be dispatched to actual gunshot events, while time is spent trying to determine if there really is an actual gunshot.
A high level of “false positive” gunshot alerts will likely cast doubt in the mind of citizens, police officers, and political leaders about the overall reliability of gunshot detection technology. There can be a negative perception for Police technology and training in general, and ultimately overall First Responder competence. For such public confidence and other reasons, gunshot detection technology is preferably highly accurate, with minimal gunshot false positives.
Prior art gunshot detection efforts are seen in U.S. Pat. No. 5,917,775 (Salisbury) patent and U.S. Pat. No. 10,089,845 (Skorpik). Generally speaking, these references disclose using acoustic energy as a basis for deciding if a sound event is a gunshot. The Salisbury '775 patent uses a Piezo Electric Microphone to capture sound energy level which is converted to digitized binary codes. The binary codes are compared with certain gunshot detection criteria to determine if a gunshot has been detected. The Skorpik '845 patent is focused only on gunshot recognition in confined spaces, and uses band pass filtering to isolate the sound energy level within a narrow frequency band. In these and other such instances, a gunshot determination is made solely on exceeding certain discretely measured values, a one-dimensional pre-set total energy value. In both of the Salisbury and Skorpik patents, any captured sound that exceeds a pre-defined acoustic energy value threshold is considered to be a gunshot. However, other sound generating events that are not gunshots will also generate waveforms and energy levels similar to a gunshot, particularly within the human frequency hearing range of 20 Hz to 20 kHz. Such references, even to the extent that a device according to either reference is able to detect a gunshot event, disclose a methodology that also undesirably promotes reporting gunshot false positives.
Yet other prior art gunshot detection efforts are seen U.S. Pat. No. 6,847,587 (Patterson) and U.S. Pat. No. 7,961,550 (Calhoun). Generally speaking, these references are directed to a network of audio microphones to recognize the location of acoustic events, including gunshots. They are not seeking to reliably determine a gunshot, or acknowledge, decrease, or eliminate gunshot false positives or false negatives, capabilities taught by the current invention.
The Patterson '587 reference generally discloses a “known acoustic event” that is identified by receiving acoustic waves at a sensor, and then compares those waves to a stored envelope and spectral characteristics of an acoustic event (gunshot). If there is a minimum pre-determined correlation (of sound envelope points and spectral characteristics), then the “acoustic event” location is estimated based upon triangulation between microphones. Many sounds that are not gunshots will have a high correlation using this methodology.
The Calhoun '550 reference generally describes a system and method to segregate data from different gunshot events that are in close time proximity. More particularly, the Calhoun '550 reference focuses on transforming gunshot sound data into time pulse subsets, and matching the time pulse subsets to known gunshot time pulse subsets. There is processing that purports to distinguish between multiple gunshots in close time proximity, where long distance and echoes off hard surfaces and the relatively slow speed of sound can result in the sound pulse subsets to overlap each other. For example, some portion of a sound from gunshot 1 may arrive at a distant microphone after a sound from gunshot 2 arrives at the microphone. The Calhoun '550 patent generally describes a system and method to segregate data from different gunshot events that are in close time proximity. Once again, this reference does not address the inherent limitations of such a system to overcome the gunshot false positives and false negatives that will occur given similar sounds within the time domain. In both of the Patterson '587 and Calhoun '550 patents, the primary teaching is on a triangulating methodology and related disclosures for determining the physical location of a gunshot-like sound. As before, even to the extent that a device according to either reference is able to identify an actual gunshot, another sound event that generates a sound decibel level and waveform similar to a gunshot will correlate and subsequently result in generating the location of a gunshot false positive.
While these prior art references purport to be sound-based gunshot detection systems, in truth any short-duration sound containing high-energy content that spans the frequencies from 20 Hz to 20 KHz, the human hearing range, may be misclassified as a gunshot by such prior art acoustic-based systems. And because responding to a gunshot detection alert has a very high social and monetary cost, the market is ripe for a solution that is reliable, inexpensive, and autonomous.
The challenge is and has been identifying true gunshots out of a range of events that generate similar short-duration, high-energy audio wave patterns. Currently, there are no highly reliable systems on the market that can autonomously and accurately meet this challenge by only using acoustic information. Programmatic solutions that have sought to do so are often confused by short-duration, high-energy audio wave patterns and therefore both gunshot false positives and false negatives are the result. In an effort to improve results, human analysis was introduced into an otherwise autonomous ecosystem. While this effort has been shown to improve the reliability over artificial intelligence (AI) or algometric methodologies alone, the resulting 85% reliability rate is still far from desirable. This human escalation step significantly increases the cost and necessarily adds unwanted delay to the time required for proper classification. Also, the resulting reliability becomes variable due to the reviewer's limitations—their innate hearing ability, and their experience may significantly impact whether a response is justified or not and this threatens the viability of the entire system if their assessment is wrong. It is therefore desirable to have a more reliable autonomous system that would achieve the same or better results without the reliance on the human factor given its unavoidable variability and higher fixed costs.
Systems that have had limited commercial success, like ShotSpotter™, come with a high price tag and are not autonomous as they ultimately depend upon a person's judgement as the final arbiter. The addition of the human screening was done to improve the system's overall reliability—allowing for the market to accept, purchase, deploy, and continue to use such a system. In an effort to achieve competitive results within confined spaces, other systems have sought to augment the sound-based approach with the addition of other sensors. Examples include light sensors that seek to detect the muzzle flash associated with a gunshot, and then using the confluence of sensor information to increase classification accuracy. While this is an improvement, it also compounds the requirements; the muzzle flash must be observed in order to correlate the events and obviously many factors may cause this confluence to fail.
Artificial Intelligence (AI) has also been tasked with the classification of acoustic events. The reliability of any AI-based system to properly classify acoustic samples is limited by the quantity and quality of its training set and how it was implemented. The current invention asserts that AI classification methods currently yield gunshot false positives and false negatives, largely because of poor-quality training sets derived from the flawed sampling methods used by prior art systems—garbage in garbage out (GIGO).
An over reliance upon the research of others is believed to have led to the current acceptance of the status quo. As an example, a frequency plot of a gunshot sound published in a Physics Forum article incorrectly shows a linear decay to background energy levels as the frequency plot approaches 20 kHz. The Physics Forum analysis asserts there is no gunshot sound energy within the ultrasound range, specifically above 20 kHz. This is due to the fact that they used equipment that attenuated, filtered out, and through digital conversion ultimately lost the higher frequency data content of a true gunshot.
Many whitepapers including IEEE technical articles have focused on the classification of sounds including gunshots using only acoustic information. Notwithstanding all of the effort, conclusions drawn by these efforts pertaining to gunshots are profoundly flawed. These studies disclosed using off-the-shelf sound equipment including studio acoustic microphones, frequency filters, amplifiers, and audio recorders to capture sounds. Comparing the data from the current invention to that being published, the filters used by researchers have caused signal attenuation within certain frequencies and because of the slow sampling rates, ultrasonic information is being lost from the resulting digital signal during the analog-to-digital conversion step. Unfortunately, the unique information that is so relevant and necessary for the proper classification of gunshots was obscured by the equipment and methodology of the researchers. In effect, these in-depth studies are undermined by their ill-suited methodologies. Over time the same type experiments yielded the same results, and ultimately the results and studies were published. It would be counterintuitive to assume such research was wrong, but it was fundamentally flawed. As a result, at least with respect to gunshot recognition, the purported facts being circulated have actually stymied progress and invention.
The current invention teaches and discloses that the tell-tale acoustic signature of a gunshot hides within the ultrasonic spectrum; it is characterized as a very short-duration, high-energy, wide-spectrum burst that cannot be heard by the human ear. This discovery has not been taught by any prior art invention. Accurately detecting such a burst of sound as a gunshot requires advanced analytics and equipment capable of sampling, storing, and processing sound data at sampling rates and frequencies not supported by most digital recording equipment including smartphones. The information contained within the ultrasonic burst is critical for the proper detection and classification of gunshots and the current invention and its various embodiments are the first to teach and take full advantage of a gunshot's ultrasonic idiosyncrasies and in doing so provide performance levels far superior to prior art alternatives.
The disclosed invention provides a system and method by which to detect and analyze a gunshot event in a highly reliable manner so as to reduce and/or minimize instances of gunshot false positives and gunshot false negatives.
The current invention teaches and discloses that the tell-tale acoustic signature of a true gunshot resides within its acoustic waveform, specifically within its ultrasonic components, and is characterized by a fleeting high-energy, wide-spectrum burst that cannot be heard by humans. Detecting such a burst requires advanced analytics and equipment capable of sampling, processing, and storing sound including human audible and ultrasonic frequency ranges. The required hardware and programmatic access to the resulting signals is beyond the capabilities of most digital audio recording equipment including smartphones.
The information contained within the ultrasonic burst allows for the proper detection and classification of gunshots. The current invention and its various embodiments are the first to teach and take full advantage of a gunshot's ultrasonic idiosyncrasies and in doing so provide gunshot recognition performance levels far superior to prior art alternatives.
With regard to sampling, we refer to how the sound data is collected in the first place. The microphone used must have the ability to reproduce the frequency content of the waveform. In specific application, a true gunshot produces a highly complex analog waveform having components that range from 20 hz to well above 20 kHz, reaching high into the ultrasonic spectrum. In order to not lose the high-frequency content of the analog waveform, it is preferable that the analog-to-digital conversion (ADC), according to the Nyquist Theorem, provide a sampling rate of at least twice that of the component frequency sought to be capture digitally. And ideally this sampling rate should be twice fmax, the highest frequency component measured in hertz for a given analog signal. When sampling is less than 2fmax, the highest frequency components of the gunshot are lost. A gunshot is a highly complex waveform having a short duration and to faithfully reproduce its initial ultrasonic burst requires a very high sampling rate. Given the bandwidth requirements, an ADC capable of approximately 300 kHz should be used. To put this sampling rate in proper perspective, typical sampling rates of consumer quality acoustic systems are set to 44.1 kHz, often referred to as CD quality sound, since audio compact discs use the 4.1 kHz sampling rate.
Sampling also pertains to how our library of representative data is collected for use in AI and multiple embodiments of the current invention. Classification is further improved by leveraging the fact that a gunshot's sound varies based upon angle from the shooter and other known factors. Creating a library and subsequently utilizing AI training set of sounds requires capturing thousands of discrete samples while varying the common acoustic variables associated with a gunshot's sound. For efficiency's sake, recording stations set at various angles and distances obtain samples from a plethora of ammunition and weaponry. Each sample collected has its associated metadata recoded including: distance, angle, caliber, barrel length and any other information deemed advisable for reliably determining a gunshot.
The resulting library of sounds may be further processed to obtain templates in the form of Spectrograms, where a typical representation for each combination is obtained. Spectrograms provide time, frequency, and intensity information and can be represented as images where each pixel of the resulting spectrogram image represents the time, frequency, and intensity of the signal. Spectrograms contain the data that is conducive to both correlation and AI classification methods. Regardless of the methodology used by a particular embodiment of the current invention, the present invention preferably contemplates that the aforementioned ultrasonic burst is included within the dataset for the classification step to yield an accurate result. Prior art systems do not capture this information, so they cannot leverage the information contained therein.
With regard to processing, there are actually many processing requirements and steps. In real-time a multi-level gating analysis process is continuously run against the digital sample to determine if a possible gunshot warrants advanced processing. Initially, “the net is cast widely” by performing, for example, a continuous high-level audio analysis looking for an impulse. It's not important to be discriminating at this stage. This step promotes signal processing efficiency, allowing for the reduction of unwarranted advanced and more costly processing. If the result of this first stage yields a candidate, the system applies a second stage, which includes analysis of an audio waveform Spectrogram that includes ultrasonic frequency data. The frequency information of the Spectrogram may be determined in a number of ways, including amongst others, utilizing a Fast Fourier Transformation analysis. This process essentially corresponds to computing the magnitude of the short-time Fourier transforms (STFT) of the signal. By calculating the frequency components of the signal over slices of time separate pieces are calculated and these windows may overlap in time and may be assembled or transformed.
With regard to storage, it is important to store the raw sampling of audio data for gunshot and for non-gunshot audio events during the collection phase. Those data are then compiled into a library that edge devices can quickly use to make fast and efficient gunshot/non-gunshot decisions, using the gating, correlation, and machine learning methods mentioned above. Additionally, edge devices store and forward to a central repository raw samples of potential gunshot audio events. The central repository is then used to further refine the processing library and algorithm to further enhance the overall system and methodology.
There could be several gunshot recognition algorithm embodiments depending upon the connectivity, processing power, and storage capacity available on the gunshot detection device, and whether recognition is performed by the gunshot detection device as a local edge processor, or by sending raw audio waveform data to a remote processor and storage for analysis and recognition feedback. Recognition algorithm embodiments could include simpler or more complex Spectrogram Image Signature Pattern Analysis and Correlation, Spectrogram Pixel Array Histogram Correlation, Spectrogram AI Model Edge Processing, other methods, or combinations thereof depending upon engineering tradeoffs of processing power, microphone frequency recognition and sampling capability, storage capacity, response time performance, real-time connectivity, security, device dimensions, battery life, durability, and cost. For lower cost embodiments the system and method can utilize correlation against a calculated Spectrogram and a representative sample. For the AI Model edge processing embodiment, the training process can be automated by having all of the metadata correct the classification “on the fly” in near real-time. Another embodiment would be to perform both Spectrogram representative sample correlation and AI Model edge processing, and use policy-based rules to make a gunshot recognition decision when two or more systems and methods agree or disagree.
Our methodology and circuitry does not use bandpass filters. Instead, our system and method uses ADC and math processing. Our preferred embodiment does not require the use of bandpass filters to distinguish between events (e.g., gunshot vs. not a gunshot).
An embodiment of the present invention may also be able to transmit gunshot detection events as real-time situational alerts, including over wireless communications such as 4G-LTE, BlueTooth, WiFi, 900 Mhz, and other wireless connectivity. Such transmissions could be sent to Police officers, Corrections staff, security guards, First Responders and/or associated vehicles, churches, synagogues, schools, shopping malls, restaurants, retail stores, sports stadiums, smart cities and their associated devices. Ultimately, 911 Dispatch Centers, local video integration centers, Federal, State, and Regional emergency monitoring and alert centers; fire stations; emergency medical response centers; hospitals; national and local vendor security monitoring services; cloud and local server artificial intelligence-based security monitoring and management systems; centrally-monitored industrial, commercial, and/or residential video and security monitoring centers; standalone un-monitored home security systems; and any number of other mobile security data gathering and management solutions, will have near real-time access to the resulting metadata produced by the invention.
There may also be areas where a user would not want a gunshot detection device to record or report a gunshot. One example is a Police department may not want a gunshot detector, recording or other device to report a gunshot detection event from within a police gun practice range. And similarly, an entity may only want gunshot detection to operate within a specified time period, such as a designated date, day of week, and time range or an enterprise may want users to have the option to place gunshot detection into a manually selected “Off-Duty” mode that would ignore all possible gunshot events. This could be useful for police training at gun ranges, where there is typically a high level of gunshot activity that should not generate a gunshot detection emergency alert. Thus, a preferred embodiment would accommodate such policy-based requests.
It could also be useful for possible gunshot detection events to be used to automatically activate a camera or another gunshot detection device, and broadcast an alert and/or a live audio stream to a local or remote monitoring system, or to other Bluetooth or WiFi locally connected devices. A person wearing or carrying a gunshot detection device would preferably be able to send, but nonetheless may be unable to send (or be unwilling to further excite an active shooter) a “Shots Fired” alert. A silent alert or a live audio stream could allow local other First Responders and/or Law Enforcement, Command Staff to get an early notification of a possible active shooter situation, as they could listen to a live audio stream as an additional data source to determine if there is an active shooter situation, or a gunshot detection “false positive” event.
Moreover, the identity of a person utilizing a gunshot detection device could be identified on an electronic map, such as Teacher X is assigned to Classroom 1. This information could also provide real-time situational awareness of the location of the active shooter. Similarly, a preferred gunshot detection device could be provided with GPS or other location reporting capability to provide real-time situational awareness of the location of device and possible a nearby gunshot or active shooter event. In a like manner, a preferred gunshot detection device could include an emergency alert or “Panic Button” capability. One could manually send a “weapon situation” alert before any shots were fired (or knife, ax, sword, bomb, etc. were used as the weapon). The gunshot detection device could broadcast a loud siren or emergency alert sound. A preferred gunshot detection may also be able to take and upload photographs, and/or start live audio and/or video streaming to a local and/or central monitoring system to provide a real-time situational awareness view of audio, visual, and location metadata in a location where a gunshot was identified.
A preferred gunshot detection device could further serve as an individual component or combination microphone and edge processor, and as such may be able to locally identify gunshot events, and screen out gunshot false positives. It would be advantageous for nearby gunshot detection devices to communicate with each other, and on a “Crowdsource” basis further confirm that a gunshot event has occurred. Such confirmation could thus collectively improve classification. When seconds can mean the difference between life and death in an active-shooter situation, any time delay occasioned by transmitting a possible gunshot audio sound file to a third-party location, having the sound recording being placed into a review wait queue, and/or waiting some amount of time for a next available human analyst to listen to, classify, and report a possible gunshot event, should be avoided to the maximum extent possible. Data communications network bottlenecks and failures could prevent a possible gunshot sound file from ever getting transmitted to a remote third-party location where the file would be placed into a wait queue for centralized automated or human analysis. A gunshot detection system that is wide area network communications dependent, and/or requires remote human analysis or remote artificial intelligence engine or programmatic processing, could delay detection, and further delay an appropriate response (or even fail) when it was needed most.
Accordingly, there is a need for highly accurate gunshot detection that significantly improves upon the high rate of gunshot false positives and gunshot false negatives. The current invention in its various embodiments provide for such detection, enable better evidence gathering and, through its various communications and response capabilities, allows for meaningful response such that Police and other Emergency Services providers are able to respond more appropriately to the threat and with better situational awareness.
In one of its embodiments, inexpensive purpose-built acoustic hardware may be paired with devices that have certain computational capabilities, such as smartphones. The eventual incorporation of the current invention's circuitry and analytic methodology within current off-the-shelf devices is contemplated and expected by this filing. Thus, it is to be understood that the present invention may be used with or incorporated within mobile video and audio recording devices such as personal cameras, smartphones, broadcast media mobile news video cameras and audio recording devices, consumer-grade still and video cameras, audio recorders, and any other electronic mobile devices where an acoustic but proximity constrained gunshot detection alert capability might be desired.
Prior art attempts have been made to use smartphones to detect gunshots. However, in order for any platform to utilize or otherwise practice the disclosed invention, support for sound sampling rates sufficient to obtain the required requisite data are preferably implemented and exposed by the platform. In the alternative, an unmodified smartphone or similar computing platform may overcome innate limitations by having a secondary device paired with or directly connected to the platform that incorporates the invention's teaching.
FBI statistics between 1988 and 2003 show that 93% of the time, the distance between a shooter and a police officer killed by a gunshot occurred at distance of 50 feet or less. NYPD data from 1854 to 1979 shows that 90% of officers were killed within 15 feet from the shooter. For gunshot events between 1970 and 1979 where NYPD officers survived, the shooting distance in 75% of cases was less than 20 feet. Anecdotal reports from several recent school, church, and synagogue multiple gunshot events indicate they generally occurred after a gunman entered into a classroom, sanctuary, or hallway of relatively small dimensions. Long distance gunshot events are rare. It would thus be preferable that gunshot detection would be reliably determined within the historical distance range of many of gunshot events, which is 50 feet or less.
The present invention, in a preferred embodiment, is able to detect a gunshot, preferably at distances of as much as 200 meters. While the present invention has application at distances greater than 200 meters, it is most efficacious at distances of 200 meters or less due to the fact that ultrasonic energy decays rapidly and is largely undetectable for such purposes at distances over 200 meters from the source. Even so, as mentioned above, FBI and NYPD statistics show that many gunshot events occur at a distance of 50 feet or less. Gunshot detection utilizing data from at least a portion of the ultrasonic frequency range will be highly reliable within the historical distance range of the vast majority of gunshot events regardless of whether the gunshot event occurred in open space or within an enclosed space.
Devices constructed and methods practiced in accordance with the present invention could also be implemented as a standalone dedicated fixed location gunshot detection sensors, with or without wireless or wired connectivity, in all the locations and types of entities already identified. The present invention may further be applied in a wide variety of existing types of fixed location sensor and “internet of things” (IoT) technology devices such as wired or wireless security cameras, security systems, perimeter security light and motion sensors, doorbells, thermostats, aircraft and train controllers and sensors, fire, smoke, and carbon monoxide alarms, kitchen appliances, industrial machinery controllers, electric and gas meters, electric distribution and substation transformers, high voltage transmission line sensors, pipeline pumping station controllers, traffic lights, street lights, toll booths, other smart cities devices, gasoline pumps, retail point of sale systems, and any number of other mobile and fixed location devices where having a gunshot detection capability might be desired.
Devices and methods according to the present invention can also provide a highly reliable “crowdsourced” network ability to quickly identify and more precisely report the location of a gunshot event, without the delay of relying upon an imperfect human audio review, or rough triangulation between distant microphones.
A fixed or known device location of a device in accordance with the invention may be used to provide real-time situational awareness. For example, location information from device including an internal GPS sensor, or location information such as a known or assigned location such as Teacher A is assigned to Classroom 1, may be utilized to provide real-time situational awareness of approximately where in a school, office, or other facility one or more gunshots have occurred. The present invention further provides that gunshot detection sound waveform analysis can provide the approximate distance of a gunshot from a detection device. So, by reference to a fixed or known location, the approximate real-time location of an active shooter can be estimated with significant reliability in accordance with the invention.
Some personal cameras and other potential gunshot detection devices may be constructed in accordance with the present invention so as to have local communications capabilities. Examples of such capabilities include BlueTooth and WiFi real-time wireless communications. As a result, such devices could communicate in real-time. A gunshot false positive could be further identified (including confirmed or rejected as such) in accordance with the invention by real-time correlation and polling of other nearby detection devices. If multiple independent devices in local proximity also detect a gunshot, then the “crowdsourcing” hive of multiple positive gunshot event reports will increase the overall collective confidence level that a gunshot actually has been detected. Conversely, if one gunshot detection device according to the invention concludes that a gunshot was detected, but a plurality of other nearby gunshot detection devices have not detected a gunshot, then the likelihood of a gunshot false positive increases. The gunshot false positive business rules and processing logic can identify when to ignore a reported gunshot detection event from other detection devices, even within close proximity to each other. Precedence logic could be implemented to give a higher value to a gunshot event reported in the same room as a gunshot detection device according to the invention.
If a plurality of gunshot detection devices in accordance with the invention are within meaningful proximity report a gunshot event, metadata about decibel level and audio waveform patterns from each device may be used to identify which devices are closer to or more distant from the location of the gun barrel. As previously mentioned, it may be possible to determine if a gunshot detection device is located inside or outside of the same enclosed space as the gun barrel. Audio sound wave patterns are different if the gunshot occurred on the other side of a wall around an enclosed room. Identifying that a preferred gunshot detection device is in the same classroom, hallway, room, or other enclosed space as the gun barrel is of high value to Police and other First Responders who are attempting to determine the location of the active shooter. Seconds can mean the difference between life or death in active shooter situations, so real-time situational awareness has very high value for ending active shooter situations as quickly as possible.
More advanced devices in accordance with the present invention have policy-based processing logic that can automatically start video recording based upon combinations of events. Gunshot detection can be one of these policy-based video recording start events. In many cases a video recording start from any combination of policy-based events causes pre-event video to be pre-pended to the camera's video segment. In the case of a gunshot event, the policy-based recording engine determines whether to pre-pend both video and audio to the camera video being stored and/or transmitted. In addition, a personal camera in accordance with the present invention or data recording device or apps captures GPS, accelerometer, gyroscope, and other metadata that may be embedded in the video file and/or stored once a gunshot has been detected. A gunshot detection device in accordance with the present invention may also generate and transmit gunshot detection sound wave files, metadata, and alerts to remote administrators, local and remote security systems, Incident Command Centers, Police Central Dispatch units, Video Integration Centers, 911 Centers, and other remote systems that could be alerted in the event a gunshot is detected.
In accordance with the invention, gunshot detection metadata and alerts from one or more preferred gunshot detection devices can then be transmitted to real-time situational awareness systems (such as the commercial product known as AVaiLWeb™). The present invention could then make gunshot detection metadata available to First Responder and Resource Officer Dispatch Centers, School Administration workstations, Video Integration Centers, or used in association with web browser map-based views of a facility or area (e.g., a campus or business). Such real-time situational awareness views and alerts may be provided to smartphones, tablets, laptops, computer monitors, Police Computer Aided Dispatch and Video Integration Center monitors, and other web-browser capable display systems.
Further, the present invention includes that gunshot detection metadata and alerts may be transmitted to other gunshot detection devices in local proximity, or all devices within a designated GeoFence boundary. All preferred gunshot detection devices could receive “Be On the Lookout” emergency alert messages with audio alerts, text messages, active shooter photographs, and classroom or office lockdown instructions. A gunshot detection device in accordance with the present invention may provide on-going alerts, status messages, and all-clear messages to teachers, supervisors, or other personnel who have a gunshot detection device.
Similarly, a gunshot detection device in accordance with the present invention may have a messaging capability for device owners to send text messages, photographs, and video clips to central administration, Police Dispatch Centers, Video Integration Centers, and other Incident Command Staff. A preferred gunshot detection device would be able to facilitate real-time situational awareness communications from persons such as teachers, supervisors, and other management personnel that may be somehow involved or affected by an active shooter.
Referring to
It is to be further understood that the apparatus 1 may be placed in various locations, either fixed or mobile. For example, a smartphone may be provided with a microphone that is able to capture gunshot sounds. A smartphone (or other device) may also be provided with an accelerometer that monitors motion and/or orientation of the device. Thus, for example, the accelerometer can monitor the smartphone for motion spikes in any axis (X, Y or Z) that could indicate that the device has hit a hard surface so that the sound associated therewith is not determined to be a gunshot. A preferred embodiment further includes the application of motion filtering rules 4 to identify device falls and hits on hard surfaces that might generate a gunshot-like sound with a high energy level at the microphone. In such a case where the device is mobile or otherwise provided with an accelerometer, both factors are applied 5 to determine if a Spectrogram waveform analysis should be performed (i.e., stage 2).
A second stage of the present invention expressly includes a consideration of ultrasonic sound waves. More particularly, a Spectrogram analysis 6 of the information captured by monitoring the audio stream, is performed. The resulting pattern is evaluated at a high level 7 to determine if a gunshot has been detected. More particularly, the present invention contemplates that a spectrogram of the potential gunshot sound is prepared such as that shown, for example, in
The present invention also utilizes the fact that a gunshot exhibits a high energy response across an entire frequency band, albeit for a brief time, and utilizes spectrograms to look at more than signal intensity and frequency response. Where prior art devices have considered frequency and intensity for a discrete time period (see, for example, the '845 patent), the present invention considers sonic and ultrasonic frequency response over a greater period of time without averaging across the entire period of time. The spectrogram analysis of the present invention differs from signal energy/intensity analysis of the prior art and thus allows for detection of a gunshot in both open and continued environments. More particularly, the preferred spectrogram of the present invention provides information regarding time (x-axis), frequency (y-axis) and intensity (e.g., high energy; z-axis). As shown in the “Frequency Analysis” image shown in
The present invention further includes looking at a larger time period than that known to exist in the prior art. Thus, by virtue of the spectrogram, the present invention considers high energy (i.e., loud) sounds of more intensity than the prior art, and further considers such factors for a greater length of time and in smaller increments than that previously accomplished in the prior art, at both sonic and ultrasonic levels. Thus, for example, where a prior art detection method may have considered results over 250 or 500 milliseconds, the present invention addresses time increments in as little as 100 milliseconds (0.1 sec) and considers the spectrogram information (time, frequency, intensity) for substantially the entire length of the captured gunshot sound—as much as a full second or more. Thus, under the present invention, key distinguishing energy readings are found in the first 100 milliseconds, but not averaged into a larger time period (such as 200 milliseconds) where the results would be averaged out and potentially missed.
If the above-described spectrogram comparison of the captured gunshot to a known gunshot spectrogram reveals an actual gunshot, that information my be relayed to third parties such as law enforcement, first responders, etc. Further, that information is preferably captured, and the spectrogram and sound profile is transferred to an AI engine 9, where the profile is added to the stored profiles, thus allowing for further comparison and further reduction or minimization of “false positives” and “false negatives.”
To the extent possible, further metadata such as distance from the microphone and gun type and caliber 10 are determined by Spectrogram Signature Pattern Analysis and Correlation, Pixel Array Histogram Correlation, AI Model edge processing, or other means. Once a gunshot event 10 is confirmed, gunshot metadata is published 11 to a metadata repository 12 for audit trail and chain of custody reporting. In the case of a BodyWorn camera, Video Recording 13 is started. Gunshot detection event notifications 14 are sent to Central Dispatch and any other predetermined authorized metadata recipients. To the extent possible, video, audio, and metadata is lived streamed to authorized recipients.
The Gunshot Detection closed loop process continues while the Gunshot Detection device is operational.
The present invention will increase active-shooter real-time situational awareness both outside and inside buildings, while minimizing the negative effects of Gunshot Detection False Positives and False Negatives. Real-time edge processing and correlation of metadata from multiple sensors can correctly identify gunshot events that audio only gunshot energy or triangulation analysis would miss.
Information rich Spectrogram result signature patterns can uniquely identify metadata beyond whether a gunshot occurred. The signature pattern can further identify gun type, caliber, and distance and angle from the gun barrel.
The present invention provides highly accurate real-time situational awareness reporting from inside schools, offices, shopping malls, and other inside locations where legacy outside sound triangulation systems can't. At the same time, the Spectrogram analysis provides a high degree of gunshot recognition capability outside in clear air. In both cases First Responders are able to respond faster and more effectively to an active shooter event.
“Crowdsourcing” gunshot sensor data from multiple devices located in a school, church, synagogue, campus and other locations further increases the range and accuracy of real-time situational awareness reporting. The risk of “Friendly Fire” tragedies can be reduced, and situations can conclude sooner.
Having thus described exemplary embodiments of the present invention, those skilled in the art will appreciate that the within disclosures are exemplary only and that various other alternatives, adaptations, and modifications may be made within the scope of the present invention. Accordingly, the present invention is not limited to the specific embodiments as illustrated herein, but is only limited by the following claims.
This application claims priority to U.S. Provisional Application No. 62/853,437, filed May 28, 2019, the contents of which are expressly incorporated herein by reference in its entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 2009013 | Karplus | Jul 1935 | A |
| 5504717 | Sharkey | Apr 1996 | A |
| 5917775 | Salisbury | Jun 1999 | A |
| 5973998 | Showen | Oct 1999 | A |
| 6167192 | Heo | Dec 2000 | A |
| 6356872 | Leung | Mar 2002 | B1 |
| 6847587 | Patterson et al. | Jan 2005 | B2 |
| 7003358 | Eastty | Feb 2006 | B2 |
| 7577259 | Iwata | Aug 2009 | B2 |
| 7961550 | Calhoun | Jun 2011 | B2 |
| 8199923 | Christoph | Jun 2012 | B2 |
| 8730040 | Donovan | May 2014 | B2 |
| 9111047 | Shridhar | Aug 2015 | B1 |
| 9830932 | Gunderson | Nov 2017 | B1 |
| 10089845 | Skorpik | Oct 2018 | B2 |
| 10115410 | Craven | Oct 2018 | B2 |
| 10627292 | Fowler | Apr 2020 | B1 |
| 10763828 | Craven | Sep 2020 | B2 |
| 10810854 | Correnti | Oct 2020 | B1 |
| 20080168895 | Duong | Jul 2008 | A1 |
| 20110218952 | Mitchell | Sep 2011 | A1 |
| 20120300587 | Azimi-Sadjadi | Nov 2012 | A1 |
| 20140269199 | Weldon | Sep 2014 | A1 |
| 20140327543 | Showen | Nov 2014 | A1 |
| 20140361886 | Cowdry | Dec 2014 | A1 |
| 20150081579 | Brown | Mar 2015 | A1 |
| 20150194045 | Edwards | Jul 2015 | A1 |
| 20160042767 | Araya | Feb 2016 | A1 |
| 20160157032 | Kane | Jun 2016 | A1 |
| 20160191163 | Preston | Jun 2016 | A1 |
| 20160232774 | Noland | Aug 2016 | A1 |
| 20160260307 | Skorpik | Sep 2016 | A1 |
| 20170110141 | Craven | Apr 2017 | A1 |
| 20170169686 | Skorpik | Jun 2017 | A1 |
| 20180053394 | Gersten | Feb 2018 | A1 |
| 20180159473 | Nestler | Jun 2018 | A1 |
| 20190080577 | Raz | Mar 2019 | A1 |
| 20190180606 | Pirkle | Jun 2019 | A1 |
| 20190186875 | Pirkle | Jun 2019 | A1 |
| 20190259378 | Khadloya | Aug 2019 | A1 |
| 20190311604 | Morehouse | Oct 2019 | A1 |
| 20190371160 | Pace | Dec 2019 | A1 |
| 20200051189 | Williams | Feb 2020 | A1 |
| 20200088832 | Jarrett | Mar 2020 | A1 |
| 20200211361 | McSchooler | Jul 2020 | A1 |
| 20200234554 | Rehfeld | Jul 2020 | A1 |
| 20200278239 | Jax | Sep 2020 | A1 |
| 20200381006 | Davis | Dec 2020 | A1 |
| 20200402378 | Connell, II | Dec 2020 | A1 |
| Number | Date | Country |
|---|---|---|
| 107862279 | Mar 2018 | CN |
| 3006908 | Apr 2016 | EP |
| Entry |
|---|
| First Action Interview issued for U.S. Appl. No. 17/198,568, dated May 26, 2021. |
| Non-Final Office Action issued for U.S. Appl. No. 17/198,568, dated Jun. 8, 2021. |
| Prince, et al., Deploying Acoustic Detection Algorithms on Low-Cost, Open-Source Acoustic Sensors for Environmental Monitoring (Year: 2019). |
| International Search Report and Written Opinion issued for Application No. PCT/US2020/035011, dated Nov. 13, 2020. |
| Number | Date | Country | |
|---|---|---|---|
| 20200381006 A1 | Dec 2020 | US |
| Number | Date | Country | |
|---|---|---|---|
| 62853437 | May 2019 | US |
