Patent Application
20240221487
The present disclosure relates to spectrogram-based alarm signal detection.
Various premises security devices are equipped to detect audible audio alarm signals that are emitted, for example, by fire alarms, smoke alarms, and carbon monoxide alarms. This feature enables designated parties who may be off-premises to be notified of the detected event.
Some alarm signals consist of a tone, such as an audible square wave, that has a fixed fundamental frequency and is pulsed on and off according to a specific temporal pattern that corresponds to the type of alarm. The exact frequency of the tone may be unknown to a premises security device, though the frequency value may be specified to fall within one or more ranges. Temporal pattern specifications may allow for a tolerance of, for example, +/−10%, in the duration of each pulse-on and pulse-off phase.
A more complete understanding of the present disclosure, and the attendant advantages and features thereof, may be more readily understood by reference to the following detailed description when considered in conjunction with the accompanying drawings wherein:
The present disclosure relates to providing spectrogram-based feature extraction for pattern detection decisions, as described herein.
Before describing in detail exemplary embodiments, it is noted that the embodiments may reside in combinations of apparatus components and processing steps related to spectrogram-based alarm signal detection. Components may be represented by conventional symbols in the drawings, focusing only those specific details that facility understanding the embodiments so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.
As used herein, relational terms, such as “first” and “second,” “top” and “bottom,” and the like, may be used solely to distinguish one entity or element from another entity or element without necessarily requiring or implying any physical or logical relationship or order between such entities or elements. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the concepts described herein. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises,” “comprising,” “includes,” “including,” “has,” and “having,” when used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
In embodiments described herein, the joining term, “in communication with” and the like, may be used to indicate electrical or data communication, which may be accomplished by physical contact, induction, electromagnetic radiation, radio signaling, infrared signaling or optical signaling, for example. One having ordinary skill in the art will appreciate that multiple components may interoperate and modifications and variations are possible of achieving the electrical and data communication.
In some embodiments described herein, the term “coupled,” “connected,” and the like, may be used herein to indicate a connection, although not necessarily directly, and may include wired and/or wireless connections.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the concepts described herein.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It will be further understood that terms used herein should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The apparatuses, methods, devices, and systems described herein perform to spectrogram-based alarm signal detection.
Referring now to the drawing figures in which like reference designators refer to like elements there is shown in
System 10 may include detection device 26 which is in communication with one or more of network 22, remote monitoring center 24, security control device 20, among other entities in system 10. Detection device 26 includes detection unit 28 which is configured to perform one or more detection device 26 functions described herein such as with respect to spectrogram-based alarm signal detection as described herein. In one or more embodiments, detection device 26 is part of premises security system 12 or separate from the premises security system 12 where detection device 26 may be configured to detect audible alarm signals from one or more of premises devices 18 (e.g., premises device 18a that is able to emit an audible signal) and 3rd party devices (e.g., 3rd party life safety devices such as alarm sensor 27) that may not be in communication with security control device 20.
User interface device 16 may be a wireless device that allows a user to communicate with security control device 20. User interface device 16 may be a portable control keypad/interface 12a, computer, mobile phone and tablet, among other devices that allow a user to interface with security control device 20 and/or one or more premises devices 18. User interface device 16 may communicate at least with security control device 20 using one or more wired and/or wireless communication protocols well known to those of ordinary skill in the art. For example, portable control keypad 16a may communicate with security control device 20 via a ZigBee based communication link, e.g., network based on Institute of Electrical and Electronics Engineers (IEEE) 802.15.4 protocols, and/or Z-wave based communication link, or over the premises' local area network, e.g., network-based on IEEE 802.11 protocols.
Premises devices 18 may include one or more types of life safety related sensors, control and/or image capture devices. For example, the types of sensors may include various safety related sensors such as motion sensors, infrared sensors, fire sensors, heat sensors, carbon monoxide sensors, flooding sensors and contact sensors, among other sensor types that are known in the art. That is, life safety features may correspond to security system functions and settings associated with premises conditions that may result in life threatening harm to a person such as carbon monoxide detection, fire detection, smoke detection, intrusion detection, etc. One or more premises devices 18 such as a life safety based premises device 18 may be configured to generate an audible alarm signal that may be detected by the spectrogram-based feature extraction described herein.
The premises devices 18 may include, for example, one or more lifestyle (e.g., home automation) related devices configured to adjust at least one premises setting such as lighting, temperature, energy usage, door lock and power settings, among other settings associated with the premises or devices on the premises. Image capture devices may include a digital camera and/or video camera, among other image captures devices. Premises device 18 may communicate with security control device 20 via proprietary wireless communication protocols and may also use Wi-Fi. Other communication technologies can also be used, and the use of Wi-Fi is only for example purposes. Those of ordinary skill in the art will also appreciate that various additional sensors and control and/or image capture devices may relate to life safety or lifestyle depending on both what the sensors, control and image capture devices do and how these sensors, control and image devices are used by system 10.
Security control device 20 may provide one or more management functions and monitoring functions, analysis functions, control functions such as power management, premises device management and alarm management/analysis, among other functions to premises security system 12. In particular, security control device 20 may manage one or more life safety and lifestyle features. Security control device 15 may communicate with network 22 via one or more communication links. In particular, the communications links may be broadband communication links such as a wired cable modem or Ethernet communication link, and digital cellular communication link, e.g., long term evolution (LTE) and/or 5G based link, among other broadband communication links known in the art. Broadband as used herein may refer to a communication link other than a plain old telephone service (POTS) line. Ethernet communication link may be an IEEE 802.3 or 802.11 based communication link. Network 22 may be a wide area network, local area network, wireless local network and metropolitan area network, among other networks known in the art. Network 22 may provide communications between security control device 20 and remote monitoring center 24. In one or more embodiments, security control device 20 may be part of premises device 18 or user interface device 16.
While security control device 20 is illustrated as being a separate device from user interface device 16 (e.g., control panel) and premises device 18, in one or more embodiments, security control device 15 may be integrated with one or more user interface devices 16 and/or premises devices 18 and/or other entity/device located at premises associated with premises security system 12.
Example implementations, in accordance with one or more embodiments, of control device 15 and imitation device 17 discussed in the preceding paragraphs will now be described with reference to
With respect to
In the embodiment shown, the hardware 30 of the detection device 26 further includes processing circuitry 36. The processing circuitry 36 may include a processor 38 and a memory 40. In particular, in addition to or instead of a processor, such as a central processing unit, and memory, the processing circuitry 36 may comprise integrated circuitry for processing and/or control, e.g., one or more processors, processor cores, field programmable gate arrays (FPGAs), and/or application specific integrated circuits (ASICs) adapted to execute instructions. The processor 38 may be configured to access (e.g., write to and/or read from) the memory 40, which may comprise any kind of volatile and/or nonvolatile memory, e.g., cache, buffer memory, random access memory (RAM), read-only memory (ROM), optical memory and/or erasable programable read-only memory (EPROM). Further, processing circuitry 36 may include and/or provide the functionality of an analog-to-digital converter for converting the detected analog signal to a digital signal for analysis.
Thus, the control device 15 further has software 42 stored internally in, for example, memory 40, or stored in external memory (e.g., database, storage array, network storage device, etc.) accessible by the detection device 26 via an external connection. The software 42 may be executable by the processing circuitry 36. The processing circuitry 36 may be configured to control any of the methods and/or processes described herein and/or to cause such methods, and/or processes to be performed, e.g., by detection device 26. Processor 38 corresponds to one or more processors 38 for performing detection device 26 functions described herein. The memory 40 is configured to store data, programmatic software code and/or other information described herein. In some embodiments, the software 42 may include instructions that, when executed by the processor 38 and/or processing circuitry 36, causes the processor 38 and/or processing circuitry 36 to perform the processes described herein with respect to detection device 26. For example, processing circuitry 36 of the detection device 26 may include detection unit 28 that is configured to perform one or more detection device 26 functions described herein, such as with respect to spectrogram-based alarm signal detection as described herein.
Although
Detection device 26 is configured to perform (Block S106) edge detection on the selected one of the plurality of spectral bins, as described herein. Detection device 26 is configured to perform (Block S108) pattern detection based on the edge detection, as described herein. Detection device 26 is configured to determine (Block S110) the audio signal corresponds to an audio alarm signal based on the pattern detection, as described herein. Detection device 26 is configured to trigger (Block S112) an action in response to determining that the audio signal corresponds to an audio alarm signal, as described herein.
According to one or more embodiments, each of the plurality of spectral bins corresponds to a respective frequency range over a predefined time period in the spectrogram.
According to one or more embodiments, the processing circuitry 36 is further configured to: determine a set of pulse-edge characteristics for the selected one of the plurality of spectral bins based on the edge detection, and perform the pattern detection further based on the set of pulse-edge characteristics.
According to one or more embodiments, the set of pulse-edge characteristics comprises a plurality of pulse-edge times and directions, and where the processing circuitry 36 is further configured to perform the pattern detection based on the set of pulse-edge characteristics.
According to one or more embodiments, the set of pulse-edge characteristics comprises a sample index and a value of each signal peak that exceeds a threshold.
According to one or more embodiments, the spectrogram is configured to contain at least one time period of a predefined temporal pattern associated with the audio alarm signal.
According to one or more embodiments, the action comprises causing transmission of a notification that indicates an alarm sensor has triggered an alarm.
According to one or more embodiments, the device 26 is not configured to be in data communication with the alarm sensor.
According to one or more embodiments, the pattern detection is performed based on a pattern detection criterion associated with one of a plurality of audio alarm signal types where each audio alarm signal type is associated with a respective pattern detection criterion. For example, detection device 26 may be configured to perform a separate detection for each audio alarm signal type such that detection device 26 may, for example, perform pattern detection for a fire alarm using a first pattern detection criterion and then perform pattern detection for a carbon monoxide alarm using a second pattern detection criterion different from the first pattern detection criterion. Each pattern detection criterion may define pulse-edge characteristics of a respective alarm type.
According to one or more embodiments, the one of the plurality of audio alarm signal types comprises a fire alarm, a smoke alarm, or a carbon monoxide alarm.
According to one or more embodiments, the processing circuitry 36 is further configured to determine a type of audio alarm of the audio alarm signal based on the pattern detection.
According to one or more embodiments, signal presence detection may be performed optionally within a spectral bin of the spectrogram such as to determine whether further processing of spectral bin is needed or warranted prior to pattern detection and/or alarm type determination. However, signal presence detection may not be required as a step such that pattern detection may be performed without first determining a signal (e.g., audio signal) has been detected.
Another example process for audio alarm detection is as follows. Detection device 26 generates a spectrogram based on blocks of input samples of a detected signal. In one or more embodiments, the sample rate may be selected according to a Nyquist criterion. The time extent of the spectrogram is selected or configured to contain at least one period of the temporal pattern of interest. The degree of overlap is selected for sufficient time resolution in the detection of the pulse-on and pulse-off phases. For example, to detect a pulse of a specified width of 0.5 s+/−10%, the spectrogram may need sufficient resolution to determine whether the pulse width is between 0.45 s and 0.55 s. Selecting a degree of overlap such that the spectrogram is calculated at a rate of 1/0.05 s=20 frames per second would be sufficient in this example. If the system sample rate is 8 kHz and the FFT size is 1024, that would correspond to a degree of overlap of approximately 61%.
The use of the spectrogram facilitates detection of a low-level signal of interest in the presence of a higher-level interfering signal at a nearby frequency.
For each spectral bin in the frequency range of interest, detection device 26 performs edge detection across the corresponding block of magnitude-versus-time samples of the spectral bin. A set of features is derived from the result as pairs consisting of sample index and value of each peak whose magnitude exceeds a threshold, such that each feature represents a candidate pulse-edge time and direction.
Detection device 26 performs pattern detection where a pattern-detection decision is made based on analysis of the feature set. In another example, pattern detection is performed using a neural network. In one example, pattern detection is performed using an ad-hoc tree-structure-based detector. Example programming code for the ad-hoc-tree-structure-based detector is reproduced below.
Hence, the process described herein is able to detect an audible alarm signal even if there are temporal-pattern phase-duration deviations that would otherwise result in a false-negative decision in the case of a matched filter detector implementation.
The detected temporal signal in
Further, in one or more embodiments, the threshold setting are configured or set based on the type of alarm. For example, in the case of a T3 signal, the thresholds may be set at one standard deviation away from the mean of the block of samples comprising the output of the edge detector. In the case of a T4 signal, the thresholds may be set at two standard deviations.
Example programming code for the ad-hoc-tree-structure-based detector is reproduced below.
/* Compute confidence metric [0.0, 1.0] for an edge pattern of alternating direction, constant nominal distance between edges, and a specified deviation in the actual distance between each pair of adjacent edges, as in the T3 and T4 alarm pattern specifications
As will be appreciated by one of skill in the art, the concepts described herein may be embodied as a method, data processing system, computer program product and/or computer storage media storing an executable computer program. Accordingly, the concepts described herein may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects all generally referred to herein as a “circuit” or “module.” Any process, step, action and/or functionality described herein may be performed by, and/or associated to, a corresponding module, which may be implemented in software and/or firmware and/or hardware. Furthermore, the disclosure may take the form of a computer program product on a tangible computer usable storage medium having computer program code embodied in the medium that can be executed by a computer. Any suitable tangible computer readable medium may be utilized including hard disks, CD-ROMs, electronic storage devices, optical storage devices, or magnetic storage devices.
Some embodiments are described herein with reference to flowchart illustrations and/or block diagrams of methods, systems and computer program products. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer (to thereby create a special purpose computer), special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer readable memory or storage medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instruction means which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
The functions/acts noted in the blocks may occur out of the order noted in the operational illustrations. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality/acts involved. Although some of the diagrams may include arrows on communication paths to show a primary direction of communication, communication may occur in the opposite direction of the depicted arrows.
Computer program code for carrying out operations of the concepts described herein may be written in an object oriented programming language such as Python, Java® or C++. However, the computer program code for carrying out operations of the disclosure may also be written in conventional procedural programming languages, such as the “C” programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer. In the latter scenario, the remote computer may be connected to the user's computer through a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).
Many different embodiments have been disclosed herein, in connection with the above description and the drawings. It will be understood that it would be unduly repetitious and obfuscating to literally describe and illustrate every combination and subcombination of these embodiments. Accordingly, all embodiments can be combined in any way and/or combination.
The present disclosure is not limited to what has been specifically shown and described herein above. In addition, unless mention was made above to the contrary, the accompanying drawings are not to scale. A variety of modifications and variations are possible in light of the above teachings without departing from the scope and spirit of the invention.
