The present disclosure relates to a presence detection system, and more particularly to a cooperative radar and infrared presence detection system.
Ensuring safety and efficient energy management in, for example, buildings, requires accurate counting of building occupants. In traditional occupancy counting applications a device may be used to quantify the number and direction of occupants traversing, for example, an entrance or exit point. The accuracy and resolution of such a device may depend on the employed technology. Different forms of technologies have been used in developing occupancy counting devices, such as a passive infrared sensor (PIR), a laser beam-break sensor, and a two-dimensional camera in the visible or thermal spectrum. Unfortunately, such known devices are high in cost, require considerable energy, are low in accuracy, have range limitations, and may lack the intelligence to verify that the target is indeed a human.
A presence detection system according to one, non-limiting, embodiment of the present disclosure includes an infrared detector; a radar detector; and a computer processor configured to receive respective signals from the infrared and radar detectors and process the signals via execution of an algorithm.
Additionally to the foregoing embodiment, the algorithm is a sensor fusion algorithm.
In the alternative or additionally thereto, in the foregoing embodiment, the infrared detector includes a focal plane array including a plurality of radiant energy sensors configured to convert radiant energy into an electrical signal.
In the alternative or additionally thereto, in the foregoing embodiment, the radar detector includes at least one antenna and a switching module configured to steer the at least one antenna for presence detection.
In the alternative or additionally thereto, in the foregoing embodiment, the radar detector is configured to perform one of an initial presence detection and a presence confirmation, and the infrared detector is configured to perform the other of the initial presence detection and the presence confirmation.
In the alternative or additionally thereto, in the foregoing embodiment, the radar detector includes a quadrature mixer module configured to communicate with the switching module and the computer processor and output a down-mixed signal, a heartbeat/respiration detector configured to receive the down-mixed signal to distinguish vital signs of a detected presence, and a localization module configured to receive the down-mixed signal to localize the detected presence.
In the alternative or additionally thereto, in the foregoing embodiment, the radar detector includes a human presence module configured to receive an output signal from the heartbeat/respiration detector to determine if the detected presence is human.
In the alternative or additionally thereto, in the foregoing embodiment, the human presence module is configured to output an affirmative signal to a FMCW mode module of a controller that includes the computer processor, and wherein the FMCW mode module is configured to separate multiple detected presence in space to assist execution of the algorithm by the computer processor to selectively reject presence detections.
In the alternative or additionally thereto, in the foregoing embodiment, the human presence module is configured to output a negative signal to the controller if a human presence is not detected.
In the alternative or additionally thereto, in the foregoing embodiment, the infrared detector is configured to operate in a sleep mode upon the absence of motion.
In the alternative or additionally thereto, in the foregoing embodiment, the radar detector is configured to measure vital signs of a presence.
In the alternative or additionally thereto, in the foregoing embodiment, the vital signs include at least one of heartbeat and respiration.
A method of operating a presence detection system according to another, non-limiting, embodiment includes detecting a presence by an infrared detector at a location; steering an antenna of a radar detector toward the location; and measuring vital signs of the presence by a radar detector.
Additionally to the foregoing embodiment, a computer processor receives a signal from the infrared detector indicative of the presence and the location and outputs an associated signal to a switching module for further interrogation of the presence.
In the alternative or additionally thereto, in the foregoing embodiment, the computer processor executes a sensor fusion algorithm.
In the alternative or additionally thereto, in the foregoing embodiment, the infrared detector includes a Doppler mode of operation.
In the alternative or additionally thereto, in the foregoing embodiment, the infrared detector includes a FMCW mode of operation.
In the alternative or additionally thereto, in the foregoing embodiment, the radar detector is steered by the computer processor indicative of presence detection by the infrared detector.
In the alternative or additionally thereto, in the foregoing embodiment, the presence detection system is part of a building management system.
In the alternative or additionally thereto, in the foregoing embodiment, the method includes reverting the infrared detector to a sleep mode upon loss of motion by the presence; and continued interrogation by the radar detector of the presence.
The foregoing features and elements may be combined in various combinations without exclusivity, unless expressly indicated otherwise. These features and elements as well as the operation thereof will become more apparent in light of the following description and the accompanying drawings. However, it should be understood that the following description and drawings are intended to be exemplary in nature and non-limiting.
Various features will become apparent to those skilled in the art from the following detailed description of the disclosed non-limiting embodiments. The drawings that accompany the detailed description can be briefly described as follows:
Referring to
The ambient air temperature control system 22 may be a forced air system such as a heating, ventilation, and air conditioning (HVAC) system, a radiant heat system and others. The security system 24 may be configured to detect intruders and provide various forms of alerts and notifications. The lighting system 26 may control and/or monitor lighting in each one of the predefined spaces 34 based on any number of factors including natural background lighting, occupancy and others. The transportation system 28 may include the control and/or monitoring of elevators, escalators and other transportation devices associated with and/or within the building 32. The safety system 30 may include the detection of conditions that may pose a risk or health hazard to occupants of the building 32. All of these systems 22, 24, 26, 28, 30 may require a variety of devices to perform any variety of functions including detection, monitoring communication, data referencing and collection, user control and others. Many devices may be shared between systems.
The building management system 20 may further include a computing device 36 that controls and/or supports each system 22, 24, 26, 28, 30. The computing device 36 may include a processor 38 (e.g., microprocessor) and a computer readable and writeable storage medium 40. It is further contemplated and understood that the building management system 20 may include more than one computing device 36 with any one computing device being dedicated to any one of the systems 22, 24, 26, 28, 30. Computing device 36 may be implemented in a physically co-located, or in a physically distributed manner. The computing device need not be physically contained within the building 32.
The building management system 20 includes at least one presence detection system 42 (i.e., a presence detection system 42 designated for a space 34). The presence detection system 42 may utilize, in-part, low complexity, low cost and low resolution infrared sensors assisted by computer vision algorithms to accurately detect, classify, track and count a moving presence or occupants (e.g., humans) in a given space 34 using minimal energy consumption. In one embodiment, the presence detection system 42 may supplement functions of the building management system 20 (e.g., HVAC system 22, lighting system 26, security system 24 and others). For example, the computing device 36 may receive a signal (see arrow 44) over a wired or wireless pathway(s) 46 from the presence detection system 42 indicative of a number of intruders in a given space 34. Upon such a signal 44, the computing device 36 may output a command signal (not shown) to the security system 24 for initiating a security response that may be an alert, an alarm, and/or other initiations.
Referring to
The infrared detector 48 may be configured to monitor substantially all of the designated space 34 for the detection of a presence that may be a living presence (e.g., a human). The detection function performed by the infrared detector 48 may include classification, tracking and counting of the presence. The detection function of performed by the infrared detector 48 is not limited to presence detection, but may also include safety and security functions. To monitor the entire space 34, the infrared detector 48 may be located in a top portion of the space 34 (e.g., mounted to a ceiling or corner mounted) and may be substantially centered to the top portion, or upon the ceiling.
Referring to
The pyroelectric FPA 62 may be an infrared FPA configured to sense and detect radiated heat emitted by the presence. The FPA 62 may include a row decoder 78, a column decoder 80 (which are part of a Read-Out Integrated Circuit (ROIC)), and a plurality of pixels or sensors 82 that may be infrared sensors arranged in a series of rows and columns (i.e., six rows and nine columns illustrated in
The memory module 64 of the infrared detector 48 is generally a computer readable and writeable storage medium and is configured to communicate with the processor 68 and generally stores intensity data from the sensors 82 for later processing, stores executable programs (e.g., algorithms) and their associated permanent data as well as intermediate data from their computation. The memory module 64 may be a random-access memory (RAM) that may be a ferroelectric RAM (FRAM) having relatively low power consumption with relatively fast write performance, and a high number of write-erase cycles. It is further contemplated and understood that the memory module 64 may be integrated, in-part, with the controller 52 and/or computing device 36 that may also perform, at least in-part, a portion of the data processing of data received from the FPA 62.
The radiant energy intensity information/data received by the decoders 78, 80 may be conditioned via a signal conditioning circuit (not shown) and then sent to the processor 68. The signal conditioning circuit may be part of the ROTC. Signal conditioning may include analog-to-digital converters and other circuitry to compensate for noise that may be introduced by the sensors 82. The processor 68 may be configured to provide radiometric correction as well as interpolation of the decoded pixels as is generally known in the art. The processor 68 is generally computer-based, and examples may include a post-processor, a microprocessor and/or a digital signal processor. It is further contemplated and understood that the processor 68 may be integrated, in-part, with the controller 52 and/or computing device 36 that may also perform, at least in-part, a portion of the data processing of data received from the FPA 62. This integration is generally called a system on a chip (SoC). The SoC is an integration of most or all of the components of a computing system onto a single substrate or carrier. The SoC may contain digital, analog, mixed-signal, and wireless communication functions and is typically low complexity, low cost, and physically smaller than the non-integrated equivalent system.
The sensor data compression block 66 of the infrared detector 48 is known to one having skill in the art and is generally optional with regard to the present disclosure.
The communication module 70 of the infrared detector 48 is configured to send and receive information and commands relative to the operation of the infrared detector. The communication module 70 may include a network coding engine block 84, an ADC 86, a receiver 88 (e.g. wireless), and a transmitter 90 (e.g., wireless). As is well-known in the art, the transmitter 90 and receiver 88 may be implemented as a transceiver or could be replaced by a well-known wired communication link (not shown). Also known in the art, the transmitter 90 may perform a digital to analog conversion (DAC) when the communication channel is inherently analog (e.g., wireless). The network coding engine block 84 is configured to interface the input and output of the processor 68 to transmitter 90, receiver 88 (through ADC 86), provide encoding (e.g., for error detection and correction), security via encryption or authentication, and other features.
The ADC 86 of the infrared detector 48 is configured to convert received analog information to digital information for eventual use by the processor 68. The network coding engine 84 provides any decoding necessary to, for example, facilitate error detection and correction, and/or for security (e.g., for encryption, decryption, or authentication.
The receiver 88 and the transmitter 90 of the infrared detector 48 are configured to respectively receive and transmit communications to and from other systems or components such as the controller 52 of the presence detection system 42 and/or the computing device 36 of the building management system 20. Such communications may be conducted over pathways that may be wired or wireless.
The power management module 72 of the infrared detector 48 is configured to control the power acquisition and power consumption of the infrared detector by controlling both the power source 74 and power consuming components. Such power consuming components may include the processor 68, the optional data compression block 66, the memory 64, the FPA 62 and the communication module 70 (e.g., transmitter 90, receiver 88, and ADC 86). It is contemplated and understood that other energy consuming components of the infrared detector 48 may be controlled. Such control may simultaneously maintain the infrared detector functionality while maximizing life (i.e., the length of time the infrared detector 48 can remain functional). In one embodiment, this control is achieved by receding horizon control (optimization). In alternative embodiments other control strategies such as model predictive control may be used. In one embodiment, the power consumption of the processor 68 and the memory 64 may be controlled by the power management module 72 by reducing the clock rate. It is further contemplated and understood that the power management module 72 and/or the power source 74 may provide power to the radar detector 50.
The power source 74 of the infrared detector 48 provides power to the other components of the detector, and may include at least one of a super capacitor 96, a battery 97 and a solar cell 98. The power management module 72 is configured to draw power from any one of the power sources as dictated by the needs of the system. The power management module 72 may also facilitate a power scheduling function that controls the simultaneous use of the various component functions to minimize unwanted current spikes. It is contemplated and understood that other short-term energy storage devices may be used in place of the super capacitor 96, other long-term energy storage devices may be used in place of the battery 97, and other energy harvesting or recharging devices may be used in place of the solar cell 98 including power from a power grid.
Referring to
By placing individual subsystem components on the same die or substrate platform 100, signal integrity, resistive losses, and security is generally improved through elimination of interconnects and sources of extraneous electrical and radiative noise typically present in systems with similar functionality but that use several individually packaged integrated circuits (IC's). Moreover, by placing all components on the same substrate platform 100, economy of scale is achieved that enables chip-scale cost reduction. Yet further, power management and consumption may be optimized potentially achieving long life battery operation, and facilitating packaging of various circuitry components on a single substrate platform 100. The infrared detector 48 may be built upon a ferroelectric memory platform using either active or passive detection; and, may be built upon a thermal isolator rather than a MEMS bridge, thereby improving yield, reducing across device response variations, and may be compatible with wafer production having small feature sizes.
By placing the individual subsystem components on the same die or substrate, it is also possible to control the number of pixels 82 being interrogated at any given time. This versatility in utilizing focal plane array 62 active pixel count is another means of power management.
Tracking algorithms may be used and executed by the processor 68 to estimate the number of occupants and track their movements within an established field of view of the infrared detector 48. The pyroelectric materials used in making the sensors 82 of the FPA 62 may only respond to moving objects, thus minimizing or eliminating signals resulting from background clutter. Motion detectors constructed from the ferroelectric class of materials are referred to as pyroelectrics. They have a volume polarization due to the nature of their crystal structure, which is non-centrosymetric. When operated significantly below their Curie point and previously poled, the net polarization varies approximately linearly with the temperature of the device. Thus, these materials can be metalized to form a capacitor structure. Upon temperature change of the device (i.e., either through deliberate heating or exposed to the radiant heat of a person, etc), the polarization changes and charge (Q) is either deposited or removed from the metal surfaces comprising the electrodes of the capacitor; whereupon a voltage (V) may appear across the capacitor (C), such that:
V=Q/C
This voltage is slowly bled off due to the small but finite internal conductivity of the pyroelectric. Thus pyroelectric imagers respond best to transitory heating and cooling, and therefore respond when a person enters, moves within, or leaves an imaged scene. Occupants that are initialized within the field of view and establish trackable motion within the array may be tracked and counted. Targets that are initialized in one of the sensors 82 and that do not establish movement within the array 62 will not be tracked by the algorithm, but may still be interrogated (i.e., measured) by the radar detector 50 of the presence detection system 42.
Referring to
The storage medium 114 of the presence detection system 42 may be part of the memory module 64 and/or is generally a computer readable and writeable storage medium configured to communicate with the processor 112, and generally stores data from the infrared and radar detector 48, 50 for later processing, stores executable programs (e.g., algorithms) and their associated permanent data as well as intermediate data from their computation. The storage medium 114 may be a random-access memory (RAM) that may be a ferroelectric RAM (FRAM) having relatively low power consumption with relatively fast write performance, and a high number of write-erase cycles. It is further contemplated and understood that the storage medium 114 may be integrated, at least in-part, with the controller 52 and/or computing device 36 that may also perform, at least in-part, a portion of the data processing of data received from the radar detector 50.
The quadrature mixer 108 of the radar detector 50 is configured to: receive an input signal (see arrow 126) from the oscillator 110, which may be a voltage controlled oscillator; communicate with the switching module 106 via communication signals (see arrow 128, and provide down-mixed signals (see arrows 130) to heartbeat/respiration detector 132 and localization module 134. Heartbeat/respiration detector 132 performs feature extraction on down mixed signals 130 to more readily distinguish vital signs in down mixed signals 130 from clutter and noise. Human presence module 136 uses the features from heartbeat/respiration detector 132 to determine if a human is present and outputs this determination to controller 102 via signal 122 if not, and to FMCW mode module 140 via signal 120 if so. The features produced by heartbeat/respiration detector 132 may include a frequency, an amplitude, a phase, a correlation, and/or the output of a filter. Localization module 134 also uses down mixed signals 130 to localize, particularly in range, the presence of possible humans in space 34 and outputs this localization to controller 102 via signal 124. More generally, the quadrature mixer 108 together with heartbeat/respiration detector 132, localization module 134, and human presence module 136 are generally configured to process data generated by the antennas 104 into two categories with the first category being relative to vital signs of the occupant(s) and the second category relative to a location of the occupant in the space 34.
The controller 102 of the radar detector 50 may be configured to selectively operate in a Doppler mode (see Doppler mode module 142 in
Each one of the plurality of antennas 104 may be positioned to monitor a pre-selected location within the space 34. The switching module 106 may be configured to energize a selected one of the plurality of antennas 104 when the infrared detector 48 detects an occupant in a location associated with the antenna 104 to be energized. Alternatively, the switching module 106 may be configured to steer one or more antennas toward any one of a plurality of locations, thus tracking the occupant when first detected by, for example, the infrared detector 48. It is contemplated that the steering may be entirely electrical (e.g., phased array beamsteering), may be entirely mechanical steering by physically moving the antenna, may be by selecting one of several antennas pointed in different directions, or may be a combination of electrical, selecting, and mechanical steering. It is further contemplated and understood that the plurality of antennas 104 may be an array.
The controller 102 may further provide a platform upon which sensor fusion algorithms may be deployed. The fusion of multiple modes of human presence detection (i.e., from radiant energy detector 48 and from radar detector 50), will provide better human presence detection that has a lower missed detection rate and/or a lower false alarm rate. This fusion may be achieved by controller 102 through the sequential use of the radiant energy detector 48 and the radar detector 50, which may be in any order, where the first to be used performs an initial detection and the second to be used provides a confirmation. In an alternate embodiment controller 102 fuses the output of the radiant energy detector 48 and the radar detector 50 simultaneously in a joint estimation of human presence. The joint estimation may be performed by Bayesian Estimation using probability density functions (pdfs) of the output of the radiant energy detector 48 and the radar detector 50. In alternative embodiments, alternative linear or non-linear joint estimation techniques may be employed such as maximum likelihood (ML), maximum a priori (MAP), non-linear least squares (NNLS), and others.
The various components, detectors and modules including the quad mixer 108, the oscillator 110, the heartbeat/respiration detector 132, the localization module 134, the human presence module 136, the FMCW mode module 140, the Doppler mode module 142, the linearizer 144, and others may be, at least in-part, software-based and generally executed by, for example, the processor 112 with source codes and relevant data being writeable and readable from, for example, the storage medium 114.
In operation and referring to
Benefits of the present disclosure include a low complexity, low cost infrared detector 48 (i.e., a FPA of few sensors), a device that utilizes little energy since the sensors are only activated by the detection circuit when an occupant moves within the device's field of view, and a simplified tracking and counting algorithm that provides high accuracy and limits false alarms.
Other benefits include the ability to confirm a human presence based on vital-signs over a long range (i.e., about ten meters or more), a relaxation of individual requirements on the infrared and radar detectors while creating additional functionality through the combination of both sensing domains, a system capable of functioning with either the infrared or the radar detectors as detection requirements change and also to adapt to power requirements, and the enablement of future applications relative to building management systems including target classification, counting, and intent detection. The system is additionally robust to electromagnetic interference, vibration, and other sources of nuisance alarms.
While the present disclosure is described with reference to exemplary embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the scope of the present disclosure. In addition, various modifications may be applied to adapt the teachings of the present disclosure to particular situations, applications, and/or materials, without departing from the essential scope thereof. The present disclosure is thus not limited to the particular examples disclosed herein, but includes all embodiments falling within the scope of the appended claims.
This application claims the benefit of PCT/US2017/016649 filed Feb. 6, 2017, which claims priority to U.S. Provisional Application No. 62/293,541 filed Feb. 10, 2016, which is incorporated herein by reference in its entirety.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/US2017/016649 | 2/6/2017 | WO | 00 |
Publishing Document | Publishing Date | Country | Kind |
---|---|---|---|
WO2017/139215 | 8/17/2017 | WO | A |
Number | Name | Date | Kind |
---|---|---|---|
5517429 | Harrison | May 1996 | A |
6190377 | Kuzdrall | Feb 2001 | B1 |
6313743 | Abraham-Fuchs et al. | Nov 2001 | B1 |
6343534 | Khanna et al. | Feb 2002 | B1 |
6351234 | Choy | Feb 2002 | B1 |
6439162 | Van Den Berg | Aug 2002 | B1 |
6791610 | Butler et al. | Sep 2004 | B1 |
6992291 | Porter et al. | Jan 2006 | B2 |
7679509 | Royer | Mar 2010 | B2 |
7807010 | Pitault | Oct 2010 | B2 |
7811234 | McGrath | Oct 2010 | B2 |
7842922 | Leneke et al. | Nov 2010 | B2 |
7924212 | Benitez et al. | Apr 2011 | B2 |
8026842 | Fox et al. | Sep 2011 | B2 |
8102261 | Wu | Jan 2012 | B2 |
8120524 | Zemany | Feb 2012 | B2 |
8378820 | Micko | Feb 2013 | B2 |
8615420 | Holliday | Dec 2013 | B2 |
8731241 | Johnson et al. | May 2014 | B2 |
8814805 | Lin et al. | Aug 2014 | B2 |
8884813 | Bangera et al. | Nov 2014 | B2 |
10247818 | Hjelmstad | Apr 2019 | B2 |
20030193429 | Campana | Oct 2003 | A1 |
20120066168 | Fadell et al. | Mar 2012 | A1 |
20120276849 | Hyde et al. | Nov 2012 | A1 |
20120319819 | Tkachenko | Dec 2012 | A1 |
20130002434 | Cuddihy et al. | Jan 2013 | A1 |
20130135444 | Stein et al. | May 2013 | A1 |
20140163343 | Heneghan et al. | Jun 2014 | A1 |
20160266256 | Allen | Sep 2016 | A1 |
20170299710 | Shin | Oct 2017 | A1 |
Number | Date | Country |
---|---|---|
102798473 | Nov 2012 | CN |
203568607 | Apr 2014 | CN |
104484968 | Apr 2015 | CN |
204405190 | Jun 2015 | CN |
0137000 | May 2001 | WO |
Entry |
---|
ISR/WO, dated May 19, 2017, 12 pages. |
Number | Date | Country | |
---|---|---|---|
20190049570 A1 | Feb 2019 | US |
Number | Date | Country | |
---|---|---|---|
62293541 | Feb 2016 | US |