Patent Grant
11288945
The disclosure generally relates to procedural compliance of staff in a medical treatment facility, and more particularly to systems and methods for monitoring and/or correcting procedural compliance of staff in a medical treatment facility to reduce the risk of Hospital Acquired Infections (HAI).
Hospital Acquired Infections (HAI) and/or Surgical Site Infections (SSI) are infections caused by virus, bacteria and other environmental factors acquired within hospitals or other medical treatment facilities. It is estimated that HAI and SSI infections cost the healthcare industry nearly $40 billion annually. HAI and SSI infections can be transmitted in multiple ways, including, but not limited to, surface contamination, airborne particulates and aspiration. HAI and SSI infections can be reduced by following strict procedures, including procedures regarding decontamination practices, hand-hygiene/antisepsis procedures, and other procedures. However, rather low compliance with such procedures among staff have been reported.
Hand-hygiene compliance monitoring systems can be utilized in homes, daycares, schools, restaurants, health care facilities including hospitals. In some cases, hand washing compliance is a metric that hospitals and other facilities would be interested in tracking and reporting. Currently, there are several methods of gathering hand washing compliance data. These methods are often ineffective, inaccurate, costly, and/or hard to install and maintain. For example, the secret shopper method is one of the most common methods of gathering hand washing compliance data. In the secret shopper method, personnel are hired to periodically monitor hand-hygiene compliance of staff. This method can be inaccurate due to human error and the presence of a monitoring person can modify staff behavior. Another prevalent method is a real time location system (RTLS) approach where staff are tracked through an infrastructure of location beacons and anchors. This system, aside from being expensive, can be inaccurate, as the system only ensures that the staff person has passed through the vicinity of a washing station, but not that they actually washed their hands.
What would be desirable is a method and system to help improve compliance with infection control procedures such as hand-hygiene compliance.
This disclosure relates to procedural compliance of staff in a medical treatment facility, and more particularly to systems and methods for monitoring and/or correcting procedural compliance of staff in a medical treatment facility to reduce the risk of Hospital Acquired Infections (HAI).
For instance, in one example, a hand-hygiene monitor may be provided that includes an accelerometer to detect vibration within a first range, a microphone to detect sound within a second range, and a transmit element to transmit data detected by the accelerometer and the microphone. Additionally, or alternatively, the hand-hygiene monitor can include one or more additional sensors. For example, the hand-hygiene monitor can include, but is not limited to, a light sensor, a radar, a humidity sensor, a temperature sensor, a motion detector, a magnetometer, or a combination thereof. The additional sensors can provide additional data and/or increase accuracy of the hand washing compliance data.
In some examples, the hand-hygiene monitor may be included in a system. The system may also include a processor, which may receive the data from the hand-hygiene monitor. The processor can, for example, be configured to determine whether a person has washed their hands by analyzing data from the hand-hygiene monitor. In some embodiments, the processor can use a machine learning model to improve the accuracy of the system over time. In some embodiments, the processor can be included in a cloud server.
The system may also include a tracking device. The tracking device may be included in an identification card, for example. In some examples, the tracking device may be used in combination with the data from the hand-hygiene monitor to identify the person, the person's location, and/or whether or not they are in compliance with washing their hands.
In a first example, a system for monitoring hand-hygiene of staff in a facility may comprise a plurality of sensors positioned adjacent a hand washing station. Each of the plurality of sensors may be free from imaging pixels that form a visually perceptible image of the hand washing station. Each of the plurality of sensors is configured to provide a corresponding sensor output signal that is indicative of whether a person is washing their hands at the hand washing station or not, and at least two of the plurality of sensors are of a different sensor type. A controller may be configured to fuse the sensor output signals from each of two or more of the plurality of sensors and to determine using the fused sensor output signals whether the person has washed their hands at the hand washing station in compliance with one or more predetermined criteria or not. The controller may be further configured to output a notification when the controller has determined that the person has washed their hands at the hand washing station in compliance with the one or more predetermined criteria.
In some cases, the plurality of sensors may comprise an accelerometer and/or a microphone, and optionally one or more of a light sensor, a radar sensor, a humidity sensor, a temperature sensor, a motion detector, and/or a magnetometer. In some cases, the controller may fuse the sensor output signals from the accelerometer and the microphone.
In some cases, the one or more predetermined criteria may comprise two or more distinct hand washing steps which may include a hand scrubbing step and a dispensing of soap step. The two or more distinct hand washing steps may also comprise approaching the hand washing station, dispensing of soap at the hand washing station, hand scrubbing at the hand washing station, and departing from the hand washing station.
In some cases, the controller may be further configured to identify and record an identity of the person washing their hands at the hand washing station, and whether the hand washing was in compliance with the one or more predetermined criteria.
In some cases, the system may further comprise an RFID reader for reading an RFID tag carried by the person. The controller may be operatively connected to the RFID reader and may be configured to identify the person washing their hands based on the RFID tag read by the RFID reader.
In some cases, the controller may be configured to determine if the hand scrubbing step lasted at least a minimum duration of time with at least a minimum level of vigor.
In another example, a system for monitoring hand-hygiene of staff in a facility may comprise a plurality of sensors positioned adjacent a hand washing station including an accelerometer and a microphone. Each of the plurality of sensors may be configured to provide a corresponding sensor output signal that is indicative of whether a person is washing their hands at the hand washing station or not. A controller may be configured to use the sensor output signals from the accelerometer and the microphone to determine whether the person has washed their hands at the hand washing station in compliance with one or more predetermined criteria or not. The controller may be further configured to output a notification when the controller has determined that the person has washed their hands at the hand washing station in compliance with the one or more predetermined criteria.
In some cases, the plurality of sensors may further comprise one or more additional sensors. The one or more additional sensors may comprise one or more of a light sensor, a radar sensor, a humidity sensor, a temperature sensor, a motion detector, and/or a magnetometer. The controller may be configured to use the sensor output signals from one or more of the additional sensors to determine whether the person has washed their hands at the hand washing station in compliance with the one or more predetermined criteria.
In some cases, the controller may be configured to fuse the sensor output signals from the accelerometer, the microphone and one or more of the additional sensors to determine whether the person has washed their hands at the hand washing station in compliance with the one or more predetermined criteria or not.
In some cases, the one or more predetermined criteria may comprise two or more distinct hand washing steps. The two or more distinct hand washing steps may comprise a hand scrubbing step and a dispensing of soap step.
In another example, a method for monitoring hand-hygiene of staff in a facility may comprise determining whether a person has washed their hands at a hand washing station in compliance with one or more predetermined criteria or not using two or more sensor outputs from two or more sensors positioned adjacent the hand washing station. Each of the two or more sensors may be free from imaging pixels that form a visually perceptible image of the hand washing station. At least two of the two or more sensors are of a different sensor type. The one or more predetermined criteria may comprise two or more distinct hand washing steps including a dispensing of soap step and a hand scrubbing step. The method may further comprise reporting whether the person has washed their hands at the hand washing station in compliance with the one or more predetermined criteria.
In some cases, the two or more sensors may comprise an accelerometer and/or a microphone, and one or more of a light sensor, a radar sensor, a humidity sensor, a temperature sensor, a motion detector, and/or a magnetometer.
In some cases, the method may further comprise fusing two or more sensor outputs from the two or more sensors to increase a confidence level in the determination of whether the person has washed their hands at the hand washing station in compliance with the one or more predetermined criteria or not.
In some cases, the method may further comprise determining whether the two or more distinct hand washing steps occurred in a predetermined order and/or whether one or more of the two or more distinct hand washing steps lasted for at least a predetermined time period.
The preceding summary is provided to facilitate an understanding of some of the features of the present disclosure and is not intended to be a full description. A full appreciation of the disclosure can be gained by taking the entire specification, claims, drawings, and abstract as a whole.
The disclosure may be more completely understood in consideration of the following detailed description of various embodiments in connection with the accompanying drawings, in which:
While the disclosure is amenable to various modifications and alternative forms, specifics thereof have been shown by way of example in the drawings and will be described in detail. It should be understood, however, that the intention is not to limit aspects of the disclosure to the particular embodiments described. On the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the disclosure.
The following detailed description should be read with reference to the drawings in which similar elements in different drawings are numbered the same. The detailed description and the drawings, which are not necessarily to scale, depict illustrative embodiments and are not intended to limit the scope of the disclosure. The illustrative embodiments depicted are intended only as exemplary. Selected features of any illustrative embodiment may be incorporated into an additional embodiment unless clearly stated to the contrary.
These embodiments are described in sufficient detail to enable those of ordinary skill in the art to practice one or more embodiments of this disclosure. It is to be understood that other embodiments may be utilized and that process changes may be made without departing from the scope of the present disclosure.
As will be appreciated, elements shown in the various embodiments herein can be added, exchanged, combined, and/or eliminated so as to provide a number of additional embodiments of the present disclosure. The proportion and the relative scale of the elements provided in the figures are intended to illustrate the embodiments of the present disclosure and should not be taken in a limiting sense.
As used herein, “a” or “a number of” something can refer to one or more such things. For example, “a number of devices” can refer to one or more devices.
Hospital Acquired Infections (HAI) and/or Surgical Site Infections (SSI) are infections caused by virus, bacteria and other environmental factors acquired within hospitals or other medical treatment facilities. HAI and SSI infections can be transmitted in multiple ways, including, but not limited to, surface contamination, airborne particulates and aspiration. HAI and SSI infections can be reduced by following strict procedures, including procedures regarding decontamination practices, hand-hygiene/antisepsis procedures, and other procedures. This may include, but not limited to, operating room personnel, physicians, nursing staff, custodial staff, etc. However, rather low compliance to such procedures among staff have been reported.
One example of a procedure where compliance can be important is in an operating room (OR). In the case of an OR, all sterile team members should perform a hand and arm scrub before entering the surgical suite. The basic principle of the scrub is to wash the hands thoroughly, and then to wash from a clean area (the hand) to a less clean area (the arm). A systematic approach to the scrub is an efficient way to ensure proper technique. There are typically two methods of scrub procedure. One is a numbered stroke method, in which a certain number of brush strokes are designated for each finger, palm, back of hand, and arm. An alternative method is a timed scrub, and each scrub should last from three to five minutes, depending on facility protocol.
This disclosure generally relates to method and systems for monitoring procedural compliance of staff in a facility such as hand-hygiene compliance, while maintaining patient and/or staff privacy, and for providing real time notification to the staff when a procedure is performed incorrectly or not at all. In one example, the present disclosure discloses methods, devices, systems, and computer-readable media useful for monitoring hand-hygiene. In some instances, hand-hygiene compliance data is gathered using a hand-hygiene monitor. The hand-hygiene monitor may include distributed synthetic sensors that are minimally obtrusive to detect and alert hand-hygiene incidents. The sensors may be placed so that a lapse in handwashing procedures can be positively identified and alerted. The sensors used may include, for example, microphones, accelerometers, ambient light, and/or other sensors. In some cases, such sensors may be used in conjunction with edge analytics to detect a faucet open and hand washing events. In some cases, a radiofrequency (RF) radar based sensor may be used to detect the presence of a person at a hand washing station, and in some instances, a hand washing motion of the hands. Fusion of the results of various sensing modalities may help identify hand washing compliance with greater certainty, while maintaining patient and/or staff privacy (e.g. free from imaging pixels that together can be used to form a visually perceptible image of the hand washing station). While the system is described with respect to a medical facility, it is contemplated that the system may be used in other environments to ensure procedural compliance, such as, but not limited to, senior care facilities, nursing homes, restaurants, hotels, office buildings, etc.
In some cases, the hand-hygiene monitor 12 may be a separate system that is placed in a particular area (e.g., near a hand washing station) or worn by a person as an identification card, a smart phone, or a wearable (e.g., pendant, smart watch, smart glasses, etc.), for example. When worn by a person, some or part of the hand-hygiene monitor 12 may be worn on both wrist and/or both hands. For example, some or part of the hand-hygiene monitor 12 may be incorporated into a watch, bracelet, ring, etc. and may be worn on one arm/hand or both arms/hands.
The illustrative hand-hygiene monitor 12 may include a communications port 26 for operatively coupling to a computing device 14, a wearable device 56, a user device 58, and/or a tracking device 62 when present. It is contemplated that the communications port 26 may be wired and/or wireless. When the communications port 26 is wireless, the communications port 26 may include a wireless transceiver, and the computing device 14, the wearable device 56, the user device 58, and/or the tracking device 62 may include a compatible wireless transceiver. It is contemplated that the wireless transceivers may communicate using a standard and/or a proprietary communication protocol. Suitable standard wireless protocols may include, for example, cellular communication, ZigBee, Bluetooth, WiFi, IrDA, dedicated short range communication (DSRC), EnOcean, or any other suitable wireless protocols, as desired.
The hand-hygiene monitor 12 may include one or more controllers or processors 24 that execute instructions stored in the system memory 28. In some cases, the controller 24 may include a programmable microprocessor. Such a programmable microprocessor may allow a user to modify the control logic of the hand-hygiene monitor 12 even after it is installed in the field (e.g., firmware update, application update). The system memory 28 of the hand-hygiene monitor 12 can include computer system readable media in the form of volatile memory, such as random access memory (RAM) and/or cache memory. The hand-hygiene monitor 12 may further include other removable/non-removable, volatile/non-volatile computer system storage media. By way of example only, the storage system can be provided for reading from and writing to a non-removable, non-volatile magnetic media (not shown and typically called a “hard drive” or flash memory). The system controller 24 and/or memory 28 may include at least one program/utility having a set of program modules that are configured to receive an input from or transmit an output to a remote computing device 14, a wearable device 56, a user device 58, and/or a tracking device 62.
In one example, the program/utility may be stored in the system memory 28 and may include one or more application program modules (e.g., software), such as a procedural compliance module 30. The procedural compliance module 30 may define one or more hand washing procedures that should be performed in a particular room of a facility in order to reduce the risk of infection. Each defined procedure may include one more predefined criteria, conditions, or actions that should be taken in order for the procedure to have been considered as performed correctly or to a set standard, as will be described in more detail herein. It is contemplated that the defined procedures may be different for different types of rooms and/or a category type of the person entering the room. For example, the defined procedures for an Operating Room (OR) may be different than the defined procedures for a patient room. In some cases, the procedural compliance module 30 may be a part of the hand-hygiene monitor 12 or may be executed on the remote computing device 14, as will be described in more detail herein.
The illustrative hand-hygiene monitor 12 may include a plurality of sensors 16. In some instances, the plurality of sensors 16 may work together to provide a synthetic sensor. A synthetic sensor can be considered a plurality of sensors that sense different conditions within an environment (e.g., motion, humidity, vibration, etc.), and where the different sensed conditions are fused to detect certain actions or conditions. A plurality of sensors 16 may be positioned adjacent to a hand washing station. In some embodiments, the hand-hygiene monitor 12 may include at least two sensors with at least two different sensor types. In some implementations, the plurality of sensors 16 of the hand-hygiene monitor 12 may include at least an accelerometer 18 and a microphone 20, an accelerometer 18 and a radar 54, and/or a microphone 20 and a radar 54. These are just some examples. The hand-hygiene monitor 12 may include additional sensors as well, including, but not limited to a light sensor 44, a humidity sensor 46, a temperature sensor 48, a magnetometer 50, a motion detector 52, etc. or a combination thereof. To help protect patient and staff privacy, it is contemplated that each of the sensors 16 may be free from imaging pixels that form a visually perceptible image of the hand washing station and/or surrounding area. As will be described in more detail herein, each sensor of the plurality of sensors 16 may provide a corresponding sensor output signal that is indicative of whether a person is washing their hands at the hand washing station or not. For example, the sensor output signal for an accelerometer 18 fixed to the sink of the hand washing station may collect a vibration signature that is indicative of running water, a vibration signature that is indicative of hand washing under running water, etc. The sensor output signal for an accelerometer 18 fixed to a bracelet of the staff member may collect a motion signature that signifies that the person is moving their hand(s) in a hand washing motion. In another example, the sensor output signal for the microphone 20 may collect a sound signature that is indicative of running water and/or hand washing under running water. In yet another example, the sensor output signal for the radar 54 may collect a radar signature that is indicative of running water and/or hand washing under running water. The sensor output signal for the humidity sensor 46 may detect an increase in humidity that is indicative of running water and/or hand washing under running water. When such sensor output signals are fused together, sometimes with various weighting factors, compliance with a set hand washing procedure may be detected with greater certainty, while protecting patient and/or staff privacy. That is, the controller 24 and/or procedural compliance module 30 may use the fused sensor signal output to determine whether the person has washed their hands at the hand washing station in compliance with one or more predetermined criteria or not. It contemplated that the controller 24 and/or procedural compliance module 30 may be configured to output a notification when the controller 24 and/or procedural compliance module 30 has determined that the person has washed their hands at the hand washing station in compliance with the one or more predetermined criteria and/or when the person has not washed their hands at the hand washing station in compliance with the one or more predetermined criteria.
In one specific example, when the accelerometer 18 is fixed to the sink base, the accelerometer 18 can detect water flow. As water flows from the faucet and contacts the sink base, the accelerometer 18 can detect the vibrations generated by the flowing water. Thus, the accelerometer 18 can determine whether the faucet is running based on the vibration the faucet creates when the faucet is on and allowing water to flow. It is further contemplated that the accelerometer 18 can also detect when hands are interrupting the flow of water. For example, when hands are interrupting the water flow, different vibration signature may be produced than when the water is flowing uninterrupted from the faucet to the sink basin. In some cases, the accelerometer 18 may discern a difference between a person holding their hand(s) stationary under the flow of water and making a scrubbing motion with their hands under the flow of water. This can be discerned because the scrubbing motion creates a different vibration signature at the accelerometer 18 than holding one or both hands stationary under the faucet.
In some embodiments, the accelerometer 18 and/or the processor 24 may also record how long the faucet was running and/or how long the person was washing their hands based on the length of the various detected vibration signatures. In some examples, a person could be deemed noncompliant with hand washing procedures when the person did not actively wash their hands for at least a minimum period of time. The vibration signature of the faucet running and/or the vibration signature of hands interrupting the flow of water can be used to determine whether a person washed their hands for at least the minimum period of time necessary to be compliant with the defined hand washing procedures. In some examples, the accelerometer 18 may also detect the vibration due to soap dispensing, a hand dryer blowing, a person drying their hands, or a paper towel dispensing, for example. Similar to the faucet, the accelerometer 18 can detect vibration signatures that are unique to each of these actions. As detailed further herein, other sensor outputs (e.g. microphone 20, light sensor 44, humidity sensor 46, temperature sensor 48, magnetometer 50, motion detector 52 and radar 54) may be fused with the accelerometer output to increase the certainty of the compliance decision. In some cases, each of the following actions or steps may be detected using the accelerometer 18:
1) water flowing from a faucet into a sink basin with no interference;
2) soap dispensing;
3) water flowing from a faucet into a sink basin with stationary hands positioned within the water flow;
4) water flowing from a faucet into a sink basin with moving hands (e.g., scrubbing) positioned within the water flow;
5) a hand dryer blowing;
6) a person drying their hands in a hand dryer; and
7) a paper towel dispensing.
The preceding list of actions and/or steps of a handwashing procedure (compliant or otherwise) is not intended to be inclusive of all actions and/or steps that can be used to determine if a handwashing procedure is compliant or not, but rather indicative of some of the types of actions that can be recognized and used.
The microphone 20 of the hand-hygiene monitor 12, when provided, can detect sound within a particular frequency range, and in some cases a direction of the sound. In some cases, a plurality of microphones 20 may be distributed around a hand washing sink, but this is not required. The microphone(s) 20 can detect water contacting the sink basin from the faucet and/or the water contacting the sink basin from the person's hands. Each of these events can create a unique sound signature. Moreover, a scrubbing motion of the hands under the flow of water can create a unique sound signature relative to a person simply holding their hands stationary under the flow of water. In some instances, the microphone 20 can detect the sound of a soap dispenser being activated or pumped. Identification of each of these events may be enhanced when more than one microphone is distributed about the hand washing station. Using more than one microphone may also help determine the direction of the sound, which may be used to help identify and/or distinguish between events. A person could be deemed noncompliant with washing their hands if the person did not use soap. In some cases, the microphone 20 can detect the sound of a hand dryer blowing, a person drying their hands, and/or a paper towel dispensing, etc. In some cases, each of the following actions or steps may be detected using the microphone 20:
1) water flowing from a faucet into a sink basin with no interference;
2) soap dispensing;
3) water flowing from a faucet into a sink basin with stationary hands positioned within the water flow;
4) water flowing from a faucet into a sink basin with moving hands (e.g., scrubbing) positioned within the water flow;
5) a hand dryer blowing;
6) a person drying their hands in a hand dryer; and
7) a paper towel dispensing.
The preceding list of actions and/or steps of a handwashing procedure (compliant or otherwise) is not intended to be inclusive of all actions and/or steps that can be used to determine if a handwashing procedure is compliant or not, but rather indicative of some of the types of actions that can be recognized and used.
The motion detector 52 and/or the radar 54, when present, may be used to detect the presence of a person in an area. The motion detector 52 can sense any movement within a particular area. The particular area can be a bathroom or an area in proximity to a faucet and/or sink. The motion detector 52 data can indicate that an object was moving for a period of time, which can suggest that a person was in the bathroom or in front of a faucet for a particular amount of time. This time can be compared to a threshold quantity of time to determine if a person may have been washing hands and/or whether the amount of time was sufficient. For example, movement in front of the faucet for twenty seconds could indicate that a person washed their hands for a sufficient period of time.
The radar 54, unlike the motion detector 52, can detect a particular location of one or more people. The radar 54 can also detect the location of a person without the use of tags worn on the person. However, it is contemplated that radiofrequency tags can be used to determine an identity of a person and/or to place the person at a specific location (near an RF reader, for example). In some cases, the radar 54 can detect whether a person is standing within a particular proximity to a faucet. The radar 54 may also detect whether another person is standing within a particular proximity to another faucet. As such, radar 54 can be used in areas with one or more sinks and can collect location data on multiple people at a time. Various types of radar 54 can be included in the hand-hygiene monitor 12. In some embodiments, the radar 54 can be a pulse-doppler radar. In other embodiments, the radar 54 can be a frequency-modulated continuous wave radar. It is contemplated that the radar 54 may be used to detect a person entering hand washing area and/or a person exiting the hand washing area. It is also contemplated that the radar 54, when directed at the sink, may be used to detect water flow without hand in the water flow, water flow with hands in the water flow, a hand washing motion of the hands, a scrubbing motion of the hands/arms, etc. Some illustrative procedural compliance systems utilizing radar tracking are described in commonly assigned U.S. patent application Ser. No. 16/522,326 titled “Methods and Systems for Improving Infection Control in a Facility,” the disclosure of which is hereby incorporated by reference.
It is contemplated that the processor 24 and/or procedural compliance module 30 of the hand-hygiene monitor 12 may combine or fuse the sensor output signals from the accelerometer 18, the microphone 20, the radar 54 and/or other sensor. In one example, both the microphone 20 and/or the accelerometer 18 are used to detect the faucet running and a hand washing motion under the running water, but if neither sensor detected soap dispensing, the data collected from the sensors of the hand-hygiene monitor 12 could suggest that the person did not use soap. This could result in the procedural compliance module 30 determining that the person is noncompliant with the hand washing procedures. It is further contemplated that the location of the person (e.g., if the person has entered and/or exited the hand washing area) may be verified or determined by the radar 54. Also, the radar 54 may be used to detect hand washing motion under a water flow, which may be used to confirm a hand washing motion detected by the accelerometer 18 and/or microphone 20. This is just one example.
In some instances, a time duration that one or more of these activities occur (e.g., a time duration of each action or step) can be tracked and compared to a threshold quantity of time appropriate for a specific activity to help determine whether the hand washing was in compliance with hand washing procedures. For example, a vibration signature indicative of interrupting the flow of water for twenty seconds could indicate that a person washed their hands for a sufficient period of time (e.g. greater than twenty seconds).
As described herein, sensors 16 in addition to (or in place of) the accelerometer 18 and/or microphone 20 may also be provided in the hand-hygiene monitor 12. For example, as described above, the hand-hygiene monitor 12 may include a light sensor 44, a humidity sensor 46, a temperature sensor 48, a magnetometer 50, a motion detector 52, a radar 54, and/or a combination thereof.
A light sensor 44 can detect whether a light is on. If the light is on in, for example, a bathroom this may be indicative of a person being present in the bathroom. For example, many people turn lights off in a bathroom to conserve energy when the bathroom is not in use. In some examples, the lights in a bathroom are coupled to a motion detector and the lights turn off after no motion has been detected after a particular period of time. As such, data collected from a light sensor 44 can indicate a person is not present, in response to the light sensor 44 detecting less than a threshold amount of light and/or no light.
A humidity, temperature and/or magnetic forces in an area can change in response to a faucet running. In some examples, the humidity, temperature, and/or magnetic forces in an area can change in response to a person being present. As such, it is contemplated that data from the humidity sensor 46, temperature sensor 48, and/or magnetometer 50 can be used to determine whether a faucet is running and/or whether a person is present, for example. Other sensors can provide additional data and/or signatures to help increase the accuracy of the hand washing compliance determination. The sound and/or vibration signatures independently, together, or in combination with additional sensors can be used to determine whether or not a person washed their hands. The sound and/or vibration signatures independently, together, or in combination with other sensors can also be used to determine how long the person washed their hands.
In some examples, analyzing a combination of the sound signatures, vibration signatures, and/or a combination of sound signatures, vibration signatures and other sensor data can increase the overall accuracy of the processor 24 and/or procedural compliance module 30 in determining whether a person is compliant with washing their hands.
The hand-hygiene monitor 12 may further include a transmit element 22. The transmit element 22 may be a transmitter or transceiver. For example, the transmit element 22 can be a radio frequency (RF) transceiver. In some examples, the transmit element 22 can be wired or wirelessly coupled to a processor and/or controller 24. The transmit element 22 can transmit data detected and collected by sensors 16 of the hand-hygiene monitor 12 and/or a determination of compliance/non-compliance with established hand washing procedures. The transmit element 22 can transmit the data to an external computing device 14 and/or to another device 56, 58, 62.
In some examples, the processor 24 and/or procedural compliance module 30 of the hand-hygiene monitor 12 can use a machine learning model to determine whether a person washed their hands. For example, machine learning methods may be used (for example, a support vector machine, deep neural networks, multi-way support vector machine, etc.) to detect hand washing events through the microphone 20, accelerometers 18, and/or other sensors 16 data. In parallel, a point cloud from a radar 54 may be used to deduce the position and identity of the personnel. The output of these two systems may be fused to come to a conclusion (e.g., the person did or did not wash their hands). In some cases, the sensor 16 data including the radar 54 may be input to a combined machine learning model that determines the presence, identity and hand washing compliance all together. In yet another example, different machine learning models are created as an ensemble and run in parallel—the results are then fused and used to establish whether the hand washing was in compliance.
The hand-hygiene monitor 12 may include a compliance indicator 60. Although the compliance indicator 60 is shown included in the hand-hygiene monitor 12, the compliance indicator 60 can be outside of the hand-hygiene monitor 12. For example, the compliance indicator 60 can be a separate device, built into a wearable device 56, built in to a user device 58, and/or a part of the computing device 14. The compliance indicator 60 can provide a visual, audible and/or haptic signal to a person to indicate whether the person has complied with, or is in compliance with, a hand washing procedure. Hand washing compliance can be tracked for a single hand washing activity or, in some embodiments, can be tracked over a period of time, such as a period of hours, days, weeks, months, years, etc. The signal of the compliance indicator 60 can be, but is not limited to, a light, a symbol, a tone, a haptic vibration, and/or a combination thereof.
In some embodiments, the compliance indicator 60 can display a green light in response to the processor 24 and/or procedural compliance module 30 determined that a person washed their hands in compliance with hand washing standards. In some examples, the compliance indicator 60 can display a red light in response to the processor 24 and/or procedural compliance module 30 determined that a person did not wash their hands in compliance with the hand washing standards. In some embodiments, the system 10 can include multiple compliance indicators that each indicate compliance with a particular sub-activity involving in compliant hand-washing, such as a first indicator for washing and a different indicator for using soap. Alternatively, the indicator could have multiple states at which it changes to a different state each time a sub-activity is accomplished (e.g., a first state for hand washing, a second state for using soap, a third state for using a towel, etc.).
In some embodiments, the compliance indicator 60 can change visual signals in response to a person washing their hands, washing their hands for a particular amount of time, or once soap is dispensed. For example, the compliance indicator 60 can display a red light until the processor 24 and/or procedural compliance module 30 indicates that the person has washed their hands for thirty seconds, at which time the compliance indicator 60 can display a green light to let the person know that they can stop washing their hands.
As described herein, the computing device 14 may be operatively coupled to the hand-hygiene monitor 12. The computing device 14 may be a system specific controller, a desktop computer, a laptop computer, a tablet, a cloud based computing device, a cloud server, etc. When the hand-hygiene monitor 12 is equipped with a processing device 24 and memory 28, the computing device 14 may not be required for the function of the system 10. When the computing device 14 is provided, the computing device 14 may be located in a same room as the hand-hygiene monitor 12, in a different room but a same building as the hand-hygiene monitor 12, or off-site (e.g., geographically different location) from the hand-hygiene monitor 12, as desired. The computing device 14 may include a communications port 32 for communicating with the hand-hygiene monitor 12 (or other devices, such as but not limited to, cell phones, wearable devices, etc.), the wearable device 56, the user device 58, and/or tracking device 62. In some cases, the computing device 14 may include more than one communications port 32 for communicating over more than one network (e.g., wireless LAN, wired LAN, the Internet, short range wireless communications, etc.). In some instances, the communications port 32 may be a radiofrequency (RF) transceiver.
The computing device 14 may further include a processor 34 (e.g. microprocessor, microcontroller, etc.) and a memory 36. The computing device 14 may also include a user interface 40. The user interface 40 may be a display or other means for allowing a user to interact with the computing device 14. The user interface 40 may be part of a personal computer, tablet computer, smart phone, laptop computer, or may include a standalone display. In some instances, the computing device 14 may include a user input 42 for receiving a user input from a user. For example, the user input may include a keyboard, mouse, actuatable buttons, or a touchscreen display. These are just examples.
The memory 36 may be used to store any desired information, such as the aforementioned procedural compliance module. The memory 36 may be any suitable type of storage device including, but not limited to, RAM, ROM, EPROM, flash memory, a hard drive, and/or the like. In some cases, the processor 34 may store information within the memory 36, and may subsequently retrieve the stored information from the memory 36. In some cases, the memory 36 may store one or more application program modules (e.g., software), such as a procedural compliance module 38. The procedural compliance module 38 may be the same as or different from the procedural compliance module 30 associated with the hand-hygiene monitor 12. The procedural compliance module 38 may include one or more defined procedures that should be performed in a particular room of a facility in order to reduce the risk of infection. Each defined procedure may include one more conditions or actions that should be met in order for the procedure to have been considered as performed correctly or to a standard, as will be described in more detail herein. It is contemplated that the defined procedure may be different for different types of rooms and/or a category type of the person entering the room. In some cases, only one of the hand-hygiene monitor 12 or the computing device 14 is provided with a procedural compliance module 30, 38. In other cases, both the hand-hygiene monitor 12 and the computing device 14 are provided with a procedural compliance module 30, 38.
In some embodiments, a weighting value can be applied to one or more sensor data values if it is determined that a particular type of data is more predictive of a person having good hand-hygiene. For example, accelerometer data may be more reliable than microphone data and therefore a higher weighting value could be applied to one or more accelerometer data values relative to microphone data values. This is just one example. Other weighting schemes may be used, as desired. Alternatively, or in addition, a machine learning model can be used by the processor 24 and/or procedural compliance module 30 of the hand-hygiene monitor 12 and/or the processor 34 and/or procedural compliance module 38 of the computing device 14 to determine whether a person was complaint in washing their hands. Data from the accelerometer 18, data from the microphone 20, and/or data from other sensors 16 can be input into the machine learning model to determine whether a person properly washed their hands. The machine learning model can be, but is not limited to, a support vector machine, a deep neural network, a multi-way support vector machine, or any other suitable machine learning and/or artificial intelligence model or algorithm.
The processor 24 and/or procedural compliance module 30 of the hand-hygiene monitor 12 and/or the processor 34 and/or procedural compliance module 38 of the computing device 14 can use a machine learning model including statistical techniques to improve the accuracy of the system 10 over time. In some examples, machine learning and/or artificial intelligence can be used to perceive an environment, such as a bathroom, and adjust sensors and/or adjust a weighting value that the processor 24 and/or procedural compliance module 30 of the hand-hygiene monitor 12 and/or the processor 34 and/or procedural compliance module 38 of the computing device 14 gives data from particular sensors to improve probability of accurately determining whether a person complied with hand washing procedures.
The processor 24 and/or procedural compliance module 30 of the hand-hygiene monitor 12 and/or the processor 34 and/or procedural compliance module 38 of the computing device 14 can include an iterative learning component. The machine learning model can be updated periodically to incorporate idiosyncrasies of the environment that the hand-hygiene monitor 12 is in. In some examples, the machine learning model can be updated to suppress particular sounds, for example, the sound of doors opening and/or closing. Moreover, the particular hand washing procedures may differ depending on the location within the facility. For example, more rigorous hand washing/scrubbing may be required at a and washing station servicing an operating room relative to a hand washing station servicing a patient room. The machine learning model can learn to identify the particular hand washing procedures associated with a particular location.
The processor 24 and/or procedural compliance module 30 of the hand-hygiene monitor 12 and/or the processor 34 and/or procedural compliance module 38 of the computing device 14 can save the processor executable instructions that provide the machine learning component in the respective memory 28, 36. The processor 24 and/or procedural compliance module 30 of the hand-hygiene monitor 12 and/or the processor 34 and/or procedural compliance module 38 of the computing device 14 can also save compliance data in the respective memory 28, 36. In some cases, the memory 28, 36 can include statistical data on people washing their hands, people washing their hands for a particular time, people washing their hands with soap, and/or a combination thereof.
The wearable device 56, when provided, may be device that is worn by or carried by a person (e.g., staff member). In some cases, the wearable device 56 may allow the system 10 to identify the staff member or a staff member classification (e.g., physician, nurse, food service, etc.) when they enter a room. The identity of the staff member may then be used to determine which procedure the staff member is expected to comply with when entering a particular room, if any. In some instances, the wearable device 56 may be used to help identify the location of the person in the room. For example, the location of the person may be identified when the person is in proximity to another device in the room capable of receiving a signal transmitted by the wearable device 56 (e.g. beacon, RF tag reader, etc.).
In one example, the wearable device 56 may be a network or internet enabled bracelet or watch. It is contemplated that in addition to providing information regarding the staff member, a bracelet may be equipped with gesture tracking technology and/or other sensors such as but not limited to an accelerometer, a gyroscope, a magnetometer, a humidity sensor, a temperature sensor, a moisture sensor, a microphone, etc. When so provided and worn on a wrist, a wearable device 56 may be used to provide information which may be used to determine if certain procedures are being followed (e.g., hand sanitizing, hand washing, etc.), as will be described in more detail herein. In other examples, the wearable device 56 may be an employee badge with a radiofrequency (RF) tag. It is contemplated that other wearable devices 56 may also be used, including, but not limited to, rings, necklaces, cell phones, etc. In some cases, the wearable device 56 may be include a wireless transceiver to wirelessly communicate with the hand-hygiene monitor 12, the computing device 14, the user device 58 and/or any other suitable device. When so provided, it is contemplated that the wireless transceivers may communicate using a standard and/or a proprietary communication protocol. Suitable standard wireless protocols may include, for example, cellular communication, ZigBee, Bluetooth, WiFi, IrDA, dedicated short range communication (DSRC), EnOcean, or any other suitable wireless protocols, as desired. In some embodiments, the system 10 may not include a wearable device 56.
The user device 58, when provided, may be a device carried by a staff member as they are performing their duties. The user device 58 may include, but is not limited to, mobile devices including smart phones, tablet computers, laptop computers, wireless network-enabled key fobs, e-readers, and/or the like. The user device 58 may be configured to receive messages from the hand-hygiene monitor 12 and/or the computing device 14 regarding a procedure that is expected to be performed in a particular room. In some cases, the messages may be informing the staff member of the procedural requirements. In other cases, the messages may be informing the staff member that they have failed to perform a procedure or performed it incorrectly, as will be described in more detail herein. In some embodiments, the system 10 may not include a user device 58.
In some instances, the system 10 may include a tracking device 62. It is contemplated that the tracking device 62 may be a part of the wearable device 56 and/or the user device 58. For example, the tracking device 62 can be included in an identification card, for example. In other embodiments, the tracking device 62 may be a separate entity. In some cases, the tracking device 62 can be a vector or real-time location system (RTLS).
In some examples, the tracking device 62 can be used in combination with the data from the hand-hygiene monitor 12 to identify a person and whether or not the person was in compliance with washing their hands. For example, the processor 24 and/or procedural compliance module 30 can match the hand-hygiene monitor 12 data at a particular time with a particular person in response to the tracking device 62 placing the person in front of the sink where the hand-hygiene monitor 12 is located at the particular time. The memory 28 can also include such tracking device 62 data.
For example, when washing hands, a person may be expected to wash their hands at a sink. Thus, in some cases, the criteria may include locating the person near the sink. The hand washing steps may include, but are not limited to:
1) approaching the hand washing station or sink;
2) turning on the water;
3) dispensing of soap (at the hand washing station or sink);
4) scrubbing of hands;
5) turning off the water;
6) drying of hands (e.g., through the use of a hand dryer, paper towels, etc.)
7) departing the hand washing station or sink.
It is contemplated that not all of the above steps are required to be performed for the user to be considered in compliance with the predetermined criteria. Further, additional steps may be required depending on the location of the hand-hygiene monitor 12. For example, an operating room may have more rigorous standards than a patient room. Some or all of the hand washing steps may include an associated length of time or minimum threshold time that the action should be performed. For example, a person may be required to scrub their hands for a minimum of 20 seconds or more to be considered as complying with the requirements of the scrubbing action or step. The predetermined criteria may be defined or programmed during commissioning of the hand-hygiene monitor 12 and/or computing device 14. In some cases, the predefined criteria may be updated or changed after commissioning. It is further contemplated that the some or all of the aspects of the predefined criteria may be learned using multi-factor machine learning or other machine learning technique. For example, the hand washing procedure may be performed one or more times while the system 10 is in a commissioning mode to allow the system 10 to recognize the vibrations, sounds, locations, and/or other signals that indicate a user is washing their hands in accordance with a set procedure. It is further contemplated that during commissioning and/or standard operation, the location of the person and/or the hand scrubbing motions may be verified using the radar 54. The predefined criteria may be updated periodically to incorporate any idiosyncrasies of the environment the hand-hygiene monitor 12 is operating in.
Once the defined procedures and/or predefined criteria have been stored, the system 10 may be used to monitor for procedural compliance. The system 10 (e.g., the processor 24 and or procedural compliance module 30 of the hand-hygiene monitor 12 and/or the processor 34 and/or procedural compliance module 38 of the computing device 14) may be configured to receive sensor output signals from at least two more sensors 16, as shown at block 104. The two or more sensors 16 may include an accelerometer 18 and a microphone 20, an accelerometer 18 and a radar, an microphone and a radar 54, etc. In some cases, one or more additional sensors, such as, but not limited to, a light sensor 44, a humidity sensor 46, a temperature sensor 48, a magnetometer 50, a motion detector 52, a radar 54, or a combination thereof, may be used in combination with, for example, the accelerometer 18 and/or the microphone 20. As described above, the sensor output signals may be unique and indicative of various actions being performed in the proximity of the sensors 16.
The system 10 (e.g., the processor 24 and or procedural compliance module 30 of the hand-hygiene monitor 12 and/or the processor 34 and/or procedural compliance module 38 of the computing device 14) may fuse or combine the sensor output signals from the at least two or more sensors 16, as shown at block 106. It is contemplated that fusing the sensor output signals may increase the confidence that a particular step or action is correctly identified. For example, the signals may be analyzed together to determine location, hand washing, and/or identity.
The system 10 may be configured to determine whether or not the one or more predefined criteria or conditions of the defined procedure have been met, as shown at block 108. The system 10 may be configured to compare a location of the person, an action or hand washing step performed by the person, a length of time the person has performed the action or step, to the one or more conditions of the defined procedures. This may include tracking the order of occurrence of the detected hand washing step.
Regardless of whether or not the person has met the one or more conditions of the defined procedure (e.g., location, action or step, and/or time), the system 10 may issue a notification or provide an alert, as shown at block 110. It is contemplated that the notification may be provided in real time so that the person has a chance to correct their mistake if the hand washing criteria were not met. In some cases, the alert may be provided directly to the user device 58 and/or the wearable device 56 of the person in the room. An illustrative alert may remind the user to perform the defined procedure (e.g., hand-hygiene). In other cases, an alert may be provided to a central location, such as, but not limited to, a nurses' station, supervisor, management, etc. In some cases, the room itself may have an alerting device, such as, but not limited to the compliance indicator 60 described herein, that can annunciate the alert to the person in the room.
In an illustrative example, an alert may be provided when a nurse or physician enters a patient room but fails to pause or stop at the hand sanitizing station or at the sink, fails to turn on the water, fails to dispense soap, etc. The alert may remind the user of the hand-hygiene procedure. It is contemplated that the alert may be a plain text alert (e.g., please wash your hands or please return to the hand sanitizer, etc.), a visual alert (e.g., an illuminated light), an audible alert (e.g., a beep or series of beeps), a haptic alert (e.g., a vibration of the wearable device 56 or the user device 58), etc.
The hand-hygiene monitor 202 may include a communications port 212 for operatively coupling to the computing device 204, the wearable device 206, the user device 208, and/or the tracking device 210. It is contemplated that the communications port 212 may be wired and/or wireless. When the communications port 212 is wireless, the communications port 212 may include a wireless transceiver, and the computing device 204, the wearable device 206, the user device 208, and/or the tracking device 210 may include a compatible wireless transceiver. It is contemplated that the wireless transceivers may communicate using a standard and/or a proprietary communication protocol. Suitable standard wireless protocols may include, for example, cellular communication, ZigBee, Bluetooth, WiFi, IrDA, dedicated short range communication (DSRC), EnOcean, or any other suitable wireless protocols, as desired.
The hand-hygiene monitor 202 may include one or more controllers or processors 214 that execute instructions stored in the system memory 216. In some cases, the controller 214 may include a programmable microprocessor. Such a programmable microprocessor may allow a user to modify the control logic of the hand-hygiene monitor 202 even after it is installed in the field (e.g., firmware update, application update). The system memory 216 of the hand-hygiene monitor 202 can include computer system readable media in the form of volatile memory, such as random access memory (RAM) and/or cache memory. The hand-hygiene monitor 202 may further include other removable/non-removable, volatile/non-volatile computer system storage media. By way of example only, the storage system can be provided for reading from and writing to a non-removable, non-volatile magnetic media (not shown and typically called a “hard drive” or flash memory). The system controller 214 and/or memory 216 may include at least one program/utility having a set of program modules that are configured to receive an input from or transmit an output to a remote computing device 204, a wearable device 206, a user device 208, and/or a tracking device 210.
In one example, the program/utility may be stored in the system memory 216 and may include one or more application program modules (e.g., software), such as a procedural compliance module 218. The procedural compliance module 218 may include one or more defined hand washing procedures that should be performed in a particular room of a facility in order to reduce the risk of infection. Each defined procedure may include one more predefined criteria, conditions, or actions that should be performed in order for the procedure to have been considered as performed correctly or to a standard, as will be described in more detail herein. It is contemplated that the defined procedure may be different for different types of rooms and/or a category type of the person entering the room. For example, the defined procedures for an operating room (OR) may be different than the defined procedures for a patient room. In some cases, the procedural compliance module 218 may be a part of the hand-hygiene monitor 202 or may be executed on the remote computing device 204, as will be described in more detail herein.
The hand-hygiene monitor 202 may include an array of thermal imaging sensors 220. In some instances, the array of thermal imaging sensors 220 may be used in combination with any or all of the sensors 16 of the system 10 described above. The array of thermal imaging sensors 220 may include a two-dimensional array of pixels which allow for a temperature detection of a two-dimensional area. It is contemplated that the array of thermal imaging sensors 220 may include four or more individually readable pixels arranged in two or more rows and two or more columns, and may be used to acquire a thermal image. In some cases, the array of thermal imaging sensors 220 may include a two dimensional array of passive infrared pixels. Example two dimensional arrays of passive infrared pixels include a 2×2 array, a 2×3 array, a 2×4 array, a 3×3 array, a 4×4 array, a 6×6 array, an 8×8 array, a 16×16 array, a 32×32 array, a 64×64 array, a 128×128 array, and/or any other suitable two dimensional array as desired. The array of thermal imaging sensors 220 may be an infrared array sensor based on advanced microelectromechanical systems (MEMS), such as, but not limited to the Infrared Array Sensor Grid-EYE manufactured Panasonic with a North American base in Secaucus, N.J.
Thermal images may be stored, at least temporarily, in the memory 216 of the hand-hygiene monitor 202. The resolution of the thermal image may be determined by the number of individually readable pixels of array of thermal imaging sensors 220. For example, an 8×8 array of individually readable pixels will result in 64 different temperature data points per frame acquired. The array of thermal imaging sensors 220 may have a relatively low resolution to allow it to be used in facilities where typical cameras are not allowed for privacy reasons. For example, the array of thermal imaging sensors 220 may not output a visually perceptible image of the hand washing station.
Referring additionally to
The hand washing station 250 may further include a soap dispenser 252. In some cases, the soap dispenser 252 may include an RFID reader which is configured to identify a user and/or verify the user was within proximity of the soap dispenser 252. The RFID reader may be configured to read an RFID tag associated with the user's identification badge or other user device 208 to determine the identity of the user. It is further contemplated that additional sensing methods may be used to determine the soap has been dispensed. For example, microphones may be used to listen for the sound of the soap being dispensed. This is just one example. Other sensing modalities may be used, as desired.
While not explicitly shown the array of thermal imaging sensors 220 may also be positioned adjacent to a waterless hand sanitizing station. In such a configuration, the array of thermal imaging sensors 220 may function in a similar manner to that described with respect to the hand washing station 250.
Returning to
The controller 214 and/or procedural compliance module 218 may be configured to combine or fuse the sensor output signals from two or more sensors, if so provided. For example, sensors, such as, RFID readers/tags, radars, microphones, etc. may be used to increase the confidence the system has made the appropriate determination. The controller 214 and/or procedural compliance module 218 may then use the fused sensor signal outputs to determine whether the person has washed their hands at the hand washing station in compliance with one or more predetermined criteria or not. Alternatively, the controller 214 and/or procedural compliance module 218 may determine whether the person has washed their hands at the hand washing station in compliance with one or more predetermined criteria or not based on the thermal images. It is further contemplated that the controller 214 and/or procedural compliance module 218 may be configured to output a notification when the controller 214 and/or procedural compliance module 218 has determined that the person has washed their hands at the hand washing station in compliance with the one or more predetermined criteria.
The controller 214 and/or procedural compliance module 218 may be configured to analyze the thermal images and/or sensor data to determine if a sequence of events has occurred in a particular order. In one illustrative example, a sequence of events may include:
1) Soap dispensed;
2) Water flowing;
3) Hand scrubbing; and
4) Dwell time.
The preceding list of actions and/or steps of a handwashing procedure (compliant or otherwise) and/or the order thereof is not intended to be inclusive of all actions and/or steps that can be used to determine if a handwashing procedure is compliant or not, but rather indicative of some of the types of actions that can be recognized and used. For example, the sequence of events and/or dwell time can vary based on the type of hand-hygiene procedure.
The detection of one or more of these events can also be to determine whether a person is compliant with washing their hands. For example, if the array of thermal imaging sensors 220 detected water running but did not detect hand scrubbing, the data collected by the array of thermal imaging sensors 220 can suggest and the procedural compliance module 218 (or processor 214) can determine that a person did not wash their hands.
It is contemplated that combining the data from the array of thermal imaging sensors 220 with data from other sensors can increase the accuracy of the procedural compliance module 218 by providing the processor 214 and/or procedural compliance module 218 with more data to make a more informed decision on whether a person washed their hands. For example, in some embodiments, a moisture sensor and/or humidity sensor may be positioned in the wearable device 206 (e.g., a wrist worn device). The moisture sensor and/or humidity sensor may be configured to detect a change in moisture or humidity adjacent to the wearable device 206 that can be associated with wet hand (e.g., from washing hands). It is further contemplated that the location of the person (e.g., if the person has entered and/or exited the hand washing area) may be verified or determined by a radar.
In some embodiments a time period for one of these activities occur (e.g., a time duration of each action or step) can be compared to a threshold quantity of time appropriate for a specific activity to determine if a person may have been engaging in the activity and/or whether the amount of time was sufficient. For example, a thermal images which indicate scrubbing hands for twenty seconds could indicate that a person washed their hands for a sufficient period of time. Additionally, or alternatively, the length of time a person remains at the hand washing station 250, or dwell time, may be used.
As described herein, sensors in addition to (or in place of) the array of thermal imaging sensors 220 may also be provided in the hand-hygiene monitor 202. For example, the hand-hygiene monitor 202 may include a light sensor, a humidity sensor, a temperature sensor, a magnetometer, a motion detector, a radar, or a combination thereof. Other sensors can provide additional data and/or increase the accuracy of the hand washing compliance data. The thermal images alone or in combination with additional sensors can be used to determine whether or not a person washed their hands and/or how long the person washed their hands.
The hand-hygiene monitor 202 may further include a transmit element 222. The transmit element 222 may be a transmitter or transceiver. For example, the transmit element 222 can be a radio frequency (RF) transceiver. In some examples, the transmit element 222 can be wired or wirelessly coupled to a processor and/or controller 214. The transmit element 222 can transmit data (e.g., images) detected and collected the array of thermal imaging sensors 220 of the hand-hygiene monitor 202. The transmit element 222 can transmit the data to the processor 214, to an external computing device 204, and/or to another device 206, 208, 210. In a number of embodiments, the processer 214 can be included in the hand-hygiene monitor 202. In some embodiments, the processor 214 can be configured to receive the data from the transmit element 222, as described above. The processor 214 and/or procedural compliance module 218 can use the data received from the transmit element 222 to determine whether a person washed their hands by analyzing the data detected and/or collected by the array of thermal imaging sensors 220 and/or other suitable sensors. In some examples, the processor 214 and/or procedural compliance module 218 can use a machine learning model to determine whether a person washed their hands.
For example, in one embodiment machine learning methods may be used (for example, a support vector machine) to detect the hand washing event through the array of thermal imaging sensors 220 and/or other sensors. In parallel, the point cloud from a radar may be used to deduce the position and identity of the personnel. The output of these two systems may then be fused to come to a conclusion (e.g., the person did or did not wash their hands). In other embodiments, the array of thermal imaging sensors 220 and radar data or both are input to a combined machine learning model that determines the presence, identity and hand washing all together. In yet another embodiment all three models above are created as an ensemble and run in parallel—the results from the three models are then fused and used to establish the predefined criteria. Finally, some embodiments may use different machine learning and artificial intelligence methods such as a support vector machine, deep neural networks, multi-way support vector machine etc. and create ensembles of these different models.
The hand-hygiene monitor 202 may further include a compliance indicator 224. Although the compliance indicator 224 is shown included in the hand-hygiene monitor 202, the compliance indicator 224 can be outside of the hand-hygiene monitor 202. For example, the compliance indicator 224 can be a separate device, built into a wearable device 206, built in to a user device 208, or a part of the computing device 204. The compliance indicator 224 can provide a visual and/or audible signal to a person to indicate whether the person has complied with, or is in compliance with, a hand washing procedure. Hand washing compliance can be tracked for a single hand washing activity or, in some embodiments, can be over a period of time, such as a period of hours, days, weeks, etc. The signal of the compliance indicator 224 can be, but is not limited to, a light, a symbol, a tone, or a combination thereof. The compliance indicator 224 may be similar in form and function to the compliance indicator 60 described above.
As described herein, the computing device 204 may be operatively coupled to the hand-hygiene monitor 202. The computing device 204 may be a system specific controller, a desktop computer, a laptop computer, a tablet, a cloud based computing device, a cloud server, etc. When the hand-hygiene monitor 202 is equipped with a processing device 214 and memory 216, the computing device 204 may not be required for the function of the system 200. When the computing device 204 is provided, the computing device 204 may be located in a same room as the hand-hygiene monitor 202, in a different room but a same building as the hand-hygiene monitor 202, or off-site (e.g., geographically different location) from the hand-hygiene monitor 202, as desired. The computing device 204 may include a communications port 226 (or receive element) for communicating with the hand-hygiene monitor 202 (or other devices, such as but not limited to, cell phones, wearable devices, etc.), the wearable device 206, the user device 208, and/or tracking device 210. In some cases, the computing device 204 may include more than one communications port 226 for communicating over more than one network (e.g., wireless LAN, wired LAN, the Internet, short range wireless communications, etc.). The communications port 226 may be a receiver or a transceiver. In some instances, the communications port 226 may be a radiofrequency (RF) transceiver.
The computing device 204 may further include a processor 228 (e.g. microprocessor, microcontroller, etc.) and a memory 230. The computing device 204 may also include a user interface 232. The user interface 232 may be a display or other means for allowing a user to interact with the computing device 204. The user interface 232 may be part of a personal computer, tablet computer, smart phone, laptop computer, or may include a standalone display. In some instances, the computing device 204 may include a user input 234 for receiving a user input from a user. For example, the user input may include a keyboard, mouse, actuatable buttons, or a touchscreen display. These are just examples.
The memory 230 may be used to store any desired information, such as the aforementioned procedural compliance module. The memory 230 may be any suitable type of storage device including, but not limited to, RAM, ROM, EPROM, flash memory, a hard drive, and/or the like. In some cases, the processor 228 may store information within the memory 230, and may subsequently retrieve the stored information from the memory 230. In some cases, the memory 230 may store one or more application program modules (e.g., software), such as a procedural compliance module 236. The procedural compliance module 236 may be the same as or different from the procedural compliance module 218 associated with the hand-hygiene monitor 202. The procedural compliance module 236 may include one or more defined procedures that should be performed in a particular room of a facility in order to reduce the risk of infection. Each defined procedure may include one more conditions or actions that should be met (sometimes in a specific order) in order for the procedure to have been considered as performed correctly or to a standard, as will be described in more detail herein. It is contemplated that the defined procedure may be different for different types of rooms and/or a category type of the person entering the room. In some cases, only one of the hand-hygiene monitor 202 or the computing device 204 is provided with a procedural compliance module 218, 236. In other cases, both the hand-hygiene monitor 202 and the computing device 204 are provided with a procedural compliance module 218, 236.
In some embodiments, a weighting value can be applied to one or more sensor data values if it is determined that a particular type of data is more predictive of a person having good hand-hygiene. For example, accelerometer data may be more reliable than microphone data and therefore a positive weighting value could be applied to one or more accelerometer data values or a negative weighting value could be applied to one or more microphone data values. This is just one example. Other weighting schemes may be used, as desired.
As discussed above, a machine learning model can be used by the processor 214 and/or procedural compliance module 218 of the hand-hygiene monitor 202 and/or the processor 228 and/or module 236 of the computing device 204 to determine whether a person was complaint in washing their hands. Data from the array of thermal imaging sensors 220 and/or data from other sensors can be input into the machine learning model to determine whether a person washed their hands. The machine learning model can be, but is not limited to, a support vector machine, a deep neural network, or a multi-way support vector machine, for example.
The processor 214 and/or procedural compliance module 218 of the hand-hygiene monitor 202 and/or the processor 228 and/or module 236 of the computing device 204 can use a machine learning model including statistical techniques to improve the accuracy of the system 200 over time. In some examples, machine learning and/or artificial intelligence can be used to perceive an environment, such as a bathroom, and adjust sensors and/or adjust a weighting value the processor 214 and/or procedural compliance module 218 of the hand-hygiene monitor 202 and/or the processor 228 and/or module 236 of the computing device 204 gives data from a particular sensor to improve probability of accurately determining whether a person complied with washing their hands.
The processor 214 and/or procedural compliance module 218 of the hand-hygiene monitor 202 and/or the processor 228 and/or module 236 of the computing device 204 can include an iterative learning component. The machine learning model can be updated periodically to incorporate idiosyncrasies of the environment that the hand-hygiene monitor 202 is in. In some examples, the machine learning model can be updated to suppress particular sounds, for example, the sound of doors opening and/or closing.
The processor 214 and/or procedural compliance module 218 of the hand-hygiene monitor 202 and/or the processor 228 and/or module 236 of the computing device 204 can save the processor executable instructions that provide the machine learning component in the respective memory 216, 230. The processor 214 and/or procedural compliance module 218 of the hand-hygiene monitor 202 and/or the processor 228 and/or module 236 of the computing device 204 can also save compliance data in the respective memory 216, 230. For example, the memory 216, 230 can include statistical data on people washing their hands, people washing their hands for a particular time, people washing their hands with soap, and/or a combination thereof.
The wearable device 206 may be device that is worn by or carried by a person (e.g., staff member). In some cases, the wearable device 206 may allow the system 200 to identify the staff member or a staff member type (e.g., physician, nurse, food service, etc.) when they enter a room. The identity of the staff member may then be used to determine which procedure the staff is expected to comply with when entering a particular room, if any. In some embodiments, the wearable device 206 may be used to identify the location of the person in the room. For example, the location of the person may be identified determining when the person is in proximity to another device in the room capable of receiving a signal transmitted by the wearable device 206.
In one example, the wearable device 206 may be a network or internet enabled bracelet or watch. It is contemplated that in addition to providing information regarding the staff, a bracelet may be equipped with gesture tracking technology and/or other sensors such as but not limited to humidity or moisture sensors. When so provided and worn on a wrist, a wearable device 206 may be used to provide additional information which may be used to determine if certain procedures are being followed (e.g., hand sanitizing, hand washing, etc.), as will be described in more detail herein. In other examples, the wearable device 206 may be an employee badge with a radiofrequency (RF) tag. It is contemplated that other wearable devices 206 may also be used, including, but not limited to, rings, necklaces, cell phones, etc. In some cases, the wearable device 206 may be include a wireless transceiver to wirelessly communicate with the hand-hygiene monitor 202, the computing device 204 and/or the user device 208. When so provided, it is contemplated that the wireless transceivers may communicate using a standard and/or a proprietary communication protocol. Suitable standard wireless protocols may include, for example, cellular communication, ZigBee, Bluetooth, WiFi, IrDA, dedicated short range communication (DSRC), EnOcean, or any other suitable wireless protocols, as desired. In some embodiments, the system 200 may not include a wearable device 206.
The user device 208 may be a device carried by a staff member as they are performing their duties. The user device 208 may include, but is not limited to, mobile devices including smart phones, tablet computers, laptop computers, wireless network-enabled key fobs, e-readers, and/or the like. The user device 208 may be configured to receive messages from the hand-hygiene monitor 202 and/or the computing device 204 regarding a procedure that is expected to be performed in a particular room. In some cases, the messages may be informing the staff of the procedural requirements. In other cases, the messages may be informing the staff they have failed to perform a procedure or performed it incorrectly, as will be described in more detail herein. In some embodiments, the system 200 may not include a user device 208.
The system 200 can include a tracking device 210. It is contemplated that the tracking device 210 may be a part of the wearable device 206 and/or the user device 208. For example, the tracking device 210 can be included in an identification card, for example. In other embodiments, the tracking device 210 may be a separate entity. For example, the tracking device 210 can be a vector or real-time location system (RTLS).
In some examples, the tracking device 210 can be used in combination with the data from the hand-hygiene monitor 202 to identify a person and whether or not they were in compliance with washing their hands. For example, the processor 214 and/or procedural compliance module 218 can match the hand-hygiene monitor 202 data at a particular time with a person in response to the tracking device 210 placing the person in front of the sink where the hand-hygiene monitor 202 is located at the particular time. The memory 216 can also include such tracking device 210 data.
For example, when washing hands, a person may be expected to wash their hands at a sink. Thus, in some cases, the criteria may include locating the person near the sink. The hand washing steps may include, but are not limited to:
1) approaching the hand washing station or sink;
2) turning on the water;
3) dispensing of soap (at the hand washing station or sink);
4) scrubbing of hands;
5) turning off the water;
6) drying of hands (e.g., through the use of a hand dryer, paper towels, etc.)
7) departing the hand washing station or sink.
It is contemplated that not all of the above steps are required to be performed for the user to be considered in compliance with the predetermined criteria. Further, additional steps may be required depending on the location of the hand-hygiene monitor 202. For example, an operating room may have more rigorous standards than a patient room. Some or all of the hand washing steps may include an associated length of time or minimum threshold time that the action should be performed for. For example, a person may be required to scrub their hands for a minimum of 20 seconds or more to be considered as complying with the requirements of the scrubbing action or step. The predetermined criteria may be defined or programmed during commissioning of the hand-hygiene monitor 202 and/or computing device 204. In some cases, the predefined criteria may be updated or changed after commissioning. It is further contemplated that the some or all of the aspects of the predefined criteria may be learned using multi-factor machine learning or other machine learning technique. For example, the hand washing procedure may be performed one or more times while the system 200 is in a commissioning mode to allow the system 200 to recognize a thermal profile, locations, and/or other signals that indicate a user is washing their hands in accordance with a set procedure. It is further contemplated that during commissioning and/or standard operation, the location of the person and/or the hand scrubbing motions may be verified using a radar. The predefined criteria may be updated periodically to incorporate any idiosyncrasies of the environment the hand-hygiene monitor 202 is operating in.
Once the defined procedures and/or predefined criteria have been stored, the system 200 may be used to monitor for procedural compliance. The system 200 (e.g., the processor 214 and or procedural compliance module 218 of the hand-hygiene monitor 202 and/or the processor 228 and/or module 236 of the computing device 204) may be configured to receive thermal images from array of thermal imaging sensors 220, as shown at block 304. In some cases, the system 200 may receive data from one or more additional sensors, such as, but not limited, a radar, and RFID reader/tag, an accelerometer, a microphone, a light sensor 44, a humidity sensor 46, a temperature sensor 48, a magnetometer 50, a motion detector 52, or a combination thereof. As described above, the sensor output signals (e.g., thermal images) may be unique and indicative of various actions being performed in the proximity of the array of thermal imaging sensors 220.
The system 200 (e.g., the processor 214 and or procedural compliance module 218 of the hand-hygiene monitor 202 and/or the processor 228 and/or module 236 of the computing device 204) may fuse or combine the sensor output signals if two or more sensors output signals are received, as shown at block 306. It is contemplated that fusing the sensor output signals may increase the confidence that a step or action is correctly identified. For example, the signals may be analyzed together to determine location, hand washing, and/or identity. However, output signals from more than one sensor is not required.
The system 200 may be configured to determine whether or not the one or more predefined criteria or conditions of the defined procedure have been met, as shown at block 308. The system 200 may be configured to compare a location of the person, an action or hand washing step performed by the person, a length of time the person has performed the action or step, etc. to the one or more conditions of the defined procedures. Regardless of whether or not the person has met the one or more conditions of the defined procedure (e.g., location, action or step, and/or time), the system 200 may issue a notification or provide an alert, as shown at block 310. It is contemplated that the notification may be provided in real time so that the person has a chance to correct their mistake if the hand washing criteria were not met. In some cases, the alert may be provided directly to the user device 208 and/or the wearable device 206 of the person in the room. An illustrative alert may remind the user to perform the defined procedure (e.g., hand-hygiene). In other cases, an alert may be provided to a central location, such as, but not limited to, a nurses' station, supervisor, management, etc. In some cases, the room itself may have an alerting device, such as, but not limited to the compliance indicator 224 described herein, that can annunciate the alert to the person in the room.
In an illustrative example, an alert may be provided when a nurse or physician enters a patient room but fails to pause or stop at the hand sanitizing station or at the sink, fails to turn on the water, fails to dispense soap, etc. The alert may remind the user of the hand-hygiene procedure. It is contemplated that the alert may be a plain text alert (e.g., please wash your hands or please return to the hand sanitizer, etc.), a visual alert (e.g., an illuminated light), an audible alert (e.g., a beep or series of beeps), a haptic alert (e.g., a vibration of the wearable device 206 or the user device 208), etc.
In a first example, a hand-hygiene monitor may comprise an accelerometer to detect vibration within a first range, a microphone to detect sound within a second range, and a transmit element to transmit data detected by the accelerometer and the microphone.
Alternatively or additionally to any of the examples above, in another example, the hand-hygiene monitor may further comprise a light sensor to detect whether a light is on to determine whether a person is present.
Alternatively or additionally to any of the examples above, in another example, the hand-hygiene monitor may further comprise a radar to detect whether a person is standing within a particular proximity of a faucet.
Alternatively or additionally to any of the examples above, in another example, the hand-hygiene monitor may further comprise a humidity sensor to detect a humidity change to determine whether a faucet is running.
Alternatively or additionally to any of the examples above, in another example, the hand-hygiene monitor may further comprise a temperature sensor to detect a temperature change to determine whether a faucet is running.
Alternatively or additionally to any of the examples above, in another example, the hand-hygiene monitor may further comprise a motion detector to detect whether a person has entered an area.
Alternatively or additionally to any of the examples above, in another example, the hand-hygiene monitor may further comprise a magnetometer to detect a change in magnetic forces to determine whether a faucet is running.
Alternatively or additionally to any of the examples above, in another example, the hand-hygiene monitor may further comprise a processor to determine whether a person washed their hands by analyzing the data detected by the accelerometer and the microphone using a machine learning model.
Alternatively or additionally to any of the examples above, in another example, the hand-hygiene monitor may further comprise the hand-hygiene monitor is worn by a person.
In another example, a system for a hand-hygiene monitor may comprise a hand-hygiene monitor including an accelerometer to detect vibration within a first range, a microphone to detect sound within a second range, and a transmit element to transmit data detected by the accelerometer and the microphone. The system may further comprise a processor configured to receive the data from the transmit element and determine whether a person washed their hands by analyzing data from the accelerometer and the microphone using a machine learning model.
Alternatively or additionally to any of the examples above, in another example, data from one or more other sensors or data from a radar may be used by the processor to determine whether the person washed their hands.
Alternatively or additionally to any of the examples above, in another example, the processor may determine a sound of at least one of: a hand dryer blowing, the person drying their hands, a paper towel dispensing, soap dispensing, or a faucet running.
Alternatively or additionally to any of the examples above, in another example, an identity of the person may be determined using a tracking device.
Alternatively or additionally to any of the examples above, in another example, the tracking device may be a vector or real-time location system (RTLS).
Alternatively or additionally to any of the examples above, in another example, the tracking device may be included in an identification card.
Alternatively or additionally to any of the examples above, in another example, the processor may be included in a cloud server.
Alternatively or additionally to any of the examples above, in another example, the machine learning model may be a support vector machine, a deep neural network, or a multi-way support vector machine.
In another example, a system for a hand-hygiene monitor may comprise a hand-hygiene monitor including an accelerometer to detect vibration of a faucet, a microphone to detect sound of hand washing, and a radar to detect a presence of a person within a particular proximity of the faucet. The system may further comprise a processor configured to receive the data from the transmit element and determine whether the person washed their hands by analyzing data from the accelerometer, the microphone, and the radar.
Alternatively or additionally to any of the examples above, in another example, the radar may be a pulse-doppler radar.
Alternatively or additionally to any of the examples above, in another example, the radar may be a frequency-modulated continuous-wave radar.
Those skilled in the art will recognize that the present disclosure may be manifested in a variety of forms other than the specific embodiments described and contemplated herein. Accordingly, departure in form and detail may be made without departing from the scope and spirit of the present disclosure as described in the appended claims.
This application claims the benefit under 35 U.S.C. § 119(e) of U.S. Provisional Application No. 62/727,480, filed Sep. 5, 2018, which is incorporated herein by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 191512 | Bennett et al. | Jun 1877 | A |
| 4009647 | Howorth | Mar 1977 | A |
| 4375637 | Desjardins | Mar 1983 | A |
| 4918615 | Suzuki et al. | Apr 1990 | A |
| 4939922 | Smalley et al. | Jul 1990 | A |
| 5566084 | Cmar | Oct 1996 | A |
| 5727579 | Chardack | Mar 1998 | A |
| 5745126 | Jain et al. | Apr 1998 | A |
| 5751916 | Kon et al. | May 1998 | A |
| 5777598 | Gowda et al. | Jul 1998 | A |
| 5973662 | Singers et al. | Oct 1999 | A |
| 6065842 | Fink | May 2000 | A |
| 6139177 | Venkatraman et al. | Oct 2000 | A |
| 6144993 | Fukunaga et al. | Nov 2000 | A |
| 6157943 | Meyer | Dec 2000 | A |
| 6229429 | Horon | May 2001 | B1 |
| 6238337 | Kambhatla et al. | May 2001 | B1 |
| 6334211 | Kojima et al. | Dec 2001 | B1 |
| 6353853 | Gravlin | Mar 2002 | B1 |
| 6369695 | Horon | Apr 2002 | B1 |
| 6375038 | Daansen et al. | Apr 2002 | B1 |
| 6429868 | Dehner, Jr. et al. | Aug 2002 | B1 |
| 6473084 | Phillips et al. | Oct 2002 | B1 |
| 6487457 | Hull et al. | Nov 2002 | B1 |
| 6580950 | Johnson et al. | Jun 2003 | B1 |
| 6598056 | Hull et al. | Jul 2003 | B1 |
| 6619555 | Rosen | Sep 2003 | B2 |
| 6704012 | Lefave | Mar 2004 | B1 |
| 6720874 | Fufido et al. | Apr 2004 | B2 |
| 6741915 | Poth | May 2004 | B2 |
| 6796896 | Laiti | Sep 2004 | B2 |
| 6801199 | Wallman | Oct 2004 | B1 |
| 6816878 | Zimmers et al. | Nov 2004 | B1 |
| 6876951 | Skidmore et al. | Apr 2005 | B2 |
| 6882278 | Winings et al. | Apr 2005 | B2 |
| 6904385 | Budike, Jr. | Jun 2005 | B1 |
| 6907387 | Reardon | Jun 2005 | B1 |
| 6911177 | Deal | Jun 2005 | B2 |
| 6993403 | Dadebo et al. | Jan 2006 | B1 |
| 6993417 | Osann, Jr. | Jan 2006 | B2 |
| 7023440 | Havekost et al. | Apr 2006 | B1 |
| 7031880 | Seem et al. | Apr 2006 | B1 |
| 7062722 | Carlin et al. | Jun 2006 | B1 |
| 7110843 | Pagnano et al. | Sep 2006 | B2 |
| 7139685 | Bascle et al. | Nov 2006 | B2 |
| 7164972 | Imhof et al. | Jan 2007 | B2 |
| 7183899 | Behnke | Feb 2007 | B2 |
| 7200639 | Yoshida | Apr 2007 | B1 |
| 7222111 | Budike, Jr. | May 2007 | B1 |
| 7222800 | Wruck | May 2007 | B2 |
| 7257397 | Shamoon et al. | Aug 2007 | B2 |
| 7280030 | Monaco | Oct 2007 | B1 |
| 7292908 | Borne et al. | Nov 2007 | B2 |
| 7295116 | Kumar et al. | Nov 2007 | B2 |
| 7302313 | Sharp et al. | Nov 2007 | B2 |
| 7308323 | Kruk et al. | Dec 2007 | B2 |
| 7308388 | Beverina et al. | Dec 2007 | B2 |
| 7313447 | Hsiung et al. | Dec 2007 | B2 |
| 7346433 | Budike, Jr. | Mar 2008 | B2 |
| 7356548 | Culp et al. | Apr 2008 | B1 |
| 7379782 | Cocco | May 2008 | B1 |
| 7383148 | Ahmed | Jun 2008 | B2 |
| 7434742 | Mueller et al. | Oct 2008 | B2 |
| 7447333 | Masticola et al. | Nov 2008 | B1 |
| 7466224 | Ward et al. | Dec 2008 | B2 |
| 7496472 | Seem | Feb 2009 | B2 |
| 7512450 | Ahmed | Mar 2009 | B2 |
| 7516490 | Riordan et al. | Apr 2009 | B2 |
| 7548833 | Ahmed | Jun 2009 | B2 |
| 7551092 | Henry | Jun 2009 | B1 |
| 7557729 | Hubbard et al. | Jul 2009 | B2 |
| 7567844 | Thomas et al. | Jul 2009 | B2 |
| 7596473 | Hansen et al. | Sep 2009 | B2 |
| 7610910 | Ahmed | Nov 2009 | B2 |
| 7626507 | LaCasse | Dec 2009 | B2 |
| 7664574 | Imhof et al. | Feb 2010 | B2 |
| 7682464 | Glenn et al. | Mar 2010 | B2 |
| 7702421 | Sullivan et al. | Apr 2010 | B2 |
| 7729882 | Seem | Jun 2010 | B2 |
| 7755494 | Melker et al. | Jul 2010 | B2 |
| 7761310 | Rodgers | Jul 2010 | B2 |
| 7774227 | Srivastava | Aug 2010 | B2 |
| 7797188 | Srivastava | Sep 2010 | B2 |
| 7819136 | Eddy | Oct 2010 | B1 |
| 7822806 | Frank et al. | Oct 2010 | B2 |
| 7856370 | Katta et al. | Dec 2010 | B2 |
| 7978083 | Melker et al. | Jul 2011 | B2 |
| 7984384 | Chaudhri et al. | Jul 2011 | B2 |
| 7986323 | Kobayashi et al. | Jul 2011 | B2 |
| 8024666 | Thompson | Sep 2011 | B2 |
| 8086047 | Penke et al. | Dec 2011 | B2 |
| 8099178 | Mairs et al. | Jan 2012 | B2 |
| 8151280 | Sather et al. | Apr 2012 | B2 |
| 8176095 | Murray et al. | May 2012 | B2 |
| 8218871 | Angell et al. | Jul 2012 | B2 |
| 8219660 | McCoy et al. | Jul 2012 | B2 |
| 8271941 | Zhang et al. | Sep 2012 | B2 |
| 8294585 | Barnhill | Oct 2012 | B2 |
| 8302020 | Louch et al. | Oct 2012 | B2 |
| 8320634 | Deutsch | Nov 2012 | B2 |
| 8334422 | Gutsol et al. | Dec 2012 | B2 |
| 8344893 | Drammeh | Jan 2013 | B1 |
| 8375118 | Hao et al. | Feb 2013 | B2 |
| 8476590 | Stratmann et al. | Jul 2013 | B2 |
| 8516016 | Park et al. | Aug 2013 | B2 |
| 8558660 | Nix et al. | Oct 2013 | B2 |
| 8639527 | Rensvold et al. | Jan 2014 | B2 |
| 8698637 | Raichman | Apr 2014 | B2 |
| 8816860 | Ophardt et al. | Aug 2014 | B2 |
| 8869027 | Louch et al. | Oct 2014 | B2 |
| 8904497 | Hsieh | Dec 2014 | B2 |
| 8936944 | Peltz et al. | Jan 2015 | B2 |
| 8947437 | Garr et al. | Feb 2015 | B2 |
| 8950019 | Loberger et al. | Feb 2015 | B2 |
| 9000926 | Hollock et al. | Apr 2015 | B2 |
| 9030325 | Taneff | May 2015 | B2 |
| 9098738 | Bilet et al. | Aug 2015 | B2 |
| 9105071 | Fletcher et al. | Aug 2015 | B2 |
| 9175356 | Peltz et al. | Nov 2015 | B2 |
| 9240111 | Scott et al. | Jan 2016 | B2 |
| 9280884 | Schultz et al. | Mar 2016 | B1 |
| 9292972 | Hailemariam et al. | Mar 2016 | B2 |
| 9320662 | Hayes et al. | Apr 2016 | B2 |
| 9370600 | DuPuis et al. | Jun 2016 | B1 |
| 9373242 | Conrad et al. | Jun 2016 | B1 |
| 9396638 | Wildman et al. | Jul 2016 | B2 |
| 9311807 | Schultz et al. | Aug 2016 | B2 |
| 9406212 | De Luca et al. | Aug 2016 | B2 |
| 9418535 | Felch et al. | Aug 2016 | B1 |
| 9418536 | Felch et al. | Aug 2016 | B1 |
| 9449219 | Bilet et al. | Sep 2016 | B2 |
| 9477543 | Henley et al. | Oct 2016 | B2 |
| 9497832 | Verberkt et al. | Nov 2016 | B2 |
| 9513364 | Hall et al. | Dec 2016 | B2 |
| 9526380 | Hamilton et al. | Dec 2016 | B2 |
| 9526806 | Park et al. | Dec 2016 | B2 |
| 9536415 | De Luca et al. | Jan 2017 | B2 |
| 9558648 | Douglas | Jan 2017 | B2 |
| 9591267 | Lipton et al. | Mar 2017 | B2 |
| 9613518 | Dunn et al. | Apr 2017 | B2 |
| 9618224 | Emmons et al. | Apr 2017 | B2 |
| 9640059 | Hyland | May 2017 | B2 |
| 9672360 | Barkan | Jun 2017 | B2 |
| 9710700 | Bilet et al. | Jul 2017 | B2 |
| 9715242 | Pillai et al. | Jul 2017 | B2 |
| 9721452 | Felch et al. | Aug 2017 | B2 |
| 9729945 | Schultz et al. | Aug 2017 | B2 |
| 9784464 | Yamamoto et al. | Oct 2017 | B2 |
| 9843743 | Lewis et al. | Dec 2017 | B2 |
| 9856634 | Rodenbeck et al. | Jan 2018 | B2 |
| 9872088 | Fadell et al. | Jan 2018 | B2 |
| 9875639 | Bone et al. | Jan 2018 | B2 |
| 9911312 | Wildman et al. | Mar 2018 | B2 |
| 9940819 | Ferniany | Apr 2018 | B2 |
| 9940826 | Divakara et al. | Apr 2018 | B1 |
| 9956306 | Brais et al. | May 2018 | B2 |
| 9986175 | Frank et al. | May 2018 | B2 |
| 10087608 | Dobizl et al. | Oct 2018 | B2 |
| 10223894 | Raichman | Mar 2019 | B2 |
| 10228837 | Hua et al. | Mar 2019 | B2 |
| 10235865 | Thyroff | Mar 2019 | B2 |
| 10251610 | Parthasarathy et al. | Apr 2019 | B2 |
| 10303843 | Bitran et al. | May 2019 | B2 |
| 10332382 | Thyroff | Jun 2019 | B2 |
| 10514817 | Hua et al. | Dec 2019 | B2 |
| 10565844 | Pourmohammad et al. | Feb 2020 | B2 |
| 10602474 | Goldstein | Mar 2020 | B2 |
| 10607147 | Raykov et al. | Mar 2020 | B2 |
| 20020111698 | Graziano et al. | Aug 2002 | A1 |
| 20020130868 | Smith | Sep 2002 | A1 |
| 20030028269 | Spriggs et al. | Feb 2003 | A1 |
| 20030030637 | Grinstein et al. | Feb 2003 | A1 |
| 20030046862 | Wolf et al. | Mar 2003 | A1 |
| 20030071814 | Jou et al. | Apr 2003 | A1 |
| 20030078677 | Hull et al. | Apr 2003 | A1 |
| 20030083957 | Olefson | May 2003 | A1 |
| 20030103075 | Rosselot | Jun 2003 | A1 |
| 20030171851 | Brickfield et al. | Sep 2003 | A1 |
| 20030214400 | Mizutani et al. | Nov 2003 | A1 |
| 20030233432 | Davis et al. | Dec 2003 | A1 |
| 20040001009 | Winings et al. | Jan 2004 | A1 |
| 20040064260 | Padmanabhan et al. | Apr 2004 | A1 |
| 20040143474 | Haeberle et al. | Jul 2004 | A1 |
| 20040153437 | Buchan | Aug 2004 | A1 |
| 20040168115 | Bauernschmidt et al. | Aug 2004 | A1 |
| 20040233192 | Hopper | Nov 2004 | A1 |
| 20040260411 | Cannon | Dec 2004 | A1 |
| 20050010460 | Mizoguchi et al. | Jan 2005 | A1 |
| 20050119767 | Kiwimagi et al. | Jun 2005 | A1 |
| 20050143863 | Ruane et al. | Jun 2005 | A1 |
| 20050267900 | Ahmed et al. | Dec 2005 | A1 |
| 20060004841 | Heikkonen et al. | Jan 2006 | A1 |
| 20060009862 | Imhof et al. | Jan 2006 | A1 |
| 20060017547 | Buckingham et al. | Jan 2006 | A1 |
| 20060020177 | Seo et al. | Jan 2006 | A1 |
| 20060028471 | Kincaid et al. | Feb 2006 | A1 |
| 20060029256 | Miyoshi et al. | Feb 2006 | A1 |
| 20060058900 | Johanson et al. | Mar 2006 | A1 |
| 20060067545 | Lewis et al. | Mar 2006 | A1 |
| 20060067546 | Lewis et al. | Mar 2006 | A1 |
| 20060077255 | Cheng | Apr 2006 | A1 |
| 20060184326 | McNally et al. | Aug 2006 | A1 |
| 20060231568 | Lynn et al. | Oct 2006 | A1 |
| 20060265664 | Simons et al. | Nov 2006 | A1 |
| 20060279630 | Aggarwal et al. | Dec 2006 | A1 |
| 20070016955 | Goldberg et al. | Jan 2007 | A1 |
| 20070055757 | Mairs et al. | Mar 2007 | A1 |
| 20070055760 | McCoy et al. | Mar 2007 | A1 |
| 20070061046 | Mairs et al. | Mar 2007 | A1 |
| 20070067062 | Mairs et al. | Mar 2007 | A1 |
| 20070088534 | MacArthur et al. | Apr 2007 | A1 |
| 20070090951 | Chan et al. | Apr 2007 | A1 |
| 20070091091 | Gardiner et al. | Apr 2007 | A1 |
| 20070101433 | Louch et al. | May 2007 | A1 |
| 20070114295 | Jenkins | May 2007 | A1 |
| 20070120652 | Behnke | May 2007 | A1 |
| 20070139208 | Kates | Jun 2007 | A1 |
| 20070216682 | Navratil et al. | Sep 2007 | A1 |
| 20070219645 | Thomas et al. | Sep 2007 | A1 |
| 20070239484 | Arond et al. | Oct 2007 | A1 |
| 20070268122 | Kow et al. | Nov 2007 | A1 |
| 20080001763 | Raja et al. | Jan 2008 | A1 |
| 20080027885 | Van Putten et al. | Jan 2008 | A1 |
| 20080036593 | Rose-Pehrsson et al. | Feb 2008 | A1 |
| 20080062167 | Boggs et al. | Mar 2008 | A1 |
| 20080099045 | Glenn et al. | May 2008 | A1 |
| 20080103798 | Domenikos et al. | May 2008 | A1 |
| 20080120396 | Jayaram et al. | May 2008 | A1 |
| 20080144885 | Zucherman et al. | Jun 2008 | A1 |
| 20080183424 | Seem | Jul 2008 | A1 |
| 20080194009 | Marentis | Aug 2008 | A1 |
| 20080198231 | Ozdemir et al. | Aug 2008 | A1 |
| 20080209342 | Taylor et al. | Aug 2008 | A1 |
| 20080222565 | Taylor et al. | Sep 2008 | A1 |
| 20080224862 | Cirker | Sep 2008 | A1 |
| 20080242945 | Gugliotti et al. | Oct 2008 | A1 |
| 20080250800 | Wetzel | Oct 2008 | A1 |
| 20080279420 | Masticola et al. | Nov 2008 | A1 |
| 20080280275 | Collopy | Nov 2008 | A1 |
| 20080303658 | Melker et al. | Dec 2008 | A1 |
| 20080306985 | Murray et al. | Dec 2008 | A1 |
| 20080320552 | Kumar et al. | Dec 2008 | A1 |
| 20090001181 | Siddaramanna et al. | Jan 2009 | A1 |
| 20090024944 | Louch et al. | Jan 2009 | A1 |
| 20090065596 | Seem et al. | Mar 2009 | A1 |
| 20090083120 | Strichman et al. | Mar 2009 | A1 |
| 20090096791 | Abshear et al. | Apr 2009 | A1 |
| 20090125337 | Abri | May 2009 | A1 |
| 20090125825 | Rye et al. | May 2009 | A1 |
| 20090144023 | Seem | Jun 2009 | A1 |
| 20090157744 | McConnell | Jun 2009 | A1 |
| 20090160673 | Cirker | Jun 2009 | A1 |
| 20090322782 | Kimchi et al. | Dec 2009 | A1 |
| 20100048167 | Chow et al. | Feb 2010 | A1 |
| 20100058248 | Park | Mar 2010 | A1 |
| 20100064001 | Daily | Mar 2010 | A1 |
| 20100070089 | Harrod et al. | Mar 2010 | A1 |
| 20100073162 | Johnson et al. | Mar 2010 | A1 |
| 20100123560 | Nix et al. | May 2010 | A1 |
| 20100134296 | Hwang | Jun 2010 | A1 |
| 20100156628 | Ainsbury et al. | Jun 2010 | A1 |
| 20100156630 | Ainsbury | Jun 2010 | A1 |
| 20100188228 | Hyland | Jul 2010 | A1 |
| 20100223198 | Noureldin et al. | Sep 2010 | A1 |
| 20100249955 | Sitton | Sep 2010 | A1 |
| 20100286937 | Hedley et al. | Nov 2010 | A1 |
| 20100318200 | Foslien et al. | Dec 2010 | A1 |
| 20100324962 | Nesler et al. | Dec 2010 | A1 |
| 20110010654 | Raymond et al. | Jan 2011 | A1 |
| 20110057799 | Taneff | Mar 2011 | A1 |
| 20110077779 | Fuller et al. | Mar 2011 | A1 |
| 20110083094 | Laycock et al. | Apr 2011 | A1 |
| 20110087988 | Ray et al. | Apr 2011 | A1 |
| 20110112854 | Koch et al. | May 2011 | A1 |
| 20110126111 | Gill et al. | May 2011 | A1 |
| 20110154426 | Doser et al. | Jun 2011 | A1 |
| 20110161124 | Lappinga et al. | Jun 2011 | A1 |
| 20110169646 | Raichman | Jul 2011 | A1 |
| 20110184563 | Foslien et al. | Jul 2011 | A1 |
| 20110202467 | Hilber et al. | Aug 2011 | A1 |
| 20110273298 | Snodgrass et al. | Nov 2011 | A1 |
| 20110291841 | Hollock et al. | Dec 2011 | A1 |
| 20110298301 | Wong et al. | Dec 2011 | A1 |
| 20110316703 | Butler et al. | Dec 2011 | A1 |
| 20110320054 | Brzezowski | Dec 2011 | A1 |
| 20120022700 | Drees et al. | Jan 2012 | A1 |
| 20120039503 | Chen et al. | Feb 2012 | A1 |
| 20120062382 | Taneff | Mar 2012 | A1 |
| 20120075464 | Derenne et al. | Mar 2012 | A1 |
| 20120109988 | Li et al. | May 2012 | A1 |
| 20120112883 | Wallace et al. | May 2012 | A1 |
| 20120131217 | Delorme et al. | May 2012 | A1 |
| 20120158185 | El-Mankabady et al. | Jun 2012 | A1 |
| 20120216243 | Gill et al. | Aug 2012 | A1 |
| 20120224057 | Gill et al. | Sep 2012 | A1 |
| 20120259466 | Ray et al. | Oct 2012 | A1 |
| 20120262472 | Garr et al. | Oct 2012 | A1 |
| 20120272146 | D'souza et al. | Oct 2012 | A1 |
| 20120291068 | Khushoo et al. | Nov 2012 | A1 |
| 20120303652 | Tseng | Nov 2012 | A1 |
| 20120310418 | Harrod et al. | Dec 2012 | A1 |
| 20130055132 | Foslien | Feb 2013 | A1 |
| 20130060794 | Puttabasappa et al. | Mar 2013 | A1 |
| 20130082842 | Balazs et al. | Apr 2013 | A1 |
| 20130086152 | Hersche et al. | Apr 2013 | A1 |
| 20130091631 | Hayes et al. | Apr 2013 | A1 |
| 20130110295 | Zheng et al. | May 2013 | A1 |
| 20130122807 | Tenarvitz et al. | May 2013 | A1 |
| 20130169681 | Rasane et al. | Jul 2013 | A1 |
| 20130184880 | McMahon | Jul 2013 | A1 |
| 20130187775 | Marsden et al. | Jul 2013 | A1 |
| 20130204570 | Mendelson et al. | Aug 2013 | A1 |
| 20130229276 | Hunter | Sep 2013 | A1 |
| 20130268293 | Knudson et al. | Oct 2013 | A1 |
| 20130289774 | Day et al. | Oct 2013 | A1 |
| 20140032157 | Khiani | Jan 2014 | A1 |
| 20140040998 | Hsieh | Feb 2014 | A1 |
| 20140046490 | Foslien et al. | Feb 2014 | A1 |
| 20140046722 | Rosenbloom et al. | Feb 2014 | A1 |
| 20140058539 | Park | Feb 2014 | A1 |
| 20140167917 | Wallace et al. | Jun 2014 | A2 |
| 20140207291 | Golden et al. | Jul 2014 | A1 |
| 20140292518 | Wildman et al. | Oct 2014 | A1 |
| 20140307076 | Deutsch | Oct 2014 | A1 |
| 20140309757 | Le Sant et al. | Oct 2014 | A1 |
| 20140316582 | Berg-Sonne et al. | Oct 2014 | A1 |
| 20140320289 | Raichman | Oct 2014 | A1 |
| 20140342724 | Hill et al. | Nov 2014 | A1 |
| 20140375457 | Diaz | Dec 2014 | A1 |
| 20150025329 | Amarasingham et al. | Jan 2015 | A1 |
| 20150032264 | Emmons et al. | Jan 2015 | A1 |
| 20150056909 | Chien | Feb 2015 | A1 |
| 20150070174 | Douglas | Mar 2015 | A1 |
| 20150077258 | Nelson et al. | Mar 2015 | A1 |
| 20150113462 | Chen et al. | Apr 2015 | A1 |
| 20150153918 | Chen et al. | Jun 2015 | A1 |
| 20150161874 | Thyroff et al. | Jun 2015 | A1 |
| 20150167995 | Fadell et al. | Jun 2015 | A1 |
| 20150168949 | Hua et al. | Jun 2015 | A1 |
| 20150194043 | Dunn et al. | Jul 2015 | A1 |
| 20150198707 | Al-Alusi | Jul 2015 | A1 |
| 20150212717 | Nair et al. | Jul 2015 | A1 |
| 20150213222 | Amarasingham et al. | Jul 2015 | A1 |
| 20150213379 | Nair et al. | Jul 2015 | A1 |
| 20150216369 | Hamilton et al. | Aug 2015 | A1 |
| 20150253748 | Brun et al. | Sep 2015 | A1 |
| 20150281287 | Gill et al. | Oct 2015 | A1 |
| 20160061476 | Schultz et al. | Mar 2016 | A1 |
| 20160061477 | Schultz et al. | Mar 2016 | A1 |
| 20160061794 | Schultz et al. | Mar 2016 | A1 |
| 20160061795 | Schultz et al. | Mar 2016 | A1 |
| 20160063833 | Schultz et al. | Mar 2016 | A1 |
| 20160066067 | Schultz et al. | Mar 2016 | A1 |
| 20160116181 | Aultman et al. | Apr 2016 | A1 |
| 20160139067 | Grace | May 2016 | A1 |
| 20160140831 | Hermann | May 2016 | A1 |
| 20160253897 | Wildman et al. | Sep 2016 | A1 |
| 20160255516 | Hill et al. | Sep 2016 | A1 |
| 20160298864 | Ekolind et al. | Oct 2016 | A1 |
| 20160306934 | Sperry et al. | Oct 2016 | A1 |
| 20160314683 | Felch et al. | Oct 2016 | A1 |
| 20160328948 | Ferniany | Nov 2016 | A1 |
| 20160335731 | Hall | Nov 2016 | A1 |
| 20160367925 | Blackley | Dec 2016 | A1 |
| 20170024986 | Austin | Jan 2017 | A1 |
| 20170193792 | Bermudez Rodriguez et al. | Jul 2017 | A1 |
| 20170256155 | Sengstaken, Jr. | Sep 2017 | A1 |
| 20170280949 | Wildman et al. | Oct 2017 | A1 |
| 20170294106 | Thyroff | Oct 2017 | A1 |
| 20170365024 | Koch et al. | Dec 2017 | A1 |
| 20180016773 | Chandler et al. | Jan 2018 | A1 |
| 20180047277 | Thyroff | Feb 2018 | A1 |
| 20180151054 | Pi | May 2018 | A1 |
| 20180218591 | Easter | Aug 2018 | A1 |
| 20180293038 | Meruva et al. | Oct 2018 | A1 |
| 20180293873 | Liu | Oct 2018 | A1 |
| 20180301014 | Worral et al. | Oct 2018 | A1 |
| 20180313695 | Shim et al. | Nov 2018 | A1 |
| 20180365957 | Wright et al. | Dec 2018 | A1 |
| 20190051138 | Easter | Feb 2019 | A1 |
| 20190139395 | Rogachev et al. | May 2019 | A1 |
| 20190209719 | Andersen et al. | Jul 2019 | A1 |
| 20200009280 | Kupa et al. | Jan 2020 | A1 |
| 20200074836 | Kolavennu et al. | Mar 2020 | A1 |
| 20200090089 | Aston et al. | Mar 2020 | A1 |
| 20200146557 | Cheung et al. | May 2020 | A1 |
| 20200200420 | Nayak et al. | Jun 2020 | A1 |
| Number | Date | Country |
|---|---|---|
| 2387100 | Nov 2003 | CA |
| 2538139 | Mar 2005 | CA |
| 103110410 | May 2013 | CN |
| 103970977 | Aug 2014 | CN |
| 105116848 | Dec 2015 | CN |
| 108961714 | Dec 2018 | CN |
| 110009245 | Jul 2019 | CN |
| 110084928 | Aug 2019 | CN |
| 110827457 | Feb 2020 | CN |
| 1669912 | Jun 2006 | EP |
| 2310981 | Apr 2011 | EP |
| 7085166 | Mar 1995 | JP |
| 11024735 | Jan 1999 | JP |
| 11317936 | Nov 1999 | JP |
| 2001356813 | Dec 2001 | JP |
| 2005242531 | Sep 2005 | JP |
| 2005311563 | Nov 2005 | JP |
| 1172747 | Aug 2012 | KR |
| 101445367 | Oct 2014 | KR |
| 1499081 | Mar 2015 | KR |
| 9621264 | Nov 1996 | WO |
| 2004029518 | Apr 2004 | WO |
| 2005045715 | May 2005 | WO |
| 2008152433 | Dec 2008 | WO |
| 2008157755 | Dec 2008 | WO |
| 2009012319 | Jan 2009 | WO |
| 2009079648 | Jun 2009 | WO |
| 2010106474 | Sep 2010 | WO |
| 2011025085 | Mar 2011 | WO |
| 2011043732 | Apr 2011 | WO |
| 2011057173 | May 2011 | WO |
| 2011123743 | Oct 2011 | WO |
| 2013062725 | May 2013 | WO |
| 2013178819 | Dec 2013 | WO |
| 2014009291 | Jan 2014 | WO |
| 2014098861 | Jun 2014 | WO |
| 2014135517 | Sep 2014 | WO |
| 2016123536 | Aug 2016 | WO |
| 2017057274 | Apr 2017 | WO |
| 2019046580 | Mar 2019 | WO |
| 2020024553 | Feb 2020 | WO |
| Entry |
|---|
| Olken et al., “Object Lessons Learned from a Distributed System for Remote Building Monitoring and Operation,” ACM SIGPLAN Notices, vol. 33, No. 10, pp. 284-295, Oct. 1998. |
| Proliphix, Inc., “Proliphix IP Devices: HTTP API,” 28 pages, Jan. 23, 2006. |
| Proliphix, Inc., Remote Management User Guide, 12 pages, prior to Aug. 27, 2007. |
| Richard Rogan et al., “Smart and Final Food Stores: A Case Study in Web Based Energy Information and Collection,” Web Based Energy Information and Control Systems: Case Studies and Application, Chapter 6, p. 59-64, 2005. |
| Sharp, “Actius AL3DU 3D LC Display High Performance 3D Visualization,” 2 pages, prior to Mar. 17, 2006. |
| So et al., “Building Automation on the Information Superhighway,” ASHRAE (American Society of Heating Refrigerating, and Air Conditioning) Transactions, vol. 104, Part 2, pp. 176-191, 1998. |
| So et al., “Building Automation Systems on the Internet,” Facilities vol. 15, No. 5/6, pp. 125-133, May/Jun. 1997. |
| Talon, “Raptor Controller,” 6 pages, Oct. 2003. |
| Talon, “Workstation Software,” 4 pages, Nov. 2002. |
| Trane, “System Programming, Tracer Summit Version 14, BMTW-SVP01D-EN,” 623 pages, 2002. |
| Lucid Design Group, Inc., “Building Dashboard,” 2 pages, Printed May 30, 2013. |
| “America's Largest Managed Security Services Provider Launches Comprehensive, Integrated Covid-19 Safety Program for Office Buildings and Suites,” KastleSafeSpaces, 5 pages, May 11, 2020. |
| “Biometric Door Reader With Body Temperature Detection,” Kintronics, 9 pages, accessed May 21, 2020. |
| “Body Surface Temperature Screening with Alarm Function TVS-200IS/TVS-500IS,” Nippon Avionics Co., 3 pages, accessed May 21, 2020. |
| “BriefCam announces video analytics innovation for contact tracing, physical distancing, occupancy management and face mask detection,” BriefCam LTD, 11 pages, Jun. 5, 2020. |
| “Thermal Imaging SmartPhone Can Be used for Temperature Screening of People,” CAT, 3 pages, accessed Jul. 13, 2020. |
| “Contact Tracing Now Available on Identiv's Hirsch Velocity Access Control Platform,” IDENTIV, 5 pages, May 21, 2020. |
| Silva et al., “Cough localization for the detection of respiratory diseases in pig houses,” ScienceDirect, 7 pages, May 28, 2008. |
| Oey et al., “Evaluation of Isolation Compliance Using Real Time Video in Critical Care,” North Shore University Hospital, 1 page, Oct. 9, 2015. |
| “Facial Attendace System With Temperature Screening Now in India,” IANS, 5 pages, Mar. 19, 2020. |
| “Plan to Re-Open,” EHIGH, 16 pages, accessed Jun. 13, 2020. |
| “How Smarter AI-Powered Cameras Can Mitigate the Spread of Wuhan Novel,” AnyConnect, 22 pages, 2020. |
| “How to fight COVID-19 with machine learning,” DataRevenue, 20 pages, accessed May 25, 2020. |
| “INNControl 5,” Honeywell, 2 pages, Aug. 8, 2018. |
| “IP Door Access Control,” Kintronics, 21 pages, 2014. |
| “Kogniz AI Health Response Platform,” KOGNIZ, 9 pages, accessed May 21, 2020. |
| “Machine Learning Could Check If You're Social Distancing Properly at Work,” MIT Technology Review, 7 pages, Apr. 17, 2020. |
| Punn et al., “Monitoring COVID-19 social distancing with person detection and tracking via fine-tuned YOLO v3 and Deepsort techniques,” 10 pages, May 6, 2020. |
| “NEC launches dual face biometric and fever detection system for access control,” Biometric Update, 4 pages, May 8, 2020. |
| “Remote temperature monitoring,” AXIS Communication, 10 pages, 2014. |
| “FebriEye-AI Based Thermal Temperature Screening System,” vehant, 1 page, 2020. |
| “See the World in a New Way Hikvision Thermal Cameras,” HIKVision, 12 pages, 2017. |
| Allain, “Trying out the iPhone Infrared Camera: The FLIR One,” Wired, 15 pages, 2014. |
| Dasgupta, “Your voice may be able to tell you if you have Covid,” Hindustan Times, 4 pages, Apr. 16, 2020. |
| Ganguty, “Gurugram-based startup Staqu has modified AI-powered JARVIS to battle coronavirus,” YourStory, 7 pages, Mar. 31, 2020. |
| Iuliana Bocicor et al. “Wireless Sensor Network based System for the Prevention of Hospital Acquired Infections”, arxiv.org, Cornell University Ithaca, NY 14853, May 2, 2017, XP080947042, (Abstract). |
| Shhedi Zaid Ali et al., “Traditional and ICT Solutions for Preventing the Hospital Acquired Infection”, 2015 20th International Conference on Control Systems and Computer Science, IEEE, May 27, 2015, pp. 867-873, XP033188038. |
| Extended European Search Report, EP application No. 20151295.1, pp. 13, dated May 26, 2020. |
| U.S. Appl. No. 14/109,496, filed Dec. 17, 2013. |
| www.geappliances.com/home-energy-manager/about-energy-monitors.htm, “Energy Monitor, Home Energy Monitors, GE Nucleus,” 2 pages, printed Jan. 15, 2013. |
| www.luciddesigngroup.com/network/apps.php#homepage, “Lucid Design Group—Building Dashboard Network—Apps,” 7 pages, Jan. 15, 2013. |
| Preuveneers et al., “Intelligent Widgets for Intuitive Interaction and Coordination in Smart Home Environments,” IEEE Eighth International Conference on Intelligent Environments, pp. 157-164, 2012. |
| Wu et al., “A Web 2.0 Based Scientific Application Framework,” 7 pages, prior to Jul. 24, 2014. |
| “4.0 Today's Activities, The Home Dashboard,” CRBM info@hand website, 46 pages, prior to Apr. 25, 2013. |
| “Free Facilities Dashboards,” eSight Energy Website, 2 pages, prior to Apr. 25, 2013. |
| Alerton Building Controls, Gallery Prints, 7 pages, Dec. 19, 2013. |
| Carter, “Industrial Energy Management Dashboards Require a Toolkit,” Cross Automation, 11 pages, Nov. 4, 2013. |
| U.S. Appl. No. 14/169,071, filed Jan. 30, 2014. |
| U.S. Appl. No. 14/169,083, filed Jan. 30, 2014. |
| U.S. Appl. No. 14/461,188, filed Aug. 15, 2014. |
| U.S. Appl. No. 14/482,607, filed Sep. 10, 2014. |
| e-homecontrols.com, “e-Home Controls Website,” link to actual website no longer works, 1 page, prior to Dec. 19, 2013. |
| http://www.ccbac.com, “C&C (/)—Omniboard,” 5 pages, Dec. 19, 2013. |
| http://www.domcontroller.com/en/, “DomController Home Automation Software—Control Anything from Anywhere,” 11 pages, printed Jan. 6, 2015. |
| http://www.novar.com/ems-bas/opus-building-automation-system, “Novar OPUS BAS,” 1 page, prior to Feb. 13, 2013. |
| Instituto Superior Tecnico, “A 3D Interactive Environment for Automated Building Control,” Master's Dissertation, 120 pages, Nov. 2012. |
| Panduit Corp., “Enable a Building Automation with Panduit Enterprise Solutions,” 4 pages, Nov. 2012. |
| “WEBs-AX Web-Enabled Building Solutions,” sales brochure, Honeywell International Inc., Mar. 2009. |
| “Attune Advisory Services,” press release, Honeywell International Inc., Mar. 20, 2012. |
| EnteliWEB product from Delta Controls, web pages retrieved on May 9, 2013 from http://deltacontrols.com/products/facilities-management/supervisory-software et seq. by the Internet Archive at web.archive.org. |
| “BACnet Protocol Implementation Conformance Statement” for enteliWEB, Delta Controls, Jul. 17, 2013. |
| Castle, “7 Software Platforms that Make Building Energy Management Easy,” http://greentechadvocates.com/2012/11/28/7-software-platforms-that-make-building-energy-managment-easy/, Nov. 28, 2012. |
| EnteliWEB catalog sheet, Delta Controls, Inc., 2012. |
| EnteliWEB catalog sheet, Delta Controls., 2010. |
| “Intelligent Building Management Systems in Miami,” Advanced Control Corp., Mar. 7, 2013. |
| “The Ohio State University,” BACnet International Journal, vol. 5, p. 4, Jan. 2013. |
| Bobker et al., “Operational Effectiveness in Use of BAS,” Proceedings of the 13th International Conference for Enhanced Building Operations, Oct. 8, 2013. |
| Castelo, “A 3D Interactive Environment for Automated Building Control,” Elsevier, Nov. 8, 2012. |
| “Creston Special Report: How Intelligent building management solutions are reducing operational costs,” Creston, 2012. |
| “Building Automation Software Solutions,” Iconics, 2013. |
| Lacey, “The Top 10 Software Vendors Connecting Smart Buildings to the Smart Grid,” http://www.greentechmedia.com/articles/read/the-top-10-companies-in-enterprise-smart-grid, Jul. 18, 2013. |
| “NiagraAX Product Model Overview,” Tridium, Inc., 2005. |
| “An Overview of NiagraAX: A comprehensive software platform designed to create smart device applications,” Tridium, Inc., 2005. |
| “Phoenix Controls Portal,” Phoenix Controls, Inc., 2013. |
| Quirk, “A Brief History of BIM,” Arch Daily, Dec. 7, 2012. |
| Samad et al., “Leveraging the Web: A Universal Framework for Building Automation,” Proceedings of the 2007 American Control Conference, Jul. 11, 2007. |
| Sinha et al., “9 Key attributes of energy dashboards and analytics tools,” https://www.greenbiz.com/blog/2013/08/28/9-key-attributes-energy-dashboards-and=analytics-tools, Aug. 28, 2013. |
| Sinopoli, “Dashboards for Buildings,” http://www/automatedbuildings.com/news/dec10/articles/sinopoli/101119034404sinopoli.html, Dec. 2010. |
| Sinopoli, “Modeling Building Automation and Control Systems,” http://www.automatedbuildings.com/news/jun13/articles/sinopoli/130521122303sinopoli.html, Jun. 2013. |
| Zito, “What is Tridium Part 1,” http://blog.buildingautomationmonthly.com/what-is-tridium/, May 12, 2013. |
| Zito, “What is Tridium Part 2,” http://blog.buildingautomationmonthly.com/tridium-part-2/, Sep. 10, 2013. |
| Search Report and Written Opinion from related International PCT Application No. PCT/US2018/025189 dated Jul. 17, 2018 (12 pages). |
| “Data analytics and smart buildings increase comfort and energy efficiency”, https://www.microsoft.com/itshowcase/Article/Content/845/Data-analytics-and-smart-buildings-increase-comfort-and-energy-efficiency, Dec. 19, 2016, 8 pages. |
| Donnelly, “Building Energy Management: Using Data as a Tool”, http://www.buildingefficiencyinitiative.org/sites/default/files/legacy/InstituteBE/media/Library/Resources/Existing-Building-Retrofits/Using-Building-Data-as-a-Tool.pdf, Oct. 2012, 9 pages. |
| “ASHRAE Dashboard Research Project,” 29 pages, Aug. 28, 2008. |
| “Energy Manager User Guide,” Release 3.2, Honeywell, 180 pages, 2008. |
| “Fuzzy Logic Toolbox 2.1, Design and Stimulate Fuzzy Logic Systems,” The MathWorks, 2 pages, May 2004. |
| “Junk Charts, Recycling Chartjunk as junk art,” 3 pages, Oct. 2, 2006. |
| “Model Predictive Control Toolbox 2, Develop Internal Model-Based Controllers for Constrained Multivariable Processes,” The MathWorks, 4 pages, Mar. 2005. |
| Honeywell, “Product Guide 2004,” XP-002472407, 127 pages, 2004. |
| “Statistics Toolbox, for Use with Matlab,” User's Guide Version2, The MathWorks, 408 pages, Jan. 1999. |
| “Vykon Energy Suite Student Guide,” Tridium Inc., 307 pages, Mar. 3, 2006. |
| “Web Based Energy Information Systems for Energy Management and Demand Response in Commercial Buildings,” California Energy Commission, 80 pages, Oct. 2003. |
| Andover Controls, Network News, vol. 2, No. 2, 8 pages, 1997. |
| Andover Controls World, 4 pages, Spring 1997. |
| Bell, Michael B. et al., “Early Event Detection—Results from A Prototype Implementation,” AICHE Spring National Meeting, 15 pages, Apr. 2005. |
| CADGraphics, “The CADGraphics User's Guide,” 198 pages, 2003. |
| Carrier Comfort Network CCN Web, “Web Browser User Interface to the Carrier Comfort Network,” 2 pages, 2002. |
| Carrier Comfort Network CCN Web, Overview and Configuration Manual, 134 pages, Apr. 2006. |
| Carrier Comfort Network CCN Web, Product Data, 2 pages, Apr. 2006. |
| Carrier, “i-Vu Powerful and Intuitive Front End for Building Control,” 2 pages, Aug. 2005. |
| Carrier, “i-Vu Web-Based Integrated Control System,” 3 pages, 2005. |
| Carrier, Demo Screen Shots, 15 pages, prior to Aug. 27, 2007. |
| Carrier, i-Vu CCN 4.0, Owner's Guide, 20 pages, Jul. 2007. |
| Carrier, i-Vu CCN, 7 pages, 2007. |
| Chen, Tony. F., “Rank Revealing QR Factorizations,” Linear Algebra and It's Applications, vol. 88-89, p. 67-82, Apr. 1987. |
| Circon, “i-Browse Web-Based Monitoring and Control for Facility Management,” 2 pages, prior to Aug. 27, 2007. |
| Published Australian Application 2009904740, 28 pages, Application Filed on Sep. 29, 2009. |
| Echelon, “Energy Control Solutions with the i.Lon SmartServer,” 4 pages, 2007. |
| Echelon, “i.Lon 100e3 Internet Server Models 72101R-300, 72101R-308, 72102R-300, 72103-R300 . . . ” 5 pages, copyright 2002-2007. |
| Echelon, “i.Lon 100e3 Internet Server New Features,” 15 pages, Sep. 2006. |
| Echelon, “i.Lon SmartServer,” 5 pages, 2007. |
| Honeywell News Release, “Honeywell's New Sysnet Facilities Integration System for Boiler Plant and Combustion Safety Processes,” 4 pages, Dec. 15, 1995. |
| Honeywell, “Excel Building Supervisor—Integrated R7044 and FS90 Ver. 2.0,” Operator Manual, 70 pages, Apr. 1995. |
| Honeywell, “Introduction of the S7350A Honeywell WebPAD Information Appliance,” Home and Building Control Bulletin, 2 pages, Aug. 29, 2000; Picture of WebPad Device with touch screen, 1 Page; and screen shots of WebPad Device, 4 pages. |
| Honeywell, Excel 15B W7760B Building Manager Release 2.02.00, Installation Instructions, 28 pages, Dec. 2004. |
| Honeywell, The RapidZone Solution, Excel 5000 Open System, Application Guide, 52 pages, Jan. 2004. |
| http://pueblo.lbl.gov/˜olken . . . , “Remote Building Monitoring and Operations Home Page,” 5 pages, prior to Aug. 27, 2007. |
| http://www.commercial.carrier.com/commercial/hvac/productdescription . . . , “Carrier: i-Vu CCN,” 1 page, printed Mar. 11, 2008. |
| http://www.commercial.carrier.com/commercial/hvac/productdescription . . . , “Carrier: 33CSCCNWEB-01 CCN Web Internet Connection to the Carrier Comfort Network,” 1 page, printed Mar. 11, 2008. |
| http://www.docs.hvacpartners.com/idc/groups/public/documents/techlit/gs-controls-ivuccn.rtf, “Products,” 5 pages, printed Jul. 3, 2007. |
| http://www.lightstat.com/products/istat.asp, Lightstat Incorporated, “Internet Programmable Communicating Thermostats,” 1 page, printed Mar. 13, 2007. |
| http://www.sharpsystems.com/products/pc_notebooks/actius/rd/3d/, “Actius RD3D Desktop Replacement Notebook with Industry-Breakthrough 3D Screen,” Sharp, 1 page, printed Jun. 16, 2005. |
| http://www2.sims.berkeley.edu/courses/is213/s06/projects/lightson;final.html, “Lights on a Wireless Lighting Control System,” 11 pages, printed Mar. 22, 2007. |
| I.Lon 100e3 Internet Server, 1 page, prior to Aug. 27, 2007. |
| I.Lon, SmartServer, 2 pages, prior to Aug. 27, 2007. |
| I-stat, Demo Screen Shots, 9 pages, printed Mar. 13, 2007. |
| I-stat, The Internet Programmable Thermostat, 2 pages, prior to Aug. 27, 2007. |
| Jeffrey Ball, “Green Goal of ‘Carbon Neutrality’ Hits Limit,” TheWall Street Journal, 7 pages, Dec. 30, 2008. |
| Johnson Controls, Network Integration Engine (NIE) 3 pages, Nov. 9, 2007. |
| Johnson Controls, Network Integration Engine (NIE), Product Bulletin, pp. 1-11, Jan. 30, 2008. |
| Kourti, Theodora, “Process Analysis and Abnormal Situation Detection: From Theory to Practice,” IEEE Control Systems Magazine, p. 10-25, Oct. 2002. |
| Mathew, Paul A., “Action-Oriented Benchmarking, Using CEUS Date to Identify and Prioritize Efficiency Opportunities in California Commercial Buildings,” 26 pages, Jun. 2007. |
| Morrison, Don et al., “The Early Event Detection Toolkit,” Honeywell Process Solutions, 14 pages, Jan. 2006. |
| Narang, “WEBARC: Control and Monitoring of Building Systems Over the Web,” 53 pages, May 1999. |
| CA Second Office Action, CA Application No. 3,054,216, dated Jul. 8, 2021 (5 pgs). |
| Number | Date | Country | |
|---|---|---|---|
| 20200074836 A1 | Mar 2020 | US |
| Number | Date | Country | |
|---|---|---|---|
| 62727480 | Sep 2018 | US |
