The present invention relates generally to wireless biosensors.
It is an object of this invention to remotely monitor physiological parameters of a patient, and—in real time—to analyze data gathered by this monitoring to recognize patterns that indicate the need for intervention, and then—also in real time—to deliver therapeutic stimuli to the patient via a transducer in a handheld or wearable device.
There are many situations where therapeutic stimuli can be used to help a person mitigate or prevent undesirable health events or unhealthy behaviors. Such stimuli are often described in the broader sense as “therapeutic interventions.” The present invention may be implemented with a network architecture that enables automatic detection of an undesirable state and enables a corresponding appropriate health intervention to be delivered in real-time.
As one example, consider a person with Autism Spectrum Disorder (ASD) that may experience episodes of extreme stress or panic attacks, better-known as “meltdowns.” Such episodes are associated with a high degree of electrodermal activity (EDA), otherwise known as skin conductance or “galvanic skin response.” It has been demonstrated that the use of touch, weighted blankets or other haptic stimuli can be used to provide calming comfort in these situations and reduce recovery time. [Champagne, T., and Mullen, B. (2005) The Weighted Blanket: Use and Research in Psychiatry. MAOT 2005] According to principles of this invention, this meltdown state can be automatically detected through the use of wearable physiological sensors to detect electrodermal activity, and a therapeutic stimulus can be automatically applied through the use of wearable transducers.
In another example, consider a veteran suffering from post-traumatic stress disorder (“PTSD”). The onset of a PTSD “dream episode” causes a pattern of physiological changes in the veteran. In an illustrative implementation of this invention, this pattern is recognized in real time. A wristband worn by the veteran delivers, in real time, a vibration or sound to the veteran. This therapeutic stimulus (vibration or sound) may trigger a startle response that helps the veteran “snap out” of the episode and re-engage with reality.
Also, consider the problem of apnea in very young infants. At this stage of development, the central nervous system is not always fully developed and some infants will temporarily stop breathing (apnea), which can lead to death if breathing is not restored. This invention may be implemented in such a way that one or more physiological sensors are used to detected signs of apnea (reduction in breathing, change in heart rate, and decrease in blood oxygen saturation). These signs can be recognized in real time and a foot vibrator activated to trigger the infant to resume breathing.
Or, for example, consider a patient who has become addicted to a “prescription opioid” such as morphine, codeine, oxycodone, hydrocodone, hydromorphone, or Tramadol® (a synthetic analog of codeine), and is experiencing the onset of a drug craving. Changes in the patient's physiological parameters indicate the start of this craving. In an illustrative implementation of this invention, these changes may be recognized in real time. A message may be sent in real time to the patient's cell phone, with a picture of the patient's daughter and a message that says “Dad, you promised not to take morphine again. I love you”. These therapeutic stimuli (the picture and message), delivered in this timely way, may help the patient resist the drug craving.
In some applications, the detection or classification of a particular physiological state or condition can be implemented by monitoring a single physiological parameter (e.g. electrodermal activity) and using a simple threshold function. However, in most cases which involve multiple sensor inputs and multiple degrees of freedom, it is necessary to use one or more processors that can analyze the physiological data using a real-time machine learning algorithm that can implement state classification in a high multidimensional space.
In an exemplary implementation of this invention, the patient wears a wearable washable biosensor, such as that described in United States non-provisional patent application Ser. No. 12/386,348 filed on Apr. 16, 2009.
This invention may be implemented using different kinds of physiological data. For example, data regarding heart beat, breathing, skin conductance, biopotentials, body core temperature, optical reflectance of blood vessels or limb movement may be monitored.
In exemplary implementations of this invention, a patient (e.g. an infant) wears a battery-powered, comfortable biosensor for real-time, remote monitoring of physiological parameters such as electro-dermal activity (EDA), heart rate, and blood oxygen saturation. The biosensor wirelessly transmits this data using the IEEE 802.15.4 protocol, which is well-suited for low-power, low-duty cycle transmission. In proximity to the patient is a small data hub that acts as a bridge between IEEE 802.15.4 and Bluetooth™ signals. The data hub transmits the data using a Bluetooth™ radio to a Bluetooth™-enabled device (such as a smartphone, computer or radio base station) for further transmission over a second wireless network or the Internet. A remote processor analyzes this transmitted physiological data to identify states that require intervention. For example, the processor may recognize, in real time, a pattern in the data that indicates the onset of apnea in a premature infant. The processor outputs instructions for delivering therapeutic stimuli to a patient, in real time, via at least one transducer that is also worn by the patient.
Alternately, this invention may be implemented so that Bluetooth™ is incorporated directly in the wearable sensor, thus allowing the sensors to communicate directly to the mobile phone. In this approach, the sensor can perform some basic filtering of the sensor data and only wirelessly connect to the mobile phone periodically (two times per second) in order to save power. The phone acts as the wireless “hub” to relay data to an external server using the external cell phone data network or via SMS or MMS messaging. This approach is advantageous in some contexts in which a patient is ambulatory.
According to principles of this invention, a transducer used to deliver therapeutic stimuli may be wearable, such as a small vibrating motor. Alternately, the transducers used to deliver therapeutic stimuli may be housed in a handheld device, such as a mobile phone. For example, a remote server can send an MMS message to the patient via a mobile phone, in the form of an image or audio file that has a psychological impact on the patient.
In some embodiments of this invention, the processor may engage in machine learning to improve pattern recognition. This machine learning may be used to customize pattern identification for a specific patient or specific medical condition. The machine learning may employ one or more methods. However, the exemplary implementation makes use of Dynamic Bayesian Networks, which is particularly suited for analyzing temporal characteristics in high-dimensional parameter spaces.
This invention may be implemented in such a way that it can handle multiple patients at the same time. In such an embodiment, the remote processor analyzes data gathered from multiple patients and multiple hub devices. When the processor recognizes a state that requires immediate intervention, therapeutic stimuli are delivered to the corresponding patient in real time.
This invention may be implemented in such a way as to function as a “reverse hotline”. In this implementation, upon recognition of a pattern requiring intervention, the system notifies one or more humans, and facilitates the delivery of a live message from at least one person directly to the patient. This live human social input delivered in real time may have a powerful therapeutic effect.
Here are some ways that this invention may be implemented:
This invention may be implemented as a system comprising, in combination: (a) at least one sensor for measuring physiological data, (b) at least one hub adapted for receiving wireless signals that are communications directly or indirectly from said sensor and for transmitting wireless signals that are communications to a network, wherein said received signals comply with a wireless protocol that is not identical to the wireless protocol with which said transmitted signals comply, and (c) one or more processors for recognizing at least one pattern in said physiological data, and based upon said recognition, for outputting instructions for at least one transducer to produce therapeutic stimuli. Furthermore: (1) at least some of said transmitted signals or said received signals may comply with a Bluetooth™ wireless protocol, (2) at least one said processor may be adapted for analyzing data transmitted by a plurality of hubs, (3) said pattern may be indicative of a drug craving or panic state, (4) said therapeutic stimuli may comprise a verbal message, (5) said therapeutic stimuli may comprise haptic or thermal stimuli, (6) at least one said sensor may be wearable, (7) said system may further comprise at least one said transducer, (8) at least one said transducer may be housed or carried in a wearable device, (9) at least one of said processors may be further adapted for machine learning that uses at least part of said physiological data as training data, (10) said system may be adapted to repeatedly: (I) accept updated physiological data, (II) use at least some of said updated physiological data as training data for machine learning, (III) based on said machine learning, revise at least one pattern recognition algorithm, and (IV) use said revised pattern recognition algorithm in a processor housed onboard a wearable or handheld device, (11) one or more of said processors may be further adapted for analyzing said physiological data regarding a user, accessing audio or visual content stored in machine-readable form in memory, and outputting instructions for at least one said transducer to deliver said content to said user in a form perceptible to said user, (12) said system may be adapted for performing said pattern recognition in real time, and (13) said system may be further adapted for soliciting at least one live human communication and outputting instructions for said communication to be delivered to a person wearing said at least one sensor.
This invention may be implemented as a system comprising, in combination: (a) at least one sensor for measuring physiological data regarding a user, and (b) one or more processors for recognizing at least one pattern in said physiological data, and based upon said recognition, for outputting instructions for obtaining at least one live human communication and causing it to be delivered to said user in a form perceptible by said user. Furthermore, said pattern recognition and output of instructions may occur in real time relative to at least one said pattern.
This invention may be implemented as a method comprising the following steps, in combination: (a) the receipt by a hub of wireless signals directly or indirectly from a sensor, (b) the transmission by said hub of wireless signals, wherein said received signals comply with a wireless protocol that is not identical to the wireless protocol with which said transmitted signals comply, and (c) the processing of said transmitted signals in such a way as to recognize at least one pattern in physiological data measured by said sensor, and based on said recognition, to output instructions for at least one transducer to deliver therapeutic stimuli to said user, wherein said pattern recognition occurs in real time. This method may further comprise the step of performing machine learning that uses at least part of said physiological data as training data.
A processor 11 in a remotely located computer analyzes this physiologic data in real time, after it has been transmitted over the wireless network 9. The processor 11 uses pattern detection algorithms to detect patterns in this data that indicate that therapeutic intervention is appropriate. For example, a processor 11 may detect a pattern that indicates that the user is experiencing the onset of a drug craving, or the onset of a PTSD “dream” episode, or the onset of an autistic meltdown episode, or a panic state.
Upon detecting such a pattern, the processor 11 outputs instructions for delivering therapeutic stimuli to the user via at least one transducer in a handheld or mobile device. These instructions may be wirelessly transmitted over a wireless network 9 to a mobile phone 7. Pursuant to these instructions, transducers in the mobile phone 7 may display an audio or visual message such as a text message that comprises the therapeutic stimuli. For example, if the processor 11 has detected the onset of a drug craving, the cell phone may display an inquiry “how strong are you feeling right now?” and then a picture of the patient's daughter and a text message that says “Dad, you promised not to take morphine again. I love you”.
This invention may be implemented so that a processor selects one out of several pre-recorded messages for a transducer to display. The message that is displayed may first ask the person how he/she is doing and then display encouragement or other support, e.g. a reminder of cognitive-behavior therapy, depending on the pattern detected and the patient's response to this inquiry.
Alternately, pattern analysis and other processing may be divided up, so that some is done onboard a sensor, and some is done offboard.
In the example shown in
A wireless hub 5 that “bridges” a Bluetooth™ network and a IEEE 802.15.4 network has at least four advantages, in exemplary embodiments of this invention. First, it allows long battery life and small size/weight for the sensors. Second, it allows integration with Bluetooth™-enabled mobile devices. Third, a wireless hub provides the function of local data storage, which allows the system to be operated without any mobile phone at all. Fourth, integration with mobile phones (over Bluetooth™) allows information to be both sent and received via a cell phone network and Internet. Thus for data-logging or experience sampling applications, the physiological data can be wirelessly transmitted and logged on a remote internet server. However, without a cell phone, the data can be logged on the internal memory of the wireless hub.
In the example shown in
In
In
In
In
In
This invention may be implemented with multiple transducers for delivering therapeutic stimuli. For example, a cell phone may have an on-board microphone, display screen and haptic vibrator, each of which transducers may be used to deliver therapeutic stimuli. Or, for example, a user may wear an arm band with multiple transducers that may deliver vibrations or sounds to the user. Alternately, a transducer may deliver a thermal stimuli (e.g., to create a warm, comforting sensation).
Also, this invention may be implemented with multiple hubs. For example, hubs may be located in different rooms of a house or other building.
Likewise, this invention may be implemented with multiple sensors. For example, there may be multiple sensors on different parts of a user's body (e.g., right wrist, left wrist, and upper arm). Or, for example, the sensors may be located on multiple human users. One of the sensors might be co-located with or physically attached to the hub.
In exemplary embodiments of this invention, sensor radio modules do not continuously transmit a stream of data. Instead, a packet based wireless data protocol and wireless channel sharing are employed. These allow multiple sensor radio modules to be operated at the same time. The packet-based protocol allows multiple transmitters to share a channel through either CSMA/CA (Carrier Sense Multiple Access/Collision Avoidance) in the MAC layer, or through a hub-mediated beacon-based time division multiplexed approach, both of which are well known in the field of wireless sensor networks. The wearable sensors implementing the IEEE 802.15.4 radio transmission employ direct-sequence spread spectrum and generally use one of 16 possible 5 MHz channels in the 2.4 GHz ISM band. However, if necessary, multiple frequency channels can be used in orders to improve data throughput or to support a greater number of sensors.
In many cases, a packet-based protocol alone is insufficient to meet the specialized needs of physiological data monitoring because the physiological data must be sampled relatively frequently (˜tenths of a second), yet the wireless transmission must occur relatively infrequently (˜seconds or tens of seconds). According to principles of this invention, this problem may be solved though the use of a local memory cache. By locally storing and processing the data and then transmitting it in one large burst rather than many short transmissions, protocol overhead is minimized and throughput maximized in the communications channel.
For example, short-term memory storage may be provided onboard the sensor radio modules (e.g. with about 128 Kbytes of EEPROM memory). Or, for example, longer-term data storage may be achieved with a larger capacity flash memory storage (e.g., Gigabytes) onboard a wearable data hub, described below.
In an exemplary implementation of this invention, biosensors may gather data regarding one or more of heart beat, breathing, skin conductance, biopotentials, body core temperature, optical reflectance of blood vessels, or limb movement.
In many applications—such as helping former addicts resist drug cravings, it is desirable for the biosensors to operate on a long-term, round-the-clock basis. In that case, it is desirable for the biosensors to be worn in a comfortable, washable form factor, such as a wristband, armband, ankle-band or sock.
Alternatively, the electrode comprises electro-conductive thread or yarn embroidered into fabric or other material. For example, a stainless steel electro-conductive thread sold by Bekaert (Winston Salem, N.C.) can be used. This enables greater comfort and durability since the conductive thread exhibits less strain fatigue than traditional metal wires. Alternatively, electrically conducting elastomers or polymers may be used for the electrodes. Poly(3,4-ethylenedioxythiophene), also known as PEDOT, is an example of such a conducting elastomer. Carbon-impregnated rubber is an example of such a conducting polymer. These conductive elastomers and polymers are not generally breathable and thus less desirable. This problem may be solved in some cases by aeration (i.e., hole-punching) that makes the material more breathable. For example, carbonized rubber may be aerated in that fashion. Alternatively, standard medical metal electrodes may be used. For example, silver-silver chloride electrodes (such as those commonly used in electrocardiographs) may be used. These provide good contact with the skin. The metal electrodes may be detachably mounted on the fabric using pop-in snaps or the like.
Metal snaps 104 may be used to connect the electrodes (or leads from them) to the circuit (or lead from it). When the snaps are snapped together, the electrodes and circuitry are electrically connected; when they are snapped apart, they are not electrically connected. These snaps thus enable the circuitry to be repeatedly attached to and detached from the wearable band with electrodes. The wearable band with electrodes can then be easily washed or replaced. The placement of the metal snaps 104 may vary. For example, the snaps may be near the electrodes, or near the circuitry instead. Alternatively, other electrical connectors may be used instead of the metal snaps. In some implementations, the electrical connector is light-weight and at least one part of the connector is washable.
In the example shown in
In
In the example shown in
A flexible closure 106 is used to fasten the two ends of the band together. For instance, the flexible closure may comprise Velcro® strips and a metal snap fastener.
The wrist is not a conventional location for measuring EDA since the sweat glands there tend to be less sensitive than those on the palm or fingers, where EDA is traditionally measured. This issue, coupled with the use of dry electrodes, means that in this example it may take at least 15 minutes (depending on humidity and the individual's temperature) before the moisture buildup between the skin and electrodes is sufficient to show a range of responsiveness on the wrist. The main advantage of sensing EDA from the wrist is that the sensor can be comfortably worn for long periods of time (days and weeks) by adults and by small children (ages 3-6) without interfering with daily activities, such as sleeping, washing hands, or typing. If desired, the wrist-worn strap can be slid up onto the palm for a more sensitive EDA response.
Alternately, the EDA sensor circuit may be implemented using a digitally controlled variable gain amplifier to maximize dynamic range. However, this requires the use of an external microcontroller that adds greater cost, complexity, and power consumption.
In the example shown in
In some embodiments of this invention, an EDA circuit performs a time-domain measurement of skin conductance by employing an oscillator circuit whose oscillation frequency is dependent on the skin conductance. By measuring this frequency instead of measuring the skin resistance directly, it is possible to perform a more precise measurement given the low power rails and limited dynamic range of the voltage.
In order to maximize battery life and maintain a stable voltage rail for the op-amps and sensors, a low-power low-noise regulator (LM1962, National Semiconductor, Santa Clara, Calif.) may be added. This regulator has a power enable pin that can be used to only momentarily provide power to the sensor module and power it off when it is not in use, thereby reducing the power consumption of the entire EDA sensor module to less than 20 microwatts.
This invention may be implemented with a wearable biosensor that includes at least one photoplethysmograph (PPG) for measuring heart rate (HR) and heart rate variability (HRV). Since the light absorption of blood is wavelength dependent, if two different wavelength LEDs are used, then it is also possible to measure the relative blood oxygen level using the ratio of readings between the two color LED's.
Conventional PPG devices employ a single LED light. However, this invention may be implemented with a PPG device that has multiple LEDs.
In some embodiments of this invention, a PPG photodiode absorbs light reflected from the skin. In other embodiments, a PPG photodiode absorbs light transmitted through tissue.
In the example shown in
The first stage of the circuit is a transimpedance amplifier, which converts the current produced by the photodiode into voltage. A 1 MΩ resistor is placed at the positive input of the op-amp because it doubles the value of the input signal without doubling the gain.
The second stage is a bandpass filter with corner frequencies 4.8 Hz and 10.6 Hz, a bias voltage of 1.08 V, and a gain of 45. The low cutoff value was chosen because it eliminated the DC bias and removed baseline frequencies without deforming the QRS complex of the heart beat. Since this is only a first order filter, the high cutoff value was chosen at 10 Hz to ensure the elimination of 60 Hz noise. This simple RC first order filter was implemented instead of more complicated higher order filters to minimize the noise introduced by additional stages. Since within the QRS complex, the R wave has higher amplitude than both the Q and S segments, the bias voltage was set a bit lower than 1.65 V, which is half of Vcc, to prevent signal saturation.
The signal obtained via PPG on the wrist has very small amplitude; therefore, a high gain is necessary in order to place the sensor's output signal within the proper range so that the microcontroller can detect the R wave peaks. The value of this gain, however, varies from user to user. For instance, a male adult with thicker skin will need a higher gain than a child with small wrists. A 50 KΩ wheel potentiometer was placed in the third stage of the PPG heart rate sensor in order to enable users to adjust the gain so that the signal falls within the proper range. A wheel potentiometer, as opposed to a rotary potentiometer or a rheostat, was chosen so that the user would be able to easily adjust the gain without the use of a screw driver or similar tools.
The fourth stage of the PPG heart rate sensor is a buffer with a bias of 3.3 V. Since the output of the third stage is an inverted heart beat signal, a purpose of the buffer is to invert the signal so that the output of the overall circuit properly represents a heart beat signal. It is desirable that the output has the correct polarity so that the comparator in the microcontroller can properly detect the R wave peaks. The output of the heart rate sensor is then connected to an 8-bit microcontroller, specifically an ATmega168 (Atmel Corporation, San Jose, Calif.), which detects the R wave peaks in the signal. It operates very similarly to a comparator: when the input signal is higher than 1.5 V, it counts it as beat and ignores the signal as it decreases back to its baseline value. As the R wave peak reoccurs, the signal increases yet again. As it crosses the 1.5 V threshold, the microcontroller records another beat. It then subtracts the two obtained time stamps to determine the time between the two R waves. Next, the time difference values between ten consecutive R waves are averaged and used to calculate the user's heart rate. As such, in this example, it is desirable to adjust the gain in the third stage of the circuit to a value such that the R wave peak crosses the 1.5 V threshold level. In many cases, the gain should not set too high, so that the amplitude of the input signal's other components remains below 1.5 V. For instance, if the gain is too high, then (a) the R wave could be saturated at 3.3 V while the smaller, secondary PPG waveform also crosses the 1.5 V threshold, and (b) the microcontroller, in turn, could count both waveforms in determining the user's heart rate and could produce a heart rate value that is too high.
Alternately, the EDA sensor circuit, PPG sensor circuits and LED circuit could be designed in other ways than the implementations described above.
This invention may be implemented in such a way that one or more motion sensors are included. For example, an analog motion sensor (SQ-SEN-200, Signal Quest, Lebanon, N.H.) with an integrator circuit may be used. Advantages of this analog sensor, over a 3-axis accelerometer, are that it draws less than 1 microamp of current and is inexpensive to purchase. Alternatively, various types of motion sensors may be used, including 3 axis digital accelerometers. For example, the motion sensor may be any of various types of micro electro-mechanical systems (MEMS) consisting essentially of a proof mass on a damped spring, that measure the deflection of the proof mass in an analog or digital manner. For example, the deflection may be measured by piezoresistors attached to the spring, or by changes in capacitance between fixed beams and beams attached to the proof mass. Also, for example, the accelerometer may have a small heated dome of gas and measure the deflection of the center of the dome.
A motion sensor can also be used to gate the PPG signal so that heart rate data during motion can be ignored or cleaned. It should be noted, however, that there are many times during the day or night when a person's wrists are still, thus allowing for snapshots of HR and HRV. The combination of motion, EDA and HR/HRV are particularly relevant for recognizing sleep stages and conditions such as apnea. In some embodiments, multiple PPG sensors are employed. The multiple PPG signals are combined using signal processing, which reduces noise caused by motion artifacts. In some versions of the invention, logarithmic detection is used, which also helps handle motion artifacts.
The wearable biosensor may include a temperature sensor. For example, a low-power temperature sensor (LM60, National Semiconductor, Santa Clara, Calif.) is used.
This invention may be implemented in such a way that one or more sensors (such as PPG heart rate sensors, motion sensors and temperature sensors) are removable in their entirety from the wearable biosensor. This allows the sensors to be easily removed or replaced, for example, when the band or other host material for the sensor is washed. In other embodiments, one or more of these sensors are coated in plastic or another waterproof or water-resistant material, so that they can remain with the wrist band (or other wearable garment or material) when it is washed. In the case of PPG sensors, this coating is preferably transparent to the wavelength of light (including red or infrared light) emitted by the LEDs and absorbed by the photodiode. In the case of any temperature sensor, this coating preferably has a high thermal conductivity. In versions where these sensors remain with a band (or other wearable garment or material) when it is washed, leads may be used to connect the sensors with the removable circuitry, including the radio module and antenna. Metal snaps or other electrical connectors may be used to enable the sensors (or leads from them) to be repeatedly attached to or detached from the removable circuitry (or leads from it).
In some embodiments, rechargeable batteries are used to power the sensors. This not only eliminates the need to purchase hundreds of batteries that may be needed for long-term use, but enables the battery to be completely embedded inside the wearable package for weatherproofing and safety reasons (e.g., for infant use).
In some embodiments, the biosensors also include a low-power radio module and antenna, and transmit using a wireless protocol that is appropriate for low-power radio hardware. For example, the biosensors may transmit using the IEEE 802.15.4 protocol.
In some implementations of this invention, the Zigbee radio protocol is used for the sensor radio modules. Zigbee is a multi-hop transport protocol based on the IEEE 802.15.4 physical layer protocol.
Although higher-level transport protocols such as Zigbee support multi-hop routing and mesh networking, some implementations of the invention adopt a star topology for the network in order to minimize processing overhead and power consumption. This radio hardware also provides a wireless operating range of 25 meters.
Some conventional wireless sensors operate in the UHF range, e.g. 433 MHz, 915 MHz. However, this invention may be implemented with sensor radio modules that operate with 2.4 GHz in order to enable a smaller antenna size and achieve better indoor radio propagation.
In some embodiments, the sensor radio module comprises an ATmega168V microcontroller (Atmel Corporation, San Jose, Calif.) and a Chipcon CC2420 RF transceiver (Texas Instruments, Dallas, Tex.). In some versions, the sensor radio module exposes six 10-bit A/D ports on the microcontroller for interfacing with the sensor module. The reference voltage on these inputs can be configured via wireless commands from an external device. In some embodiments, a processor on-board the sensor can map patterns of the physiological and motion data to signals or alerts such as a likely seizure, drug craving, or other states that the wearer would like to know about or use to alert other people or devices for assistance. Alternatively, pattern analysis can be performed in a device that directly or indirectly receives the wireless data.
This invention may be implemented in such a way that an IEEE802.15.4 sensor radio module has firmware with independent sampling and transmission intervals that can be set via wireless commands from an external device. For example, every transmission cycle, the radio module may wake up and then in turn activate the power enable pin on the sensor module to power up the sensors. After a 10 ms delay, the radio module may capture a 10-bit A/D sample from each of the sensors, transmit the data packet wirelessly, and then go back to sleep. Alternately, data samples from sensors may be stored in a cache or buffer before being transmitted.
In some implementations, to reduce cost and provide omnidirectional performance, a sensor has an integrated printed circuit board antenna. For example, some embodiments employ a bent-dipole, horseshoe-shaped antenna that results in a compact design having a nearly isotropic radiation pattern.
IEEE 802.15.4 communication hardware supports a 250 kbps data rate. However, since EDA data has a relatively low rate of change, it is desirable in some implementations to use a slow sampling rate of 2 Hz and packetized data transmission to allow a very low operational duty cycle and long battery life.
This invention may be implemented in such a way that sensor radio modules employ channel-sharing wireless protocols to enable the transmission of data from multiple users and multiple sensors (e.g., both sides of the body, wrists or hands and feet, or multiple people). For example, as part of the firmware implementation of the IEEE 802.15.4 MAC (Media Access Channel) Layer, the CSMA (Carrier Sense Multiple Access) algorithm may be employed. This algorithm provides exponential back-off in the case of colliding transmissions between two or more radio modules. Different types of the CSMA protocol may be used in the invention, including 1-persistent, p-persistent and non-persistent CSMA. Alternatively, other contention network protocols may be used. For example, any of the following protocols may be used to avoid or reduce data collisions in the wireless network: TDMA (time division multiple access), slotted ALOHA, reservation ALOHA, OFDMA (orthogonal frequency-division multiple access), and MACA (multiple access with collision avoidance).
In an illustrative implementation of this invention, the sensor radio module includes a CC2420 radio IC with a maximum transmission power of 1 milliwatt (0 dBm). This provides a wireless detection range of 50-75 meters in free space using a 5 dBi gain receiver antenna. Indoor range is significantly less and depends on the building layout, but is approximately 15-20 meters for the module with integrated antenna and 5-10 meters for the version with external antenna. These wireless operating distances are sufficient for many applications, including transmission to a hub that is being worn by the same person that is wearing the transmitting biosensor. The sensor radio module may also have controllable output power, so the operating distance can be reduced to less than 1 meter as one of several ways to address data privacy. Alternatively, sensor radio modules may employ one or more ICs other than the CC2420.
Privacy and data security are important concerns in many applications. In addition to controlling the radio output power, the CC2420 radio IC includes hardware support for 128-bit AES encryption, which can be turned on as an option. In some implementations of this invention, a sensor device contains a user controlled ON/OFF switch so the user can choose to turn off the data transmission when desired.
In exemplary implementations of this invention, after a hub receives data from sensor radio modules, the hub transmits the data to a Bluetooth™-enabled device. From there, the data is transmitted directly or indirectly to a computer. A variety of different wireless or wired networks may be used for these transmissions, as discussed in more detail below.
After the computer receives the data, at least one processor in the computer analyzes the data in real time, using pattern recognition algorithms to identify patterns in the data that indicate the need for therapeutic intervention.
In some implementations of this invention, a simple classification scheme that does not involve machine learning may be used to recognize a data pattern. For example, in such a scheme, data may be classified based on criteria derived by simply averaging or aggregating the physiological patterns of multiple users. This scheme may be modified for a particular user's physiology by adjustment-to-baseline.
However, in many applications, pattern recognition is more accurate if machine learning is used. For example, machine learning allows a classification algorithm to be customized to take into account differences in affect or context, or cross-user differences in physiology (in a more nuanced manner than merely adjustment-to-baseline). Machine learning algorithms learn from a limited number of examples, where the data may be noisy and contain complex patterns which elude human detection.
It must be stressed that use of a learning machine allows a classification scheme to adapt in response to data. In some embodiments, this gives the processor great flexibility to adjust to complex data patterns that may, for instance, vary from user to user or that may vary within a user over different contexts.
In exemplary implementations of this invention, machine learning with Dynamic Bayesian Networks (DBNs) is employed to better recognize patterns in physiological, affective, and contextual data. It is advantageous to use DBNs for the following four reasons:
First, DBNs are well-suited for modeling a complex dynamic system. For example, they can be used to model behavioral states confounded by time-varying comorbidities that may come into play in the moments before drug relapse. DBNs are designed to manage noisy data, unknown quantities and uncertain events. A DBN has the power to describe not only instantaneous correlations among variables, but also how their values change over time.
Second, DBNs can generalize from limited data because the learning algorithm stresses balancing performance with model complexity. An overly complex model might be able to explain a data set (such as continuous physiology monitoring data) perfectly, but fails to generalize because it is explaining the data's idiosyncrasies (e.g., the humidity that day) of the specific data set. By penalizing model complexity, the algorithm finds the simplest acceptable explanation of the patterns—which are more robust to noise in existing data and tend to generalize better to future data.
Third, individual subjects have varying physiology. DBNs are well suited to devising hierarchical models (where data is organized into branching patterns that describe one-to-many relationships) that allow the prediction of physiological changes of an individual person.
Fourth, computation in a DBN is efficient: the time required is linear in the length of the sequence and may be performed in real time. Although the complexity of computation does grow with the complexity of the network, the learning algorithm strives to produce a simple network for generalization performance; as a consequence, computation is kept efficient. Thus, a classifier derived from a DBN performs minimal computation to produce an accurate result. This computational efficiency is particularly advantageous if the processor is deployed onboard a mobile device, such as a cell phone.
This invention may be implemented in such a way that a pattern recognition algorithm incorporates prior knowledge (in addition to training data). For instance, prior knowledge may include knowledge of transformation-invariance or knowledge about the data.
In an illustrative embodiment of this invention, a DBN learning algorithm incorporates prior knowledge into a suitable prior distribution over structures, which guides the search toward models that are physiologically relevant while also favoring simple models. Furthermore, the DBN's conditional probability tables (CPT's) are parameterized in a way that incorporates domain-specific knowledge. In an illustrative embodiment of this invention, cross-validation is used to set the tunable model parameters. In cross-validation, a portion of the data is withheld from training and instead used for testing; this is repeated across the entire data set.
In some implementations of this invention, the result of the learning algorithm is a structure and parameter set for a DBN: while the training data indicates physiology and context associated with prescription opioid cues, the goal is a classifier to predict State X of relapse risk; this corresponds to using the learned DBN with the relapse status node left unobserved. Prediction of this variable is then made using the Belief Propagation (BP) algorithm, a simple message passing algorithm which operates on the learned network. An advantage of using a DBN is that the computation time required for BP is linear in the length of the sequence, and thus presents no obstacle to implementation in a low-power deployable system.
In illustrative implementations of this invention, a learning algorithm can be trained using data to produce a fully specified DBN: the output consists of both the graph structure determining how variables are interrelated, as well as the CPTs that determine how each variable is influenced by its immediate causes in the model. An advantage of using DBNs is that the resulting models are readily interpretable, in contrast to black box approaches such as neural networks.
In illustrative implementations of this invention, user feedback may be part of the data used to train the algorithm. This feedback may be obtained in a wide variety of ways.
For example, in an application to help a user resist a prescription opioid craving, a mobile computing device may display an Annotate Panel and Activity Panel. These panels may be used to gather user feedback, as described below.
The Annotate Panel is a graphical user interface (GUI) comprising multiple screens. It allows users to self-report stress, depression, pain exacerbations, frustration, feeling deprived or the need to reward one's self, prescription opioid craving, or any other feeling, behavior, or event they consider interesting. Color-coded buttons increase recognition and indicate a requested response: Dark Blue-“None” (I don't need anything); Green-“Barely”; Yellow-“So-so” (I can control things); Orange-“High”; Red-“Way High” (I need some help). The Annotate Panel also allows a user to self-report his or her response to relapse prevention interventions. The user enters data for the Annotate Panel on a mobile computing device, so there is no threat of disclosure to another person Annotations can be completed in any location in which the participant has confidence, and all data is securely stored and transmitted.
An Activity Panel is a GUI that allows a user to self-report his or her current activities, such as when experiencing stress. For example, an Activity Panel may allow a user to select Commute, Working, Personal, Fun, Exercise, Relaxing, Eating, Meeting, Talking or Other. After the user selects one of these broad categories, the user is given an opportunity to select a more detailed subcategory. For example, pressing the “Working” button leads to subcategories such as “Argued with boss” or “Just made a sale”. In this example, the Activity Panel is generally organized with more popular activities at the top of the screen (and therefore easier to identify by the user). However activities most associated with stress and drug craving are placed in easily recognized locations.
Alternately, this invention may be implemented with other approaches to machine learning instead of DBNs. For example, it may be implemented with neural networks, conditional random fields, hidden Markov models, Kalman filters, fuzzy logic, kernel estimation, k-nearest neighbor, learning vector quantization, Gaussian models, RBF (radial basis function) classifiers and other statistical classification approaches.
This invention may be implemented as a method comprising the following steps, as shown in
In exemplary implementations of this invention, one or more wireless networks are employed to transmit data and instructions. If a mobile phone is employed as the wireless hub, the wireless network may be a standard GSM or CDMA network, and the data may be transmitted over 3G or 4G mobile telecommunication data standards, UMTS (Universal Mobile Telecommunications System), or EDGE (Enhanced Data Rates for GSM Evolution). Alternately, this wireless network may comprise a IEEE 802.15.4 network, a Wi-Fi™ network that complies with an IEEE 802.11 standard, or a Wi-MAX network that complies with an IEEE 802.16 standard.
The system architecture may vary, depending on the way in which this invention is implemented. Some examples are shown in
In
In
In
In
This invention may be implemented as shown in
In the example shown in
In the example shown in
In the example shown in
In the example shown in
Alternately, this invention may be implemented as shown in
In some embodiments of this invention, a radio base station or reader is used to collect data from one or more radio modules and sensors. For example, the base station can be the ZR-USB, which has a USB interface to plug into PC's and laptops. This base station may comprise an ATmega168V microcontroller, a CC2420 radio IC, and a FTDI232BQ USB interface chip (Future Technology Devices International, Ltd., Glasgow, United Kingdom). In some implementations, a 50-Ohm antenna port permits a variety of commercially available 2.4 GHz antennas to be used.
This invention may be implemented in such a way that a mobile phone is used as the wireless hub, and the IEEE 802.15.4 wireless protocol is not used. In this approach, Bluetooth™ is used directly to the phone, and then the phone transmits out through the cell phone network to the remote server.
Alternately, this invention may be implemented in such a way that data is sent to a separate wireless hub (e.g., using the IEEE 802.15.4 protocol), and then this hub communicates to the phone. In that case, the patient does not have to wear or carry two different devices (the phone plus the wireless hub). For example, this approach is advantageous where the patient is not very mobile (such as a baby monitor).
In many cases, it is desirable to set an ID code for each sensor radio module and to configure its transmission or sampling rate. In some embodiments, to meet these needs, a command protocol is added using a “command and response” paradigm. Such a command protocol may operate as follows, in an illustrative implementation of this invention. To send a command to a sensor radio module, the command is first sent to another device (such as a cell phone, hub or radio base station). The command is stored in this other device's “command queue” until the sensor radio module wakes up and transmits its data packet to this other device. The command is then transmitted to the sensor radio module, which receives the command and immediately executes it before going back to sleep.
This invention may be implemented in such a way that data from multiple radio modules (e.g. on both left and right wrists) is monitored. In those cases, it may be desirable to synchronize time between sensors. For example, in some embodiments in which the ad-hoc asynchronous nature of the network does not automatically provide a common time base, a radio base station (or other device that receives radio transmissions from sensors) may be programmed to time stamp arriving data packets in order to generate a proper time base of the measurements. Alternately, the sensors themselves may include a real-time clock. Or the devices may auto-synch with a designated node of the networked system
In some implementations of this invention, a background process runs on a smartphone or other mobile computing device. This background process automatically collects data from the sensors and relays this data to a central server. For example, the information relayed to the server may be raw data or calculated parameters.
In some embodiments of this invention, self-reports are collected from a patient on a periodic basis (e.g., as a daily assessment). For example, a smartphone may have a chron job that periodically opens a pre-loaded form and prompts the patient to answer some questions. (For instance, the first screen on this form may display “It's time to ask you a few questions. Can we do this now? [YES] [WAIT 5 min] [NO]” If No, then the app will wait half hour or so and try again later.) These forms can be dynamically created and pushed to the phone from the server.
In some embodiments, a server: (a) stores individual profile data for patients, (b) stores content to be used for their interventions (e.g., photos, text messages, audio clips), and (c) keeps track of the interventions they have received. According to principles of this invention, content (such as prerecorded audio or visual content) may be stored in machine-readable form in the memory of a server. This content may be accessed and delivered to a user as therapeutic stimuli in a form perceptible to a user.
In some embodiments, GUIs (a) display an administrator control panel that can be used to enroll new patients or de-activate old ones, and (b) enable a patient and doctor to select or create the content (photos, messages, sounds) to be used for their individual treatment (intervention).
In some embodiments, a software application allows content to be delivered automatically (on a time schedule) to the patients. This content may include following: (a) messages that are sent at regular intervals, and (b) new forms to push to the phone that are used for periodic self-reporting by the patient. For example, these messages may be emails or SMS/MMS messages and these forms may be XForms. Also, for example, some of these messages may provide patients with daily status updates to let them know how they are doing in their therapy. Some messages may simply provide fun activities (e.g. a short game or puzzle) or topics of interest (perhaps news or a tip addressing a patient's hobby or special interest) to the patient, to encourage continued engagement with the system.
In some embodiments, a server analyzes physiological data from the sensors by applying a pattern recognition algorithm to classify the data and identify specific cases. Based on the results of this classification, the server modifies the content and also the parameters of the “content pusher” module (e.g., changes the images or text messages that are sent, changes the frequency that the messages are sent, or changes the “sensitivity” of the algorithm that detects the specific abnormality).
In some implementations of this invention, data is gathered by patients by using IVR (interactive Voice Response).
This invention has numerous applications, in addition to those described above. Here are four examples:
First, this invention may be implemented in such a way that a person who has experienced multiple seizures (epilepsy) wears a sensor on one or both wrists with customized pattern recognition software in a on-board processor that analyzes patterns in autonomic nervous system activation to predict the onset of a seizure. This information can be communicated to a nearby helper, and may also be used to trigger various automatic interventions.
Second, in some embodiments, a person wears a wireless sensor and a processor recognizes the wearer's detectable patterns of physiological change related to threat such as the autonomic nervous system changes that happen in a bank robbery or other high-risk life-threatening situation. The device can be discretely integrated into clothing (wrist watch, shoe, etc.) so that it is not visible. The wireless components allow for multiple individuals (e.g. bank employees) to communicate their state simultaneously, e.g., to somebody who can help intervene on behalf of their safety.
Third, this invention may be implemented for triage in an emergency response to a disaster, such as an earthquake or explosion, with many casualties. In a mass casualty situation, emergency workers generally perform triage, and it is not possible to deploy large and expensive medical equipment to each patient who is injured. The present invention may be implemented so that it is small and portable and can be easily attached to a large number of patients. Data transmitted from each patient to a wireless hub can be analyzed in real time to detect specific states such as hemorrhagic (hypovolemic) shock due to loss of blood, or septic shock due to onset of infection. More specifically, certain physiological signs for the human body inflammatory response have been defined by the American Society of Chest Physicians and Society of Critical Care Medicine (consensus conference 1992) to include the following: (a) body temperature less than 36° C. or greater than 38° C., (b) tachycardia—heart rate greater than 90 bpm, (c) tachypnea—respiration greater than 20 breaths per minute. In this implementation, since the emergency workers are very busy attending to other patients, a machine learning algorithm running on the wireless hub or mobile phone automatically monitors patients wearing the sensors, and automatically detects these dangerous states and triggers a message to be sent to one of the emergency workers.
Fourth, this invention may be implemented to help patients with severe diabetes who are susceptible to diabetic shock (hypoglycemia). In this case, before the patient becomes unconscious, the patient exhibits increased heart rate and high skin conductance (electrodermal activity). In this implementation, the wireless processor “hub” or mobile phone is used to analyze data in real time and initiate a message to a caregiver in the form of an e-mail, SMS text message, or even a phone call.
A few definitions, for clarity:
The term “hub” means an apparatus comprising a plurality of radios, which apparatus is at least capable of receiving radio signals that comply with one wireless protocol and transmitting radio signals that comply with a different wireless protocol. For example, a Bluetooth™-enabled cell phone with a 3G wireless link is a hub.
The term “mobile computing device” means a smartphone, smartbook, notepad computer, subnotepad computer, netbook, tablet computer, internet tablet, personal digital assistant, palmtop computer, laptop computer, mobile phone, cell phone or other portable or handheld computing device.
The term “physiological data” means data relating to a physical state, physical property or physical parameter of a living organism, or a time at which such state, property or parameter occurs. Examples of physiological data include heart rate, breathing rate, skin conductance, biopotentials, core body temperature, and limb movement, and parameters derived from them (e.g., heart rate variability). Physiological data may relate to any living organism, including humans, animals, and also plants (e.g. photosynthesis measurements). Physiological data includes data regarding the physical activity of a living organism. For example, data regarding the position, acceleration or movement of body parts (or GPS data regarding the geographical position or movement of a person) is “physiological data” as defined herein.
The term “sensor” means an electronics module comprising a radio, and one or more sensor elements and circuits for measuring physiological data, such as EDA, heart rate, or temperature. This electronics module may further comprise a data storage module, power supply, and GPS receiver.
The term “therapeutic” means pertaining to the treatment of an adverse medical condition or treatment of unhealthy behavior. “Therapeutic stimuli” may include, for example (a) vibrations or flashes of light, or (b) images, audio files, text messages, and other communications content having psychological impact. “Therapeutic stimuli” may also include stimuli delivered in or as a therapeutic intervention.
The term “verbal” refers to communication that involves, at least in part, words. Verbal communication is not limited to audible communication, but may be perceptible by any other sense, such as visual (e.g. a text or handwritten message) or tactile (e.g., a Braille message).
The term “wireless protocol” means a standard, specification or protocol, in each case for wireless communication. For example, the Bluetooth™ standard and the IEEE 802.15.4 wireless data standard. However, wireless protocol may also include protocols that do not conform to an industry standard, such as a proprietary standard.
It is to be understood that the methods and apparatus which have been described above are merely illustrative applications of the principles of the invention. Numerous modifications may be made by those skilled in the art without departing from the scope of the invention. The scope of the invention is not to be limited except by the claims that follow.
This application is a continuation-in-part of U.S. non-provisional patent application Ser. No. 12/386,348 filed on Apr. 16, 2009, the entire disclosure of which is incorporated by reference herein.
Number | Date | Country | |
---|---|---|---|
Parent | 12386348 | Apr 2009 | US |
Child | 12834903 | US |