Patent Application
20250025040
The subject matter described herein relates to determining physical characteristics and actions of a subject. More particularly, the subject matter described herein relates to determining physical characteristics and actions of a subject using illumination from behind the eye, also referred to herein as back-illumination of the subject's eye.
Biometric monitoring is the observation of conscious and unconscious physiological responses to stimuli. Tracking pupil size and location, respiratory, and heart rate dynamics are key measurements that altogether characterize physiological arousal responses. Pupillometry is a non-invasive technique that measures pupil size and position over time and traditionally relies on a camera to capture videos of the pupil illuminated externally with infrared (IR) light7-9. This approach requires bulky equipment placed near the eye that occludes the field of view, preventing subjects from performing the full repertoire of natural behaviors11. Extracting pupil size dynamics from the video data requires computationally expensive segmentation algorithms to be applied to each frame. Sampling rates are limited by the available camera frame rate, typically at or below 100 fps, preventing the capture of high speed physiological events, and real-time operation is computationally challenging. Certain gaze tracking and pupillometry systems replace the camera in favor of infrared emitters and sensors placed in front of the eyes10, however these systems do not segregate pupillary responses arising from arousal changes from those caused by fluctuations in the ambient light.
The pupillary light reflex is an important metric of autonomic nervous system function that has been exploited for a wide range of clinical applications to diagnose multiple neurological disorders2. However, these measurements are taken across brief periods of time (seconds) and longitudinal tracking of pupillary light responses across longer periods of time (minutes, hours, days, months) have not been studied. Furthermore, these systems are limited to measuring pupil dynamics while heart and respiratory biometrics must be acquired using stress inducing neck collars12 or invasive methods that require surgical implantation of a device inside the body13. To capture all relevant biometrics, several separate and often expensive devices must measure signals from the subject simultaneously, leading to noise, errors in data synchronization, and an increased cost for the user. In light of these and other difficulties, there exists a need for improved methods, systems, and computer readable media for determining physical characteristics and actions of a subject using illumination of the eye(s) of the subject.
A system for determining a physical characteristic or action of a subject using internal illumination of the back of an eye of a subject includes at least one light source configured for illuminating an eye of a subject using light from within a head of the subject. The system includes at least one light sensor positionable outside of the head of the subject for sensing light from the at least one light source exiting the subject through the eye of the subject. The system further includes a controller coupled to the at least one light source and at least one light sensor for recording an indication of the light while controlling the illuminating.
According to another aspect of the subject matter described herein, the at least one light source comprises an infrared light source.
According to another aspect of the subject matter described herein, the infrared light source comprises an infrared emitter circuit configured to generate pulses of infrared light.
According to another aspect of the subject matter described herein, the infrared emitter circuit is configured to generate continuous illumination of pulses of light at instantaneous power levels ranging from 1 milliwatt to 10 Watts.
According to another aspect of the subject matter described herein, the at least one light source comprises a housing configured to be inserted into an ear canal or a nostril of the subject or positioned adjacent to a surface of the skin of the subject.
According to another aspect of the subject matter described herein, the at least one light source comprises an implantable device for implantation within the head of the subject.
According to another aspect of the subject matter described herein, the at least one light source comprises a primary light source for illuminating the eye of the subject from within the head of the subject and a secondary light source for controlling a pupil of the eye of the subject by illuminating the pupil externally to the head of the subject.
According to another aspect of the subject matter described herein, secondary light source comprises a visible light source.
According to another aspect of the subject matter described herein, the system includes a head mountable frame for holding the at least one light source.
According to another aspect of the subject matter described herein, the head mountable frame is configured to hold the at least one light sensor.
According to another aspect of the subject matter described herein, the system includes a mirror for directing the light exiting the subject through the eye of the subject to the at least one light sensor.
According to another aspect of the subject matter described herein, the at least one light source comprise an infrared light source and the mirror comprises an infrared mirror.
According to another aspect of the subject matter described herein, the at least one light sensor comprises a light sensor array including a plurality of infrared receiver circuits positionable around an exterior of the eye of the subject.
According to another aspect of the subject matter described herein, the infrared receiver circuits each comprise an optoelectronic device for detecting the light and generating a current proportional to the detected light.
According to another aspect of the subject matter described herein, the optoelectronic device comprises a photodetector.
According to another aspect of the subject matter described herein, the system includes a head mountable frame for holding at least one light sensor on the head of the subject.
According to another aspect of the subject matter described herein, the head mountable frame has a glasses-like or goggles-like form factor.
According to another aspect of the subject matter described herein, the head mountable frame comprises an annulus for holding the at least one light sensor near the eye of the subject.
According to another aspect of the subject matter described herein, the controller includes an analog-to-digital converter for producing digital values based on a signal generated by the at least one light sensor.
According to another aspect of the subject matter described herein, the controller records at least one signal generated by the at least one light sensor while synchronously controlling the at least one light source to generate pulses of light.
According to another aspect of the subject matter described herein, the system includes a biometric measurement module for determining a physical characteristic of the subject based on the light sensed by the at least one light sensor.
According to another aspect of the subject matter described herein, the biometric measurement module is configured to determine at least one of eye movement, eye blinking, and pupil size variation based on the light sensed by the at least one light sensor.
According to another aspect of the subject matter described herein, the biometric measurement module is configured to determine at least one of heart rate, heart beats, blood flow, pulse oximetry, breathing rate, and respirations of the subject based on the light sensed by the at least one light sensor.
According to another aspect of the subject matter described herein, the system includes a calibration module that implements an algorithm configured to calibrate the biometric measurement module.
According to another aspect of the subject matter described herein, the controller is configured to subtract a representation of a signal generated by the at least one light sensor when the at least one light source is off from a representation of the signal generated by the at least one light sensor when the at least one light source is on for measuring the amount of light exiting the eye regardless of ambient light levels.
According to another aspect of the subject matter described herein, the at least one light sensor is configured to sense light exiting the subject through a pupil of the eye of the subject.
According to another aspect of the subject matter described herein, a method for determining a physical characteristic or measuring an activity of a subject using back-illumination of an eye of the subject is provided. The method includes positioning at least one light source for illuminating an eye of a subject using light from within a head of the subject. The method further includes controlling the at least one light source to illuminate the eye of the subject from within the head of the subject. The method further includes sensing, using at least one light sensor located external to the head of the subject, light from the at least one light source exiting the subject through the eye of the subject. The method further includes recording an indication of the light sensed by the at least one light sensor while controlling the illuminating.
According to another aspect of the subject matter described herein, positioning the at least one light source includes positioning the at least one light source in an ear canal, in a nostril, or adjacent to a surface of the skin of the subject.
According to another aspect of the subject matter described herein, positioning the at least one light source includes positioning the at least one light source within or on the head of the subject.
According to another aspect of the subject matter described herein, recording the indication of the light sensed by the at least one sensor while controlling the illuminating includes recording at least one signal generated by the at least one sensor while synchronously controlling the at least one light source to generate pulses of light.
According to another aspect of the subject matter described herein, the method includes determining a physical characteristic or action of the subject based on the light sensed by the at least one light sensor.
According to another aspect of the subject matter described herein, determining the physical characteristic or measuring the activity includes measuring at least one of eye movement, eye blinking, and pupil size variation based on the light sensed by the at least one light sensor.
According to another aspect of the subject matter described herein, the method includes determining a physical characteristic or activity of the subject based on the light sensed by the at least one light sensor.
According to another aspect of the subject matter described herein, the physical characteristic includes at least one of eye movement, eye blinking, and pupil size variation based on the light sensed by the at least one light sensor.
According to another aspect of the subject matter described herein, the physical characteristic or activity includes at least one of heart rate, heart beats, blood flow, pulse oximetry, breathing rate, and respirations of the subject based on the light sensed by the at least one light sensor.
According to another aspect of the subject matter described herein, the method includes subtracting a representation of a signal generated by the at least one light sensor when the at least one light source is off from a representation of the signal generated by the at least one sensor when the at least one light source is on for measuring the amount of light exiting the pupil regardless of ambient light levels.
According to another aspect of the subject matter described herein, sensing the light exiting the eye of the subject includes light exiting the subject through a pupil of the eye of the subject.
According to another aspect of the subject matter described herein, a non-transitory computer readable medium having stored thereon executable instructions that when executed by a processor of a computer controls the computer to perform steps is provided. The steps included controlling at least one light source to illuminate the eye of the subject from within the head of the subject. The steps further include sensing, using at least one light sensor located external to the head of the subject, light from the at least one light source exiting the subject through the eye of the subject. The steps further include recording an indication of the light sensed by the at least one light sensor while controlling the illuminating.
The subject matter described herein can be implemented in software in combination with hardware and/or firmware. For example, the subject matter described herein can be implemented in software executed by a processor. In one exemplary implementation, the subject matter described herein can be implemented using a non-transitory computer readable medium having stored thereon computer executable instructions that when executed by the processor of a computer control the computer to perform steps. Exemplary computer readable media suitable for implementing the subject matter described herein include non-transitory computer-readable media, such as disk memory devices, chip memory devices, programmable logic devices, and application specific integrated circuits. In addition, a computer readable medium that implements the subject matter described herein may be located on a single device or computing platform or may be distributed across multiple devices or computing platforms.
Exemplary implementation of the subject matter described herein will now be explained with reference to the accompanying drawings, of which:
The
We have developed a lightweight low-cost technology for recording multiple biometric indicators of arousal simultaneously, continuously, and in real-time. The subject matter described herein includes a device that comprises a low-power, pulsed source of infrared light that illuminates the back of the eye from behind it, with infrared light entering the body from the back or the side, or inside the brain (in animals) that diffuses through tissue, bone and skin. Pulsed illumination is synchronized with photodetectors placed near the eye that continuously measure the amount of diffused infrared light exiting through the pupil across multiple directions. The subject matter described herein includes custom computational methods to acquire the raw data and process the data to extract quantitative measurements of ocular biometrics and other physical characteristics of a subject. The subject matter described can measure pupil size and direction (eye movement), heart and breathing rates, the ambient light level, and head motion, simultaneously, in real-time, and at multi-kilohertz sampling speeds. Our technology is safe to be utilized both in animals and humans, and can be built with inexpensive, lightweight, and low-power electronics.
We anticipate that our technology will be of interest to (1) the scientific community, for the study and development of treatment models for neurological and neuropsychiatric disorders affecting arousal, mood, sleep, and attention. We also foresee (2) clinical and medical applications since many neurological disorders have symptomatology that affect autonomic properties. Here, our technology can facilitate the early detection of neurological disorders, of adverse events and symptoms, and help address them as needed with precisely timed treatments. In addition, it can be used to detect onset of hypo- and hyper-arousal state changes in patients suffering from neuropsychiatric disorders related to mood and anxiety. Our technology is also perfectly suited for (3) consumer-level applications such as video games, advertisement, and the broader self-care industry where the ability to monitor biometrics in real-time can provide feedback to optimize content and/or to validate or enhance the desired user's reaction. Finally, our technology can be implemented in (4) smart personal protection equipment to track warning signs of fatigue, attention to tasks, and help prevent human errors in critical situations.
Biometric monitoring is the observation of conscious and unconscious physiological responses to stimuli. Tracking pupil size and location, respiratory, and heart rate dynamics are key measurements that altogether characterize physiological arousal responses. Pupillometry is a non-invasive technique that measures the eye's pupil size and position over time1. Variations of the pupil size happen, as a reflexive process in response to variations in the surrounding light level, but also through a voluntary mechanism driven by physiological arousal and higher-level cognition. For example, positive arousal and cognitively demanding tasks generally lead to an increase in pupil size. Hence, rapid pupillary responses are a reliable indicator of changes in emotional and attentional states. Measuring pupil size fluctuations provides valuable information about how mammalian species interact with their surrounding environments. Therefore, pupillometry data provides important insights for research across human and preclinical animal models of health and disease.
There is a growing body of literature indicating that abnormal pupil responses occur in patients experiencing neurological and neuropsychiatric disorders1-3. Studies have highlighted that abnormal pupil responses can be early indicators of neurological diseases such as Parkinson's, Autism, and Multiple Sclerosis4. Early detection of neurological disease is at the forefront of biomedical research studies to improve patient outcomes through prompt medical intervention. Detecting adverse neurological events is also highly valuable in clinical settings to improve treatment plans with a precisely timed intervention, for instance in combination with the delivery of fast acting drugs. Our technique could be very useful for patients suffering from neuropsychiatric disorders where both patient and clinician can detect early phases of hypo- or hyper-arousal states before they escalate to either a panic attack or suicide.
One of the main advantages of the pupillometry technique we propose is that our system is non-invasive, and portable. This allows any user to perform longitudinal studies, and gather data without disturbing the animal or patient for extended periods. Also, our technology can be used across several species without any substantial hardware modifications which facilitates translational research. Pupillometry has also been implemented to improve the performance of brain-computer interfaces for patients that have deficits in motor control and are nonverbal. Utilizations include locked-in-patients (pseudocoma) that rely on clinicians and family members to monitor pupil movement towards letters to communicate. Recently, one study found that using changes in pupil diameter that occur when eyes are focused on letters leads to more accurate communication from locked-in-patients compared to only using eye movement5. This indicates that our technology can answer a need for real-time pupillometry technology to reliably track pupil responses and enhance the performance of brain-computer interfaces.
Additionally, the use of pupillometry data extends beyond basic research and clinical applications. Since changes in pupil response occur when a subject is performing mental challenging tasks, one study used pupillometry as a way to measure difficulty levels of educational video games amongst students6. This information was helpful for instructors to assess student learning. Pupillometry can also provide feedback in video games or other forms of entertainment, to optimize the user's experience. Our technology is also useful to gather feedback with study groups when evaluating the effectiveness of advertisement campaigns. Finally, our technology can be integrated into safety goggles, to create smart monitors that track sleep or loss of attention and reduce accidents caused by human errors in job sites where personal safety is critical.
The subject matter described herein includes a portable device that measures the size and location of the pupil with photodetectors placed near the eye. As opposed to existing techniques that illuminate the front of the eye with light to capture pupil information, our approach relies on a back-illumination method with infrared light diffusing through bones and tissue to illuminate the back of the eye (
More particularly,
The amount of light captured by an individual light sensor 106 increases when one of eyes 102 is pointed at the light sensor 106 and decreases when the eye 102 is pointed away from the light sensor 106. By mapping the amount of light captured by several of light sensors 106, and with a calibration step, we can estimate the eye orientation. By measuring the total amount of light captured by all of light sensors 106, we can then estimate the size of the pupil, independently of its orientation.
As indicated above, infrared (IR) illumination can be achieved by placing the source of IR light in the ear, nose, neck, or on the head and through the skull (
The photodetectors of our device detect infrared light, which includes the IR light emitted by our illumination source, as well as IR light present in the ambient light. The flow of ambient light adds a positive baseline to our measurements of the IR photon flow through the pupil and fluctuates depending on head orientation and on the surrounding lighting environment. To separate ambient light from the detected signal, our technology pulses the IR illumination source. The detectors are synchronized with the pulsed light to capture one baseline data point when the IR light is off, directly followed by another datapoint when the IR light is on. This captures the baseline and the signal together. By subtracting the two sequentially recorded values we obtain a quantitative measurement of the signal, i.e. the flow of IR light emitted by the pulsed source and reaching each photodetector. The signal is a measurement of the pupil size and orientation, with fluctuations that are caused by the pulsed blood flow, and by head motion. The signal is further processed to extract ocular biometrics (2, 3, 4, 5, 6). The baseline is also recorded for each photodetector. The total amount of baseline signal recorded across all photodetectors is a quantitative measure of ambient light. Fluctuations of the ambient light are compensated by modulations of the pupil size that are needed to maintain a suitable range of brightness on the retina, but these modulations of the pupil size do not relate to the arousal or behavior. By identifying synchronous events between the ambient light level and the signal, our technique classifies changes in pupil size and identifies events that can be attributed to behavior and arousal independently of physiological pupil size adjustments that only occur to match variations of the ambient light levels.
Differential amounts of light across photodetectors in the baseline signal occurs when the subject changes location, which introduces local variations in the illumination environment perceived by each photodetector. Differential signals in the baseline will be computed to track subject motion, providing yet another indicator of arousal.
External factors (2, 6) all introduce modulations in the signal. To measure eye motion and detect blinking (5,6), either involuntary, or intentional (e.g., eye saccades, twitch, gaze tracking), we compare the amount of signal detected on each photodetector. The signal detected on any detector increases when the pupil is pointed towards it and decreases when pointing away from it (see
To quantify pupil size (2), we calculate the average amount of light captured by all the detectors around the eye. This quantity no longer depends on pupil location, but scales linearly with the surface of the pupil, with small perturbations induced by blood flow (3) and tissue motion (e.g. from breathing) (4).
To separate these quantities, we filter the pupil size data (2) sampled at kilohertz speeds with temporal band-pass filters at frequencies centered around predicted heart and breathing rates. The peak frequency in the filtered signal corresponds to the heart and breathing rates. Since our sampling speeds are significantly faster than all these events, the signal temporal resolution is sufficient to detect these events, even in the presence of noise.
Since our technology generates raw data that requires a significant amount of nonlinear processing, our invention also features a deep learning model to process raw data in real time and extract quantitative indicators of behavior and arousal (e.g., pupil size, pupil orientation, heart rate, heart beats breathing rate, respirations, subject motion, and ambient light).
For pulse oximetry through the eye, our system would be equipped with several IR illumination sources with distinct peak emission wavelengths, as in a finger-attached pulse oximeter. Here, our invention has the advantage of enabling measurements through the eye, and yields results that are not affected by skin color. Recent studies have shown that finger-clip pulse oximeters are more often likely to return false or misleading results when patients have dark skin color14. Our technology would avoid such issues. Our technology is scalable to enable use in both animals (
In head-fixed cases, the animal is restrained, and light is delivered through the brain to come out of the pupils. The IR light source in these embodiments is typically either an LED placed flush with the skull, or the laser diode is coupled with an optic fiber for easy integration with pre-existing optogenetic protocols (
The optical system is connected to an electronic controller placed away from the subject. We have already implemented the signal processing device with an Arduino, using transistor amplifiers to drive the LED with enough current, and with operational amplifiers to pick up weak currents from the photodiodes.
We have tested our first prototype with an IR led placed on the skull of a restrained mouse, and by placing the photodetector near the eye. For validation purposes, we recorded data with our invention while simultaneously recording video data of the pupil with a fast camera. An example test setup used to validate pupil size, heart beat, and respiration signals generated using the output from the biometric optical photometer is shown in
Experimental results show that our technique successfully measures pupil size and that our experiments (shown by the solid lines in
In particular,
Preliminary results for the implementation of our device as wearable glasses (as in
The electronic circuits for IR light pulsing and synchronous acquisition of the photon flux through pupils are shown in
Emitter circuit 500 includes an infrared source 506, which in the illustrated example is an infrared light emitting diode (LED) configured to emit infrared light at a wavelength of 940 nm. Emitter circuit 500 further includes an operational amplifier 507 configured as a voltage follower circuit to amplify and follow the pulsed control signal generated by controller 504. Emitter circuit 500 further includes a transistor 508 and associated resistors 510 and 512. Transistor 508 also functions as an amplifier, which amplifies the current passing through light source 506. In one example, controller 504 is capable of driving light source to generate pulses of ultrasound light in the kilohertz range, for example at a frequency in a range from 10 to 50 kilohertz at instantaneous power levels ranging between 1 milliwatt and 10 Watts.
Receiver circuit 502 includes an optoelectronic device, which in the illustrated example is a photodiode 514 for generating a current proportional to the intensity of detected light from emitter circuit 500 after exiting through the pupil of the subject. In an alternate example, the optoelectronic device may be implemented using another type of photodetector, such as a phototransistor. Receiver circuit 502 also includes an operational amplifier 516 that converts the photocurrent captured by the photodiode 514 into a voltage signal. A variable resistor 518 allows for fast adjustment of the gain of the current input to operational amplifier 516.
The output of operational amplifier 516 is an analog signal that is synchronously sampled by controller 504. A bandpass filter 519 may be included to filter the output along signal from operational amplifier 516 to remove the baseline (DC) component from the signal. Controller 504 includes an analog to digital (A/D) converter 520 which digitizes the signal and provides real time digitized data indicative of the detected light for plotting and analysis via a wired or wireless communications module 521 to a personal computer (PC) 522. Wired or wireless communications module 521 may be a wireless communications chip, such as a Wi-Fi, BlueTooth, or ultrawideband chip or a wired communications chip, such as an Ethernet or universal serial bus (USB) chip. Controller 504 also includes an LED pulse control signal generator 523 for generating the signal for controlling pulses generated by light source light source 506 synchronously with the sampling of the signal output by receiver circuit 502.
PC 522 includes at least one processor 524 and a memory 526. A biometric measurement module 528 reads the data received from controller 504 and generates real time data plots and output indicative of a physical characteristic or activity of the subject. A calibration module 530 performs the steps described herein for calibrating biometric measurement module 528, for example, to generate output indicative of light exiting the subject's eyes independently of the ambient light level.
In one optional configuration, emitter circuit 500 and receiver circuit 502 may be components of a two-photon microscope 532, which scans the brain of a subject using two infrared lasers to generate neural activity data. In such an implementation, controller 504 may be configured to record the indication of the light exiting the subject through the eye of the subject synchronously with neural imaging data produced by two-photon microscope 532.
In one example, biometric measurement module 528 may calculate an arousal index for a subject. Biometric ocular photometry allows gathering of multiple biometrics: pupillary, heart, respiratory dynamics. It is possible that an overarching latent arousal variable is a better metric than any of these metrics, individually. Therefore, we will develop an arousal-index (latent construct) designed to extract and integrate the most informative components of arousal garnered from the simultaneously acquired biometrics. For each individual arousal biometric, a unique function will be identified that maps that biometric into a lower dimensional latent space that fully expresses the temporal dynamics of that biometric. This function will be implemented using a 1-dimensional convolutional or recurrent neural network that is trained on the captured data from multiple subjects. This function is referred to as the encoder. Theoretically, this mapping should be identical across different subjects, therefore the data recorded from all subjects can be used to train the parameters of this mapping. The latent space for each biometric will be separable into a subspace that corresponds to an arousal index, and another complementary subspace describing the biometric specific dynamics. The arousal signal predicted by different encoders should reflect the same value and during the training of the encoder this subspace is shared and enforced to be equal for different encoders. Since the latent variables cannot be explicitly defined or experimentally measured, the encoder cannot be trained with supervised learning methods. Instead, we will use unsupervised algorithms to train our neural networks. To enable unsupervised training, we will use a separate network, termed decoder, that will predict the exact values of each biometric from the latent space corresponding to that biometric. The unsupervised loss will be calculated by quantifying the mismatch between the predictions of decoder and the ground truth biometric measurements and used to train both the encoder and the decoder.
Emitter and receiver circuits 500 and 502 are simultaneously controlled by controller 504 to synchronize digital pulsing of the IR light and analog recordings of the photocurrent in the photodiode(s). Receiver circuit 502 is replicated for each photodiode in the circuit, and the emitter circuit 500 is replicated for each desired illumination wavelength.
Our portable high-speed inexpensive arousal monitoring device has applications in medicine, research, marketing, and entertainment and will be of interest to a broad range of users. Manufacturers of miniature acquisition or photostimulation hardware can easily enhance their existing products with our device, gaining far more functionality with little additional cost. Open source lab instrumentation companies may also be interested in making the detection devices described herein. Rapid biometric monitoring is key to employ cutting edge neuroscience research which may entice major manufacturers of behavioral instrumentation equipment. Another application of the subject matter described herein is pupillometry and eye tracking for video games.
The following are potential applications of biometric optical photometry as described herein:
In this implementation of the invention, the BOP LED and sensors are held in a static position with respect to a head-fixed animal, restrained to an experimental setup. The LED, sensors, and other optical components are brought individually or together at their appropriate location. Each component is then wired to the controller.
In this implementation, the optical components of the BOP are assembled on a small 3D printed frame that is surgically glued to the skull of the animal. After recovery, the implanted device is connected to the control board by means of a wire tether allowing free displacements within the range of the experimental assay. This implementation can be modified to be compatible with other neuroscience technologies that target neural ensembles (i.e. optogenetics, 1 photon miniscopes). The geometry of the head implant can be adjusted to leave clear access to specific brain regions to implement these other modalities.
In this implementation, the optical components of the BOP are assembled on a small 3D printed frame and connected to a circuit board that is surgically glued to the skull of the animal. The circuit board includes a wireless chip (e.g., a Wi-Fi, BlueTooth, or ultrawideband chip), control electronics, signal recording and digitization, as well as a small detachable battery. Once the battery is in place, the circuit can be turned on and controlled remotely, usually via bluetooth protocols. This modality allows experimental recordings from freely-moving, untethered, animals.
This implementation enables the BOP signal and calcium activity from neural ensembles to be captured concurrently. By routing the signal from the photodiode to a channel that is normally connected to one of the two photomultiplier tubes (PMT) in a 2 photon microscope, BOP data can be read in synchronously with neural activity, all within the same data acquisition system. The data is then processed to extract biometric information. In this modality, the infrared laser used for two-photon imaging also serves as an illumination source for BOP measurements. This modality involves an additional computational step to compensate for the scanning of the laser beam over the imaged brain region.
In this implementation, the optical components of BOP and multiple sensors are used to track the position of the pupil in real-time. Differences in intensity collected from each sensor are used to infer pupil position along the horizontal and vertical directions. This implementation can be added to any other modality.
The technology can be utilized for the sole purpose of tracking eye motion in substitution for any currently used eye tracking technology. Eye tracking is used in behavioral studies, for user experience feedback, or in consumer applications to control a touchless screen. It can also be used as an assistive device for patients suffering from heavy paralysis and rely on eye-controlled keyboards to communicate.
The BOP technology can be used to monitor pupil size through a closed eyelid. The eye lid adds a static amount of absorption, but infrared light can still be detected through it. Operating a BOP sensor through a closed eye lid continues to capture fluctuations of the pupil size. Hence, this modality can be used to measure the pupillary reflex in sleep studies or during surgery, to estimate the level of awareness (e.g. patient is awake) in real time.
The following are features of the subject matter described herein.
Back-illumination. Instead of recording signals without a light source or with light originating from the front of the face, our approach illuminates the back of the eye using diffused light passing through bone, skin and tissue, enabling the collection of hemodynamics similar to pulse oximeters. The wavelength we operate with is selected to allow maximal transmission through tissue and is not detected by the human eye.
Camera-free technique. Many pupillometry techniques measure the size of the eye from camera images. Our camera-free design facilitates portability. Cameras are slower than our technique and generate large datasets that need substantial processing. Our device outputs analog signals, achieves kilohertz sampling speeds, and does not require substantial processing to live-stream the raw data in real-time.
Simultaneous parallel acquisition of eye and body biometrics and environmental cues. Our technique not only measures eye properties such as motion and pupil size, but also body metrics (heart and breathing rate, and properties of the environment (ambient light, head motion). All existing technologies only allow a fraction of these measurements and from distinct devices.
Continuous tracking across changing environments of pupillary dynamics and markers of arousal. Our invention does not reduce the field of view and relies on invisible light that the eye cannot detect. Low power operation also allows for continuous and comfortable operation for several hours and without disturbing behavior.
Classification of pupillary events. Since our device simultaneously captures both the level of ambient light, and pupil size in real time, our algorithm can reliably identify if any detected pupillary change results from either an environmental factor, i.e., a variation in the ambient light, or if the event occurs in response to behavioral changes or arousal responses.
Fashionable device with a compact form factor. Our technology can be integrated within safety equipment, or as a standalone modified pair of glasses. The detectors can be easily concealed in a pair of glasses' frame, and the IR source can be built as a small ear-plug or a skin-contact device placed at the edge of the glasses. The device can be made both comfortable for implementation in personal protective equipment, and as a fashionable device for consumer-grade utilizations.
Head mountable device for research in freely moving animals. Our technology employs a low weight device that allows animals to still engage in their full repertoire of naturalistic behavior, as opposed to heavier technologies that employ bulky camera systems.
Self-calibrating algorithms with real time operation. We will implement machine learning models trained offline with phantoms to directly extract the relevant data from the raw signal measured by the sensors. This strategy has the advantage to merge calibration and processing into the same pipeline and facilitate the utilization of the same device on various people and/or animal models.
Compatibility with existing neuroscience technology. Our system can be used alongside other optical brain interfacing technology including calcium, voltage imaging methods as well as optogenetic techniques, allowing its implementation as an add-on to many existing experimental protocols, and without disturbing their respective operations.
In step 902, the process includes controlling the at least one light source to illuminate the eye of the subject from within the head of the subject. For example, controller 504 may control light source(s) 100 to generate pulses of infrared light to illuminate the eye of the subject from within the subject's head.
In step 904, the process includes sensing, using at least one light sensor located external to the head of the subject, light from the at least one light source exiting the subject through the eye of the subject. For example, one or more sensors 106 may detect light exiting the subject's eyes through the pupils of the subject's eyes. It should be noted that the sensing performed in this step can detect light exiting the subject's eyes when the subject's eyes are open or closed. That is, if a subject is sleeping or undergoing anesthesia, for example, sensors 106 may detect light exiting the subject's eyes through the subject's closed eyelids, which may be partially transparent to some wavelengths of light, such as infrared light wavelengths.
In step 906, the process includes recording an indication of the light sensed by the at least one sensor while controlling the illuminating. For example, controller 504 may digitize and record data detected by sensors 106 as they detect light exiting the user's eyes. The recording may be performed synchronously with the pulsing of light by light source 100.
In step 908, the process includes determining a physical characteristic or activity of the subject based on the light detected by the light sensors. For example, the biometric measurement module may determine at least one of eye movement, eye blinking, pupil size variation, heart rate, heart beats, blood flow, pulse oximetry, breathing rate, and respirations of the subject based on the light sensed by the at least one light sensor.
The disclosure of each of the following references is incorporated herein by reference in its entirety.
It will be understood that various details of the subject matter described herein may be changed without departing from the scope of the subject matter described herein. Furthermore, the foregoing description is for the purpose of illustration only, and not for the purpose of limitation, as the subject matter described herein is defined by the claims as set forth hereinafter.
This application claims the priority benefit of U.S. Provisional Patent Application Ser. No. 63/287,482, filed Dec. 8, 2021, the disclosure of which is incorporated herein by reference in its entirety.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/US2022/052302 | 12/8/2022 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 63287482 | Dec 2021 | US |
