Patent Application
20220133212
Embodiments of the invention(s) herein relate to systems and/or methods to observe head position and/or motion, eye position and/or motion, pupil size, and/or eyeblinks and use these observations to measure ocular parameters such as saccades, vestibulo-ocular reflex, vestibulo-ocular reflex cancellation, vergence, smooth pursuit, nystagmus, dynamic visual acuity, pupil size, and/or eyeblinks. These ocular parameter measurements can be used to determine human health status, including:
Eye and head tracking, can have significant value in determining human health such as concussions (traumatic brain injury), vision impairment, neurocognitive disorders (NCD), alertness, and fatigue. Eye movement responses can also indicate physiologic and biochemical impairments such as impairments due to alcohol and drugs.
Historically, human health status has been diagnosed in a clinical setting. New sensors and electronic technologies enable the development of portable systems for non-clinical environments. Such advancements facilitate increases in speed and accuracy for eye and head movement observations to measure ocular parameters such as saccades, vestibulo-ocular reflex (VOR), vestibulo-ocular reflex cancellation (VORC), vergence, smooth pursuit (SP), nystagmus, dynamic visual acuity (DVA), pupil size, and/or eyeblinks. These ocular parameter measurements can more effectively determine human health status.
Human health status. It has been demonstrated that neurologic disorders, such as traumatic brain injury, microvascular ischemic disease, stroke, Alzheimer's disease, and Parkinson's disease can produce measurable changes in one or more of the following ocular parameters: VOR, saccades, smooth pursuit, vergence, nystagmus, and/or pupillometry. It has also been demonstrated that biochemical health impairments including alcohol and drug use, electrolyte, and hormone deficits (associated with dehydration, renal failure, and diabetes), and pulmonary deficits (resulting in hypercapnia or hypoxia), can cause measurable changes in one or more of the following ocular parameters: VOR, saccades, smooth pursuit, dynamic visual acuity, vergence, pupillometry, and/or eyeblinks. Additionally, it has been demonstrated that physiologic health impairments such as alertness/fatigue, spatial disorientation, vertigo, motion sickness, effects of electromagnetic waves due to exposure, and cardiac hypotension can cause measurable changes in one or more of the following ocular parameters: pupillometry, saccades, VOR, nystagmus, and smooth pursuit. Thus, having a portable device that can be used in a non-clinical environment for measuring one or more of the above ocular parameters would be highly beneficial. Each of the aforementioned human health detriments can affect different areas of the neurologic system and each of the ocular parameters to be measured can assess different anatomical regions and neural pathways of the brain. Human health status and certain health disorders or impairments may be more accurately detected by different ocular parameter measurements or by using a combination of ocular parameter measurements.
For example, the vestibulo-ocular reflex maintains eye fixation during head rotation. It is the head rotation that generates this reflex. Measurement of this reflex assesses the neural pathway response from the inner ear balance structures (e.g., utricle, saccule, semicircular canals) to the vestibular nerves, the vestibular nucleus in the brainstem (medulla and pons), and cranial nerves III, IV and VI, which move the extraocular muscles. The superior branch of the vestibular nerve innervates the lateral semicircular canal, anterior canal, and utricle. The inferior branch innervates the posterior semicircular canal and saccule. It is important to assess VOR function of each individual semicircular canal, as different impairments can affect different semicircular canals. For example, TBI can affect one semicircular canal function and not the others. Other less thorough studies, such as rotary chair testing or videonystagmography, evaluate only the function of the lateral canal. Measuring each of the semicircular canals as described herein at varying frequencies, particularly between 2 and 10 Hz is much more representative of natural head movements and can provide more information regarding impairments affecting the inner ear (e.g., vestibular, or labyrinthine impairments).
Smooth pursuit another ocular parameter, is a voluntary eye movement which requires attention on a moving visual target. It is designed to keep a moving visual target on the fovea, the region in the retina of the eye where visual acuity is highest. The goal of the smooth pursuit system is to generate eye velocities that are like the target speed, as the target is tracked. Visual information from the retina is transmitted to the middle temporal area (MT) and from this area motion signals are transmitted through the parietal sensory-motor cortex to the smooth eye movement region of the frontal eye fields (FEF) in the frontal lobe of the brain. From these 3 areas, the MT, the parietal sensory-motor cortex, and the FEF, signals are transmitted through a variety of brainstem relay nuclei to at least two regions of the cerebellum, the oculomotor vermis and the floccular complex. From these areas, signals are then relayed to the vestibular nuclei in the brainstem and lastly project to the extraocular muscles, to move the eyes.
Saccades represent another ocular parameter and are described as rapid conjugate eye movements that are under both voluntary and reflex control. Saccades are mainly used for orienting gaze towards a visual object of interest or target. The eyes execute a series of very rapid movements from one point to another, stopping briefly at each point, the fixation point, to assess the visual image. Examples of voluntary saccades are self-directed eye movements in response to skimming a text. The sudden appearance of a peripheral visual object or an abnormal sound may evoke a reflex saccade in the direction of the stimulus. Visual information from the retina generates a signal which is sent to the primary visual cortex in the occipital lobe of the brain. Signal information is then sent to the frontal eye fields (a region in the frontal lobe), the parietal cortex and the superior colliculus of the midbrain. The superior colliculus activates the horizontal and vertical brainstem gaze centers in the pons. The horizontal gaze center and the sixth nerve nucleus allows for coordinated abduction and adduction of the eyes to shift within the horizontal axis by movement of the lateral rectus and medial rectus muscles. The vertical gaze center is in the rostral part of the midbrain which sends signals to the nuclei in the pons, which subsequently provides movement of the superior oblique, inferior oblique, superior rectus, and inferior rectus muscles. Activation of the horizontal and vertical gaze centers in concert allows for oblique eye movements. The frontal eye field of one hemisphere controls voluntary saccadic eye movements that are directed toward the contralateral visual hemifield (i.e., the right frontal eye field directs the eyes to the left).
Pupillometry is another ocular parameter, which is used to record and measure reactions of the pupil, such as size, position and movement with minute fluctuations seen in response to a stimulus. The pupillary light reflex constricts the pupil in response to light, and pupillary constriction is achieved through the innervation of the iris sphincter muscle. Light reaching the retina is converted into neuronal signals. Information travels through the optic nerve, then goes through the optic chiasm to reach the pretectal nucleus in the midbrain, where it communicates with parasympathetic nuclei in the midbrain called Edinger-Westphal nuclei, which controls the efferent pupillary pathway. Efferent parasympathetic fibers travel via the oculomotor nerve to directly innervate the iris sphincter muscles. The contraction of the iris sphincter muscles leads to pupillary constriction (miosis). It is also at the optic chiasm that some nerve fibers cross to the contralateral side of the optic tract. As a result of the crossing fibers, there is not only a direct pupillary reflex but also a consensual pupillary light reflex. The pupillary dilation pathway is a sympathetically driven response to stimuli and is a three-neuron pathway. The first-order neuron begins in the hypothalamus and descends to the spinal cord in the lower cervical area and upper thoracic area. The second-order neuron exits the spinal cord and ascends through the thorax onto the superior cervical ganglion. The superior cervical ganglion, a third-order neuron, enters the orbit via the ophthalmic division of the trigeminal nerve and ends on the dilator pupillae muscle, causing pupillary dilation.
Eye observation. As mentioned previously, ocular parameters can be measured from observations of eye and head information. Observed eye and eyelid information of interest can include horizontal eye movement (left-right when seen from the front), vertical eye movement (up-down when seen from the front), rotation of the eyes (when seen from the front), eye position, pupil size and movement, and eyeblinks. It is desired to use a system and/or method for observing these eye parameters that is as accurate, repeatable, responsive to higher frequencies, portable, and low cost as possible. In the past, non-imaging technologies, such as detectors that used scleral coils, placed on the eye have also been used. More recently, imaging technologies that detect light information reflected off the eye and convert this light information to electrical information have become the most common technologies for eye and eyelid observation. Such imaging technologies typically have a light source and a photodetector or photodiodes. The light source can be monochromatic or polychromatic, it can be ambient or generated by the device, it can be visible, or at a wavelength that is shorter than visible (ultraviolet) or longer than visible (infrared), it can be a focused and directed beam or multidirectional. Examples of imaging technologies can include video cameras that use an array of photodetectors (typically charge conducting devices known as CCDs), video cameras that scan a beam onto an image and sense this with a photodetector (such as the broadcast video cameras in common use from the 1930s to the 1990s), and scanners that direct a beam at the eye that is then sensed with a photodetector. Imaging of the eye can be used to determine information such eyeblinks, the location and size of the pupils, and/or the location of the cornea or other eye feature being measured at any time. This location information can determine gaze and eye closure. By taking multiple successive location readings at successive times, it is possible to determine movement of the eye or eyelid as a function of time. It is desirable to have this eye and/or eyelid movement information at fast enough rates to accurately resolve human physiological behavior such as eyeblinks (which can occur at a mean rate of 17 blinks per minute when resting, 26 blinks per minute in conversation and as low as 4 blinks per minute when reading) and saccades, which can occur at a rate of 900 degrees/second and involved movements of only 0.1-0.5 degrees and generally 0.1 degree in the case of microsaccades.
Head observation. The described eye observations can be combined with head observations to determine ocular parameters such as VOR, DVA, VORC, nystagmus, SP, saccades, and dynamic vergence with combinations of other ocular parameters. The primary head observations of interest are pitch, which is movement of the head up and down when looked at from the front, and yaw, which is a side-to-side movement of the head when viewed from the front. The third rotation that is sometimes of interest is roll, as rotation of the head when looked at from the front. Such head rotations can easily be observed using an inertial measurement unit (IMU), which can comprise one or more accelerometers, gyroscopes, and magnetometers. Other systems and methods for observing head position and/or movement can use optical (including infrared), acoustic, ultrasonic, and laser technologies.
It is desired to overcome some of the following challenges of prior art systems and methods that use eye and head observations to assess human health status:
The potential benefits of devices described herein is that they can change the standards of care and clinical recommendations by optimizing rapid evaluation and treatment. Since ocular parameters assess different areas of the central nervous system and different factors of the human health status, such systems and methods can function as physician extenders by detecting abnormalities, monitoring the recovery process, and establish rehabilitation programs to more rapidly recover from an abnormal ocular parameter detected. The technology can be applicable to any head-worn platform (i.e., apparatus), such as a see-through device (helmet, glasses, face shield, augmented reality, etc.), a virtual reality device, or any related technology. Such systems and methods could determine the human health status, detect a health disorder or impairment, define the characteristics of the deficit, quantify the deficit, and wirelessly transmit this information to the wearer of the device, and/or remotely. If an abnormal ocular parameter is noted for users participating in contact sports or military personnel participating in remote areas, not only can a rehabilitation program begin earlier but an accurate monitoring strategy for decisions regarding the need for extraction or returning to play or deployment be determined. Mobile testing for clinics, hospitals, emergency departments, law-enforcement environments, urgent care centers, school facilities, on-the-field locations, or in remote areas can provide greater access for evaluation of the human health status because of its portability, lower expense, scalability, and availability of rapid and accurate results for larger numbers of individuals.
The present invention will be better understood on reading the following detailed description of non-limiting embodiments thereof, and on examining the accompanying drawings, in which:
It should be understood that the drawings are not necessarily to scale. In certain instances, details that are not necessary for an understanding of the invention or that render other details difficult to perceive may have been omitted. It should be also understood that the invention is not necessarily limited to the embodiments illustrated herein.
The ensuing description provides preferred exemplary embodiment(s) only, and is not intended to limit the scope, applicability, or configuration of the disclosure. Rather, the ensuing description of the preferred exemplary embodiment(s) will provide those skilled in the art with an enabling description for implementing preferred exemplary embodiment(s). It should be understood that various changes could be made in the function and arrangement of elements without departing from the spirit and scope as set forth in the appended claims.
Specific details are given in the following description to provide a thorough understanding of the embodiments. However, it will be understood by one of ordinary skill in the art that the embodiments may be practiced without these specific details.
In a preferred embodiment, the present invention comprises systems and/or methods for ocular parameter measurement that use one or more head orientation sensors and one or more eye imaging devices (sensors) integrated into a head-worn platform (i.e., apparatus, device). Examples of head-worn platforms can include a face shield, a visor, eyeglasses, and goggles. The head-worn device can comprise a display, such as an augmented reality display or a virtual reality display, or could have no display. The eye sensor(s) can use an imaging device to generate eye information such as horizontal eye movement, vertical eye movement, eye position, pupil size, pupil position, pupil movement, eyelid movement (e.g., eyeblinks), static eye fixation, and dynamic eye fixation. The eye sensor can comprise an imaging device that uses at least one opto-electric transducer that converts light signals into electrical signals and is configured for imaging eye and or eyelid features at a plurality of times to create a moving image of the eye or eyelid. The system/methods can combine the eye information and head orientation information to measure ocular parameters such as saccades, vestibulo-ocular reflex (VOR), vestibulo-ocular reflex cancellation (VORC), vergence, smooth pursuit (SP), nystagmus, pupil size, dynamic visual acuity (DVA), and/or eyeblinks. The systems/methods can be configured to determine a human health status such as normal health, a neurologic disorder, a biochemical impairment, and/or a physiologic impairment. Embodiments of the systems and methods can be portable and can be deployed in a non-clinical setting.
The following table shows examples of the relationship between health categories (disorders or impairments) and the underlying causes, and measurable ocular parameters.
As shown in this table, neurologic disorders affecting different areas of the central nervous system, may require different ocular parameter measurements to detect an abnormality. The ocular parameter being tested must involve the neurologic pathway which was affected by the disorder. Additionally, certain health disorders have characteristic ocular parameter abnormalities. For example, abnormal microsaccade rates have been observed in numerous human health disorders or impairments. Here are some further examples:
Abnormal eye movements can precede detection of neurocognitive disorders before neuropsychological cognitive testing and can accurately assess different cognitive domains. The cognitive control of eye movements requires synchronization of circuits between frontal and parietal cortex as well as subcortical nuclei to produce appropriate context-specific responses. The oculomotor networks substantially overlie the hemispheric attentional systems in frontal, temporal, and parietal lobes interacting at many levels. Cognitive processes are most readily examined with saccades, smooth pursuit and pupillometry when measuring ocular parameters. XR and 3-D displays can be designed to provide a rehabilitation prescription (e.g., a type of electronic prescription) to improve specific cognitive visual deficits or cognitive domains. The use of such visual displays and measuring ocular parameters can provide information of the cognitive function as well as quantify the degree of the cognitive deficit detected. The use of eye and head movement information described in this document can be valuable in cognition evaluation, care, management, and long-term planning for anyone suspected as having a neurocognitive disorder.
Based on the foregoing, it should be apparent that head-worn systems and methods that measure ocular parameters can be valuable for assessing human health status.
The definitions that follow apply to the terminology used in describing the content and embodiments in this disclosure and the related claims.
An artificial intelligence system is defined as a computer system which attempts to implement aspects of human-level intelligence, in which a machine can learn and form judgements to improve a recognition rate for information as it is used. Artificial intelligence technologies include a machine learning (deep learning) technology that uses an algorithm that classifies/learns the characteristics of input data by itself and an elemental technology that simulates functions such as recognition or judgment, like the human brain. The elemental technology can include a visual comprehension technique for recognizing objects as in human vision.
Angular velocity is defined as speed of a physical object that is moving along a circular path. The angular velocity of an object is the object's angular displacement with respect to time. Angular velocity is the rate of change of the position angle of an object with respect to time, so w=theta/t, where w=angular velocity, theta=position angle, and t=time. Angular velocity, also called rotational velocity, is a quantitative expression of the amount of rotation that a spinning object undergoes per unit time. It is a vector quantity, consisting of an angular speed component and either of two defined directions or senses.
Augmented reality (AR) is defined as technology that superimposes a computer-generated image on a user's view of the real world, thus providing a composite view. This superimposed digital overlay can, in a limited manner, interact with the environment in real-time but users cannot interact with it like they would be able to in real life. Mixed reality (MR) is a type of augmented reality in which real and virtual worlds are combined to produce new environments and visualizations where physical and digital objects co-exist and interact. In this document, the terms AR and MR are used synonymously as both represent visual display systems. Extended reality (XR) in this document is an umbrella term incorporating AR, VR and MR display systems.
Biochemical health impairment is defined as an impairment of production, regulation, structure, levels, or physical properties of the biological or chemical nature of hormones, immunoglobulins, electrolytes, gases, or metabolites. This would include proteins, carbohydrates, lipids, and nucleic acids, the mechanisms of enzyme action, the chemical regulation of metabolism, the chemistry of nutrition, the molecular basis of genetics (inheritance), the chemistry of vitamins, energy utilization in the cell, and the chemistry of the immune response. Most biochemical diseases affect the brain, and many lead to mental deficits, developmental delays, behavioral problems, or neurologic handicaps. For example, the brain requires certain levels of neurotransmitters to function. They are the molecules used by the nervous system to transmit messages between neurons, or from neurons to muscles. Biochemical health impairments associated with mental disorders have included abnormalities of hormones such as serotonin, dopamine, norepinephrine, and gamma-aminobutyric acid (GABA). The hypothalamic-pituitary-adrenal axis (HPA axis), is responsible for the release of various hormones, including cortisol, which regulate the stress response. Additionally, toxic drugs can impair the biochemical health of humans. For example, exposure to cocaine before birth may affect the way a child's brain functions many years later, due to alterations in metabolic processes that enable brain cells to use energy and function properly. Biochemical impairments which affect brain function can be detected when they cause abnormal effects on the neural pathways in the brain which are related to the ocular parameter being measured.
Biometrics can be defined as measurable physical characteristics or personal behavioral traits used to identify, or verify the claimed identity of, an individual. Iris features, facial images, fingerprints, voice recognition, behavior characteristics and handwriting samples are all examples of biometrics which can be captured visually and/or electronically with various sensors and analyzed. Fundamentally biometrics involve any metrics related to human features.
A classifier (or machine learning classifier) is defined as an algorithmic computer vision tool that takes an input data frame (an image for example), processes the pixel-level information against a target, and outputs a result. A classifier attempts to identify a pattern within the pixels in the image and compares that pattern to its target set. Classifiers can be of a machine learning type (such as a convolutional neural networks or general adversarial networks) or of a static type (such as Haar cascades), and typically require some form of training for optimization.
Clock generators are integrated circuits (ICs) that generate timing signals for electrical circuits. MEMS clock generators use micro-electromechanical systems (MEMS) technology. Clock generators typically use oscillators, which typically work on mechanical resonance, and can therefore be called resonators. They can use phase locked loops (PLLs) to produce selectable or programmable output frequencies. Clock generators can generate one output or multiple outputs that supply timing signals for electronic systems that require multiple frequencies or clock phases.
Cognition is defined as the mental action or process of acquiring knowledge and understanding through thought, experience, and the senses. It encompasses various aspects of high-level intellectual functions and processes such as attention, memory, knowledge, decision-making, planning, reasoning, judgment, perception, comprehension, language, and visuospatial function. Cognitive processes are defined as encompassing all information processing even at the subconscious level or as the ability to think and reason. Other related cognitive processes are concept formation, pattern recognition, imagery, and problem solving. Human eye movements have been shown to provide information of cognition, cognitive processing, and cognitive disorders.
Cognitive disorders are defined as any disorder disease, or condition that significantly impairs the cognitive functioning of the person to the point where normal functioning in society is impossible without treatment. These disorders are commonly detected by ocular parameter measurements. There are various causes of these disorders including traumatic brain injuries, infectious diseases or other diseases of the brain, drug effects, lifestyle factors and abnormal metabolic and/or hormonal factors which have an adverse effect on the brain. Genetic influences also play a role in many different cognitive disorders. They are a part of neurocognitive disorders. Alzheimer disease is the most well-known disorder associated with cognitive impairment. The use of eye position and eye movement measurements, related to eye fixation, can be used to assess cognitive disorders, and provide key treatment approaches. Visual and cognitive processing occurs during eye fixations which makes vision-based testing, such as with ocular parameter measurements, vital as a sensitive approach in the initial evaluation of cognitive disorders. For example, VOR, smooth pursuit, vergence, and VORC can be used to determine cognitive deficits or neurocognitive disorders. Additionally, other major eye movement parameters, such as fixations, saccades, eyeblinks, and pupillary measurements can provide distinct information about cognitive effort in response to task demand.
A concussion is a traumatic brain injury that results in temporary loss of normal brain function. It is characterized by immediate and transient alteration in brain function, including alteration of mental status or level of consciousness, that results from mechanical force or trauma. Concussions can be caused by direct trauma to the head, such as from falling, getting hit or being in an accident. They can also occur because of rapid acceleration-deceleration of the head, such as in whiplash injuries or blast injuries, such as in a war zone. A concussion can affect memory, judgment, reflexes, speech, balance, and muscle coordination and is associated with abnormal ocular parameter measures. In this document, it is used interchangeably with traumatic brain injury (TBI).
A convolutional neural network (CNN) is defined as a deep learning neural network designed for processing structured arrays of data such as images. Convolutional neural networks (CNN) are widely used in computer vision and many visual applications such as image classification. The CNN contain many convolutional layers stacked on top of each other and each layer processes an incoming image such that the series of layers progressively identify more complex features.
Corneal reflex is defined as an eye blink in response to tactile stimulation of the cornea. More specifically, it is a contraction of the orbicularis oculi in response to light touch of the cornea. It is polysynaptic, the afferent limb of the reflex being the ophthalmic division of the fifth cranial nerve, the efferent limb running in the seventh nerve. Absence of the reflex can be associated with specific diseases affecting the central nervous system.
Duction of eye movement refers to the rotation of one eye considered independently of its fellow. A prefix is attached to this word to indicate the direction in which the eye is rotated.
Dynamic eye fixation is defined as the ability to fixate on a visual target of interest, which is in motion. Static eye fixation refers to the ability to fixate on a stationary visual target of interest. In normal human activities, when viewing objects in the visual field, the head has natural motion or has movement and we follow moving objects or observe stationary visual targets of interest, while we are in motion. When observing a visual object of interest, it is important to have a focused position of the eye on the visual object when these objects are stationary or in motion, and the head is in motion. Our ability to maintain dynamic and static fixation on these visual targets while we are in motion, performing our daily activities, can provide a measure of human performance.
Dynamic visual acuity (DVA) in this document is defined as an ocular parameter measurement to quantify the visual impairment (i.e., visual acuity loss) while viewing a stationary visual target and moving the head. The total visual acuity lost (i.e., DVA loss) due to dynamic movement of the head is calculated by identifying the total difference in visual acuity between dynamic and static conditions (i.e., DVA—static visual acuity [SVA]). In subjects with a normal functioning VOR, minimal loss of visual acuity occurs between dynamic and static conditions. Visual acuity, also known as clarity of vision or sharpness of vision, refers to a person's ability to see small details.
Dynamic Visual Acuity Testing (DVAT) is defined as a computerized VOR assessment method to evaluate the peripheral vestibular system during head movements, and specifically assesses visual acuity during head movement relative to baseline static visual acuity. In this document DVAT represents DVA. In normal individuals, losses in visual acuity are minimized during head movements by the vestibulo-ocular system that maintains the direction of gaze on an external target by driving the eyes in the opposite direction of the head movement. When the vestibulo-ocular system is impaired, visual acuity degrades during head movements. Adequate function of the peripheral vestibular system, specifically the vestibulo-ocular reflex (e.g., a network of neural connections between the peripheral vestibular system and the extraocular muscles) is essential for maintaining stable vision during head movements. Decreased visual acuity resulting from an impaired peripheral vestibular system may impede balance and postural control and place an individual at risk of falling. Reduced visual acuity, as documented with DVA testing, is significantly reduced in individuals with vestibular deficits and is an important ocular parameter measurement used to detect other human health disorders, impairments, or deficits.
Electromagnetic exposure refers to human tissue penetrated by electromagnetic fields (EMF) or electromagnetic radiation (EMR). Moving electric charges produce both electric and magnetic fields. The properties of electromagnetic radiation depend strongly on its frequency and the frequency is the rate at which the radiating electromagnetic field is oscillating. Frequencies of electromagnetic radiation are given in Hertz (Hz), with radio frequency (RF) being on the lower end and gamma-ray being on the upper end of the electromagnetic spectrum. These waves of electric and magnetic energy moving together, can cause damaging electromagnetic effects to human health. The electromagnetic harmful effects, to the human body are characterized by the presence of a wide spectrum of non-specific multiple organ symptoms, disorders, and impairments, typically including the central nervous system following acute or chronic exposure to the electromagnetic field or radiation. More specifically, exposure to electromagnetic fields can induce changes in calcium signaling cascades, activate free radical processes, cause fractures of DNA, alter hormone production, and overproduce reactive oxygen species (ROS) in living cells as well as alter neurological and cognitive functions and disruption of the blood-brain barrier. Autonomic nervous system effects of EMF/EMR could also be expressed as symptoms in the cardiovascular system. Other common effects of EMF include effects on skin, microvasculature, immune and hematologic systems. Individuals can have neurologic, neuro-hormonal and neuro-psychiatric symptoms following exposure to EMF/EMR because of neural damage and over-sensitized neural responses. More recently, suspected electromagnetic exposure has been suspected with the “Havana Syndrome”, when individuals reported hearing strange sounds before feeling ill. Symptoms of exposure include headaches, vestibular disturbances such as dizziness and disorientation, ear pain, tinnitus, fatigue, confusion, insomnia, difficulty focusing, and cognitive deficits. Examination of these individuals have exhibited an array of oculomotor visual findings including convergence insufficiency, abnormal smooth pursuit, and saccadic dysfunction impairment of the vestibulo-ocular reflex. The use of head and eye tracking can provide relevant ocular parameter measurements, as described in this document, to detect abnormalities associated with this physiologic health impairment
Extended reality (XR) is defined as an umbrella term incorporating virtual reality (VR), augmented reality (AR), and mixed reality (MR).
Eyeblinks are defined as the action of closing and re-opening the eyes. Eyeblinks are either voluntary, involuntary (such as a spasm), or reflex blinks (evoked by an external stimulus). A voluntary eye involves cortical control. Blink patterns can be comprised of incomplete or partial blinks, prolonged eyelid closure time and short blink intervals. When the eyes are closed during a blink, there is no incoming visual information to process. Eyeblinks can indicate changes in attention, fatigue, and cognition. Specifically, features of eyeblinks in this document include the frequency of eyeblinks or eyeblink rate, the amplitude, velocity of blinks, blink latency, and the duration of blinks which can be measured to detect different human health disorders or impairments. This ocular parameter of eyelid performance or eyeblink information can be useful in detecting human health deficits such as with neurologic disorders, biochemical and physiologic impairments.
Eye Correcting Algorithm (ECA) is an algorithmic computer vision tool. It builds upon a classifier by attempting to account for movement between the opto-electric transducer itself and the eye being observed. This movement is typically referred to as slippage and the ECA takes the input data frame (the same image as the classifier), processes the information to determine appropriate offsets, and supplies the offset parameters as its output.
Eye Features are defined as the structural features of the eye, including the outer layer comprised of the sclera and cornea, the limbus or border between the transparent cornea and opaque sclera, the middle layer comprised of the iris and pupil, the inner layer comprised of the retina with the vessels, optic nerve as well as the macula, and the eyelids which cover the eyeball for protection.
An eye imaging device is defined as any mechanical, digital, or electronic apparatus for recording, storing, or transmitting visual images. Examples include still cameras, video cameras, and scanners. Imaging devices can comprise light sources, lenses, prisms, mirrors, and other means for converting images or light paths. These means for converting the image or light path can be passive or could be active, an example would be a micro-opto-electromechanical system (MOEM). Imaging devices also typically comprise a detector. The detector could be a photodectors (e.g., an opto-electric transducer) which converts the optical signal into an electric signal. It could also be an array of electro-optical sensors, such as the charge conducting device (CCD) arrays found in some video cameras. In this document, eye tracking, eye sensor and/or eye orientation sensor all represent an eye imaging device, and the term may be used interchangeably.
Eye tracking is defined as the process of measuring where we look, also known as point of gaze. A light source, such as near-infrared light, is directed towards the center of the eyes (pupil), causing detectable reflections in both the pupil and the cornea (the outer-most optical element of the eye). These resulting reflections, the vector between the cornea and the pupil, are tracked by an infrared camera. This is the optical tracking of corneal reflections, known as pupil center corneal reflection. The pupil provides information of gaze direction and glints inform eyeball location. These measurements can be carried out by an eye sensor or sensing unit, such as an imaging device comprised of an opto-electric transducer that detects the position and movements of the eye and converts the light signal to an electric signal.
Eyelid position is defined by its location and as being normal when in primary gaze (e.g., binocular fixation while looking straight ahead). For example, in the resting position the eyelid position may be open, partially open or closed. The upper eyelid is positioned about 1 to 2 mm inferior to the superior limbus (the junction of the white sclera and clear cornea). Measured in another manner, the normal upper eyelid position in an individual may be up to 5.5 mm above the mid-pupil (or center of the cornea).
Eyelid movement is defined as the motion of the eyelid (e.g., an eyeblink) to position the eyelid in a particular place. More specifically, it is related to the velocity of an eyeblink, the duration of the eyeblink, the amplitude, as well as the frequency of eyeblinks, and whether the eyeblink is voluntary, involuntary, or reflexive during the upward or downward motion to position the eyelid in a specific location. In this document eyelid movement represents eyeblinks and is used to determine eyelid performance.
A face shield is a device or item of personal protective equipment (PPE), which aims to protect the wearer's entire face (or part of it) from trauma, injury or hazards. In this document and the appended claims, face shields, visors, goggles, and eye shields are used synonymously. These devices can be attached to a helmet or worn separately and can function as a display for displaying visual targets to the user or can be comprised of an extended reality device to view visual targets for ocular parameter measurements.
Fixation of eyes or visual fixation is defined as maintaining the visual gaze on a single location. More specifically, it refers to a collection of relatively stable gaze points that are near in both spatial and temporal proximity. During fixation, the eyes hold steady on an object, and thus fixation reflects attention to a stimulus and strongly correlate with task performance. Because task performance is also correlated with effort expenditure there is a link between fixation frequency and cognitive effort. Fixations are those times when our eyes essentially stop scanning about the scene, holding the central foveal vision in place so that the visual system can take in detailed information about what is being looked at. Fixations are brief pauses from scanning a scene to gather a more comprehensive idea of what is being viewed. The eye remains relatively still, spending time in one location to understand the visual world. Most fixations last between 50-600 ms, but the amount of time spent on a specific fixation is dependent on both the task and stimulus. Fixations are excellent measures of visual attention and visual fixation ability on an object of interest, while the head is stationary or in motion and can be an accurate and predictable measure of human performance and the human health status.
Focused position of the eyes is defined as the position or orientation of the eyes to provide a clear image of a visual element, visual object, or target of interest on the fovea.
Foveated rendering is defined as a process which renders most of the view into a virtual world at lower resolution, except for the exact area directly in front of user's gaze. The gazed area is rendered at a higher resolution. More specifically, foveated imaging uses a digital image processing technique in which the image resolution varies across the image according to the user's gaze, and a foveated display is therefore designed to function in the context of user gaze. The head-worn device can then use a display with regions of lower and higher resolution to reduce power consumption and information bandwidth needed for the display. The electric circuit carrying the information may be shared by lower and higher resolution portions of a display, or different portions of a display with different resolutions may be supplied with different electric circuits carrying the information. Foveated rendering can be used in extended reality (XR) applications by using a foveated display and an eye tracking system to monitor the gaze of the user, and to optimize rendering to focus only where the user is actively looking on a display. If the focused position of the eye is determined, the display can show the higher-resolution picture in the individual's direct view, and the lower resolution information remains in the remainder of the display. The result is improved latency and computations.
Gaze or gaze point is defined as where the eyes are looking. More specifically, a gaze point is the instantaneous spatial location of the visual axis landing on the visual stimulus. As such, it has an (x, y) coordinate and a timestamp corresponding to its measurement. If a series of gaze points is very close, in time and/or space, this gaze cluster constitutes a fixation, denoting a period where the eyes are locked towards an object. Gaze can serve as a reliable indicator of attention and cognitive effort.
Global Shutter is defined as an imaging sensor that is capable of simultaneously scanning the entire area of an image. This is contrasted with a rolling shutter where the image area is scanned sequentially, typically from the top to bottom. Some consumer and industrial machine vision and 3D sensing need a global shutter to avoid motion blur. Target applications include facial authentication and eye tracking.
A hologram is defined as a three-dimensional image reproduced from a pattern of interference produced by a split coherent beam of radiation (such as a laser). It represents an image created by a photographic projection of a recording of a light field and appears as a three-dimensional representation on a two-dimensional object.
Kinetosis is defined as the state of being dizzy or nauseated because of motions that occur while traveling in or on a moving vehicle. This can occur with ocean travel (e.g., sea sickness), air travel (e.g., air sickness), or automobile travel (car sickness). Kinetosis describes symptoms related to simultaneous vestibular, visual, and somatosensory stimulation. These symptoms include nausea, dizziness, headache, malaise, increased salivation, vomiting, and drowsiness. The sensory conflict theory suggests kinetosis can be attributed to the conflict that occurs between the visual, vestibular apparatus (e.g., labyrinthine portion of the inner ear) and somatosensory systems resulting from real or virtual motion. Afferents from the vestibular apparatus arrive at the vestibular nuclei of the brainstem, which also receives inputs from the visual and proprioceptive systems. Efferent projections then reach the temporoparietal cortex via the posterolateral thalamus, triggering autonomic reactions and the vomiting center. When there is a discrepancy between actual versus expected patterns of vestibular, visual, and kinesthetic inputs, it initiates the cascade of motion sickness symptoms. Motion sickness can be induced in almost all subjects with a functioning vestibular apparatus given a sufficient provocative environment, and results in vestibular nystagmus. Fixation of gaze or closure of eyes generally prevents visual motion sickness, while vestibular otolithic function is eliminated in microgravity of space, indicating a predominant pathogenetic role for visuo-sensory input. The causes of kinetosis symptoms can be related to subjects with central disorders, such a with migraines, or a cerebrovascular accident such as a stroke or basilar artery occlusion. Labyrinthine impairments can also cause kinetosis symptoms of motion sickness as they can increase visual-vestibular conflict.
A Light sensor is defined as a device used to detect light and comprises many different types of light sensors responsive to different and/or multiple wavelengths or types of light sources and may function in different ways. In the spectrum of electromagnetic waves, the ranges of frequencies which can be detected by light sensors are between infrared, near infrared (NIR), short-wave infrared (SWIR) light, to visible light, and up to ultraviolet light. They can detect and convert the light energy in the form of photons to electrical energy in the form of electrons. They may also be known as image sensors, photo sensors, photo-electric devices, photodiodes, photo transistors, photo resistors, and optical detectors. A commonly used light or image sensor is the camera or video camera. The video camera can also be defined as an imaging device that has at least one opto-electric transducer configured for converting a light signal to an electrical signal. A charged coupled device (CCD) transports electrically charged signals and is used as a light sensor in digital cameras and night-vision devices. Photomultipliers detect light and multiply it. Light sensors can be classed into three types based on the physical quantity that is affected. The main classes are photo resistors, photo voltaic and photo emitters. Photo emitters generate electricity when exposed to light. Photo resistors change their electrical properties when illuminated and photo voltaic generate a potential proportional to the intensity of the light. One such photo-electric device is the photojunction diode which uses light to control the flow of electrons across the junctions. A photodiode is in the class of photojunction device which is essentially a PN junction light sensor. They are generally made from semiconductor PN junctions and are sensitive to visible light and infrared light. When light is incident on a photodiode, the electrons and holes are separated and will allow the junction to conduct.
Linear velocity is defined as the speed and direction of a physical object that is moving in a straight line. It is the rate of change of the object's position with respect to time.
Machine Learning is defined as the science of getting computers to learn and act like humans, and improve their learning over time in autonomous fashion, by feeding them data and information in the form of observations and real-world interactions. Machine learning fundamentally is the practice of using algorithms to parse data, learn from it, and then make a determination or prediction about something in the world. This entails getting computers to act without being explicitly programmed and is based on algorithms that can learn from data without relying on rules-based programming.
Margin to Reflex Distance is defined as the distance from the eyelid margin to the corneal light reflex. It is measured by having an individual fixate on a light and measuring the distance from the margin of the eyelid to the corneal light reflex. There are two margin-to-reflex distances (MRD), corresponding to the measurement from the upper and lower eyelids. MRD1 is the upper eyelid margin-to-light reflex and is the single most important measurement when evaluating ptosis. MRD2 is the lower eyelid margin-to-light reflex. MRD1+MRD2 provides the palpebral fissure height.
Near accommodative triad is a three-component reflex that assist in the redirection of gaze from a distant to a nearby object. It consists of a pupillary accommodation reflex, lens accommodation reflex, and convergence reflex.
A neurocognitive disorder is defined as a disorder characterized by a decline primarily in mental function due to a medical disease of the brain caused by a variety of acquired disorders and/or impairments such as cerebrovascular disease, Alzheimer's disease, infections, drug effects, abnormal metabolic and hormonal deficits, and trauma, such as with concussions. It is an acquired condition representing underlying brain pathology, that results in a decline in cognitive abilities such as memory, problem solving, executive function, intellect, and perception. Ocular parameter measurements, such as smooth pursuit, saccades, pupillometry, VOR, vergence, or nystagmus can be used to determine different neurocognitive disorders, based on the neurologic pathway associated with the disorder.
A neurologic disorder is defined as a disorder that affects the brain, the spinal cord and/or nerves found throughout the human body, resulting in physical dysfunction. Structural, biochemical, physiologic, or electrical abnormalities in the brain, spinal cord or other nerves can result in a large range of symptoms. There are numerous different types of neurological disorders including, but not limited to cerebrovascular/stroke, degenerative, neurocognitive disorders, demyelinating diseases, brain lesions including tumors, as well as neuro-oncology disorders and each can affect different areas of the central nervous system. Measures of ocular parameters, such as smooth pursuit, saccades, pupillometry, VOR, vergence, or nystagmus can be used to determine the many different neurologic disorders, also based on the neurologic pathway involvement associated with the disorder.
Nystagmus is a general term defined by two oscillatory involuntary eye movements, with a “slow phase” eye movement followed by a second movement (e.g., fast phase or a re-fixation phase or saccade) to stabilize images on the retina. This eye movement can be horizontal, vertical, torsional or a combination. Nystagmus can occur as a result of several factors including: a) vestibular stimulation (e.g., vestibular nystagmus); b) visual stimulation (e.g., optokinetic nystagmus); or c) it can occur without any vestibular or visual stimulation (e.g., spontaneous nystagmus). Vestibular nystagmus can be induced by head and/or body movement and due to stimulation of the vestibular apparatus in the inner ear. Optokinetic nystagmus (OKN) is induced by a visual stimulus moving across the visual field. OKN is also referred to as the optokinetic reflex (OKR). The term “optokinetic nystagmus” is synonymous with “visual nystagmus”. Spontaneous nystagmus can occur as a result of an acute labyrinthine impairment, stroke or other lesion in the brainstem or cerebellum. When present, nystagmus can be characterized by eyes that drift slowly in one direction and then jerk back in the opposite direction, called “jerk nystagmus”. Alternatively, nystagmus can be referred to as a “pendular nystagmus”, characterized by eye motion moving like a pendulum swinging back and forth. Although nystagmus is often described by the direction of its quick phases (for example, downbeat nystagmus), it is the slow phase that reflects the underlying disorder. Generally, vestibular nystagmus, optokinetic nystagmus and spontaneous nystagmus can all be associated with labyrinthine impairments or central nervous system disorders. However, specific features of the nystagmus can help to differentiate central disorders from labyrinthine impairments. For example, spontaneous vertical or pure torsional nystagmus can indicate a central lesion. Central lesions typically have a direction changing, gaze-evoked nystagmus (i.e., left-beating nystagmus on left gaze, and right-beating nystagmus on right gaze) and frequently have decreased fixation suppression. However, suppression of nystagmus with fixation is more typical of labyrinthine impairments. Additionally, the type of nystagmus (e.g., jerk, pendular, etc.) can provide a clue to the origin of the nystagmus. For example, pendular nystagmus can result from a brain disorder, such as with multiple sclerosis, and jerk nystagmus is more often related to a labyrinthine impairment. During nystagmus, vision is reduced or limited. Nystagmus can have variable amplitude and frequency, and can be worsened or improved by gaze position, fixation, vergence, or covering one eye (latent), depending on the cause. The nystagmus intensity is defined as the slow-phase velocity (SPV). Measurements include this SPV and duration as well as binocularity, frequency, and amplitude. Nystagmus can be a valuable ocular parameter to determine human health and help to differentiate disorders, such as stroke or brain lesions from physiologic impairments, including spatial disorientation, and motion sickness.
Ocular Parameters are defined as measurable factors that define and determine the components, actions, processes, behavior and functional ability of the eye, eyeball, and eyelid. Included in ocular parameters are eye position, eye and eyelid movement responses which can be detected or measured, including saccades, vestibulo-ocular reflex, vestibulo-ocular reflex cancellation, vergence, smooth pursuit, nystagmus, dynamic visual acuity, pupil size, eyeblinks and focused eye position or gaze. Reflexes included in the measured ocular parameters or eye movement responses include the pupillary light reflex, pupillary dark reflex, near accommodative triad, corneal reflex, palpebral oculogyric reflex (Bell's reflex) and the optokinetic reflex (OKR). Measuring movements of eye includes the extraocular muscles (which move/rotate the eye), the ciliary muscles (which helps to focus by changing the lens shape), the levator (which raises the eyelid), and the pupillary muscle (which dilates or constricts the pupil).
Ocular reflexes are involuntary responses that are usually associated with protective or regulatory functions They require a receptor, afferent neuron, efferent neuron, and effector to achieve a desired effect.
The oculomotor system is defined as the part of the central nervous system (CNS) centers, complex CNS connections or pathways, numerous peripheral inputs, cranial nerves III, IV and VI and the extraocular muscles, which functions mainly in maintaining visual stability, aligning, and controlling eye movements. It is made up of many brain areas that cooperate to stabilize images of interest on the high-acuity part of the retina. Assessment of deficits in oculomotor function is useful to detect visuomotor impairments due to a closed head injury and other neurologic disorders as well as biochemical and physiologic impairments.
An opto-electric transducer is defined as a device that converts an optical signal into an electrical signal. Examples of such a device include photodetectors, photosensors, charge conducting devices (CCDs), and photodiodes.
Palpebral Fissure is defined by the normal exposed area between upper and lower eyelids. The palpebral fissure vertical height (PFH) is measured between the margin of the lower eyelid to the margin of the upper eyelid. The normal palpebral fissure height is 8-11 mm. This is a key measurement that is needed when assessing ptosis.
Palpebral oculogyric reflex (Bell's reflex): The palpebral oculogyric reflex, or Bell's reflex, refers to an upward and lateral deviation of the eyes during eyelid closure against resistance, and it is particularly prominent in patients with lower motor neuron facial paralysis and lagophthalmos (i.e., incomplete eyelid closure).
A photodetector is defined as a device that turns light into an electrical signal. This can be an opto-electric transducer which converts the optical signal into an electrical signal. Multi-element photodetectors can be used for imaging. A non-imaging photodetector is a device that turns light into an electrical signal but has too few elements to produce an image. Thus, a non-imaging photodetector might comprise only one light-sensing element that turns received light into a magnitude based on the intensity of the light received. A non-imaging photodetector might comprise two light-sensing elements that allow the detection of an edge, but not an image, and therefore can be called an edge-detection photodetector. A non-imaging photodetector might comprise a two-dimensional pattern of three or four photodetectors, that allow the detection of an edge in more than one dimension, but not an image. Photodetectors could work with visible light, they could work with invisible light (such as infrared or ultraviolet), or they could work with a combination of visible and invisible light.
Physiologic health impairment is defined as impairments of the normal biological functions of organs, tissues, or cells of humans. It also includes impairments affecting the vital functions, growth and development, the absorption and processing of nutrients, the synthesis and distribution of proteins and other organic molecules, and the functioning of different tissues, organs, and other anatomic structures, such as the pulmonary, cardiac, and neurologic systems.
Pupillary light reflex refers to an autonomic reflex that constricts the pupil in response to light, thereby adjusting the amount of light that reaches the retina. Pupillary constriction occurs via innervation of the iris sphincter muscle, which is controlled by the parasympathetic system.
Pupillary dark reflex is defined as the reflex that dilates the pupil in response to dark. It can also occur due to a generalized sympathetic response to physical stimuli and can be enhanced by psychosensory stimuli, such as by a sudden noise or by pinching the back of the neck, or a passive return of the pupil to its relaxed state.
Pupillometry in this document is defined as the measures of pupil size, the (e.g., diameter of the pupil), which includes pupil movement features as a function of time, to determine pupil performance. The size of the pupil is determined by the reactivity of two antagonistic iris muscle movements: the sphincter, which reacts to parasympathetic stimuli, and the dilator, which reacts to sympathetic stimuli. Pupil size is affected by the level of retinal illumination, the accommodation state of the eye, various sensory and emotional conditions, cognitive and affective information processing as well as neurologic disorders, such TBI, biochemical impairments, such as with drug use and physiologic impairments, such as with alertness. Pupil measures includes movement features of pupil diameter, dilation information including acceleration, latency, duration of changes in size, amplitude changes, and constriction information including latency, duration as well as amplitude changes. It also includes peak and average constriction velocity from the iris sphincter muscle as well as dilation velocities of the dilator pupillae muscle under numerous stimulus conditions, including dim pulse, dim step, bright pulse, bright step, bright red step, and bright blue step. A higher frame rate (although pupil activities have a relatively low frequency) is desirable to determine time parameters with a higher precision and examine the time dependence of the phase between the pupil and other bio-signals. A higher spatial or linear resolution also can provide information of micro-fluctuations in pupil size. The pupil performance or pupillometry (which results from measures of pupil size and other pupil features described above) can be an important biomarker for neurologic disorders, biochemical and physiologic impairments.
Saccades are defined as rapid, ballistic movements of the eyes that abruptly change the point of fixation when gazing from one object to another. The eye movements between fixations are generally referred to as saccades. Like fixations, saccades are made up of multiple gaze points and they have a start and end point each with a timestamp. Measures can be made at which point saccades occurred in time and their duration. Whereas VOR, OKN, and pursuit movements all have the function of stabilizing the world, or a particular object image, the saccadic system serves to change our gaze from one point to another. The purpose of saccades is to alter the gaze from one object of interest to another under effort of will (voluntary saccades), to alter the gaze to a sudden event in the periphery (reflex saccades), to reset gaze after VOR or OKN has carried it to an extreme (fast phases of nystagmus), to correct small errors of fixation (fixational microsaccades), and to correct small errors in pursuit (catch-up or back-up saccades). Vision is disrupted during saccades, a phenomenon called saccadic omission. Saccadic omission occurs because of visual masking in which the image seen before the saccade tends to mask the image seen during the saccade. Retinal blur occurs as the images move rapidly across the retina because the retina has limited temporal resolution. Saccade parameters of measurement includes accuracy, amplitude, inhibition, latency, duration, velocity with initial acceleration and peak velocity, frequency, and number over time. These quantitative measurements of saccades are used to assess the function of the oculomotor system, to investigate the effects of drugs or lesions, and to aid diagnosis of neurologic disease or locating of brain lesions in the central nervous system.
Saccade accuracy refers to the eye's ability to quickly move and accurately shift from one target fixation to another. Accuracy refers to how well the calculated fixation location matches actual fixation location. This is expressed in degrees of visual angle (a half circle has 180° of visual angle). Saccade adaptation is a process for maintaining saccade accuracy based on evaluating the accuracy of past saccades and appropriately correcting the motor commands for subsequent saccades. An adaptive process is required to maintain saccade accuracy because saccades have too short a duration relative to the long delays in the visual pathways to be corrected while in flight.
Saccade amplitude refers to the size of the eye movement response, usually measured in degrees or minutes of arc. The amplitude determines the saccade accuracy. This is sometimes referred to as gain. It is also described as the angular distance the eye travels during the movement.
Saccade Inhibition refers to an absence or near-absence of saccades initiated around 80-120 msec following a brief visual distracting effect that interferes with the production of scanning saccades.
Saccade latency refers to the time taken from the appearance of a visual target to the beginning of an eye movement in response to that target. Saccades have a latency of typically about 200 msec. This is significantly longer than OKN, smooth pursuit or vergence, and more than ten times longer than VOR. Many factors influence saccade latency. Longer latencies occur with weak (dim or low contrast) targets, unpredictable targets, and with older subjects. Shorter latencies occur with brighter visual targets, predictable targets, with auditory stimuli, and with younger individuals.
Saccade velocity is defined as the speed measurement during the eye movement. High peak velocities and the main sequence relationship can also be used to distinguish microsaccades from other eye movements such as ocular tremor, ocular drift and smooth pursuit.
Saccades have a very high velocity, up to 800 or even 1000 degrees per second for very large saccades. Saccade velocities follow a very specific, predictable pattern such that the peak velocity of the saccade is dependent on its amplitude. Saccades are reprogrammed after each fixation period. In most cases, if a target moves during a saccade, the saccade in progress is not modified and the next saccade will not occur until one latency period after the end of the first saccade. Therefore, saccades have been called ballistic, meaning that they are determined before they are started and cannot be redirected during movement. Inaccurate control of saccades is termed saccade dysmetria, undershoots are referred to as hypometric and overshoots are termed hypermetric. Peaks corresponding to saccadic movements show a linear relationship between the peak velocity of a particular saccade and the amplitude. Once the peak velocity has been reached, the amplitude of the saccade, and therefore the final position of the eye after the saccade can be determined with a high degree of accuracy. Saccades have fixed relationships between the amplitude, duration, and peak velocity. There are main sequence parameters and relationships. Generally, in normal individuals there is a linear relationship between saccade amplitude and duration.
Sampling rate of eye tracking refers to how many times per second eye position is measured. Common sampling rates are 1,000 Hz, 500 Hz, 250 Hz, 120 Hz, 90 Hz and 60 Hz. During normal adult reading, fixation durations typically vary from about 100-800 milliseconds, with the average being approximately 250 milliseconds. Higher sampling rates produce better temporal accuracy when measuring the duration of fixations and saccades. Specifically, the average temporal error will be approximately half the duration of the time between samples. For example, a sampling rate of 1,000 Hz (sampling eye position every 1 millisecond) will lead to an average error of 0.5 millisecond and a sampling rate of 60 Hz (sampling eye position every 16.7 milliseconds) will lead to an average error of approximately 8 milliseconds.
Sensor fusion is defined as any stage in an information integration process in which there is a combination (i.e., fusion) of different sources of sensory information into a single sensory signal. In one form, sensor fusion can be an algorithm that combines sensory data or data derived from disparate sources such that the resulting information has less uncertainty than would be possible when these sources were used individually.
Sensor integration refers to the synergistic use of the information provided by multiple sensory devices to assist in the accomplishment of a task by a system. The sensors used for sensor fusion and/or sensor integration can be of the same type (such as opto-electric transducers for a stereoscopic image) or of differing types (such as combining accelerometer and gyroscopic data in a Kalman Filter).
Situational awareness (SA) is defined as being aware of one's surroundings, comprehending the present situation, and being able to predict outcomes. It is a key human skill that, when properly applied, is associated with reducing errors of human performance activities.
Slippage is defined as when an imaging device viewing a subject's eye moves out of phase with the subject's head. The slippage offset is an algorithm that can account for slippage and computes an appropriate value that can be used to synchronize sensor data.
Smooth pursuit is defined as the voluntary movement of the eyes in response to tracking a moving visual object. Such movements are under voluntary control in the sense that the observer can choose whether to track a moving stimulus. These movements are described to be smooth, continuous, conjugate eye movements with velocity and trajectory, determined by the moving visual target. However, the eyes are in continual small-scale motion, showing irregular drift and tremor, interspersed by miniature saccadic movements (less than 0.5 degrees). With the naked eye, smooth pursuit movement appears smooth, but with high-speed eye imaging devices, the movements are not entirely smooth at all, but can have an altered appearance due to the presence of saccades (covert or overt) or saccadic intrusions which can be associated with underlying neurologic disorders or other physiologic or biochemical impairments. There are separate mechanisms of control for horizontal and vertical smooth pursuit tracking. Smooth pursuit eye movement can be divided into two stages: open-loop pursuit and closed-loop pursuit. Open-loop pursuit is the visual system's first response to a moving object we want to track and typically lasts approximately 100 msec. Therefore, this stage is ballistic and visual signals have not yet had time to correct the ongoing pursuit velocity or direction. The second stage of pursuit, closed-loop pursuit, lasts until the pursuit movement has ceased. This stage is characterized by the online correction of pursuit velocity to compensate for retinal slip. In the closed-loop phase, the eye angular velocity and target angular velocity are nearly equal. Pursuit eye movements are initiated within 90-150 msec, while typical latencies for voluntary saccades are in the order of 200-250 msec. The first 100 msec of pursuit is open-loop, and during this period no visual feedback is available because of the delays in the visual system. Thereafter, visual feedback is available to close the loop, and other sources of information are also available to improve performance. These movements are slower tracking movements of the eyes, designed to keep the moving viewed stimulus on the fovea. Measures of initiation parameters can detect information about the visual motion processing required for pursuit. When a bright light appears in the periphery, the fastest it can achieve a smooth pursuit is 30°/second. It first fixes the gaze to the peripheral light and, if not more than 30°/second, will follow the target equally with the movement. Smooth pursuit can be adversely affected by numerous factors including, but not limited to, alcohol, drugs, traumatic brain injury, mental deficits, learning deficits, human performance impairment, neurologic disorders, visual defects, fatigue, alertness, electromagnetic wave energy effects, motion sickness, eye fixation ability, and visual attention. Measurements of this smooth pursuit parameter while following a moving visual target includes acceleration, accuracy, latency, and velocity.
Smooth pursuit acceleration refers to the rate of change of the eye velocity. The first approximately 20 milliseconds of pursuit tend to be the same regardless of target parameters. However, for the next 80 milliseconds or so, target speed and position have a large effect on acceleration.
Smooth pursuit accuracy is defined by the ability of the eyes to closely follow a moving object. The pursuit of visual targets moving with velocities of greater than 30°/s tends to require catch-up saccades. Smooth pursuit accuracy represents how closely the percentage of time the smooth pursuit velocity value remains within the target velocity value.
Smooth pursuit latency is defined by the time from target appearance to the beginning of pursuit. It is measured from traces of eye velocity. It is often calculated by finding the intersection between two regression functions, one fitted to velocity about the time of target appearance, and the second fitted over the initial part of the pursuit response.
Smooth pursuit velocity refers to the speed of the eye movement (velocity) which usually rises to a peak, following pursuit initiation, and then either declines slightly or oscillates around the target velocity. This peak velocity can be used to derive a value for gain (peak velocity/target velocity). It is usually near the velocity of the target. Instead of using peak velocity, measures of velocity at times relative to either target appearance or pursuit initiation can be made. For example, eye velocity up to 100 milliseconds after target appearance can be used as a measure of prediction or anticipation.
Vergence is defined as the simultaneous movement or orientation of both eyes to rapidly obtain or maintain single binocular vision or ocular fusion of the object of interest. To maintain binocular vision, the eyes must rotate around a vertical axis so that the projection of the image is in the center of the retina in both eyes. Vergence measurements can easily be performed. Normally, changing the focus of the eyes to look at an object at a different distance will automatically cause vergence and accommodation, known as accommodation-convergence reflex. Convergence is the simultaneous inward movement or orientation of both eyes toward each other, usually to maintain single binocular vision when viewing an object more closely. Divergence is the simultaneous outward movement or orientation of both eyes away from each other, usually to maintain single binocular vision when viewing an object which is further away. Typically, vergence velocity responses do not exceed 60 degrees/second. Vergence orientation movements tend to have relatively long latencies, typically on the order of 150-200 msec. Measurements of vergence can be performed while visually following the target element of interest, which moves in a smooth transition to different depths (e.g., dynamic vergence) or in a sequence of steps with the head stationary or head in motion. Such measurements can also include a binocular precision index (BPI) and binocular accuracy index (BAI) to quantify changes of convergence and divergence peak velocity, amplitude, symmetry, and latency. Like other ocular parameters, vergence can be adversely affected by neurologic disorders, biochemical and physiologic impairments.
Version refers to simultaneous movement of both eyes in the same direction; a prefix indicates the direction of the conjugate movement. The muscles in each eye that are the prime movers undergo graded contractions in accord with Hering's Law of innervation, and for each contracting muscle, there is normally a corresponding relaxation of an antagonist (Sherrington's Law). Hering's Law states that the gaze effort command to each eye is the same; if only one eye can see the target, the other eye will receive the same command.
Vestibular apparatus in this document refers to the inner ear balance organs, (also referred to as the vestibular portion of the labyrinth, vestibular portion of the inner ear, labyrinthine portion of the inner ear, and peripheral vestibular system). The vestibular apparatus is comprised of the semicircular canals with sensory elements call crista, which respond to angular acceleration and changes in angular velocity. Additionally, it is comprised of two linear acceleration sensory elements called the maculae, one responsive to movement in the horizontal plane (the utricle) and one responsive to movement in the vertical plane (the saccule). In this document, labyrinthine impairments also refer to impairments of the vestibular apparatus as defined above. The vestibular system refers collectively to the vestibular apparatus of the inner ear, the brain structures, and neural pathways that help control balance and eye movements.
Vestibulo-ocular reflex (VOR) is defined as a gaze reflex or reflexive movement of the eye, that keeps the visual image stable on the retina by producing an eye movement in the direction opposite to head movement, thus preserving the visual image in the center of the visual field. While viewing a static visual target, this reflex is initiated by rotational head movement in one direction, and information from the inner ear organs (e.g., vestibular portion of labyrinth) generate the eye response in the opposite direction. It is a short latency reflex system, which generates a rotation of the eye with an amplitude equal and opposite to the direction of a head movement because of vestibular stimulation, namely the semicircular canals, utricle, and saccule. In this document vestibulo-ocular reflex is synonymous with vestibular ocular reflex. The VOR is valuable in determining the human health status including neurologic disorders, biochemical health impairments, physiologic health impairments, and/or normal health. Measurements of the VOR include accuracy, gain (amplitude ratio of eye to head movement) or peak slow phase eye velocity/peak head velocity), phase (the timing response between the head movement and the reflexive eye response), symmetry (slow component of eye response when head is rotated to one side compared to rotation to the other side), and saccades (overt and covert). Gain of the VOR is also defined as the change in the eye angle divided by the change in the head angle during the head rotation.
Vestibulo-ocular Reflex Cancellation (VORC) is an ocular parameter measurement used to assess visual motion sensitivity and suppresses the VOR during combined eye and head tracking. It is initiated by the moving visual target, and the head moves in the same direction to follow the visual target. Measurements of the eye movement are correlated with the head orientation information. It is also referred to as vestibulo-ocular reflex suppression (VORS) in this document. Specifically, an individual is asked to rotate the head and eyes together while following a moving target, or visual element of interest, while the target moves side to side, up and down, and/or in diagonals. The VORC compensates for head movement to permit the eyes to maintain foveal fixation on moving objects in the environment. Moving objects in the environment are often tracked with head movement during which the eyes are held stable relative to the head. Under these conditions, the VOR must be suppressed to prevent loss of visual fixation on the target. Vestibulo-ocular reflex cancellation is an important ocular parameter measurement used to detect neurologic disorders, such as TBI and other physiologic health impairments.
Virtual reality (VR) is defined as near-reality and describes an artificial three-dimensional, computer-generated environment which can be explored and interacted with a person. Individuals using virtual reality become part of the virtual world or are immersed within the environment and while there, can manipulate objects or perform a series of actions in that 3-D space.
Visual-Oculomotor-Vestibular (VOV) Rehabilitation refers to rehabilitation training therapy designed to improve the health status of individuals with neurologic disorders or other impairments which affect the vision, oculomotor and/or peripheral vestibular system (e.g., inner ear or labyrinthine impairments). Affected individuals have deficits affecting the ocular parameters due the involvement of the central nervous system and/or vestibular neural pathways. In this document and claims, visual-oculomotor-vestibular rehabilitation is a system for improving human health using a display with visual training information and will be used synonymously with visual rehabilitation. Habituation therapy, gaze stabilization, and balance exercises using a visual display are three examples of types of visual-oculomotor-vestibular rehabilitation training.
A waveguide is defined as a device that directs the light wave's transference so that the light wave transmits only in a certain direction. With this device, there is minimal loss of light by restricting the transmission of the light wave to one direction.
Figures Describing Ocular Parameter-Based Human Health Determination
Referring now to the figures,
The head worn unit 602 can measure responses from the stimulus produced at step 690: eye responses that can be measured by the eye imaging device 608; and head movement responses that can be measured by the head orientation sensor 606. Eye response information can be measured and recorded as shown at step 692. This eye information can comprise eye position information, horizontal eye movement information, vertical eye movement information, pupil size information, and eyelid information (and more specifically eyeblink information). Head movement responses can be measured and recorded as shown at step 640. This head movement response can comprise head orientation change information such as pitch and yaw of a person's head, where pitch represents rotation about a first axis representing up and down movement of the person's face when the rear of the person's head moves in the opposite direction, and yaw represents horizontal movement of the person's face when looked at from the person's front about a second axis wherein the second axis is vertical, substantially aligned with the person's spine, and perpendicular to the first axis.
The eye information from step 692 and the optional head information from step 640 can be used to measure ocular parameters, as shown at step 694. Examples of ocular parameters that embodiments of the invention can be configured to measure can include saccades, vestibulo-ocular reflex, vestibulo-ocular reflex cancellation, vergence, smooth pursuit, nystagmus, dynamic visual acuity, and/or eyeblinks.
Further referring to
Among the health categories this health status can be categorized into can include:
The comparisons made with 802, 804, 806, and 808 can then be used to generate a categorized human health status, as shown at 698. The determination of human health status from 698 can then be used to train, treat, and/or rehabilitate as possible and/or necessary, as shown at 820. This training, treatment, and/or rehabilitation will be further described with reference to
When assessing cognitive deficits for human health disorders or impairments, numerous visual tasks can be performed with visual targets on a display. For example, ocular parameter measurements, including smooth pursuit, vestibulo-ocular reflex cancellation, pupillometry, eyeblink information, and dynamic visual acuity, use visual targets for testing and all provide information about cognition and inattentiveness. There are other visual cognitive function tests which can be viewed on a display and determine cognitive deficits. These tasks can be used as visual cognitive function tests, that will be further described with reference to
Embodiments of the inventions described herein can also provide supernormal enhancement of the ocular parameters discussed, where no balance disorder exists, for enhancement of athletic and vocational abilities. Embodiments can enable individuals to reach a higher level of performance in their occupation, enable them to have increased ocular performance functions when participating in their usual occupational or play activities as well as enabling cognitive training and rehabilitation.
Regarding 810 (health history), diagnosis of health status has been described as both a process and a classification scheme, or a pre-existing set of categories agreed upon by the medical profession to designate a specific condition. The working diagnosis may be either a list of potential diagnoses (a differential diagnosis) or a single potential diagnosis. Generally, there are four types of information-gathering activities in the diagnostic process: 1) taking a clinical history and interview; 2) performing a physical exam; 3) obtaining diagnostic testing; and (4) sending a patient for referrals or consultations. A subject's clinical history includes documentation of the current concern, past medical history, family history, social history, symptom history, and other relevant information, such as current medications (prescription and over-the-counter) and dietary supplements. An accurate history facilitates a more productive and efficient physical exam and the appropriate utilization of diagnostic testing. The medical history of a patient is the most useful and important element in making an accurate diagnosis, much more valuable than either physical examinations or diagnostic tests. The medical interview is the process of gathering data that will lead to an understanding of the disease and the underlying physiological process. A common maxim in medicine attributed to William Osler is: “Just listen to your patient, he is telling you the diagnosis.”
The head tracking inertial system can be mounted to the helmet 202, or any other head worn device in numerous configurations. When used with display systems discussed in this document, the head tracking inertial system can refresh on-screen images at a rate of 125-1250 frames per second (or Hz). Higher frame rates reduce movement lag. For specific applications, the refresh rate may be lower than 125 frames per second (fps) or higher than 250 (fps), depending upon the platform (e.g., apparatus) used, the application, and type of measurement or testing being performed. For performing some tests, such as the head impulse test, a sample rate or refresh rate of 250 Hz or higher might be necessary to capture the subtle eye movements, such as overt and/or covert saccades. Reducing the lag between head movement and the headset response will mitigate symptoms of motion sickness or visually induced motion sickness. The resolution use can be variable depending on the application or platform used but may be chosen as 1080×1200 or 2160×1200-2560×1440 or higher and the latency between images should be short (20 milliseconds or less).
Further referring to
The eye imaging device 406 or sensors could be a video camera or cameras. It can be appreciated and understood by those skilled in the art that a video camera is an imaging device that has at least one opto-electric transducer configured for converting a light signal to an electrical signal. In the case of professional video cameras in common use from the 1930s to the 1990s, this involved the use of a scanner to sequentially scan a projected image of a scene and record it using a single opto-electric transducer. Many modern video cameras apply this same concept by using multiple opto-electric transducers in an array (typically a charge conducting device or CCD). The scanning technology in the video cameras is like the scanning technologies that use a single beam light source that is scanned across the eye, either by moving the beam source or by using a mirror to move the point at which the beam is pointing. If a mirror is used, it could be mechanically moved by a system such as a micro-electronic mechanical system (MEMS) device. The resulting scanned reflection off the eye can then be detected using a photodetector, which is an example of an opto-electric transducer that converts an optical signal into an electrical signal.
In the embodiment shown in
Note that the embodiment of the helmet shown at 400 in
In embodiments, biochemical sensors and/or physiologic sensors in contact with the skin can provide human health information that can be correlated with ocular parameter measurements to detect the specific biochemical or physiologic abnormality. These specialized sensors can communicate with the display system to provide specific information regarding the biochemical or physiologic health status. These sensors can also be attached to the display device, in contact with the skin, and on other specific anatomic areas of the body. These sensors can be strategically placed on anatomical structures for the parameter they were designed to detect and measure. As an example, when measuring the pulse rate or blood pressure the sensors are placed over a major artery (such as the superficial temporal or occipital artery). As another example, sweat provides a significant amount of information about a person's health status and is readily accessible, making it suitable for wearable, noninvasive biosensing. Sweat contains important electrolytes, metabolites, amino acids, proteins, and hormones, which allows monitoring of metabolic diseases, or a person's intoxication level. Some anatomic areas of the body have a higher concentration of eccrine sweat glands, such as on the forehead, temporal area and back of the neck. Skin sensors strategically placed on these anatomic areas of increased sweat glands are ideal for measuring dilute electrolyte solution with primary components of bicarbonate, potassium, and sodium chloride, glucose, pyruvate, lactate, cytokines, hormones such as cortisol and immunoglobulins. The skin sensors can detect abnormal biochemical measures including simple monosaccharides (sugar) or dextrose, metabolites, proteins, and electrolyte abnormalities which can be correlated with eye and head tracking measurements to provide information about the health of the individual. Additionally, sensors used to detect and measure other physiologic properties of the user, can be anatomically positioned for the intended specific physiologic measures. These sensors can detect such abnormal physiologic measures as arterial pressure, thermal changes, changes in cardiac activity, such as arrythmia, atrial fibrillation, atrial flutter, cardiac arrest, or the rate of inhalation and expiration. Other sensing elements/transducers, in contact with the scalp or using evoked Potentials (EP), can detect and measure abnormal changes in cranial, motor, or sensory function, mental status, non-focal and focal neurologic changes, abnormal waveform frequency of the cerebral cortex, spike analysis, and electrophysiologic measurement of cerebral activity. These sensors can also communicate with the display system and can be correlated with the ocular parameter measurement to detect abnormalities of the human health.
The embodiment of the helmet shown at 400 in
Features of the system and methods described herein could also be used in a face shield-based embodiment, such as the face shield system shown at 480 in
The face shield-based system 480, of
Referring specifically to the left and right eye imaging device(s) 406 in
It should be noted that embodiments of the present invention can be implemented using video cameras for the imaging devices, shown for example at 406 in
The electronic module 410 shown in
It is possible to have other embodiments of ocular performance-based measurement systems and methods that use some of the elements shown in
Further referring to
It should be noted that the AR and VR embodiments of the inventions disclosed herein can also be implemented using computer-generated 3-dimensional synthetic information instead of the monoscopic or stereoscopic “reality” information used for the augmented reality (AR) and virtual reality embodiments discussed herein.
Embodiments of the invention can add an image or reference marker to a scene to measure ocular parameters of the types that have been discussed previously in this document.
In addition to all of the items described with regard to
Regarding the forward-facing camera, shown at 408 in
Further referring to
Further referring to
Any ocular parameter that was shown and described with reference to
Display: Like the tests described with reference to
Head: In this test, the subject is asked to keep his/her head motionless while looking at the visual target, as shown at step 612. Note that, a head orientation sensor, as described herein, can be used to ensure that the head is motionless during testing.
Eyes: The subject is instructed to keep his/her eyes focused on the moving visual target. The eye imaging device measures eye orientation changes 642, as the visual target is moving. at step 624.
This test uses a head-worn unit 602 that comprises a head orientation sensor 606, and an eye imaging device 608.
This test comprises the following configuration and steps:
There can be many additional embodiments of the ocular performance tests and other uses for the head-worn platform (e.g., apparatus) described with reference to
In the tests described with reference to
The measures shown in
The ideas expressed in the previous paragraph can best be explained by looking at some examples.
DVA testing could be performed with lettered optotypes and as the head rotates back and forth, the letters can rotate in position. Alternatively, numbers can be used as well as other familiar images of objects. The images can also be native or natural to the background environment displayed. As the head rotates back and forth, the visual target is more difficult to visualize. If there is a VOR abnormality, for example the eyes will not be able to focus on the visual target of interest and will subsequently have less fixation and more errors in identifying a visual target. Measurement can also be performed with the visual target stationary and the head in motion or both the visual target and head in motion, which would be more realistic with everyday experiences. Static visual testing (SVT) can be performed to obtain a normal visual test. The visual acuity can be obtained, while the head and the visual target, or optotype being displayed are both motionless. Like a standard eye exam, an XR platform can enable a person's static visual acuity (SVA), a component of DVA testing, by asking a person to identify a multitude of images or optotypes (letters, symbols, characters, figures of different sizes, shapes, orientation) on the visual screen.
Dynamic visual acuity (DVA), and foveal visual stability (FVS) testing can be used to determine a person's vestibulo-ocular reflex performance. Normally, DVA assessment includes identifying a series of static images or optotypes but with the addition of head movement. The displayed images could also be dynamically moving in any direction and can be random in position, appearance, and presentation. Specifically, the image or visual target to be identified can be moving in any direction, randomly or with a specified pattern of motion, and may have different shapes, features, colors, sizes, orientation, patterns, or identifying characteristics, in a specific plane of axis or in variable plane, which the person must identify while the head in motion or rotating. The subject can then provide feedback regarding what they see via an on-screen gesture, keyboard, smart device (e.g., defined as an electronic device, generally connected to other devices or networks via different wireless protocols such as Bluetooth, NFC, Wi-Fi, 3G/4G/5G cellular, etc., that can operate to some extent interactively and autonomously), eye or other physical response or by voice response. The comparison of the smallest visual image or optotypes in both the DVA and SVA tests can determine if the person has a defect in his or her vestibulo-ocular reflex performance. DVA assessment can also be performed as described in
Smooth pursuit testing can be performed with similar visual targets of interest as described previously. Smooth pursuit testing has traditionally been performed with the head motionless and the eyes following a moving light or finger moving across a visual field.
Note that any of the ocular parameter measurements described herein for any of these embodiments can be done with static targets or visual targets being viewed, or with dynamic targets. They can include more complex combinations of different ocular parameters. For example, vergence can be simultaneously measured with VORC and pupil size as the eyes and head are following a visual target moving as it transitions different depths with varying luminosities and when the target simultaneously moves in different directions or patterns as it nears or moves away from the eyes. The images or targets viewed may be familiar objects, such as balls, or objects more familiar to one's occupation. The visual target may be displayed in a manner that is native or natural to the background.
Each semicircular canal is maximally sensitive to rotations perpendicular to its canal plane. The maximum sensitivity of the horizontal canal 74 occurs when the head is tilted forward about 30 degrees. Maximum sensitivity of the left anterior canal 70L and the right posterior canal 72R occurs when the head 98 is rotated approximately 45 degrees to the right about a vertical axis (about the foramen magnum 90), and the head is pitched downward and upwards in a plane parallel to the LARP planes (86L and 86R). Maximum sensitivity of the left posterior canal 72L and right anterior canal 70R occurs when the head is rotated approximately 45 degrees to the left and the head is pitched backwards and forwards in a plane parallel to the RALP planes (88R and 88L).
Human health status related to the vestibular apparatus can be determined using the tests described previously with reference to
Systems and methods for performing visual-oculomotor-vestibular (VOV) rehabilitation can comprise any of the elements and modules described in this document. The display can be a virtual reality display, a see-through display, or any other display capable of being understood by anyone skilled in the art. As an example, VOV rehabilitation could be used for traumatic brain injuries associated with cognitive deficits. VOV can be a specific rehabilitative therapy designed to correct an abnormal ocular parameter measurement detected. For example, for a vestibulo ocular reflex (VOR) abnormality, an individual could be presented with a visual display of a stationary visual target and then asked to rotate the head back and forth in a horizontal and/or vertical manner while focusing on the visual target. This can be repeated, or other variants of the therapy can be presented, depending on the abnormality, until the ocular parameter measurement returns to normal value. The visual target activity can be changed as well as the user head movement for VOV rehabilitation, depending on the abnormal finding of the ocular parameter which was measured. This VOV rehabilitation therapy can be considered a non-pharmacological prescription (e.g., an electronic prescription) used to recover ocular parameter abnormalities. The goal of this rehabilitation using a display with visual stimuli is to use a problem-oriented training approach to promote compensation. This is achieved by customizing exercises to address the specific abnormal ocular parameter detected with each subject. Depending on the abnormal ocular parameter(s) identified, three principal training methods of visual rehabilitation can be prescribed: 1) Habituation Training, 2) Gaze Stabilization Training, and/or 3) Balance Training using the visual display. Habituation Training therapy is indicated for subjects who report increased dizziness when moving, such as quick head movements, or when changing positions. Also, habituation exercise is appropriate for subjects who report increased dizziness in visually stimulating environments. The goal of habituation exercise is to reduce the dizziness through repeated exposure to visual stimuli that provokes patients' dizziness. Gaze Stabilization Training exercises are used to improve control of eye movements so vision can be clear during head movement. These exercises are appropriate for subjects who report problems seeing clearly because their visual world appears to bounce or jump around, such as when reading or when trying to identify objects in the environment, especially when moving. There are different types of eye and head movement exercises used to promote gaze stability. The choice of the exercise(s) depends on the detected ocular parameter. For example, impairment of the VOR can be improved by targeted gaze-stability training. Gaze stability refers to the ability to hold the eyes on a fixed location while the head is in motion. Gaze-stability training requires a subject to maintain visual focus while moving his or her head and can be used in a variety of different conditions to facilitate recovery from VOR impairment. Alternatively, to improve gaze stability the head can be moving in the same direction of the visual target, or the head can move in the opposite direction, while in both conditions the eyes remain fixed on the target. For dysfunction of visual motion sensitivity, graded and systematic exposure to visually stimulating environments in the display can be used as a rehabilitative technique to habituate the subject. For subjects with symptoms of positionally induced vertigo, such as with benign paroxysmal positional vertigo, identified with ocular parameter testing like that described related to
Further referring to
With reference to step 830 in
Further referring to the table above, it should be noted that dynamic activity (walking or other movement) could be added to any of the above training protocols. Such requested movements could be performed at a prescribed metronomic pace. The above training protocols could be performed by the subject multiple times per day.
Regarding step 842 in
There can be numerous other embodiments, capable of being understood by anyone skilled in the art, using the above-described figures which use eye and head orientation sensors for measurement of ocular parameter to determine human health status, including neurologic disorders, such has traumatic brain injury and cognitive deficits; biochemical health impairments, such as alcohol and drug use; and/or physiologic health impairments, such as motion sickness, fatigue, spatial disorientation, and vertigo.
Ocular Parameter Measurement for Human Health Assessment
The measurement of eye fixation ability while performing visual tasks, can provide a rapid method to detect normal human health, concussions (traumatic brain injuries), and other human health neurologic disorders or physiologic or biochemical impairments. Such ocular parameters include saccades, vestibulo-ocular reflex, vestibulo-ocular reflex cancellation, vergence, smooth pursuit, nystagmus dynamic visual acuity, pupil size, and/or eyeblinks. As has been shown, these measurements are used to assess different areas of the brain and each disorder or impairment can adversely affect a specific brain region, the vestibular system, or its connections (e.g., the neurologic pathway).
Some health disorders or impairments are associated with abnormal eyelid performance, such as eyeblinks, and can be detected by measuring the eyelid movements. Specific health issues can affect different neurologic pathways and hence different movements of the eyelids. For example, facial paralysis prevents eyelid closure because the orbicularis oculi muscle, which is innervated by the seventh cranial nerve, severely impairs blinking but does not alter the vertical eyelid movements. Eyelid performance (eyeblink frequency and timing) can be measured to determine cognition, physiologic impairments, (such as attention deficits and fatigue), and biochemical impairments (such as with hypoxia and hypercapnia). Eyelid performance can accurately be measured with a portable head-worn device with the systems and methods discussed in this document, with an XR display or without a display using natural visual targets. In a preferred embodiment, eyelid performance, such as the frequency, amplitude, latency, duration, and timing of eyeblinks, can determine the human health status.
Ocular parameter measurement can be combined with head motion measurement for human health assessment. Head motion is virtually constant with our everyday activities. It is present with sports and with all our activities of daily living, including speaking, reading, breathing, and eating. The only time our head is not likely to be completely motionless is when the head is held in a rigid head frame. Observing objects or visual information in our visual field is most often associated with head motion. Additionally, many individuals have involuntary head movements. Genetics can have a close relationship with involuntary head movement. Other common causes of involuntary head movement can include traumatic brain injuries, strokes, brain tumors, diseases related to thyroid hormone secretion, degenerative disorders like Parkinson's disease, untreated syphilis and others. Our ocular parameters function naturally with head motion while performing everyday activities. The clinical method of testing some parameters, such as smooth pursuit, with the head placed in a head frame is unnatural and will not provide real-life measurements. To understand how an individual performs with ocular parameter measures in real-life or natural environments and determine the human health status, eye and head tracking should both be utilized.
The vestibulo-ocular reflex (VOR) is generated by head motion and produces eye movements in the opposite direction. This allows the visual image to remain stable on the retina so that the VOR gain (eye velocity/head velocity) is equal to unity (gain=1.0). The stimuli during rotational testing of the head can be passive or active impulses or sinusoidal rotations. Impulse rotations demand a rapid acceleration (usually about 100°-150°/s/s) to a constant speed and, after the vestibular provoked nystagmus fades away, a sudden stop during which the nystagmus is again recorded. Sinusoidal rotations are performed by rotating the patient's head or body from side to side so that head movement recordings appear as a series of sine waves. The frequency of the rotations refers to the number of turns per second and is expressed in Hertz. By convention, measurement of the VOR in rotational testing is expressed in terms of gain and phase shift, which is an offset in the timing of eye movement relative to head motion. A gain of 1.0 and a phase shift of 180° indicate perfect VOR, i.e., the eyes move synchronously with head movement but in the opposite direction. The VOR is at its best during head oscillations or rotations of 2 to 6 Hz as encountered in natural locomotion but is less efficient at the extremely low frequencies of head movement. In active head rotation, the subject's head is rotated from side to side horizontally or vertically, as described in this document, at frequencies ranging from 0.1 to 20 Hz. Eye movements and head movements are recorded using sensors discussed in this document attached to the head. At frequencies from 1.0 to 5.0 Hz, VOR gain is not significantly affected whether the head motion is active or passive. This is especially true if an auditory signal indicating the command for moving the head is random. The VOR allows for eye movements in the opposite direction of head movement to maintain steady gaze and prevent retinal image slip. Motion signals from the utricle, saccule, and/or semicircular canals in the inner ear (e.g., vestibular portion of the labyrinth) travel through the utricular, saccular, and/or ampullary nerves to areas in the vestibular nucleus, which sends output to cranial nerve III, IV, and VI nuclei to innervate the corresponding muscles. The VOR is an ocular parameter, which is very useful in detecting abnormalities of human health and can be an accurate measurement to predict human performance ability when motion is present. An abnormal VOR can include overt and/or covert saccades while an individual rotates the head, and it can indicate bilateral, complete, or severe (>90%) loss of vestibular function. In the technology discussed in this document, a display, such as XR, can be used to provide visual target(s) for the individual to view while the head is moving. The VOR assessment can be performed in a natural environment without a display and viewing natural elements or targets while the head is in motion. This can easily be performed with sufficiently high scan rate using eye sensing technology, such as opto-electric transducers, and scanners which can rapidly convert eye information into electrical signals and transmitting this to photodectors. In a preferred embodiment, the VOR can determine the human health status, including neurologic disorders, like TBIs, Multiple Sclerosis, biochemical impairments, which occurs with alcohol use, and physiologic impairments, such as spatial disorientation, and exposure to electromagnetic waves because such impairments affect the associated VOR neurologic pathway.
Vestibulo-ocular reflex cancellation (VORC) occurs when the visual target and head are both moving in the same direction, ideally with the same angular velocity. The vestibulo-ocular reflex (VOR) mechanism triggers eye movements, as a result of head motion, in order to keep gaze stationary relative to the world. However, in order to shift the direction of the gaze along with head motion, the VOR mechanism must be overridden (“cancelled”). Two separate mechanisms provide VOR suppression during eye-head pursuit, that is to overcome the vestibular drive that would move the eyes away from the target of interest. The first is the cancellation of the VOR by a smooth pursuit signal based on a cerebro-ponto-cerebellar pathway. The second suppression mechanism is a partial, parametric reduction of VOR gain (eye velocity/head velocity). These pathways point to the complexity of regulation of the VOR suppression, involving a large network of brain areas and, therefore, being vulnerable to the risk of age and brain lesion-associated deficits of this movement. The VORC can also accurately be measured with a portable head-worn device with the systems and methods discussed in this document, with an XR display or without a display using natural visual targets. In a preferred embodiment, the vestibulo-ocular reflex cancellation can determine the human health status, including neurologic disorders like TBI, biochemical impairments, which occurs with alcohol use, and physiologic impairments, such as dizziness, because such impairments affect specific anatomic regions and neural pathways of the brain related to VORC.
The rapid eye movement of saccades cannot consciously be controlled. After gazing on a moving target of interest, which was initially motionless, it takes about 200 milliseconds for eye movement to begin. During this delay, the position of the target with respect to the fovea is computed, and the difference between the initial and intended position, or “motor error”, is converted into a motor command that activates the extraocular muscles to move the eyes the correct distance in the appropriate direction. If the target moves again during this time (which is in the order of 15-100 milliseconds), the saccade will miss the target, and a second saccade must be made to correct the error. While visual information is not processed during saccadic eye movements, they still can provide information about viewing behavior. According to the theory of visual hierarchy a stimulus is inspected by scanning it through a sequence of visual entry points. Each entry point acts like an anchor, which allows the user to scan for information around it. According to this perspective, longer duration of saccadic eye movements could indicate increased cognitive effort in finding a suitable entry point into a visual display. One reason for the saccadic movement of the human eye is that the central part of the retina (known as the fovea) plays a critical role in resolving objects. By moving the eye so that small parts of a scene can be sensed with greater resolution, body resources can be used more efficiently. The saccade that occurs at the end of a head turn with someone who has an abnormal VOR is usually a very clear saccade, and it is referred to as an overt saccade. An overt saccade can be indicative of abnormal semicircular canal function on the side to which the head was rotated. Covert saccades are small corrective saccades that occur during the head movement of a person with abnormal inner ear function. Covert saccades reduce the need for overt saccades that the end of the head movement and are more difficult to identify than overt saccades. Covert saccades are very fast. This makes them almost impossible to detect by the naked eye, and therefore sensitive eye tracking measurements are typically required to detect covert saccades. There is a rapid deceleration phase as the direction of sight lands on the new target location. Following a very short delay, large saccades are frequently accompanied by at least one smaller corrective saccade to further approach a target location. Corrective saccades can occur even if the target has been made to disappear, further supporting the projected, ballistic nature of saccadic movements. However, corrective saccades are more frequent if the target remains visible. Saccade accuracy, amplitude, latency, and velocity can be measured with oculomotor eye movements, such as smooth pursuit, and vestibulo-ocular movements. Saccades can be elicited voluntarily, but occur reflexively whenever the eyes are open, even when fixated on a target. They serve as a mechanism for fixation, rapid eye movement, and the fast phase of optokinetic nystagmus. The rapid eye movements that occur during an important phase of sleep are also saccades. For saccadic amplitudes up to 15 or 20°, the velocity of a saccade linearly depends on the amplitude (the so-called saccadic main sequence). Saccade duration depends on saccade amplitude. In saccades larger than 60 degrees, the peak velocity remains constant at the maximum velocity attainable by the eye. In addition to the kind of saccades described above, the human eye is in a constant state of vibration, oscillating back and forth at a rate of about 60 Hz. Saccades are important ocular parameter measurements used to detect numerous health disorders, or impairments and can accurately be measured with a portable head-worn device with the systems and methods discussed in this document, with an XR display or without a display using natural visual targets. In a preferred embodiment, saccade measurement can determine the human health status, including neurologic disorders, like TBI and Parkinson's Disease, physiologic impairments, including deficits caused by electromagnetic wave exposure and motion sickness, and biochemical impairments, such with electrolyte deficits and organic solvents because such impairments affect the related saccade neurologic pathway.
When we shift our gaze from a far object to a near object, such as with vergence, our eyes converge, the lenses of our eyes modify their focus (accommodate), and our pupils often constrict. The mechanism and control of vergence eye movements involves complex neurological processes that can be compromised in individuals with traumatic brain injury, thus frequently resulting in a wide range of vergence dysfunctions and related near-work symptoms, such as oculomotor-based reading problems. Vergence requires that the occipital lobes be intact, and the pathway involves the rostral midbrain reticular formation (adjacent to the oculomotor nuclei) where there are neurons that are active during vergence activities. It comprises a complex and finely tuned interactive oculomotor response to a range of sensory and perceptual stimuli. There is an important interaction between the vergence system and vestibular labyrinth (e.g., inner ear balance organs) system. To keep the eyes focused on a visual target or object of interest, while the head is moving, the vestibular labyrinth system senses head rotation and linear acceleration which activates the eyes to counter rotate to keep gaze constant even though the head is moving. Measuring the ocular parameter vergence is important as it can be adversely affected not only by concussion or traumatic brain injury (TBI) but also by factors including aging, drug use, visual abnormalities and with some other physiologic impairments. Vergence can accurately be measured with a portable head-worn device with the systems and methods discussed in this document, with an XR display or without a display using natural visual elements. In a preferred embodiment, measures of vergence can determine the human health status, including neurologic disorders like TBI, physiologic impairments, including deficits caused by electromagnetic wave exposure, and biochemical impairments, such with drug use because these disorders or impairments affect the anatomic region and neurologic pathway related to vergence in the brain.
Pupillometry entails the measurement of minute fluctuations in pupil diameter in response to a stimulus and assessment of pupil measures includes the pupil features previously discussed. The size of the pupil is controlled by the activities of two muscles: the circumferential sphincter muscle found in the margin of the iris (innervated by the parasympathetic nervous system), and the iris dilator muscle, running radially from the iris root to the peripheral border of the sphincter (which contain α-adrenergic sympathetic receptors that respond to changes in sympathetic tonus and changes in the blood level of circulating catecholamines). The pupillary light reflex is an autonomic reflex that constricts the pupil in response to light, thereby adjusting the amount of light that reaches the retina. Pupillometry can accurately be measured with a portable head-worn device with the systems and methods discussed in this document, with an XR display or without a display using natural visual elements. In a preferred embodiment, pupil measurements can determine the human health status, including neurologic disorders, such as Alzheimer's Disease and strokes, physiologic impairments, including decreased attention and cardiac deficits, and biochemical impairments, such as with drug use, because such impairments affect the neurologic pathway related to pupil performance and activity.
Smooth pursuit, the eye movement in which the eyes remain fixated on a moving object, is an important ocular parameter measurement for detecting neurologic disorders and/or biochemical and physiologic impairments. Smooth pursuit can also be accurately measured with a portable head-worn device with the systems and methods discussed in this document, with an XR display or without a display using natural visual elements. In a preferred embodiment, measures of smooth pursuit can determine the human health status, including neurologic disorders, like TBI, and microvascular brain disease, physiologic impairments, including motion sickness, spatial disorientation, deficits caused by electromagnetic wave exposure, and biochemical impairments, such with drug and alcohol use because these disorders or impairments involve the smooth pursuit neurologic pathway in the central nervous system.
The rhythmic eye movement of nystagmus is comprised of a slow eye movement driving the eye off the target followed by a second movement that brings the eye back to the target and can be comprised of various waveforms. Nystagmus can accurately be measured with a portable head-worn device and the systems/methods discussed in this document, with or without a display. In an embodiment, nystagmus can detect the human health status, including neurologic disorders, such as strokes, and physiologic impairments, including motion sickness, spatial disorientation, and dizziness because these disorders or impairments can affect the area in the central nervous system associated with occurrence of nystagmus.
When optokinetic nystagmus (OKN) occurs, the initial eye movement is a smooth pursuit movement followed by an opposite saccade back to the primary gaze or direction of visual interest and when the optokinetic system is activated, a perception of self-motion, (i.e., vection) occurs. The neural pathways for optokinetic nystagmus include the smooth pursuit pathway and accessory optic system. This ocular parameter measure is used detect visual and neurologic disorders and can be measured with a portable head-worn device with the systems and methods discussed in this document, with or without a display using natural visual elements. In a preferred embodiment, OKN can determine the human health status, including neurologic disorders, like brain tumors of the rostral midbrain, or optic nerve lesions, biochemical impairments with metabolic abnormalities involving the cortex, brainstem and/or cerebellum, and physiologic impairments, such as inner ear or labyrinthine impairments, because such disorders or impairments affect the associated OKR neurologic pathway.
Infectious diseases can adversely affect ocular parameters. For example, Covid 19 has been associated with demyelination of the optic nerve, pupillary changes, and cranial neuropathy affecting cranial nerves III, IV and VI, resulting in paresis. In an embodiment, such impairments can be detected by pupil movement, VOR, DVA and vergence because of the affected neurologic pathway.
In one embodiment, the head worn device can be comprised of an XR display device with at least one affixed eye imaging sensor and head orientation sensor discussed herein with an electronic circuit. The eye imaging sensor can be configured to generate an electrical image in response to eye information received from the eye sensor and the head orientation sensor can be configured for generating an electric head orientation signal in response to head movement or head information. The eye information can include horizontal, vertical and tortional eye movements and eye position, pupil size and movement activity information as well as eyelid movement information, including eyeblink frequency, amplitude, latency, duration, and timing. The eye imaging sensor and head orientation sensor can be responsive to the display. The eye imaging sensor can be comprised of an imaging device constituted of at least one opto-electric transducer. This imaging device can convert a light signal into an electric signal and is configured to image the eye features or surface features of the eye and the eyelid performance. The imaging device can image the eye a plurality of times to create a moving image of the eye or surface of the eyes. The electronic circuit can be responsive to the eye imaging device electric signal and the head orientation electronic signal. The eye information received from the imaging device electric signal can be compared to information received from the head orientation electric signal to determine the measured differences. This embodiment can be configured to determine human health status, including those associated neurologic disorders, physiologic and biochemical impairments from the measured eye information responses and head orientation responses and the different measures between the eye sensor and head orientation sensor.
In another embodiment, the discussed system above can also comprise a forward-facing camera, which can communicate with the electronic circuit, eye imaging sensors and head orientation sensor and is configured to transmit video information. This can be used to determine location of eye fixation, identify and correct slippage offsets of the head worn device.
In another embodiment, the system described can be comprised of physiologic and biochemical sensors, which are in contact with the skin and can provide information from the body which can communicate with the electronic circuit and be correlated with the ocular parameter measures.
In an embodiment, the present invention can be comprised of a device which uses eye and head position and movement information to measure ocular parameters to determine human health. The eye information can be acquired from an eye sensor that is comprised of at least one opto-electric transducer configured for converting a light signal to an electrical signal and the head information can be acquired from the head orientation sensor comprised of an IMU. The head orientation sensor and eye imaging sensor(s) can be integrated into the head-worn device. The system described is configured for measuring eye position and eye movement responses and/or eye reflexes and can be comprised of at least one eye imaging sensor, a head orientation sensor and electronic circuit. The head orientation sensor senses pitch and yaw of the person's head in a range of frequencies that comprises at least one frequency greater than 0.01 Hertz and less than 20 Hertz. The head orientation sensor can be comprised of one or more accelerometer(s), magnetometer(s), and/or gyroscopes.
In the embodiments discussed herein, features including a forward-facing camera, extended reality (virtual reality or augmented reality) display, eye imaging device and head orientation sensors and components of the electronic circuit can be activated or controlled haptically, auditorily, remotely, wirelessly, with gestures or movement of the eyes, head, hands or manually with a power switch on the head worn device. As an example, a bone or air conducting sensor incorporated in the framework of the head worn device can provide auditory/acoustic signals to issue an input signal to a controller to operate the system. The electronic circuit can also be activated by placing the head worn device on the head which can issue a similar input signal when in contact with the skin and when removed from the head, the system will automatically become deactivated.
In an embodiment of the device, the system may include the user interface for providing information to the user of the device. The user interface may be associated with displayed images, a touchpad, a keypad, buttons, a microphone, a haptic device, and/or other peripheral input devices. The processor may control functions of the system based on input received through the user interface. The system and/or testing function controls and input connections can be in a head-worn device and/or in a remote device. The computing system could be a distributed computing system. The computing system could comprise cloud computing. The ocular parameter measure methods can be comprised of an application connected to a cloud-based AI infrastructure. The app can be made up of a series of tasks that are completed in a few minutes, and a user's eye movement can be recorded in data sets called Eye Movement Biomarkers (EMBs) and Gaze Mapping Biomarkers (GMBs).
In an embodiment, the head worn system can include a gyroscope, a global positioning system (GPS), magnetometer, and an accelerometer. The head worn eye and head tracking system may be configured to provide information associated with a position and an orientation to the processor. The gyroscope may include a micro-electromechanical system (MEMS) gyroscope or a fiber optic gyroscope as examples. The gyroscope may be configured to provide orientation information to the processor. The GPS unit can include a receiver that obtains clock and other signals from GPS satellites. The GPS unit can be configured to provide real-time location information to the processor. The head worn system may further include an accelerometer configured to provide motion input data to the processor.
In one embodiment, the head worn device or method can present a visual target to one eye (monocular) or both eyes (binocular). A power source can be attached to the head worn device and which can be rechargeable by a wireless interface.
In another embodiment, the head worn device described herein can measure information between position and orientation of the head and eye position, eye movement and/or eye reflexes and the ocular parameter being assessed. The data acquired can be processed by the head worn device and displayed to the user or collected data can be transmitted wirelessly to a smart phone, electronic device, or other computer source for the processing and viewing.
In an embodiment, the head worn display system can include an eye imaging and measuring system, a connected head orientation and measuring system, a power supply, a micro-processor, a memory, and a user interface. Components of the system may be configured to work in an interconnected fashion with each other and/or with other components coupled to respective systems. For example, the power supply may provide power to all the components of the system. The processor may receive information from all the affixed sensors and control the eye imaging system and the head orientation system. The processor may be configured to execute program instructions stored in the memory unit and to generate a display of images on the user interface. The display to the user can be presented as a 2D or 3D (3 dimensional) extended reality (virtual reality or augmented reality) display.
In another embodiment, the head worn system can have a manual control operating switch with an active and inactive mode. It can provide real-time feedback on the display screen, has the capability to display time and can be adjusted to fit users of different statures. It can be comprised of an imaging device, a head orientation sensor, physiologic sensors, biochemical sensors, an electronic circuit comprising a central processing unit with memory unit and a display system which can project visual targets for the user to focus upon for measuring various ocular parameters. It can also comprise an auditory input for testing instruction, signaling randomized head movement, and serving as an auditory distractor for some visual cognitive tests. Auditory signals can be presented to either ear, indicating the direction for the head to rotate. Visual instructions can also be displayed regarding information to complete the ocular parameter testing. Collected data can be transmitted to a small electronic device where easily understandable results can be seen.
In embodiments of the invention, the imaging device can comprise components configured to provide images of eye position and eye movement using features or surface features of the eye. The components can include a light source, diffracting elements to alter the light source, and opto-electric transducer configured for converting the light signal to an electrical signal. A head orientation sensor can be configured to generate electrical signals in response to head movements and both the imaging device and head orientation sensor components can be electrically coupled such that eye information can be compared to electrical head orientation signals with ocular parameter measurements.
In another embodiment, the extended reality display can be part of a face shield and may allow the wearer to view images in one or both eyes and can allow a view of the real-world environment while providing superimposed virtual images. The eye imaging device and head orientation sensor described in this document may be integrated into the electric circuit with a data storage and logging recorder.
In another embodiment, two or more eye imaging devices can be configured in a complementary fashion to increase eye sensor accuracy. Imaging devices can be configured from the following group: imaging devices of the same type across different focal lengths, imaging devices of the same type across different angular locations and/or imaging devices of differing types to provide composite images.
In another embodiment, the imaging devices can be attached to or mounted within the structural member(s) of a face shield. At least one of the imaging devices can be positioned at a sight plane below the inferior margin of the upper eyelid. It is below the upper eyelid to more easily visualize the pupil, cornea, iris, or other features of the eye used for eye tracking and measurement. Above this plane of sight, the eye sensor would have difficulty tracking the eye movements, due to obstruction of the upper lid and eyelashes.
In an embodiment, the human ocular performance measuring system can be comprised of an eye imaging device, attached to the head worn device, and configured to measure eye movement responses using different techniques of eye imaging measurement including, but not limited to use of one or multiple eye imaging devices, or simultaneous use of different types of eye imaging devices for eye tracking. In another embodiment, at least one eye imaging device can track one or more different locations simultaneously on the surface of one or both eyes (e.g., cornea, pupil, limbus, sclera) or image features from the retina (e.g., optic disc, fovea, macula, retinal vessels). In another embodiment, the eye imaging device can measure more than one corneal reflection or other eye feature using one or more different types of illumination sources simultaneously. In one embodiment, different types of illumination sources can also alternate or combine the type of illumination, depending on the light needed.
In another embodiment, eye imaging devices attached to the head worn unit can be in different positions to acquire different focal points of the eyeball, to achieve more accuracy with eye tracking. Eye imaging devices can also be configured to merge eye movement responses from different imaging devices for more accurate measurement. For example, an imaging device tracking one surface feature of the eye can be merged with the same imaging sensor, or another imaging devices, attached to different location on the head worn unit which is tracking a different eye surface feature. The merged data can provide more information regarding gaze and eye movement responses. In embodiments described, eye imaging devices can have multiple functions which enable different measurement features of the eye.
In one embodiment, a single imaging device can be used for the eye tracking. In another embodiment, multi-imaging devices are used, and the imaging devices can be in different sight planes or located at different distances from the measured area of the eye.
In embodiments of the invention, the light source can be infrared, short-wavelength infrared, near infrared, and/or visible light, such as LED, and can be directed toward one or both eyes. The eye imaging device can be used to detect and measure the reflection of the light source and visible eye features such as the pupil features, cornea reflection features, iris registration features, limbus features or for retinal data imaging. The collected data from the eye imaging device can be used to measure the movement features of the eyes or eyelids or rotation of the eye, acceleration/velocity of the eye movement, duration of the eyelid closure, rate of the eyelid closure and the direction of gaze. Additional information such as blink frequency and changes in pupil diameter can also be detected by the eye imaging device. Aggregated eye imaging device data can be written to a file for later analysis. Stored eye imaging device data can be used to analyze the visual path across an interface such as a computer screen. In this case, each eye data information is translated into a set of pixel coordinates. From there, the presence or absence of collected eye data points in different screen areas can be examined. This type of analysis can be used to determine which features are seen, when a particular feature captures attention, how quickly the eye moves, what content is overlooked and virtually any other gaze-related data. Eye position can be extracted by the eye imaging device and graphics are often generated to visualize the findings. Beyond the analysis of visual attention, stored eye data can be examined to measure the cognitive state, fatigue, alertness, or other information.
In other embodiments of the present invention, at least one beam splitter can be used to provide a guide light into the eye. Alternatively, two or more of the single prism beam splitters can be combined to form compound beam splitters that split a single beam of light into three or more different beams of light. A beam splitter can have an optical multi-layer thin film, formed by laminating numerous layers in sequence. The numerous laminated layers can each be comprised of having a different refractive index.
In other embodiment, multiple prisms can be used which can use a corrective optical element to eliminate any deviation or aberrations in the see-though viewing path. For example, in one of its aspects, the invention may include a wedge prism having only planar surfaces. This prism acts as a light guide to supply illumination light to the eye, as well as providing imaging light to the eye imaging device from the illuminated eye. In this embodiment a complementary prism can be arranged with respect to the thin prism such that the two prisms appear to the eye as a plane-parallel plate.
In an alternative embodiment, an eye imaging device can use a free-form prism between the eye and the eye imaging device. The freeform prism can include one or more surfaces with optical power, which are used both for imaging of the eye by the eye imaging device, and for optical aberration control. In certain embodiments, the freeform prism can be used in conjunction with, or exclusive of, additional focusing optics such as an eye imaging device.
In other embodiments, the head tracking can be done from sensors in an electronic device, smart phone, smart pad, from another sensor system attached to a body part, or from a remote device viewed by the user.
In another embodiment, a face shield with eye imaging device attached to the helmet covering the head can be configured for measuring and correcting slippage offsets. The measurement and correction of slippage offsets is carried out by one or more sensors selected from the group of: the existing multi-axis IMU, the existing eye imaging device, an additional IMU, and a wide field of view eye imaging device and/or forward-facing camera.
In one embodiment, the head-worn device as discussed herein can be calibrated before it is used. Calibration can also be performed by viewing a visual target on the XR display, focusing on a distant target of interest in the natural environment, viewing a visual target projected by a laser light source which is projected to a distant surface, or viewing a projected holographic image. The static image or visual target can be positioned vertically, horizontally, and centrally. The calibration can be done with the visual target located at 5 or 9 separate points. Typically, several trials can be performed to establish reproducible results. Ideally, the subject is instructed to slowly rotate the head slowly from side to side—horizontally or vertically—in the plane of each semicircular canal being evaluated while viewing the static visual target. Calibration of this type can allow gain, phase, and asymmetry to be accurately measured separately for each canal and at various frequencies. In another embodiment, the individual can be asked to follow an object at a combination of frequencies (“natural tracking”) and a Fourier transform can be used to convolve the gain, phase, and asymmetry at various frequencies directly from the complex waveform that was being followed by the individual. Natural tracking in the horizontal plane could entail focusing on a natural visual element or target moving across the horizontal visual field. Watching a moving object ascend and descend in the air can serve as a natural vertical test.
Any combination of the discussed embodiments of head inertial trackers and eye imaging systems can be used to measure the ocular muscle movement or reflex response with head movement (e.g., ocular parameters). Alternatively, in another embodiment, the visualized target required to focus upon for the ocular parameter being tested may be displayed, in the natural environment, as an AR, 3D image, hologram or some other light source image. An eye imaging device, using invisible or visible light, simultaneously can be used with head tracking. As the head moves, the eye responses can be tracked and measured by a variety of modalities. In another embodiment, the eye sensor(s) can be comprised of an array of transparent imaging devices based on graphene. In another embodiment, a hologram can be used to blend the digital world with the real world in an attached AR system, to aid in the testing and measurement of the eye movement. This can enable a more immersive see-through multi-dimensional method for all the visual or ocular parameter tests described in this disclosure. A Fourier transform or other method of analysis can be used to compare the inertial head movement and eye movement response information at various frequencies in a complex waveform and software can process the data. The processed information can be displayed remotely to another electronic device and responses can determine the health status of the user and predict potential performance of the user during an activity.
Embodiments of the invention can incorporate physiological and biochemical sensing elements/transducers in contact with the skin for detecting any abnormal physiological or biochemical properties of the user. These sensors can communicate with the electronic circuit and can be correlated with the ocular parameter measured to determine the human health status. This information can also be wirelessly transmitted to a remote electronic device for viewing.
In one embodiment, the vestibulo-ocular reflex can be measured by comparing each of the eye orientation responses to the head orientation movements, at various velocities, while viewing a stable visual target on a display. Abnormal VOR performance associated with aberrant accuracy, gain, phase, or symmetry, and/or with abnormal saccades can be used to determine neurologic disorders, including TBI, cognitive decline, Multiple Sclerosis, microvascular disease, physiologic impairments like dizziness or deficits caused by electromagnetic exposure as well as biochemical impairments with alcohol.
In another embodiment, the vestibulo-ocular reflex cancellation can be measured, by comparing the eye orientation changes of both eyes with the head orientation changes, while the subject's head and eyes follows a viewed moving visual target at various velocities on a display. A poor vestibulo-ocular reflex cancellation performance can be indicated by abnormal responses of accuracy, gain, phase (the timing response for the eye and head position), symmetry or latency, and/or with abnormal saccades. Abnormal vestibulo-ocular reflex cancellation performance can determine neurologic disorders such as TBI, Parkinson's Disease and physiologic impairments like motion sickness, dizziness, and electromagnetic effects as well as biochemical impairments with drug abuse.
In another embodiment, the angular vestibulo-ocular reflex cancellation can be measured, comparing the eye orientation changes of both eyes with the head orientation changes while the subject's head and eyes are following a viewed moving visual target at various velocities on a display. In this instance, the head can be positioned downward typically near 30 degrees (for maximal stimulation of the horizontal semicircular canal) and can move alternatively left and right with the viewed visual target which can also move in the horizontal planes. The head can also be rotated typically near 45 degrees to the right (for maximal stimulation of the left vertical and right posterior semicircular canal) and can move alternatively downwards and upwards with the viewed visual target which can also move in the same 45-degree diagonal plane. The head can also be rotated typically near 45 degrees to the left (for maximal stimulation of the right vertical and left posterior semicircular canal) and can move alternatively downwards and upwards with the viewed visual target, which can also move in the same 45-degree diagonal plane. A poor vestibulo-ocular reflex cancellation performance can be indicated by abnormal responses of accuracy, gain, phase (the timing response for the eye and head position), symmetry or latency, and/or with abnormal saccades. Abnormal angular vestibulo-ocular reflex cancellation performance and/or with saccades can determine neurologic disorders such as TBI, Parkinson's Disease and physiologic impairments like motion sickness, dizziness, and electromagnetic effects as well as biochemical impairments with drug abuse.
In an embodiment, vergence can be measured and compared in both eyes, as a visual target in a display appears to move forward and away from the subject's eyes. This movement of the visual target can be a continuous transition, or it can occur in a series of distinct stages. Poor vergence performance can be recorded, indicating abnormal changes of accuracy, convergence, divergence, peak velocity, amplitude, symmetry, or latency, and can be used to determine neurologic disorders, such as TBI, biochemical impairments such as alcohol and drug use, as well as physiologic impairments such as that caused by electromagnetic effects.
In another embodiment, vergence can be measured during continuous transition of different depths vertically, horizontally, or diagonally as the visual target gets closer or further from the user's eyes combined with dynamic motion of the head, which is moving in the same pattern or direction as the visual target.
In another embodiment, saccades can also be measured during other ocular parameter measures including vestibulo-ocular reflex, vestibulo-ocular reflex cancellation, vergence, dynamic vergence, smooth pursuit, dynamic visual acuity, and optokinetic testing. The occurrence of saccades, saccadic intrusions, or saccade dynamics on fixational eye movement during ocular parameter measure can be related to neurologic disorders or other impairments of human health.
In another embodiment, smooth pursuit can be measured while the head remains stable, and the eyes are focused on a visual target which is moving in various 2-D patterns on a display. An abnormal smooth pursuit performance can be indicated by aberrant measured eye orientation changes of gain (peak velocity/target velocity), velocity changes, accuracy of following a moving object or latency. These abnormalities can determine neurologic disorders like TBI, Alzheimer's Disease, Parkinson's Disease, microvascular disease, physiologic impairments such as motion sickness, electromagnetic effects, and biochemical impairments due to alcohol or hormonal disorders such as diabetes.
In an embodiment, pupil performance can be measured by determining pupil size on each side and features of changes of the pupil, while viewing stationary alternating bright and dim elements on a display. The visual targets can have selective wavelengths, such as with chromatic light and can be presented with various stimulus presentations such as with stepwise increases in intensity or light flashes. Alternatively in another embodiment, these visual targets can be moving toward or away from the eye, or they can be presented in different positions with different characteristics, requiring the subject to recognize the difference between visual targets. Poor pupil performance can include abnormal measures of pupil size, dilation information of acceleration, amplitude, latency or duration, and constriction information of amplitude, latency, or duration. These abnormal pupil measures can determine neurologic disorders like Alzheimer's Disease, Multiple Sclerosis, Stroke, biochemical impairments with drug use and physiologic impairment with cardiac disease, such as hypotension.
In an embodiment, eyelid performance can be measured and compared between each eye by viewing a display with a visual stimulus, at various intensities of brightness, with varied task content and at varying speeds causing eyeblinks. Abnormal eyelid performance can be associated with aberrant velocity of eyeblinks, the duration of eyeblinks, the amplitude or frequency of eyeblinks which can determine the presence of neurologic disorders such as Alzheimer's Disease or cognitive deficits, biochemical disorders associated with electrolyte or metabolic abnormalities and physiologic impairments which occurs with fatigue, or lack of alertness.
In another embodiment, dynamic visual acuity performance can be measured with an eye imaging device to determine eye orientation changes while viewing a display of a visual target, which can change attributes, as the head moves horizontally or vertically and establish DVA performance. Abnormal DVA measures of reduced visual acuity, compared to the SVA, can determine neurologic disorders, such as brainstem lesions, and physiologic impairments such as visual and vestibular impairments as well as gaze stability.
In another embodiment, the presence of abnormal nystagmus as determined by slow-phase velocity measures including duration, frequency, or amplitude and induced by visual stimulation can be used to determine spatial disorientation, motion sickness and dizziness. Alternatively, in another embodiment the abnormal presence of nystagmus with vestibular stimulation can determine neurologic disorders such as stroke and physiologic impairments with the inner ear, such as Meniere's Disease.
In another embodiment, anyone of the ocular parameter measurements, discussed in this document, can be used to determine the status of human health, and implemented for training athletes or other individuals in their occupational activities, to assume a supernormal level of performance.
In another embodiment, skin probes placed over the temporal region of the head can also be comprised of near-infrared spectroscopy to identify chemical and even structural changes in the brain, by comparing the average intensity of diffusely reflected light. These probes can be attached to the worn device and correlated with the ocular parameter measurement for assessment of a neurologic disorder.
In another embodiment, the human health device as described in this document, including the head-worn unit, comprised of an eye imaging device, head orientation sensor, display, and electronic circuit can be configured for use with machine learning such that a classifier can recognize any abnormal ocular parameter measured and provide classification of the raw gaze datasets, belonging to fixations, saccades, or other predetermined categories. The classified algorithm can be used in a decision tree to determine whether the data can be used for training or specific rehabilitation, based on the abnormal datasets, and uses a computer code that can modify an attribute of the head-worn display.
In an embodiment, the system discussed herein can be portable, autonomous, constantly sensing head and eye information with the use of an artificial intelligence (AI) platform and classifiers to determine the human health status and can provide this information to the user as well as wirelessly transmit this information to a remote electronic device. In another embodiment, this system can provide an electronic-type prescription for a non-pharmacological treatment alternative with VOV rehabilitation to treat neurologic disorders, physiologic or biochemical impairments affecting the neurologic pathway of the abnormal ocular parameter.
In an alternative embodiment, the present invention can visually rehabilitate or retrain the user when a specific ocular parameter abnormality is present. Visual-oculomotor-vestibular rehabilitation can enhance ocular parameter visual accuracy with specific visual stimulation and head movements. VOV rehabilitation can help a user of the device improve the health disorders or impairments by exercising, enhancing, and/or retraining the abnormally detected ocular parameter. This type of rehabilitation system can also provide more rapid recovery of an abnormal ocular parameter by visually stimulating the associated neurologic pathway and connections affected by the neurologic, physiologic, or biochemical deficits with repetitive ocular parameter techniques.
In another embodiment, ocular parameter assessment can be used to train the oculomotor system and brain with individualized program, which can increase accuracy of eye fixation, cognition, attention, reaction time, and treat traumatic brain injuries and cognitive deficits.
In another embodiment, an interactive ocular parameter program can be provided which uses image-based interactivities for testing, management, and rehabilitation of concussions/traumatic brain injury with periodic assessment to analyze the progress of cognitive deficits. A cognitive rehabilitative program can be used with specific identified cognitive disorders. For example, the cognitive testing can also be used for assessing the neurologic status, alertness, fatigability, inattention, deployment readiness, situational awareness, predicting human performance, stress and managing any deficits detected with a visually interactive cognitive program designed to correct those deficits.
In another embodiment, the collected eye and head movement data from ocular parameter measurements of each subject can be logged into an individual database for that subject, to provide a measure of brain health, biochemical and physiological health over a period of time and a machine learning classifier can be utilized to determine patterns to detect disease early and provide early intervention and therapy.
In an embodiment, the information collected from ocular parameter measurement(s) of a user can be logged and transmitted to the medical health care records of the health care provider(s) and/or insurance company.
In another embodiment, the collected eye and head movement data from ocular parameter measurements can use machine learning and biomarkers related to diagnosing neurodegenerative conditions such as Alzheimer's Disease, Parkinson's Disease, and strokes with abnormalities of cognitive function, mood, and behavior to automatically personalize VOV rehabilitation therapy plans. This VOV rehabilitation therapy can also access the software therapy from the cloud, through a smartphone, or other electronic device. For example, the display can be designed to provide rehabilitation prescriptions to improve specific cognitive visual deficits or cognitive domains. It can provide more immersive experience with engaging visual targets for measuring ocular parameters to assess cognitive function, while quantifying the degree of the cognitive deficit detected. Once the measured ocular parameters determine the health status of the subject, the identity proofing, privacy, and security for the subject can be established. Information regarding normal ocular parameters and/or abnormal ocular parameters can be wirelessly transmitted to the cloud. Machine learning in the cloud can establish the rehabilitation program needed for the display, based on the abnormal ocular parameter measured, or further training desired by the subject to obtain above normal performance with selected parameter. The electronic VOV rehabilitation prescriptions can be specifically designed with programs to train a subject to have above normal performance or to treat or train a subject for rehabilitation with a neurologic, biochemical, or physiologic impairment, identified by an abnormal ocular parameter, to a normal health status level.
In another embodiment, the specific electronic prescription, determined by the computer code (machine learning algorithm) in the cloud or other external electronic device, can be transmitted to the trainer/rehabilitation therapist and/or to the subject or to others, such as providers of the subject's health care. Specific programming can also be accessed and actively streamed to the user automatically, upon sensing an abnormal parameter value associated with a particular disorder or the need for training of a specific parameter desired. The subject with normal parameters desiring training can select specific programs to enhance eye fixation with activities to super-normal levels. The subject having an abnormal ocular parameter(s) can be trained with specific visual ocular tasks to rehabilitate the ocular parameter(s) which was abnormal. Eye orientation changes, pupil size and eyeblinks can be measured with head orientation changes with the VOV rehabilitation tasks or with the visual training tasks. The improvements can be determined by the measured data and wirelessly transmitted back to the cloud for data logging. The use of eye and head movement information described in this document can be valuable in visual-oculomotor-vestibular rehabilitative evaluation, care, management, and long-term planning for anyone suspected as having a neurologic disorder, biochemical or physiologic impairment.
In another embodiment of the head worn system described, any abnormal ocular parameter, biochemical or physiological impairment detected can be configured to be transmitted to a computer controller system of vehicles or other devices. Reception of this abnormal sensing data can control the operating systems of the vehicle or device, through AI and/or machine learning.
In an embodiment, this head-worn device discussed can function as a health care provider extender to detect and mitigate the long-term physical and psychological sequelae of traumatic brain injuries. This human health device can be comprised of a display to rapidly and accurately provide optimal triage and early intervention with effective rehabilitation treatment for subjects subjected to head trauma. It can transmit the collected head sensor and eye imaging sensor data information, measured while viewing an ocular parameter on the display, to an electronic device remotely and/or to the injured subject. This can provide a key treatment method of early VOV rehabilitation, as well as providing a more accurate strategy in the decision making to guide return to previous activities.
In another embodiment the human health device as described in this document, comprised of a head-worn unit, eye imaging device, head orientation sensor, and display can be used for assessing a physiologic health impairment from electromagnetic wave exposure.
The data obtained from the head worn devices and methods described herein can be transmitted by wireless communication to a remote device.
In an alternative embodiment, the head worn system can be tethered to a smart phone or computer and use their display or push eye imaging and head orientation responses to these devices.
Embodiments described herein can be used with a protective helmet including those designed for sport activities. Various embodiments can also be used for safety helmets, such as construction or industrial helmets, and helmets used by law enforcement, security and/or military forces.
In another embodiment, eye tracking and/or measurement can also be done with a contact lens. The contact lens can be embedded with electronics and eye motion sensor and can be comprised of a micro-LED display or augmented reality display, a power source with capability of being charged wirelessly. It can also be wireless connected to a smart phone, such as iPhone or another computer device. It can provide an augmented reality experience without a headset attached. Not only can the electronic contacts provide eye tracking but also can visualize biometric, biochemical, and physiologic data from remote wirelessly connected skin sensors.
In another embodiment, a windshield, attached to a vehicle, can be comprised of an eye imaging device, head orientation sensor and electronic circuit within the windshield, to detect abnormal ocular parameters, such as eye closure or attention deficits. If an abnormality is detected and measured, the data can be transmitted to the control operating system of the vehicle through AI and/or machine learning to control the vehicle. The eye tracking sensors can be fixed in a position within the windshield and below the upper eye lid, for eye imaging and head orientation sensing. In an alternative embodiment, the eye imaging sensors, can become “locked” on the eyes when starting the vehicle, and subsequently can continuously move to different locations to maintain the locked tracking feature for ocular parameters measurement, while the vehicle is being operated by the user.
In another embodiment, there can be no display affixed to the head worn device, and the visual stimulus for measuring the ocular parameters could originate from viewed natural elements.
In another embodiment, observing the eye and head information as described herein to measure ocular parameters discussed to determine human health can be performed without use of head worn device. For example, this can be performed with an eye imaging device and head orientation sensors embedded in the windshield or affixed to the dashboard or other area facing the operator in a vehicle that can image the eyes and head.
In another embodiment, ocular parameter measurement can be performed with eye sensors (e.g., eye imaging device) and head orientation sensors embedded or affixed to a smart device or smart phone that can image the eyes and head.
Areas of Application
Sports. Embodiments of the invention(s), can be used in sport environments where ocular parameter measurement can help predict player performance, player fatigue, attention, cognition, and early detection of traumatic brain injury. Additionally, if an athlete had such an abnormality and could be given rehabilitation, this can correct the abnormality and allow the athlete to return to play activities sooner. Embodiments of the invention(s) can be used to determine when the athlete is ready to return to play activities, based on improvement of the previously detected ocular parameter abnormality. One example is TBI/concussion evaluation, assessment, and management for return to play. Alcohol and drugs can also adversely affect ocular performance. Embodiments of the invention(s) can be used for drug screen screening, and predict player performance based on eye fixation ability.
Military personnel function in a high-level environment and requiring target fixation of their eyes, while performing other activities such as with head or body movement. They require normal ocular parameters, including VOR, DVA, and VORC. If these ocular parameters are abnormal, the soldier will not demonstrate peak human performance. Embodiments of the invention can be used by the military in places such as the pilot selection process or special operations community to aid in the selection of individuals without such an abnormality. This invention can provide immediate training for rehabilitation if an abnormality is detected with ocular parameter measurements. A non-pharmaceutical electronic prescription can be provided from the cloud for rehabilitation, using a visual display as described, and enabling more rapid recovery of abnormal ocular parameters in remote areas or combat zones to improve the health status of the wounded soldier.
Medical. Embodiments of the present invention can be useful to individuals who have experienced symptoms of dizziness, imbalance, unsteadiness or have had concussions. It also can be utilized by centers performing vestibular rehabilitation and athletic/vocational enhancement environments. This invention can provide objective tools for assisting in the detection of neurologic disorders, including traumatic brain injury (TBI), the long-term effects of TBI known as chronic traumatic encephalopathy (CTE), biochemical impairments or physiologic impairments which would affect the human health status.
Commercial. Embodiments can also be used in other industries where individuals are expected to perform in high activity levels, or provocative motion environments. Like the other environments, the invention can predict human health status, such as performance, and can detect neurologic disorders, physiologic and biochemical impairments which affect the neurologic pathway and their connections.
Although the invention herein has been described with reference to embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present invention. It is therefore to be understood that numerous modifications may be made to the illustrative embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims.
Further variations and modifications of the disclosed embodiments can also be used. The principles described here can also be used for applications other than sports, military, commercial and medical. While the principles of the disclosure have been described above in connection with specific apparatuses and methods, it is understood that this description is made only by way of example and not as limitation on the scope of the disclosure.
This application is a continuation-in-part of U.S. patent application Ser. No. 16/903,136, filed 16 Jun. 2020, which is a continuation in part of U.S. patent application Ser. No. 16/264,242, filed 31 Jan. 2019, now U.S. Pat. No. 10,716,469, issued 21 Jul. 2020, which is a continuation-in-part of U.S. patent application Ser. No. 15/713,418, filed 22 Sep. 2017, now U.S. Pat. No. 10,231,614, issued 19 Mar. 2019, which is a continuation-in-part of U.S. patent application Ser. No. 15/162,300, filed 23 May 2016, now U.S. Pat. No. 9,788,714, issued 17 Oct. 2017, which is a continuation-in-part of U.S. patent application Ser. No. 14/326,335, filed 8 Jul. 2014, now U.S. Pat. No. 9,370,302, issued 21 Jun. 2016. U.S. patent application Ser. No. 16/264,242 is also a continuation-in-part of U.S. patent application Ser. No. 13/749,873, filed 25 Jan. 2013. The entire disclosures of all of the aforementioned patents and applications are incorporated by reference herein.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16903136 | Jun 2020 | US |
| Child | 17576673 | US | |
| Parent | 16264242 | Jan 2019 | US |
| Child | 16903136 | US | |
| Parent | 15713418 | Sep 2017 | US |
| Child | 16264242 | US | |
| Parent | 15162300 | May 2016 | US |
| Child | 15713418 | US | |
| Parent | 14326335 | Jul 2014 | US |
| Child | 15162300 | US | |
| Parent | 13749873 | Jan 2013 | US |
| Child | 16264242 | US |
