Patent Application
20180235505
This application claims the benefit of Korean Patent Application Nos. 10-2017-0021519, filed on Feb. 7, 2017, and 10-2017-0147607, filed on Nov. 7, 2017, in the Korean Intellectual Property Office, the disclosures of which are incorporated herein in their entirety by reference.
Additional aspects will be set forth in part in the description which follows and, in part, will One or more embodiments relate to a method of inferring human physiological signals performed in a non-contact mode, and a system using the method, and more particularly, to method of detecting parameters of a brain-frequency domain from pupil rhythm video captured by a camera.
In vital signal monitoring (VSM), physiological information can be acquired by a sensor attached to a human body. Such physiological information includes electrocardiogram (ECG), photo-plethysmograph (PPG), blood pressure (BP), galvanic skin response (GSR), skin temperature (SKT), respiration (RSP) and electroencephalogram (EEG).
The heart and brain are two main organs of the human body and analysis thereof provide the ability to evaluate human behavior and obtain information that may be used in response to events and in medical diagnosis. The VSM may be applicable in various fields such as ubiquitous healthcare (U-healthcare), emotional information and communication technology (e-ICT), human factor and ergonomics (HF&E), human computer interfaces (HCIs), and security systems.
ECG and EEG use sensors attached to the body to measure physiological signals and thus, may cause inconvenience to patients. That is, the human body experiences considerable stress and inconvenience when using sensors to measure the signals. In addition, there are burdens and restrictions with respect to the cost of using the attached sensor and the movement of the subject due to attached hardware such as the sensors.
Therefore, VSM technology is required in the measurement of physiological signals by using non-contact, non-invasive, and non-obtrusive methods while providing unfettered movement at low cost.
Recently, VSM technology has been incorporated into wireless wearable devices allowing for the development of portable measuring equipment. These portable devices can measure the heart rate (HR) and RSP by using VSM embedded into accessories such as watches, bracelets, or glasses.
Wearable device technology is predicted to transit from portable devices to “attachable” devices shortly. It is predicted that attachable devices will transit to “eatable” devices.
VSM technology has been developed to measure physiological signals by using non-contact, non-invasive, and non-obtrusive methods that provides unfettered movement at low cost. While VSM will continue to advance technologically, innovative vision-based VSM technology is required to be developed also.
One or more embodiments include a system and method for inferring and detecting physiological signals by non-contact, non-invasive and non-obstructive method at low cost.
In detail, one or more embodiments include a system and method for detecting parameters of a brain frequency domain by using rhythm of pupillary variation.
Additional aspects will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the presented embodiments.
According to one or more exemplary embodiments, the method of inferring EEG spectrum based on pupillary variation comprises: obtaining moving images of at least one pupil from a subject; extracting data of pupillary variation from the moving images; extracting band data for a plurality of frequency bands to be used as brain frequency information, based on frequency analysis of the signal of pupillary variation; and calculating outputs of the band data to be used as parameters of a brain-frequency domain.
According to one or more exemplary embodiment, the data of pupillary variation comprises a signal indicating pupil size variation of the subject.
According to one or more exemplary embodiment, the frequency analysis is performed in a range of 0.01 Hz-0.50 Hz.
According to one or more exemplary embodiment, the method further comprises resampling of the data of pupillary variation at a predetermined sampling frequency, before extracting the band data based on the frequency analysis.
According to one or more exemplary embodiment, the plurality of frequency bands include at least one of: a delta range of 0.01 Hz˜0.04 Hz, a theta range of 0.04 Hz˜0.08 Hz, an alpha range of 0.08 Hz˜0.13 Hz, a beta range of 0.13 Hz˜0.30 Hz, a gamma range of 0.30 Hz˜0.50 Hz, a slow alpha range of 0.08 Hz˜0.11 Hz, a fast alpha range of 0.11 Hz˜0.13 Hz, a low beta range of 0.12 Hz˜0.15 Hz, a mid beta range of 0.15 Hz˜0.20 Hz, a high beta range of 0.20 Hz˜0.30 Hz, a mu range of 0.09 Hz˜0.11 Hz, a SensoriMotor Rhythm (SMR) wave range of 0.125 Hz˜0.155 Hz, and a total band range of 0.01 Hz˜0.50 Hz.
According to one or more exemplary embodiment, each of the outputs is obtained from a ratio of respective band power to total band power of a total band range in which the plurality of frequency bands are included.
According to one or more exemplary embodiment, the system adopting the method, comprises: video equipment configured to capture the moving images of the subject; and a computer architecture based analyzing system, including analysis tools, configured to process and analyze the moving images in the plurality of frequency bands.
According to one or more exemplary embodiment, the analyzing system is configured to perform frequency analysis in a range of 0.01 Hz-0.50 Hz.
According to one or more exemplary embodiment, the range includes at least one of: a delta range of 0.01 Hz˜0.04 Hz, a theta range of 0.04 Hz˜0.08 Hz, an alpha range of 0.08 Hz˜0.13 Hz, a beta range of 0.13 Hz˜0.30 Hz, a gamma range of 0.30 Hz˜0.50 Hz, a slow alpha range of 0.08 Hz˜0.11 Hz, a fast alpha range of 0.11 Hz˜0.13 Hz, a low beta range of 0.12 Hz˜0.15 Hz, a mid beta range of 0.15 Hz˜0.20 Hz, a high beta range of 0.20 Hz˜0.30 Hz, a mu range of 0.09 Hz˜0.11 Hz, a SMR wave range of 0.125 Hz˜0.155 Hz, and a total band range of 0.01 Hz˜0.50 Hz.
In These and/or other aspects will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings in which:
In Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout. In this regard, the present embodiments may have different forms and should not be construed as being limited to the descriptions set forth herein. Accordingly, the embodiments are merely described below, by referring to the figures, to explain aspects of the present description.
Hereinafter, a method and system for inferring and detecting physiological signals according to the present inventive concept is described with reference to the accompanying drawings.
The invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the invention to those skilled in the art. Like reference numerals in the drawings denote like elements. In the drawings, elements and regions are schematically illustrated. Accordingly, the concept of the invention is not limited by the relative sizes or distances shown in the attached drawings.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” or “includes” and/or “including” when used in this specification, specify the presence of stated features, numbers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, numbers, steps, operations, elements, components, and/or groups thereof.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and/or the present application, and will not be interpreted in an overly formal sense unless expressly so defined herein.
The embodiments described below involve processing brain frequency information from pupillary response which is obtained from video information
The present invention, which may be sufficiently understood through the embodiments described below, involve extraction brain frequency information from the pupillary response by using a vision system equipped with a video camera such as a webcam without any physical restriction or psychological pressure on the subject, Especially, the pupil response is detected from the image information and brain frequency information is extracted from it.
In the experiment of the present invention, the reliability of the parameters of the brain frequency domain extracted from the pupil size variation (PSV) acquired through moving images was compared with the EEG signal by EEG of ground truth.
The experiment of the present invention has been performed by video equipment, and computer architecture based analyzing system for processing and analyzing the moving image which includes analysis tools provided by software.
In order to cause variation of the physiological state, this experiment used sound stimuli based on the Russell's cir-complex model (Russell, 1980). The sound stimuli included a plurality of factors, including arousal, relaxation, positive, negative, and neutral sounds. The neutral sound was defined by an absence of acoustic stimulus. The steps for selecting sound stimulus are shown in
(S11) Nine hundred sound sources were collected from the broadcast media such as advertisements, dramas, and movies.
(S12) The sound sources were then categorized into four groups (i.e., arousal, relaxation, positive, and negative). Each group was comprised of 10 commonly selected items based on a focus group discussion for a total of forty sound stimuli.
(S13) These stimuli were used to conduct surveys for suitability for each emotion (i.e., A: arousal, R: relaxation, P: positive, and N: negative) based on data gathered from 150 subjects that were evenly split into 75 males and 75 females. The mean age was 27.36 years±1.66 years. A subjective evaluation was required to select each item for the four factors, which could result in duplicates of one or more of the items.
(S14) A chi-square test for goodness-of-fit was performed to determine whether each emotion sound was equally preferred. Preference for each emotion sound was equally distributed in the population (arousal: 6 items, relaxation: 6 items, positive: 8 items, and negative: 4 items) as shown in Table 1.
Table 1 shows the chi-square test results for goodness-of-fit in which the items selected for each emotion are based on comparisons of observation and expectation values.
Resurveys of the sound stimuli were conducted for relation to each emotion from the 150 subjects by using a seven-point scale based on 1 indicating strong disagreement to 7 indicating strong agreement.
Valid sounds relating to each emotion were analyzed using PCA (Principal Component Analysis) based on Varimax (orthogonal) rotation. The analysis yielded four factors explaining of the variance for the entire set of variables. Following the analysis result, representative sound stimuli for each emotion were derived, as shown in Table 2.
In Table 2, the bold type is the same factor, the blur character is the communalities <0.5, and the thick, light gray lettering with shading in the background represents the representative acoustic stimulus for each emotion.
In Seventy undergraduate volunteers of both genders, evenly split between males and females, ranging in age from 20 to 30 years old with a mean of 24.52 years±0.64 years participated in this experiment. All subjects had normal or corrected-to-normal vision (i.e., over 0.8), and no family or medical history of disease involving visual function, cardiovascular system, or the central nervous system. Informed written consent was obtained from each subject prior to the study. This experimental study was approved by the Institutional Review Board of Sangmyung University, Seoul, South Korea (2015 Aug. 1).
The experiment was composed of two trials where each trail was conducted for a duration of 5 min. The first trail was based on the movelessness condition (MNC), which involves not moving or speaking. The second trial was based on the natural movement condition (NMC) involving simple conversations and slight movements. Participants repeatedly conducted the two trials and the order was randomized across the subjects. In order to verify the difference of movement between the two conditions, this experiment quantitatively measured the amount of movement during the experiment by using webcam images of each subject. In the present invention, the moving image may include at least one pupil, that is, one pupil or both pupils image.
The images were recorded at 30 frames per second (fps) with a resolution of 1920×1080 by using a HD Pro C920 camera from Logitech Inc. The movement measured the upper body and face based on MPEG-4 (Tekalp and Ostermann, 2000; JPandzic and Forchheimer, 2002). The movement in the upper body was extracted from the whole image based on frame differences. The upper body line was not tracking because the background was stationary.
The movement in the face was extracted from 84 MPEG-4 animation points based on frame differences by using visage SDK 7.4 software from Visage Technologies Inc. All movement data used the mean value from each subject during the experiment and was compared to the difference of movement between the two trails, as shown in
In
In order to cause the variation of physiological states, sound stimuli were presented to the participants during the trails. Each sound stimulus was randomly presented for 1 min for a total of five stimuli over the 5 min trial. A reference stimulus was presented for 3 min prior to the initiation of the task. The detailed experimental procedure is shown in
The experimental procedure includes the sensor attachment S31, the measurement task S32 and the sensor removal S33 as shown in
The experiment was conducted indoors with varying illumination caused by sunlight entering through the windows. The participants gazed at a black wall at a distance of 1.5 m while sitting in a comfortable chair. Sound stimuli were equally presented in both the trials by using earphones. The subjects were asked to constrict their movements and speaking during the movelessness trial (MNC). However, the natural movement trial (NMC) involved a simple conversation and slight movement by the subjects. The subjects were asked to introduce themselves to another person as part of the conversation for sound stimuli thereby involving feelings and thinking of the sound stimuli. During the experiment, EEG signal and pupil image data were obtained. EEG signals were recorded at a 500 Hz sampling rate from nineteen channels (FP1, FP2, F3, Fz, F4, F7, F8, C3, Cz, C4, T7 (T3), T8 (T4), P7 (T5), P8 (T6), P3, Pz, P4, O1, and O2 regions) based on the international 10-20 system (ground: FAz, reference: average between electrodes on the two ears, and DC level: 0 Hz-150 Hz). The electrode impedance was kept below 3 kΩ. EEG signals were recorded at 500 Hz sampling rate using a Mitsar-EEG 202 Machine.
Hereinafter, a method of extracting or constructing (recovering) a vital sign from a pupillary response will be described.
The pupil detection procedure acquires a moving image using the infrared camera system as shown in
The pupil detection procedure required following certain image processing steps since the images were captured using an infrared camera, as shown in
Threshold=(−0.418×Bmean+1.051×Bmax)+7.973 <Equation 1>
B=Brightness value
The next step to determine the pupil position involved processing the binary image by using a circular edge detection algorithm, as shown in Equation 2 (Daugman, 2004; Lee et al., 2009).
In case that multiple pupil positions were selected, the reflected light caused by the infrared lamp was used. Then we obtained an accurate pupil position, including centroid coordinates (x, y) and a diameter.
Pupil diameter data (signal) was resampled at a frequency range of 1 Hz-30 Hz, as shown in Equation 3. The resampling procedure for the pupil diameter data involved a sampling rate of 30 data points, which then calculated the mean value during 1-s intervals by using a common sliding moving average technique (i.e., a window size of 1 second and a resolution of 1 second). However, non-tracked pupil diameter data caused by the eye closing was not involved in the resampling procedure.
The non-contact detecting or inferring of the EEG spectral index is proposed in this section.
The index includes delta (δ, 1 Hz-4 Hz), theta (θ, 4 Hz-8 Hz), alpha (α, 8 Hz-13 Hz); beta (β, 13 Hz-30 Hz), gamma (γ, 30 Hz-50 Hz), slow alpha (8 Hz-11 Hz), fast alpha (11 Hz-13 Hz), low beta (12 Hz-15 Hz), mid beta (15 Hz-20 Hz), high beta (20 Hz-30 Hz), mu (μ, 9 Hz-11 Hz), and the sensorimotor rhythm wave (SMR) (12.5 Hz-15.5 Hz) by using 19 channels to determine the pupillary response.
The EEG spectral index is related to the various physical and physiological states (Gastaut, 1952; Glass, 1991; Noguchi and Sakaguchi, 1999; Pfurtscheller and Da Silva, 1999; Niedermeyer, 1997; Feshchenko et al., 2001; Niedermeyer and da Silva, 2005; Cahn and Polich, 2006; Kirmizi-Alsan et al., 2006; Kisley and Cornwell, 2006; Kanayama et al., 2007; Zion-Golumbic et al., 2008; Tatum, 2014), as shown in Table 3.
Table 3 shows Comparison of EEG spectral index.
Referring
These BPF ranges were applied by the harmonic frequency with a 1/100 resolution. The filtered signal was processed to extract each frequency band data by using frequency analysis (e.g. FFT analysis), and the total power (X power) as outputs for the each frequency band was calculated as shown in Equation 4.
The outputs, that is, the powers (X power) of each frequency band, from delta to SMR, were calculated using the ratio between the total band power and EEG spectral index, as shown in Equation 4. This procedure was processed by the sliding window technique by using a window size of 180 sec and a resolution of 1 sec.
The EEG signals of ground truth were processed by using a BPF of 1 Hz-50 Hz range and the FFT analysis as shown in
Result
In this section, the vital signs from the cardiac time domain index, cardiac frequency domain index, EEG spectral index, and the HEP index of the test subjects were extracted from the pupillary response. These components were compared with each index from the sensor signals (i.e., ground truth) based on correlation coefficient (r) and mean error value (ME). The data was analyzed in both MNC and NMC for the test subjects.
To verify the difference of the amount movement between the two conditions of MNC and NMC, the movement data was quantitatively analyzed. The movement data was a normal distribution based on a normality test of probability-value (p) >0.05, and from an independent t-test. A Bonferroni correction was performed for the derived statistical significances (Dunnett, 1955). The statistical significance level was controlled based on the number of each individual hypothesis (i.e., α=0.05/n). The statistical significant level of the movement data sat up 0.0167 (upper body, X and Y axis in face, α=0.05/3). The effect size based on Cohen's d was also calculated to confirm practical significance. In Cohen's d, standard values of 0.10, 0.25, and 0.40 for effect size are generally regarded as small, medium, and large, respectively (Cohen, 2013).
According to the analysis results, the amount of movement in MNC (upper body, X and Y axis for the face) was significantly increased compared to the NMC for the upper body (t(138)=−5.121, p=0.000, Cohen's d=1.366 with large effect size), X axis for the face (t(138)=−6.801, p=0.000, Cohen's d=1.158 with large effect size), and Y axis for the face (t(138)=−6.255, p=0.000, Cohen's d=1.118 with large effect size), as shown in
The EEG spectral index of the brain activity, as represented by delta, theta, alpha, beta, gamma, slow alpha, fast alpha, low beta, mid beta, high beta, mu, and SMR power for the 19 channel brain regions, were extracted from the pupillary response. These components were compared with the EEG spectral index from EEG signals of ground truth. The examples of EEG spectral index extraction from the pupillary response and ECG signals are shown in
This exemplary study was able to determine the EEG spectral power (e.g., low beta in FP1, mid beta in FP1, SMR in FP1, beta in F3, high beta in F8, mu in C3, and gamma in P3) from the pupillary response by the entrainment of the harmonic frequency.
The EEG spectral index of brain activity ranged from 12 Hz to 15 Hz for low beta; 15 Hz to 20 Hz for mid beta; 12.5 Hz to 13.5 Hz for SMR; 13 Hz to 30 Hz for beta; 20 Hz to 30 Hz for high beta: 9 Hz to 11 Hz for mu; and 30 Hz to 50 Hz for gamma were closely connected with the circadian pupillary rhythm within the range of 0.12 Hz to 0.15 Hz; 0.15 Hz to 0.20 Hz; 0.125 Hz to 0.135 Hz; 0.13 Hz to 0.30 Hz; 0.20 Hz to 0.30 Hz; 0.09 Hz to 0.11 Hz; and 0.30 Hz to 0.50 Hz (harmonic frequency of 1/100f), respectively.
The exemplary process of extracting the EEG spectral index from the pupillary response in subjects, is shown in
(A): Signal of pupil size variation
(B): Signal re-sampled at 1 Hz based on sliding moving average technique (window size: 30 fps, resolution: 30 fps)
(C): Signals processed by BPF of each frequency band.
(D): Signals by FFT analysis
(E): Power signals as outputs of delta to SMR (0.01 Hz-0.50 Hz)
Followings are Frequency bands obtained from pupillary response.
1) delta: 0.01 Hz˜0.04 Hz
2) theta: 0.04 Hz˜0.08 Hz
3) alpha: 0.08 Hz˜0.13 Hz
4) beta: 0.13 Hz˜0.30 Hz
5) gamma: 0.30 Hz˜0.50 Hz
6) slow alpha: 0.08 Hz˜0.11 Hz
7) fast alpha: 0.11 Hz˜0.13 Hz
8) low beta: 0.12 Hz˜0.15 Hz
9) mid beta: 0.15 Hz to 0.20 Hz,
10) high beta: 0.20 Hz˜0.30 Hz
11) mu (μ): 0.09 Hz˜0.11 Hz, and
12) SMR: 0.125 Hz˜0.135 Hz) with harmonic frequency of 1/100f
The process of extracting the EEG spectral index from EEG raw data of ground truth in subjects, is shown in
(A): Raw signal of EEG (ground truth)
(B): Filtered EEG signal by BPF of 1 Hz-50 Hz
(C): Spectrum analysis and extraction powers of each frequency band (delta to SMR)
(D): Power signals (output) of each frequency band of EEG signal (ground truth)
In detail,
In detail,
When comparing the results of the ground truth in MNC, the EEG spectral index from the pupillary response indicated a strong correlation for all parameters where r=0.754±0.057 for low beta power in the FP1 region; r=0.760±0.056±for mid beta power in the FP1 region; r=0.754±0.059 for SMR power in the FP1 region; r=0.757±0.062 for beta power in the F3 region; r=0.754±0.056 for high beta power in the F8 region; r=0.762±0.055 for mu power in the C4 region; and r=0.756±0.055 for gamma power in the P4 region.
The difference between the mean error of all parameters was low where ME=0.167±0.081 for low beta power in FP1 region; ME=0.172±0.085 for mid beta power in the FP1 region; ME=0.169±0.088 for SMR power in the FP1 region; ME=0.160±0.080 for beta power in the F3 region; ME=0.178±0.081 for high beta power in the F8 region; ME=0.157±0.076 for mu power in the C4 region; and ME=0.167±0.089 for gamma power in the P4 region.
This procedure was processed using the sliding window technique where the window size was 180 s and the resolution was 1 s by using the recorded data for 300 s. The correlation and mean error were the mean value for the 70 test subjects (in one subject, N=120), as shown in Tables 4 and 5.
Table 5 shows average of correlation coefficient of EEG spectral index in MNC (N=120, p<0.01)
Table 6 shows average of mean error of EEG spectral index in MNC (N=120)
The correlation and mean error matrix table between brain regions and EEG frequency ranges is shown in Tables 7 and 8. Low beta, mid beta, and SMR power from the pupillary response were strongly correlated and had little difference with the EEG band power in the FP1 and FP2 regions (r>0.5, ME<1).
Beta power from the pupillary response was strongly correlated, and had little difference, with EEG band power in the F3, F4, and Fz brain regions (r>0.5, ME<1). High beta power from the pupillary response was strongly correlated, and had little difference, with EEG band power in the F7 and F8 brain regions (r>0.5, ME<1). Mu power from the pupillary response was strongly correlated, and had little difference, with EEG band power in the C3, C4, and Cz brain regions (r>0.5, ME<1).
Gamma power from the pupillary response was strongly correlated, and had little difference, with EEG band power in the P3 and P4 brain regions (r>0.5, ME<1). Other brain regions and frequency ranges were a low correlation and indicated a large difference (r<0.5, ME>1). Low beta, mid beta, SMR, beta, high beta, mu, and gamma were the higher correlations and had very little differences (r>0.7, ME<0.2) with FP1, FP1, FP1, F3, F8, C4, and P4, respectively.
Table 6 shows average of correlation matrix between brain regions and EEG frequency ranges in MNC (dark grey shade r>0.7, light grey shade r>0.5).
Table 8 shows average of mean error matrix between brain regions and EEG frequency ranges in MNC (dark grey shade ME>0.2, light grey shade ME>1).
The example of extracting the EEG spectral index from the pupillary response and ECG signals for the subjects was shown in
r=0.634, ME=0.006 for low beta in FP1
r=0.688, ME=0.106 for mid beta in FP1
r=0.656, ME=0.004 for high beta in F8
r=0.639, ME=0.020 for beta in F3
r=0.677, ME=0.055 for SMR in FP1
r=0.712, ME=0.065 for gamma in P4
r=0.714, ME=0.053 for mu in C4
When comparing the results with ground truth in NMC, the EEG spectral index from pupillary response indicated a strong correlation for all parameters where r=0.642±0.057 for low beta power in the FP1 region; r=0.656±0.056 for mid beta power in the FP1 region; r=0.646±0.063 for SMR power in the FP1 region; r=0.662±0.056 for beta power in the F3 region; r=0.648±0.055 for high beta power in the F8 region; r=0.650±0.054 for mu power in the C4 region; and r=0.641±0.059 for gamma power in the P4 region.
The difference between the mean error was of all parameters was low with ME=0.494±0.196 for low beta power in the FP1 region; ME=0.472±0.180 for mid beta power in the FP1 region; ME=0.495±0.198 for SMR power in the FP1 region; ME=0.483±0.180 for beta power in the F3 region; ME=0.476±0.193 for high beta power in the F8 region; ME=0.483±0.198 for mu power in the C4 region; and ME=0.488±0.177 for gamma power in the P4 region.
This procedure was processed by the sliding window technique, where the window size was 180 s and the resolution was 1 s by using recorded data for 300 s. The correlation and mean error were the mean value for the test 70 subjects (in one subject, N=120), as shown in Tables 9 and 10.
Table 9 shows average of correlation coefficient of EEG spectral index in MNC (N=120, p<0.01).
Table 10 shows average of mean error of EEG spectral index in NMC (N=120).
The correlation and mean error matrix table between the brain regions and the EEG frequency ranges are shown in Tables 10 and 11. Low beta, mid beta, and SMR power from the pupillary response indicated moderate correlation and had little difference compared to the EEG power band in the FP1 and FP2 regions (r>0.4, ME<1.5).
Beta power from the pupillary response indicated a moderate correlation and had little difference compared to the EEG power band in the F3, F4, and Fz brain regions (r>0.4, ME<1.5). The high beta power from the pupillary response indicated moderate correlation and had little difference compared to the EEG power band in the F7 and F8 brain regions (r>0.4, ME<1.5).
The mu power from the pupillary response indicated moderate correlation and had little difference compared to the EEG power band in the C3, C4, and Cz brain regions (r>0.4, ME<1.5).
Gamma power from the pupillary response indicated moderate correlation and had little difference compared to the EEG power band in the P3 and P4 brain regions (r>0.4, ME<1.5).
Other brain regions and frequency ranges indicated a low correlation and a large difference (r<0.4, ME>1.5). Low beta, mid beta, SMR, beta, high beta, mu, and gamma were the higher correlations and had very little difference (r>0.6, ME<0.5) with FP1, FP1, FP1, F3, F8, C4, and P4, respectively.
Table 11 shows average of correlation matrix between brain regions and EEG frequency ranges in NMC (dark grey shade r>0.6, light grey shade r>0.4).
Table 12 shows average of mean error matrix between brain regions and EEG frequency ranges in NMC (dark grey shade ME>0.5, light grey shade ME>1.5).
The real-time system for detecting human vital signs was developed using the pupil image from an infrared webcam. This system consisted of an infrared webcam, near IR (Infra-Red light) illuminator (IR lamp) and personal computer for analysis.
The infrared webcam was divided into two types, the fixed type, which is a common USB webcam, and the portable type, which are represented by wearable devices. The webcam was a HD Pro C920 from Logitech Inc. converted into an infrared webcam to detect the pupil area.
The IR filter inside the webcam was removed and an IR passing filter used for cutting visible light from Kodac Inc., was inserted into the webcam to allow passage of IR wavelength longer than 750 nm, as shown in
The conventional 12 mm lens of the USB webcam shown in
In
As described in the above, the present invention develops and provides an advanced method for non-contact measurements of human vital signs from moving images of the pupil. Thereby, the measurement of parameters in cardiac time domain can be performed by using a low-cost infrared webcam system that monitored pupillary rhythm. The EEG spectral indexes presents the low beta power, mid beta power, and SMR power in FP1 region, beta power in F3 region, high beta power in F8 region, mu power in C4 region, and gamma power in P4 region.
This result was verified for both the conditions of noise (MNC and NMC) and various physiological states (variation of arousal and valence level by emotional stimuli of sound) for seventy subjects.
The research for this invention examined the variation in human physiological conditions caused by the stimuli of arousal, relaxation, positive, negative, and neutral moods during verification experiments. The method based on pupillary response according to the present invention is an advanced technique for vital sign monitoring that can measure vital signs in either static or dynamic situations.
The proposed method according to the present invention is capable of measuring parameters in cardiac time domain with a simple, low-cost, non-invasive, and non-contact measurement system. The present invention may be applied to various industries such as U-health care, emotional ICT, human factors, HCI, and security that require VSM technology. Additionally, it should have a significant ripple effect in terms implementation of non-contact measurements.
It should be understood that embodiments described herein should be considered in a descriptive sense only and not for purposes of limitation. Descriptions of features or aspects within each embodiment should typically be considered as available for other similar features or aspects in other embodiments.
While one or more embodiments have been described with reference to the figures, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the disclosure as defined by the following claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2017-0021519 | Feb 2017 | KR | national |
| 10-2017-0147607 | Nov 2017 | KR | national |
