Patent Application
20160113539
The present application does claim priority from an Indian patent application number 3390/MUM/2014 filed on 26 Oct. 2014.
The present subject matter described herein, in general, relates to processing of bioelectric signals for diagnostic purposes, and more particularly to determining a cognitive load of a subject from Electroencephalography (EEG) signals.
Cognitive load is a total amount of mental activity imposed on a memory of a subject/human while performing any cognitive task. High cognitive load may significantly influence performance of the subject leading to poor outcome, stress, or anxiety. Now days, measurement of the cognitive load is receiving increased attention. Physiological measures like brain signals, a galvanic skin response, a functional magnetic resonance imaging (MRI), and an electroencephalogram (EEG) technique may be used to quantify the cognitive load. As compared to other available techniques, the EEG technique is relatively in-expensive, non-invasive and has excellent temporal resolution. Currently a time-domain and a frequency-domain of the EEG signals and statistical parameters have been used to measure the cognitive load.
There are various devices available in market to measure the cognitive load using the EEG technique. One type of EEG devices may include high cost and high resolution EEG devices which may fall under precise medical diagnostic devices and other type may include low cost low resolution EEG devices used for all-purpose diagnostic services. The low cost low resolution EEG devices come with lower number of EEG channels, hence may miss the EEG channels that are sensitive to the cognitive load. Further, sensitive positions of the EEG channels may be subjective and may vary from person to person. Moreover, the low cost low resolution EEG devices do not come with extra channels (EOG) to measure and remove a noise contaminating the EEG signals. The noise may pose serious issue in accurately measuring the cognitive load of the subject.
Different algorithmic approaches are available in prior art to measure the cognitive load of the subject. However these algorithms come with excessive computation and also fail to provide accurate results in measurement of the cognitive load, particularly in measurement of the cognitive load with low cost, low resolution EEG devices.
This summary is provided to introduce aspects related to systems and methods for determining a cognitive load of a subject from Electroencephalography (EEG) signals and the aspects are further described below in the detailed description. This summary is not intended to identify essential features of the claimed subject matter nor is it intended for use in determining or limiting the scope of the claimed subject matter.
In one implementation, a method for determining a cognitive load of a subject from Electroencephalography (EEG) signals is disclosed. The method comprises receiving, by a processor, the EEG signals from a set of EEG channels associated with a left-frontal brain lobe. The EEG signals are received from a low resolution EEG device comprising a maximum of fourteen EEG channels. The EEG signals are associated with the subject performing a cognitive task. The method further comprises preprocessing, by the processor, the EEG signals using a Hilbert-Huang Transform (HHT) filter to remove a noise corresponding to one or more unrelated, non-cerebral artifacts to generate preprocessed EEG signals. The method further comprises extracting, by the processor, features comprising Fast Fourier Transform (FFT) based alpha and theta band power, from the preprocessed EEG signals. The method further comprises generating, by the processor, a feature vector from the features. The method further comprises classifying, by the processor, the feature vector using a supervised machine learning technique to determine the cognitive load of the subject.
In one implementation, a system for determining a cognitive load of a subject from Electroencephalography (EEG) signals is disclosed. The system comprises a processor and a memory coupled to the processor. The processor is capable of executing programmed instructions stored in the memory to receive the EEG signals from a set of EEG channels associated with a left-frontal brain lobe. The EEG signals are associated with the subject performing a cognitive task. The processor further preprocesses the EEG signals using a Hilbert-Huang Transform (HHT) filter to remove noise corresponding to one or more unrelated, non-cerebral artifacts to generate preprocessed EEG signals. The processor further extracts features comprising Fast Fourier Transform (FFT) based alpha and theta band power, from the preprocessed EEG signals. The processor further generates a feature vector from the features. The processor further classifies the feature vector using a supervised machine learning technique to determine the cognitive load of the subject.
In one implementation, a computer program product having embodied thereon a computer program for determining a cognitive load of a subject from Electroencephalography (EEG) signals. The computer program product comprises a program code for receiving, the EEG signals from a set of EEG channels associated with a left-frontal brain lobe, wherein the EEG signals are associated with the subject performing a cognitive task. The computer program product further comprises a program code for preprocessing, the EEG signals using a Hilbert-Huang Transform (HHT) filter to remove a noise corresponding to one or more unrelated, non-cerebral artifacts to generate preprocessed EEG signals. The computer program product further comprises a program code for extracting features comprising Fast Fourier Transform (FFT) based alpha and theta band power from the preprocessed EEG signals. The computer program product further comprises a program code for generating a feature vector from the features. The computer program product further comprises a program code for classifying the feature vector using a supervised machine learning technique to determine the cognitive load of the subject.
The detailed description is described with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the drawings to refer like features and components.
Systems and methods for determining a cognitive load of a subject from Electroencephalography (EEG) signals are described. The EEG signals may be received from a set of EEG channels. The EEG signals may be associated with the subject performing a cognitive task. The EEG signals may be preprocessed to remove noise corresponding to one or more unrelated, non-cerebral artifacts to generate preprocessed EEG signals. Features comprising Fast Fourier Transform (FFT) based alpha and theta band power may be extracted from the preprocessed EEG signals. A feature vector may be generated from the features. The feature vector may be classified using a supervised machine learning technique to determine the cognitive load of the subject.
While aspects of described system and method for determining a cognitive load of a subject from Electroencephalography (EEG) signals may be implemented in any number of different computing systems, environments, and/or configurations, the embodiments are described in the context of the following exemplary system.
Referring now to
The system may extract features comprising Fast Fourier Transform (FFT) based alpha and theta band power from the preprocessed EEG signals. Subsequent to extracting the features, the system may generate a feature vector from the features. Post generating the feature vector, the system may classify the feature vector using a supervised machine learning technique to determine the cognitive load of the subject.
Although the present subject matter is explained considering that the system 102 is implemented on a server, it may be understood that the system 102 may also be implemented in a variety of computing systems, such as a laptop computer, a desktop computer, a notebook, a workstation, a mainframe computer, a server, a network server, and the like. In one implementation, the system 102 may be implemented in a cloud-based environment. It will be understood that the system 102 may be accessed by multiple users through one or more user devices 104-1, 104-2 . . . 104-N, collectively referred to as user devices 104 hereinafter, or applications residing on the user devices 104. Examples of the user devices 104 may include, but are not limited to, a portable computer, a personal digital assistant, a handheld device, a server and a workstation. The user devices 104 are communicatively coupled to the system 102 through a network 106.
In one implementation, the network 106 may be a wireless network, a wired network or a combination thereof. The network 106 can be implemented as one of the different types of networks, such as intranet, local area network (LAN), wide area network (WAN), the internet, and the like. The network 106 may either be a dedicated network or a shared network. The shared network represents an association of the different types of networks that use a variety of protocols, for example, Hypertext Transfer Protocol (HTTP), Transmission Control Protocol/Internet Protocol (TCP/IP), Wireless Application Protocol (WAP), and the like, to communicate with one another. Further the network 106 may include a variety of network devices, including routers, bridges, servers, computing devices, storage devices, and the like.
Referring now to
The I/O interface 112 may include a variety of software and hardware interfaces, for example, a web interface, a graphical user interface, and the like. The I/O interface 112 may allow the system 102 to interact with a user directly or through the client devices 104. Further, the I/O interface 112 may enable the system 102 to communicate with other computing devices, such as web servers and external data servers (not shown). The I/O interface 112 can facilitate multiple communications within a wide variety of networks and protocol types, including wired networks, for example, LAN, cable, etc., and wireless networks, such as WLAN, cellular, or satellite. The I/O interface 112 may include one or more ports for connecting a number of devices to one another or to another server.
The memory 114 may include any computer-readable medium known in the art including, for example, volatile memory, such as static random access memory (SRAM) and dynamic random access memory (DRAM), and/or non-volatile memory, such as read only memory (ROM), erasable programmable ROM, flash memories, hard disks, optical disks, and magnetic tapes. The memory 114 may include the programmed instructions and data 116.
The data 116, amongst other things, serves as a repository for storing data processed, received, and generated by execution of the programmed instructions. The data 116 may also include a system database 118.
In one implementation, at first, a user may use the client device 104 to access the system 102 via the I/O interface 112. The user may register using the I/O interface 112 in order to use the system 102. The working of the system 102 may be explained in detail in
In one embodiment, in order to determine the cognitive load of the subject, the system 102 may receive the EEG signals from a set of EEG channels linked to an EEG device 120. The EEG signals may be associated with the subject performing a cognitive task. The cognitive task may be a task in which a mental activity is imposed on a memory of the subject. For example, the cognitive task may comprise a problem solving task, a decision making task, language skills, and the like. The EEG signals may be received from low resolution EEG electrodes/channels. The low resolution EEG device may comprise a maximum of fourteen EEG channels.
Cortex or cerebrum region is the largest part of a brain. The cortex or the cerebrum region is associated with different functions of the brain like critical thinking, perception, decision making and the like. Different lobes of the cortex are responsible for different cognitive functions of the brain. For example, an occipital lobe is associated with visual perception, a temporal lobe is associated with perception and recognition of an auditory stimuli and the like. The cognitive load variations for tasks of different difficulty levels are most clearly visible if frontal and parietal lobes are considered. In one embodiment, the system 102 may receive the EEG signals from the set of EEG channels associated with a left-frontal brain lobe. The set of EEG channels may comprise four EEG channels associated with the left-frontal brain lobe. Frontal region of the brain is mainly responsible for cognitive tasks like problem solving or decision making and the like. By way of an example, subjects (participants) selected in the present experiments are right-handed human beings and hence their left-frontal brain lobe is most indicative of the cognitive load. Further, the low resolution device used in the present disclosure has only four (AF3, F7, F3 and FC5) channels/electrodes in the left-frontal brain lobe region. Hence, the set of EEG channels comprises four channels (AF3, F7, F3 and FC5) associated with the left-frontal brain lobe. For example, the combination of the four channels (AF3, F7, F3 and FC5) associated with the left-frontal brain lobe is proved as a promising combination for determining the cognitive load of the subject.
Post receiving the EEG signals, the system 102 may preprocess the EEG signals to generate preprocessed EEG signals. The system 102 may preprocess the EEG signals to remove a noise corresponding to one or more non-cerebral artifacts to generate the preprocessed EEG signals. In one embodiment, the system may preprocess the EEG signals using a Hilbert-Huang Transform (HHT) filter. The EEG signals may be susceptible to the one or more unrelated, non-cerebral artifacts. The EEG signals may be contaminated by the one or more non-cerebral artifacts. The one or more non-cerebral artifacts may comprise artifacts not directly associated with brain activity and may be associated with other body parts. The one or more non-cerebral artifacts may comprise eye-blinks, eye movements, extra-ocular muscle activity, facial muscle movements, cardiac artifacts, and the like. The one or more non-cerebral artifacts may also comprise artifacts originated from outside the body of the subject and may be associated with a machine or an environment. The one or more non-cerebral artifacts may also comprise movement of the subject, settling of EEG electrodes, spikes originating from a momentary change in an impedance of an EEG electrode, or poor grounding of the EEG electrodes.
According to an exemplary embodiment, the Hilbert Huang Transform (HHT) filter decomposes the EEG signals into empirical mode signals. The empirical mode signals can be used to visualize changes in the EEG signals in both a time domain and a frequency domain, Hence the Hilbert Huang Transform (HHT) filter enable manipulation of a frequency of the EEG signals at a given time point. Further, Hilbert Huang Transform (HHT) also provides computational simplicity over other time frequency decomposition filters like wavelet transform and the like. When at the given time point, the frequency of the EEG signals overshoots a predefined level, the Hilbert Huang Transform (HHT) filter adaptively removes the noise from the EEG signals using simple statistical measures such as replacing the frequency with a median value at the given time point.
Subsequent to generating the preprocessed EEG signals, the system 102 may extract features from the preprocessed EEG signals using techniques known in the art. The features are extracted based upon experimentation and quality of results there from. In one embodiment, the system 102 may extract features comprising Fast Fourier Transform (FFT) based alpha and theta band power from the preprocessed EEG signals.
Post extracting the features, the system may generate a feature vector from the features. Depending on an algorithmic approach used, a number of features, in different combination are used to generate the feature vector. Referring to Table 1, different algorithmic approaches, comprising various combinations of features, in path1 to path6 are explained. The features comprises a variance, Hjorth parameters, alpha (α), beta (β), theta (θ), delta (δ), gamma (γ) band powers, band power ratios such as β/θ and α/δ, and the like.
According to one embodiment, as per present disclosure, the feature vector may be generated from the features comprising the Fast Fourier Transform (FFT) based alpha and theta band power features. The selection of the Fast Fourier Transform (FFT) based alpha and theta band power features eliminate need of spatial filtration and provides sufficient quality of the features to further measure cognitive load of the subject. Since the selected features comprises only two types of features such as FFT based alpha and theta band power features, there is no need of spatial filtration. Thus by reducing the use of spatial filtration, the computation complexity in processing of the system 102 and a method 600 is considerably reduced.
Post generating the feature vector, the system may classify the feature vector using a supervised machine learning technique. The system may classify the feature vector using a supervised machine learning technique to determine the cognitive load of the subject. The supervised machine learning technique may be a Support Vector Machine (SVM) classifier.
According to an exemplary embodiment, determination of a cognitive load of a subject from Electroencephalography (EEG) signals is explained below. Experimental work and data collected during the experiments is also provided. Specifically, a group of 10 participants (subjects), aged between 25-30 yrs, are selected. All the 10 participants are right-handed male and have English as second language. The selection of the 10 participants ensures minimum variance in level of expertise and brain lateralization across all the 10 participants. Each participant is connected with a low resolution EEG device—‘Emotiv™’ headset positioned on head. The low resolution EEG device—‘Emotiv™’ headset has fourteen EEG channels. The fourteen EEG channels of ‘Emotiv™’ headset are arranged on head of the participant to probe different brain lobes. According to an exemplary embodiment, placement of 14 EEG channels of the low resolution EEG device—‘Emotiv™’ headset is shown in
In the experimental work, 10 participants are given with two sets of stimuli to work with. The two sets of stimuli are presented on a 9.7-inch iPad. The two sets of stimuli contain two high cognitive load tasks and two low cognitive load tasks. EEG signals corresponding to first set of stimuli are used as a training data and the second set of stimuli are used as a test data and vice versa. An average of results received from the two sets of stimuli is used as a final result.
The two high cognitive load tasks and two low cognitive load tasks are pertaining to reading activity. For a low cognitive load task, the subjects are asked to mentally count a number of two letter words (except ‘of’) while reading an English passage and report the number of counted words at the end. For a high cognitive load task, the subjects are asked to count two-letter words as well as three-letter words separately (except ‘of’ and ‘the’) while reading a similar English passage and report the number of counted words at the end.
While executing the two high cognitive load tasks and two low cognitive load tasks on each of the 10 participants, the EEG signals are received by the system 102 from the ‘Emotiv™’ device. As shown in
The features are extracted from the EEG signals based on the path selected from path1 to path6. A feature vector is generated from the features extracted from the EEG signals. The feature vector may comprise variance, Hjorth parameters, alpha (δ), beta (β), theta (θ), delta (δ), gamma (γ) band powers and ratios of band powers β/θ and α/δ based on selection of the path from Path1 to Path6.
Referring to
Finally, a cognitive score for a particular path for each participant is calculated using Equation (1). The cognitive score (CS) represents the cognitive load of the subject.
In Equation (1), mi is number of windows reported as class i, n is total number of windows in a test trial and wi is a weight-factor. For high cognitive load class wi=100 and for low cognitive load class wi=0. Hence for trial containing low cognitive load task CS and for trial containing high cognitive load task CS 100. The computational complexity (CC) of an algorithm is number of processing steps required for a particular input. In present disclosure, the computational complexity (CC) is defined in Equation (2) as shown below.
CC=n
c
×C
HHT)nc×(mf×F)+(nc×mf)×CCSP Equation (2)
In Equation (2), nc is a number of channels selected, L is a computational complexity for a single channel, CHHT is the computational complexity of HHT filter, mf is a number of features selected, F is a computational complexity for extracting a particular feature, CCSP is a computational complexity of using TRCSP filter. Thus Equation (1) gives a measure of the cognitive score for a particular algorithm and Equation (2) gives a measure of the computational complexity for the particular algorithm.
Table 2 provides the cognitive score of the 10 participants (CS) for executing cognitive tasks of a low cognitive load (L) and a high cognitive load (H), following Path1 through Path6. Since a result should clearly differentiate the high cognitive load (H) from the low cognitive load (L), achieving maximum separation between the high cognitive load (H) and the low cognitive load (L) is the purpose of this invention. As shown in Table 1, particularly the algorithm approach provided in path6 is disclosed in the present disclosure. As shown in path6, 2→3→4→6→8, the system 102 receives the EEG signals from a set of EEG channels associated with a left-frontal brain lobe (AF3, F7, F3, FC5). Further, system 102 preprocess the EEG signals using a Hilbert-Huang Transform (HHT) filter to remove a noise corresponding to one or more non-cerebral artifacts to generate preprocessed EEG signals. Further, as shown in path6, the system 102 extracts only the features comprising Fast Fourier Transform (FFT) based alpha and theta band power from the preprocessed EEG signals. Further, as shown in path 6, system 102 generates a feature vector from the features comprising Fast Fourier Transform (FFT) based alpha and theta band power and classifies the feature vector using a SVM classifier to calculate the cognitive score (CS) and computational complexity (CC). Table 2 contains Cognitive score (CS) of the low cognitive load (L) and high cognitive load (H) tasks for 10 participants using path1 to path6.
Referring to Equation (2), the results of the Computational Complexity (CC) are discussed below. For path1, path3 and path5, value of nc=14, further in Path2, path 4 and path6, value of nc=4. Hence by using less number of EEG channels, the Computational Complexity decreases for path2, path4 and path6. Initially for path1 and path2, number of features used are mf=11. Hence higher CC for path 1 and path2. Further for path 3 to path6, the number of features used is mf=2, hence reduced computational complexity for path 2 to path 6.
In path6, the computational complexity is further reduced by eliminating use of to TRCSP spatial filter. Hence path6 provides the least computational complexity in the experimental path1 to path6. The least computational complexity of path6 is attributed by i) Probing only left-frontal brain (AF3, F7, F3, FC5) ii) Using only alpha and theta band power features and iii) Alpha and theta band power features fed to SVM by eliminating use of TRCSP spatial filter. Since mf and nc of Equation (2) is reduced, so third component of equation (2) is also reduced to (2×3×CCSP) compared to original one in path1 (14×11×CCSP).
A higher value of F implies that samples are drawn from populations with different mean values, indicating that the samples belong to different groups. P is a probability of an observed result to be correct. The Box plot in
In a box plot, a median line is required to be at a middle of a box depicting ideal dispersion of data. Hence
29
43
100
4
97.4
34.2
17.6
84
21
100
44
65
78
71.4
29
33.3
52
77.7
23.5
100
57.5
42.3
67.3
Referring to
The Exemplary embodiments discussed above may provide certain advantages. Though not required to practice aspects of the disclosure, these advantages may include those provided by the following features.
Some embodiments enable a system and a method for determining a cognitive load of a subject from Electroencephalography (EEG) signals received from low cost and low resolution EEG devices comprising a maximum of fourteen EEG channels.
Some embodiments enable the system and the method for determining the cognitive load of the subject using EEG signals received only from four EEG channels associated with the left-frontal brain lobe.
Some embodiments enable the system and the method for determining the cognitive load of the subject using EEG signals by using optimized method having minimum algorithmic complexity, by using minimum number of FFT based features to accurately determine the cognitive load.
Some embodiments enable the system and the method for eliminating noise from the EEG signals using HHT.
Some embodiments enable the system and the method for eliminating use of spatial filters in processing of the EEG signals with improved accuracy while determining the cognitive load of the subject.
Some embodiments enable the system and the method for reducing computational complexity by using of reduced number of EEG channels, reduced number of EEG features and algorithmic simplicity.
Some embodiments enable the system and the method for determining the cognitive load of the subject from EEG signals with superior accuracy using low resolution EEG devices.
Referring now to
The order in which the method 600 is described is not intended to be construed as a limitation, and any number of the described method blocks can be combined in any order to implement the method 600 or alternate methods. Additionally, individual blocks may be deleted from the method 600 without departing from the spirit and scope of the subject matter described herein. Furthermore, the method 600 can be implemented in any suitable hardware, software, firmware, or combination thereof. However, for ease of explanation, in the embodiments described below, the method 600 may be considered to be implemented in the above described system 102.
At block 602, the EEG signals may be received. The EEG signals may be received from a set of EEG channels associated with a left-frontal brain lobe. The EEG signals may be associated with the subject performing a cognitive task. The EEG signals may be received from low resolution EEG device comprising a maximum of fourteen EEG channels. The set of EEG channels may comprise four EEG channels associated with the left-frontal brain lobe.
At block 604, the EEG signals may be preprocessed to remove a noise corresponding to one or more non-cerebral artifacts to generate preprocessed EEG signals. The EEG signals may be preprocessed using a Hilbert-Huang Transform (HHT) filter to remove the noise corresponding to the one or more non-cerebral artifacts to generate preprocessed EEG signals. The one or more non-cerebral artifacts may comprise artifacts not directly associated with brain activity and may be associated with other body parts or a machine.
At block 606, features may be extracted from the preprocessed EEG signals. The features may comprise Fast Fourier Transform (FFT) based alpha and theta band power based features.
At block 608, a feature vector may be generated from the features.
At block 610, the feature vector may be classified using a supervised machine learning technique to determine the cognitive load of the subject. The supervised machine learning technique may be a Support Vector Machine (SVM) classifier.
Although implementations of methods and systems for determining a cognitive load of a subject from Electroencephalography (EEG) signals have been described in a language specific to structural features and/or methods, it is to be understood that the appended claims are not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as examples of implementations for determining the cognitive load of the subject from the EEG signals.
| Number | Date | Country | Kind |
|---|---|---|---|
| 3390/MUM/2014 | Oct 2014 | IN | national |
