The invention relates to a method for determining the perception or perceptive quality of a subject, in particular of a human, by means of a brain-computer interface.
The prior art has disclosed a multiplicity of different methods for determining the perception or perceptive quality of subjects, with all these methods applying different types of stimuli to the subject and testing on the basis of the reaction of the subject in the form of brain waves as to whether or not the relevant subject exhibits an adequate mental activity on account of the stimuli.
A particular problem in the prior art is that it is often difficult to evaluate the reaction of the subject and that the perception or perceptive quality of the subject occasionally decreases or increases during the test; in particular, a problem arising in the case of comatose patients is that it is unclear whether the relevant patient or subject is currently asleep. In this context, it is possible for the subject to fall asleep after a training phase, during which they are awake, and the established test shows a very poor perception of the subject whereas, in principle, the subject is however by all means capable of perception. This leads to an incorrect assessment of the mental state and may occasionally have significant consequences for the relevant patient or subject, especially since such tests are only performed very infrequently.
The invention removes these problems of a method as set forth at the outset by way of the features of the patent claims.
The invention comprises a method for determining the perception or perceptive quality of a subject by means of a brain-computer interface, said method comprising the following steps:
a) At least two mutually distinguishable types of stimuli which are applicable to a subject, in particular acoustic, mechanical, electric or optical stimuli, are prescribed, wherein the subject is instructed, in particular, to carry out a specific mental act, preferably to count, when a specific stimulus is present.
b) A multiplicity of temporally successive stimuli are applied to the subject, the stimuli applied to the subject are combined to form blocks, wherein
c) Calibration data are created from the EEG data ascertained thus by virtue of a number of ascertained EEG data and stimuli assigned to these EEG data being combined to form calibration blocks, wherein the EEG data of the subject upon application of a first stimulus are respectively assigned to one place and the EEG data of the subject upon application of a further stimulus are assigned to the remaining places in each individual calibration block, and wherein a classification function is ascertained by means of a classification analysis on the basis of ascertained calibration blocks, said classification function specifying the position of the stimulus of the first type in the respective calibration block on the basis of EEG data of a block of stimuli.
d) The EEG data of a number of test blocks selected from the blocks are subjected to the ascertained classification function and a classification result is ascertained. An examination is carried out as to whether the position at which the stimulus of the first type is situated in the respective test block corresponds to the classification result. The number of test blocks for which a correspondence is identified is used as a measure for the perception of a subject at the time of recording the further EEG data.
In this context, it is particularly advantageous that the time at which the individual EEG data were ascertained only plays a subordinate role within the scope of assessing the perception or perceptive quality.
A particularly advantageous variant of the invention, by means of which comprehensive mixing of the individual EEG data recorded at different times may be achieved, provides that steps c) and d) are executed multiple times on the basis of the same ascertained blocks,
wherein, respectively during each individual execution of steps c) and d)
An advantageous procedure which facilitates continuous adaptation of the relevant perception measure and which exhibits convergence to a certain maximum value in the case of patients with complete perceptive capability provides that blocks are continuously ascertained in accordance with step b), wherein steps c) and d) are respectively executed on the basis of the previously ascertained blocks after one or more blocks were recorded,
wherein, respectively during each individual execution of steps c) and d)
In order to obtain an evaluation which is as complete as possible of all EEG data obtained within the scope of the recording, there may be provision for individual blocks to be selected in step c) as calibration blocks, in particular according to random criteria, wherein this selection, in particular, comprises at least 50% of the blocks, for of the remaining ascertained blocks, in particular all remaining blocks or at least 10% of the remaining blocks to be subjected to the classification function as test blocks in step d) and an examination to be carried out as to whether the position at which the stimulus of the first type is situated in the respective block corresponds to the classification result.
In order to be able to subsequently undertake the determination of the perception of the respective subject again in a simple and fast manner, provision may be made for a number of successive further stimuli to be applied to the subject after formation of the classification function, said further stimuli being combined in accordance with step b) to form further blocks, and for an individual examination to be carried out in each case for these as to whether the position at which the stimulus of the first type is situated in the respective further block corresponds to the classification result and the number of further blocks for which a correspondence is identified to be used as a measure for the perception of a subject at the time of recording the further EEG data.
An increase in the measure of the perception of the subject may additionally be achieved by virtue of at least one of the stimuli, in particular all stimuli or at least stimuli of the first type, being applied to the subject in a vibrotactile manner, and by virtue of an activation stimulus being applied to the subject if a classification result exceeding a threshold is present.
Here, in particular, provision may be made in a preferred embodiment of the invention for the activation stimulus applied to the subject to consist of an application of functional electrostimulation or an application with an orthosis, a prosthesis or a robot on a region of the body of the subject, by means of which the body of the subject is stimulated at a place or by means of which a part of the body of the subject, which was stimulated in a vibrotactile manner, is manipulated.
A further preferred embodiment of the invention, which may be carried out on a multiplicity of patients, even in patients without the faculty of sight, provides for the set of the types of stimuli to be prescribed by different sounds, in particular having different durations, frequencies and volumes, at frequencies which are audible for a human and the respective sound to be played to the subject, or for the set of the types of stimuli to comprise applications of vibrations to different body parts and/or with different intensities and/or durations, said applications of vibrations being applied to the subject by means of vibration units.
A further preferred embodiment of the invention, which may be carried out on a multiplicity of patients, even on patients without the faculty of hearing, provides for the set of the types of stimuli to comprise visual stimuli for one eye or both eyes and/or visual stimuli with different intensities and/or durations, said visual stimuli being applied to the subject by means of a screen or by means of luminous means, or for the set of the types of stimuli to comprise electrical stimuli at different body parts and/or electrical stimuli with different intensities and/or durations, said electrical stimuli being applied to the subject by means of electrical stimulators.
Particularly good identifiability in the EEG data may be achieved if, depending on the respective type of stimulus, the subject is tasked with one of the following mental acts:
A particularly advantageous analysis of the EEG data may be achieved by virtue of the classification analysis being carried out as follows:
For the purposes of determining the dependence of the perception and perceptive quality of the subject on external influences, provision may be made, after determining the perception or perceptive quality, for the subject to be brought into different states by cooling or heating the body or a part of the body and/or by medication or for the oxygen partial pressure to be changed in the region of the subject and the perception or perceptive quality to be repeated according to the same criteria and a new measure to be ascertained for the perception or perceptive quality and the measure and the new measure to be compared to one another.
For the same purpose, provision may advantageously be made, after formation of the classification function, for the subject to be brought into different states by cooling or heating the body or a part of the body and/or by medication or for the oxygen partial pressure to be changed in the region of the subject, and for a number of successive further stimuli to be applied to the subject, said further stimuli being combined in accordance with step b) to form further blocks, and for an individual examination to be carried out in each case for these as to whether the position at which the stimulus of the first type is situated in the respective further block corresponds to the classification result and the number of further blocks for which a correspondence is identified to be used as a new measure for the perception of a subject at the time of recording the further EEG data, and for the measure and the new measure to be compared to one another.
An advantageous feedback to the subject may be achieved by virtue of the recorded EEG data being classified by means of the previously executed classification analysis and, if an ascertained result is available from the classification, by virtue of an activation stimulus assigned to this result being applied to the subject.
In the following text, a preferred exemplary embodiment of the invention is illustrated on the basis of the figures of the drawing specified below.
In the present exemplary embodiment, the individual stimuli SA, SB have prescribed intervals between one another and between them and the respective start time of the block B1, B2, B3, B4. Particularly advantageously, the time interval between every stimulus SA, SB and the respective subsequent stimulus SA, SB is the same such that the subject 1 is unable to identify when a block B1, B2, B3, B4 ends and the subsequent block B1, B2, B3, B4 starts.
Advantageously, the subject 1 may also be instructed to carry out a specific mental act when a specific stimulus is present, in particular the subject may be instructed to count or calculate, to think of specific movements of body parts, in particular to think of the movement of the extremities. Such thoughts of the subject 1 may be recorded by means of the brain-computer interface 20 and analyzed in the test and control unit 30.
In the present exemplary embodiment, a multiplicity of temporally successive stimuli SA, SB are applied to the subject 1 (
In the present exemplary embodiment, the stimuli SA of the first type are high sounds played for the subject 1 while the stimuli SB of the second type are lower sounds played for the subject 1. In the present exemplary embodiment, each block B1, B2, B3, B4 comprises one high sound SA and seven low sounds SB in each case. The procedure depicted in
Calibration data are created from the totality of the EEG data U1, U2, U3, U4 obtained thus with the respectively assigned stimulus SA, SB by virtue of the EEG data U1, U2, U3, U4 assigned to a stimulus being combined together with the respective stimulus SA, SB to form calibration blocks KB1, KB2, KB3, KB4 (
By way of example, 80% of the available EEG data U1, U2, U3, U4 or 80% of the available recorded blocks B1, B2, B3, B4 may be used for creating the calibration blocks KB1, KB2, KB3, KB4. In a further step, the individual EEG data U1, U2, U3, U4 are subjected to a classification analysis K, wherein a classification function fK is ascertained on the basis of the ascertained calibration blocks KB1, KB2, KB3, KB4, said classification function specifying the position p of the stimulus SA of the first type in the respective block or calibration block on the basis of EEG data U1, U2, U3, U4 of a block B1, B2, B3, B4 or calibration block KB1, KB2, KB3, KB4 of EEG data U1, U2, U3, U4.
Here, it is advantageously possible for the classification analysis in respect of the calibration blocks KB1, KB2, KB3, KB4 to be carried out using one of the classification methods presented below:
In all of the methods specified above, it is possible, proceeding from the EEG data U1, U2, U3, U4 ascertained when recording a block B1, B2, B3, B4, to make a statement in respect of the position p of the stimulus SA of the first type within the respective block B1, B2, B3, B4.
The remaining ascertained blocks BT1, BT2, BT3, BT4, in particular at least 10% of the remaining blocks BT1, BT2, BT3, BT4, or the remaining EEG data U1, U2, U3, U4 are subjected to the ascertained classification function fK as test blocks. A classification result K is ascertained, said classification result specifying whether the position p at which the stimulus SA of the first type is situated in the respective test block BT1, BT2, BT3, BT4 corresponds to the classification result K. The number n of test blocks BT1, BT2, BT3, BT4 for which a correspondence is identified is used as a measure M of the perception or perceptive quality of a subject 1 at the time of recording the EEG data U1, U2, U3, U4.
In a preferred embodiment of the invention, it is possible to execute the steps of forming a classification function fK and of applying the respective classification function fK to the test blocks BT1, BT2, BT3, BT4 multiple times. Here, a separate measure M1, . . . , Mn is ascertained for the perception or perceptive quality of the subject 1 in each case for each individual execution of these steps. A further measure M* for the perception or perceptive quality may be ascertained from these individual measures M1, . . . , Mn, for example by averaging or aggregating the individual ascertained separate measures M1, . . . , Mn.
A further method in accordance with a preferred embodiment of the invention has the advantage that it can access to a continuously increasing amount of ascertained or recorded blocks B1, B2, B3, B4 (
The number n of blocks required until the ascertained measure MM1, . . . , MMn exceeds the threshold T may be considered to be a further measure M** for the perception or perceptive quality of the subject 1. Further, the mean value or median of the individual ascertained measures may be considered to be a further measure M*** for the perception or perceptive quality of the subject 1. Advantageously, the threshold T may be set in such a way that the subject 1 correctly assigns more than 90%, in particular more than 95%, of the blocks B1, B2, B3, B4 or, for this block B1, B2, B3, B4, correctly determines the position of the stimulus SA of the first type in the respective block.
A preferred embodiment of the invention, which may be combined with all aforementioned embodiment variants of the invention, provides for the test to which the subject 1 was subjected to be executed again after a prescribed time interval. What is particularly advantageous in relation to this procedure is that classification functions fK are already available for the relevant subject and it is expected that, upon application of the same stimuli SA, SB, the same subject 1 will probably supply the same results, and so there is no need for complicated determination of the classification function fK. This procedure is advantageous, in particular, if little time has passed since determining the classification function fK.
When carrying out the aforementioned tests again, the recorded EEG data U1, U2, U3, U4 and stimuli SA, SB are once again assigned to one another and combined to form further blocks B1′, B2′, B3′, B4′, wherein these further blocks B1′, B2′, B3′, B4′ have the same design as the calibration blocks KB1, KB2, KB3, KB4 and the blocks B1, B2, B3, B4 ascertained within the scope of the calibration. An individual examination is carried out in each case for each of the further blocks B1′, B2′, B3′, B4′ in respect of whether the position at which the stimulus SA of the first type is situated in the respective further block B1′, B2′, B3′, B4′ corresponds to the classification result K, i.e. the result of applying the classification function fK to the further block B1′, B2′, B3′, B4′. The number of further blocks B1′, B2′, B3′, B4′ for which a correspondence was identified is determined and used as a measure M* for the perception of a subject at the time of recording the further EEG data U′1, U′2, U′3, U′4.
Instead of using sounds with different frequencies, it is also possible, in all aforementioned exemplary embodiments of the invention, to prescribe sounds with different durations or volumes at frequencies that are audible for a human. Alternatively, it is also possible to apply vibrations to different body parts of the relevant subject 1, with intensity and duration of the vibrations differing and a distinction being possible between the first stimulus and further stimuli SA, SB on account of different intensities or durations of the applications of vibrations.
Alternatively, it is also possible for the set of the types of stimuli SA, SB to comprise visual stimuli for one eye or both eyes and/or visual stimuli with different intensities and/or durations, which are applied to the subject 1 by means of a screen or by means of a luminous means. Alternatively, the set of the types of stimuli SA, SB may also comprise electrical stimuli which are exerted on different body parts and/or with different intensities and/or different durations.
Provided that vibrotactile stimuli SA, SB are applied to the subject 1, it is possible, in a preferred embodiment of the invention after successfully determining perception or a perceptive quality exceeding a threshold, for an application of a functional electrostimulation or an application with an orthosis, a prosthesis or a robot on a region of the body of the subject 1 to exist, by means of which the body of the subject 1 is stimulated at a place or by means of which part of the body of the subject 1, which was stimulated in a vibrotactile manner, is manipulated.
Feedback may be provided to the subject using this measure. Here, the recorded EEG data are classified by means of a previously executed classification analysis. If an ascertained result from the classification is available, an activation stimulus assigned to this result is applied to the subject.
In some cases, it may be advantageous if the above-described procedure is repeated when the subject is exposed to certain influences. Such influences may include:
The test is carried out again under these influences. This renders it possible to ascertain the dependence of the perception of the subject 1 depending on the respective external influences.
Here, it is not necessary to form the classification function again in the presence of the external influence. Rather, as described above, the classification function may be maintained. Only the further blocks newly ascertained when the external influence is present are subjected to the classification function. In this manner, a new measure for the perception and perceptive quality is formed and compared to the previously ascertained measure for the perception and perceptive quality.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/AT2016/050117 | 4/29/2016 | WO | 00 |
Publishing Document | Publishing Date | Country | Kind |
---|---|---|---|
WO2017/185109 | 11/2/2017 | WO | A |
Number | Name | Date | Kind |
---|---|---|---|
6092058 | Smyth | Jul 2000 | A |
20040162489 | Richards-Kortum | Aug 2004 | A1 |
20140228701 | Chizeck | Aug 2014 | A1 |
20140347265 | Aimone | Nov 2014 | A1 |
20150230744 | Faubert et al. | Aug 2015 | A1 |
20160360992 | Guger et al. | Dec 2016 | A1 |
Number | Date | Country |
---|---|---|
2015530895 | Oct 2015 | JP |
2015533559 | Nov 2015 | JP |
2012104853 | Aug 2012 | WO |
2014052938 | Apr 2014 | WO |
2015058223 | Apr 2015 | WO |
Entry |
---|
King, Jr. et al., “Single-trial decoding of auditory novelty responses facilitates the detection of residual consciousness”, Dec. 1, 2013, pp. 726-738. |
Anuradha Saha et al.: “EEG Analysis for Olfactory Perceptual-Ability Measurement Using a Recurrent Neural Classifier”, IEEE Transactions on Human Machine Systems, vol. 44, No. 6, Dec. 2014, pp. 717-730. |
Number | Date | Country | |
---|---|---|---|
20180092566 A1 | Apr 2018 | US |