This application claims the benefit of Italian Application No. 102021000027866 filed on Oct. 29, 2021, which application is hereby incorporated herein by reference.
The present invention relates to a method for detecting an activity of eyes of a user of glasses. In particular, it relates to a detection method based on the use of electrodes configured to detect electrostatic charge variations generated by movements of the eyes, to a pair of glasses thereof comprising the electrodes and a control unit configured to implement the detection method, and to a computer program product thereof.
As known, electrooculography (EOG) is a technique to measure the corneo-retinal potential (CRP) that exists between the front portion (e.g., comprising the cornea) and the back portion (e.g., comprising the retina) of the human eye. In fact, the eye acts as a dipole where the front pole at the cornea is positive and the back pole at the retina is negative.
EOG is a commonly used measure as CRP is higher than the potentials involved in electroencelography (EEG signals) since the eye is placed outside the skull and therefore there is no bone structure that attenuates the electrical signal generated. To measure the eye movement, pairs of electrodes are placed around the eye and in contact with the skin of the face and, for example, a pair of electrodes above and below the eye and a pair of electrodes to the left and right of the eye.
In greater detail, when the eye rotates around an own center of rotation (e.g., substantially central to the same eyeball), it generates a CRP variation which is detected through the electrodes. In particular, when the eyelid closes the cornea moves in one direction while when the eyelid opens the cornea moves in the opposite direction, and this generates a CRP variation that is clearly recognizable and associable with this blink. The signal resulting from a blink generally has a low frequency, for example comprised between about 1 Hz and about 13 Hz.
Blinks have been shown to be correlated to a person's mental state, and in particular they are indicative of cognitive states such as relaxation or attention. For example, the Inter-Eye blink interval (IEBI) is a known and reliable biomarker of the degree of concentration of a person intent on performing an activity that requires visual attention, such as watching a film or performing manual work.
The detection of movements of the eyes, such as blink, may be relevant for applications aimed at determining the attention level of the user, the risks resulting from sleep and fatigue, visual disturbances or neurodegenerative diseases, or more simply to automatically activate functions in mobile and portable devices. In fact, detecting the movements of the eyes is relevant in applications such as smart glasses where knowing the direction of the gaze of the person who is wearing them is important, as well as acquiring commands provided by the person (e.g., blink, etc.). For example, eye tracking may activate advanced functions for adjusting the focal point of a camera lens or quick reading functions such as zoom and panoramic operations.
However, this known measurement technique suffers from the following problems: measurement noise due to alternating electric current at 50 Hz or 60 Hz possibly present in the environment where the measurement is carried out; noise due to head and body movements during the measurement; artifacts in the measurement caused by the operation of electrical apparatuses present in proximity to the electrodes; measurement errors due to contraction of the facial or neck muscles, or to the slipping of the electrodes on the skin due to sweat and eyelid blink.
Embodiments provide a method for detecting the activity of eyes of a user of glasses, the glasses and a computer program product that overcome the drawbacks of the prior art.
For a better understanding of the present invention, preferred embodiments are now described, purely by way of non-limiting example, with reference to the attached drawings, wherein:
The glasses 10 further comprise a main control unit 21 and one or more first electrostatic charge variation sensors electrically coupled to the main control unit 21 and fixed to the frame 12, and therefore arranged in proximity to the eyes of the user when the latter wears glasses 10.
According to an exemplary embodiment, the main control unit 21 (such as a microprocessor, a microcontroller or a dedicated calculation unit) comprises, coupled to each other, a processing unit 21a and a data storage unit 21b (such as a memory, e.g. a non-volatile memory) for storing the acquired data. For example, the main control unit 21 is integrated into the frame 12.
In the embodiment exemplarily considered hereinafter and shown in
In particular and as better shown in
In use, each electrode 22a, 22b detects a respective electrostatic charge variation caused by movements of the eyes of the user, as better discussed below, and generates a respective detection signal SR indicative of said electrostatic charge variation.
In detail, each electrode 22a, 22b may have a metal surface or be totally metal coated by dielectric material, or even have a metal surface arranged under an external case of the glasses 10. In any case, during use, each electrode 22a, 22b is electrostatically coupled to the environment having the glasses 10 present therein, and in greater detail to the user's eye which is closest to said electrode 22a, 22b, in order to detect the induced electrostatic charge variation thereof.
According to an embodiment, each electrode 22a, 22b is integrated into the external case of the glasses 10, and for example comprises a conductive track formed on, or in, a semiconductor material wafer comprised in the glasses 10. According to a different embodiment, each electrode 22a, 22b is a metal element present in the glasses 10. Optionally, when a possible use of the glasses 10 in a humid environment (more specifically in water) is envisaged, each electrode 22a, 22b is inserted inside a waterproof case or in any case it is shielded by one or more protective layers, so as to prevent direct contact of the electrode 22a, 22b with water or humidity: in this case, the waterproof case or the one or more protective layers are of a material (e.g., dielectric or insulating material, such as plastics) such that it does not shield the electrostatic charge generated by the user's eye, which needs to be acquired by the electrode 22a, 22b. Other embodiments are possible, as apparent to the person skilled in the art, so that the electrodes 22a, 22b are electrostatically coupled to the user's eyes during use.
Furthermore, according to an exemplary embodiment, the sensor control unit 15 (such as a microprocessor, a microcontroller or a dedicated calculation unit) comprises, coupled to each other: an interface unit 17 (optional and of known type) electrically coupled to the electrodes 22a and 22b to interface the latter with the sensor control unit 15 (e.g., the interface unit 17 comprises an amplification circuit and/or an analog-to-digital converter, ADC, not shown); a respective processing unit 16 for processing detection signals SR acquired through the electrodes 22a and 22b (and optionally processed through the interface unit 17); and a respective data storage unit 18 (such as a memory, e.g. a non-volatile memory) for storing the acquired data. For example, the sensor control unit 15 is integrated into the respective electrostatic charge variation sensor 20a, 20b.
In detail, each sensor control unit 15 is configured to process (in a per se known manner, for example by amplifying and converting into digital) the respective detection signals SR acquired through the electrodes 22a and 22b and to generate a respective first electrostatic charge variation signal SQ,1 indicative of a difference between the detection signals SR acquired through the first and second electrodes 22a and 22b. In particular, the first electrostatic charge variation signal SQ,1 is of digital type and is indicative of a difference between the electrostatic charge variations detected through the electrodes 22a and 22b.
According to an embodiment, the first and second electrodes 22a and 22b of each first electrostatic charge variation sensor 20a, 20b are spaced from each other. In particular, the electrodes 22a and 22b have a first mutual distance D1 from each other and, for example, are arranged on respective opposite ends of the respective support portion 12a, 12b (e.g., they are diametrically opposite to each other with respect to the respective support portion 12a, 12b of annular shape). For example, the first and second electrodes 22a and 22b are aligned with each other along a first axis 19 which joins the first and second lenses 14a and 14b (e.g., which joins the centers, e.g. the barycenter, of the lenses 14a and 14b).
In other words, as schematically shown in
According to an embodiment shown in
Each second electrostatic charge variation sensor 20c, 20d comprises respectively, instead of the first and the second electrodes 22a and 22b, a third and a fourth electrode 22c and 22d which are spaced both from each other and with respect to the first and second electrodes 22a and 22b and which are similar to the first and second electrodes 22a and 22b (and therefore are not described again in detail). In particular, the third and fourth electrodes 22c and 22d are radially internal with respect to the first and second electrodes 22a and 22b with respect to the center of the respective lens 14a, 14b. For example, the third and fourth electrodes 22c and 22d have a second mutual distance D2 from each other that is less than the first mutual distance D1 and, for example, are fixed to the respective lens 14a, 14b so as to face the eye 30 of the user when the latter wears the glasses 10. For example, the third and fourth electrodes 22c and 22d are aligned with each other and with the first and second electrodes 22a and 22b along the first axis 19, such that the third and fourth electrodes 22c and 22d are interposed between the first and second electrodes 22a and 22b. In other words, as schematically shown in
In general, since the eye 30 operates as an electric dipole with a positive pole at the cornea 30a and a negative pole at the retina (indicated in
In greater detail, it has been verified that these movements of the eyes 30 in the presence of blinks generate, in each first electrostatic charge variation signal SQ,1 and in succession to each other in a blink period with a duration of less than about 50 ms, two respective peaks having opposite sign with respect to a baseline of this signal. In particular, by exemplarily considering a zero baseline, it is possible to have a first positive peak and a second negative peak or vice versa, as a function of the direction of movement of the eyes 30 and of the positions of the first and second electrodes 22a and 22b. These first and second consecutive peaks in the blink period define a blink scheme (or pattern) that is indicative of a blink (a voluntary or involuntary blink, as better discussed below). Examples of such blink patterns are provided in
Instead, the movements of the eyes 30 in the absence of complete closure of the eyelids generate respective peaks, isolated from each other over time, in the second electrostatic charge variation signals SQ,2. In particular, such peaks in the second electrostatic charge variation signals SQ,2 may be positive or negative (the latter are also referred to as valleys) with respect to respective baselines of the second electrostatic charge variation signals SQ,2 (e.g., here exemplarily considered to be equal to 0), as a function of the direction of movement of the eyes 30 and of the position of the third and the fourth electrodes 22c and 22d. An example of such peaks in the second electrostatic charge variation signals SQ,2 is provided in
In use, the main control unit 21 implements a detection method 50 of activity (i.e. of movement or state) of the eyes 30 of the user.
An embodiment of the detection method 50 is shown in
The detection method 50 is performed iteratively, so as to update the information on the activity of the eyes 30 of the user in real time. For the sake of simplicity, an iteration of the detection method 50, also referred to as the current iteration, is described below.
At a step S10 of the detection method 50, the first electrostatic charge variation signals SQ,1 are acquired through the first electrostatic charge variation sensors 20a and 20b. Hereinafter, the first electrostatic charge variation signal SQ,1 acquired through the first left electrostatic charge variation sensor 20a is also indicated with the reference SQ,1a, while the first electrostatic charge variation signal SQ,1 acquired through the first right electrostatic charge variation sensor 20b is also indicated with the reference SQ,1b. In particular, the first electrostatic charge variation signals SQ,1a, SQ,1b are acquired through a scrolling buffer. In detail, at each iteration the first electrostatic charge variation signals SQ,1a, SQ,1b are acquired in a respective time window for example having a predefined duration equal to a time window period (e.g. equal to a few thousand ms, e.g. comprised between about 900 ms and about 2500 ms and e.g. equal to 1800 ms) greater than the blink period; in other words, at each iteration the first electrostatic charge variation signals SQ,1a, SQ,1b with a time duration equal to the time window period are considered. At each iteration, the oldest sample of each first electrostatic charge variation signal SQ,1a, SQ,1b is deleted and a new sample of each first electrostatic charge variation signal SQ,1a, SQ,1b is stored in the buffer.
In greater detail, at the current iteration the detection signals SR of the first and second electrodes 22a and 22b of the first electrostatic charge variation sensors 20a and 20b are generated and the respective first electrostatic charge variation signals SQ,1a, SQ,1b are calculated as a function of the detection signals SR (in particular, each first electrostatic charge variation signal SQ,1a, SQ,1b is equal to, or proportional to, the difference between the detection signals SR of the first and the second electrodes 22a and 22b of the respective first electrostatic charge variation sensor 20a, 20b).
At a step S12, optional and immediately consecutive to step S10, the first electrostatic charge variation signals SQ,1a and SQ,1b are filtered to remove the contribution of the alternating electric current possibly present in the environment surrounding the glasses 10. In fact, if any, the alternating electric current generates respective electrostatic charge variations in the environment, which may be detected by the first electrostatic charge variation sensors 20a and 20b generating a respective peak, in the frequency domain, in the first electrostatic charge variation signals SQ,1a, SQ,1b at the frequency of the alternating electric current (i.e. 50 Hz or 60 Hz depending on the country one is in). In particular, the performed filtering may be of low-pass type with a cut-off frequency lower than a first threshold frequency (e.g., equal to about 25 Hz), or of band-pass type with a lower cut-off frequency greater than a second threshold frequency (lower than the first threshold frequency and for example equal to about 1 Hz) and with a higher cut-off frequency lower than the second threshold frequency and for example equal to 20 Hz, or of notch type with a lower cut-off frequency lower than the second threshold frequency and for example equal to 20 Hz, and with a higher cut-off frequency greater than a third threshold frequency (greater than the first threshold frequency and for example equal to about 80 Hz).
At a step S14, optional and immediately consecutive to step S12, the first electrostatic charge variation signals SQ,1a and SQ,1b are processed to remove an offset of the latter, and in detail to subtract a respective baseline of the latter (i.e. a reference value, not necessarily constant over time, around which the respective first electrostatic charge variation signal SQ,1a and SQ,1b develops; e.g., an average value) from each of them. In this manner, subsequently the variations of the first electrostatic charge variation signals SQ,1a and SQ,1b may be taken into account with respect to a zero baseline, and possible offset thereof are not to be considered.
At a step S16, immediately consecutive to step S14, the first electrostatic charge variation signals SQ,1a and SQ,1b are processed in a per se known manner to identify any peaks thereof (i.e. positive peaks or negative peaks, the latter also referred to as valleys). In detail, at step S16, if any, a number of peaks of the first electrostatic charge variation signals SQ,1a and SQ,1b and, for each peak, a respective maximum value and a respective time position (or instant) of this maximum value are identified. Greater details regarding the identification modes of these peaks may be found in the Italian patent document identified by the reference number 102021000012665, of the present Applicant. Alternatively, the portions of the first electrostatic charge variation signals SQ,1a and SQ,1b that have a greater value than a threshold value (e.g., a predefined value for example equal to about 200 LSB or a value equal to about 10% of the maximum value of the respective first electrostatic charge variation signal SQ,1a, SQ,1b in the considered time window) are considered.
At a step S18, immediately consecutive to step S16, it is determined whether a condition on the blink patterns of the first electrostatic charge variation signals SQ,1a and SQ,1b is verified. In particular, it is determined whether at least one blink pattern has been detected in at least one of the first electrostatic charge variation signals SQ,1a and SQ,1b. In other words, it is determined whether a sum of the number of blink patterns of the first electrostatic charge variation signals SQ,1a and SQ,1b is non-zero.
If the condition on the blink patterns is not verified (i.e. said sum is equal to zero), the current iteration of the detection method 50 ends and the method returns to step S10 to process the first electrostatic charge variation signals SQ,1a and SQ,1a in a subsequent time window.
If the condition on the blink patterns is verified (i.e. said sum is greater than zero), the detection method 50 proceeds to a step S20.
At step S20, immediately consecutive to step S18, it is verified whether the detected blink patterns of the first electrostatic charge variation signals SQ,1a and SQ,1b at the current iteration are indicative of a first condition of the eyes 30 of the user (i.e. whether a voluntary blink, hereinafter also referred to as eye click or more simply click, has occurred). In greater detail, for each pair of consecutive blink patterns of the first electrostatic charge variation signals SQ,1a and SQ,1b, the first condition is verified if these two blink patterns have a relative time distance greater than a first threshold period that is smaller than the time window period and greater than the blink period (for example it is equal to a few hundred ms, e.g. comprised between about 100 ms and about 800 ms and for example equal to 500 ms). For example, the relative time distance is calculated between reference points of the two blink patterns, similarly to what has been described hereinbelow with reference to the click time distances. Therefore, when the first condition is verified, a click has been detected at the current iteration.
A step S22, immediately consecutive to step S20, is a decision block that verifies whether the first condition has been detected or not at step S20.
In the present embodiment, if the first condition has not been verified (i.e. no click has been detected) the detection method 50 proceeds to a step S24.
If the first condition has been verified (i.e. a click has been detected), the detection method 50 proceeds to a step S32.
Step S24 refers to a second condition of the eyes 30 of the user (i.e. to an involuntary blink, hereinafter also referred to as eye blink or more simply blink). In general, a blink has a shorter duration than a click as it occurs in an involuntary manner, and involves both eyes simultaneously (unlike the click that is performed with only one eye). In the present embodiment, the first and second conditions are alternative to each other, and therefore the second condition is detected when the first condition is not detected.
At step S24, an IEBI (“Inter-Eye blink interval”) parameter, of a known type, is therefore calculated as a function of the first electrostatic charge variation signals SQ,1a and SQ,2a, and in particular as a function of the peaks of the first electrostatic charge variation signals SQ,1a and SQ,1b. The IEBI parameter is indicative of a blink frequency of the eyes 30 of the user, and in particular is indicative of an average, calculated over a predefined period (e.g., 1 minute), of the distances between blinks that are consecutive to each other over time.
In particular,
In greater detail, the IEBI parameter is indicative of the average, calculated in the predefined period, of the relative distances (not shown and hereinafter also referred to as blink time distances) between two consecutive blinks, and in detail between respective reference points of the two consecutive blinks. Each reference point may for example be a value such as the time position tMK1, tMK2, tMK3 or tMK4 of the maximum value MK1, MK2, MK3, MK4 of the first (K1 and K2) or of the second (K3 and K4) peak of the first (P1) or of the second (P2) blink pattern of the blink. For example, the blink time distances used for the calculation of the IEBI parameter may be the distances between the time positions tMK1 of the maximum values MK1 of the first peaks K1 of the first blink patterns P1 of two consecutive blinks.
In particular, the IEBI parameter is updated at each iteration in which the second condition is verified. For example, at a first iteration (e.g., i=1) of the detection method 50, the IEBI parameter is set to a predefined value (e.g., equal to 5 s), and at each subsequent iteration in which the occurrence is detected of a blink (e.g., i=N) it is recalculated as a function of both the own value at the immediately preceding iteration (e.g., i=N−1) and the blink time distance calculated at the current iteration (e.g., i=N).
At a step S26, immediately consecutive to step S24, it is determined whether a condition on the IEBI parameter is verified or not, in order to determine an attention (or concentration) state of the user. In particular, it is verified whether the IEBI parameter is greater than an IEBI threshold value (e.g., comprised between about 3 s and about 12 s and for example equal to about 8 s) indicative of a threshold attention level of the user.
If the condition on the IEBI parameter is verified (i.e. the IEBI parameter is greater than the IEBI threshold value), a first activity of the eyes 30 of the user is detected (step S28). The first activity of the eyes 30 is indicative of a first attention state of the user which corresponds to a low (or lower) attention level of the user.
If the condition on the IEBI parameter is not verified (i.e. the IEBI parameter is lower than, or equal to, the IEBI threshold value), a second activity of the eyes 30 of the user is detected (step S30). The second activity of the eyes 30 is indicative of a second attention state of the user which corresponds to a high (or higher) attention level of the user.
As evident, the terms “high” and “low” mentioned in the present description with reference to the attention level are not to be understood in an absolute sense, but rather relatively to each other and in connection with the activity being performed (where the distinction is given by the IEBI threshold value).
At step S32 a relative distance (hereinafter also referred to as the click time distance) between two detected and consecutive clicks of the first electrostatic charge variation signals SQ,1a and SQ,1b is calculated as a function of the first electrostatic charge variation signals SQ,1a and SQ,1b and in particular as a function of the peaks of the first electrostatic charge variation signals SQ,1a and SQ,1b.
In particular,
In greater detail, the click time distance is defined between respective reference points of the last two detected clicks (P3 and P4 in
At a step S34, immediately consecutive to step S32, it is determined whether a condition on the click time distance is verified or not, in order to determine a click mode of the eyes 30 of the user. In particular, it is verified whether the click time distance is less than a click threshold distance (greater than the first threshold period, e.g. comprised between about 200 ms and about 1 s and for example equal to about 700 ms) indicative of a threshold level which identifies a double click (better described hereinbelow).
If the condition on the click time distance is verified (i.e. the click time distance is less than the click threshold distance), a third activity of the eyes 30 of the user is determined (step S36). The third activity is indicative of said double click which corresponds to two voluntary clicks close to each other over time, as shown in
If the condition on the click time distance is not verified (i.e. the click time distance is greater than, or equal to, the click threshold distance), a fourth activity of the eyes 30 of the user is determined (step S38). The fourth activity is indicative of a single click, i.e. of a voluntary click isolated over time, as shown in
According to an embodiment of the detection method 50, shown in
Furthermore, in the present embodiment steps S10-S16 previously described are performed not only for the first electrostatic charge variation signals SQ,1a and SQ,1b but also for the second electrostatic charge variation signals SQ,2a and SQ,2b acquired through the third and fourth electrodes 22c and 22d of the second electrostatic charge variation sensors 20c and 20d. In particular, step S16 is performed for the second electrostatic charge variation signals SQ,2a and SQ,2b in order to detect possible peaks (positive or negative) thereof, similarly to what has been previously described for the detection of blink patterns in the first electrostatic charge variation signals SQ,1a and SQ,1b. In particular,
At a step S40a, consecutive to step S22 and interposed between step S22 and step S24, it is verified whether the second electrostatic charge variation signals SQ,2a and SQ,2b acquired at the current iteration through the third and fourth electrodes 22c and 22d are indicative of a third condition of the eyes 30 of the user. In greater detail, the third condition is indicative of a movement of the eyeballs of the eyes 30 in the orbital cavity in the absence of complete closure of the eyelids; in other words, the third condition is correlated to a rotation of the eyeballs which does not require the closure of the eyelids (i.e. the mutual contact of the upper eyelid and of the lower eyelid of each eye 30). The third condition is verified if the presence of at least one single peak is identified in each second electrostatic charge variation signal SQ,2a, SQ,2b. This single peak is a peak isolated over time with respect to the other peaks of the second electrostatic charge variation signal SQ,2a, SQ,2b and which is not part of any blink pattern. In particular, the single peak of the second electrostatic charge variation signal SQ,2a, SQ,2b is a peak that has no counterpart in the first electrostatic charge variation signals SQ,1a, SQ,1b (i.e. the first electrostatic charge variation signals SQ,1a, SQ,1b have no peak at the time position of the single peak of the second electrostatic charge variation signal SQ,2a, SQ,2b).
If the third condition has not been verified (i.e. no movement of the eye 30 has been detected in the absence of complete closure of the eyelids), the detection method 50 proceeds to step S24 previously described as it is detected that the second condition is verified (i.e. a blink has been detected).
If the third condition has been verified (i.e. a movement of the eyes 30 has been detected in the absence of complete closure of the eyelids), the detection method 50 proceeds to a step S40b. Optionally, the method may also proceed simultaneously with both step S24 and step S40b.
At step S40b, consecutive to step S40a, the orientation is detected with respect to the baseline of the respective second electrostatic charge variation signal SQ,2a, SQ,2b, of the one or more single peaks detected in the second electrostatic charge variation signals SQ,2a, SQ,2b which identify respective movements of the eyeballs of the eyes 30 in the absence of complete closure of the eyelids. In detail, exemplarily considering only one second electrostatic charge variation signal SQ,2a, SQ,2b and only one single peak of the latter, at step S40b it is verified whether the single peak is positive (i.e. whether the respective maximum value is greater than the baseline, here considered zero in view of step S14) or negative (i.e. whether the respective maximum value is lower than the baseline, here considered zero in view of step S14).
A step S40c, consecutive to step S40b, is a decision step wherein it is determined whether, for each detected single peak of the second electrostatic charge variation signals SQ,2a, SQ,2b, a fourth condition which exemplarily corresponds to a predefined orientation of said single peak (e.g., positive orientation) is verified.
If the fourth condition has been verified (i.e. the single peak is positive), the detection method 50 proceeds to a step S40d.
If the fourth condition has not been verified (i.e. the single peak is negative), the detection method 50 proceeds to a step S40e.
If the fourth condition is verified, a fifth activity of the eyes 30 of the user is determined (step S40d). The fifth activity is indicative of a first movement of the eyes 30. For example, the first movement corresponds to a movement of the cornea 30a from right to left relative to the user's point of view, and/or to a movement of the cornea 30a from the third electrode 22c to the fourth electrode 22d of the second left electrostatic charge variation sensor 20c; nevertheless, the orientation of the first movement obviously depends on the position of the third and fourth electrodes 22c and 22d with respect to the eye 30, and therefore may also correspond, for example, to a movement of the cornea 30a from left to right.
If the fourth condition is not verified, a sixth activity of the eyes 30 of the user is determined (step S40e). The sixth activity is indicative of a second movement of the eyes 30. In particular, the second movement is opposite to the first movement; in other words it is performed in the opposite direction to the first movement. For example, the second movement corresponds to a movement of the cornea 30a from left to right, and/or to a movement of the cornea 30a from the fourth electrode 22d to the third electrode 22c; nevertheless, the orientation of the second movement obviously depends on the position of the third and fourth electrodes 22c and 22d with respect to the eye 30, and therefore may also correspond, for example, to a movement of the cornea 30a from right to left.
In the present embodiment, the detection method 50 comprises further steps S03-S07 preceding step S10 and the glasses 10 further comprise one or more accelerometers (not shown) and one or more gyroscopes (not shown). Hereinafter, exemplary reference is made to the case of an accelerometer and a gyroscope, although a greater number of accelerometers and/or gyroscopes may similarly be considered. In particular, the accelerometer and the gyroscope are fixed to the glasses 10 (for example to the support portions 12a and/or 12b) and are configured to detect possible movements of the user's head (e.g., lateral rotations and forward or backward bending of the head, with respect to the torso).
At a step S03, the main control unit 21 acquires, through the gyroscope, one or more angular velocity signals Sω indicative of respective angular velocities measured by the gyroscope. These one or more angular velocities are generated by movements of the user's head, for example with respect to the user's torso.
At a step S05, for example performed simultaneously with step S03, the main control unit 21 acquires, through the accelerometer, one or more linear acceleration signals Sacc indicative of respective linear accelerations measured by the accelerometer. These one or more linear accelerations are generated by movements of the user's head, for example with respect to the torso.
At a step S07, it is determined whether a fifth condition is verified, as a function of the angular velocity signals Sω and of the linear acceleration signals Sacc. The fifth condition is determined when no movements of the user's head are detected. The determination of the movements of the head from the angular velocity signals Sω and the linear acceleration signals Sacc is performed in a per se known manner (e.g., through machine learning techniques) and therefore not described in detail herein; nevertheless, details on this aspect may be found for example in the article “Absolute Orientation for Head-Tracking Using Gyroscope, Accelerometer, and Camera”, Fisher, EE 267—Virtual Reality—Stanford University—2018.
If the fifth condition is not verified (i.e. movements of the head are detected) the detection method 50 ends and steps S10-S38 are not performed (e.g., the method returns to step S03).
If the fifth condition is verified (i.e. no movements of the head, which is immobile, are detected), the detection method 50 proceeds to step S10 and then the steps shown in
According to an embodiment not shown, the detection method 50 of
The glasses 10 and the detection method 50 previously described allow, owing to the proximity of the electrodes 22a-22d to the eyes 30, the movements of the eyes 30 to be detected with high accuracy. In particular, eye activities may be detected indicative of the attention level of the user, of single and double clicks and of movements of the eye 30 in the absence of blink.
The possibility, owing to the accelerometer and to the gyroscope, to perform the detection method 50 only in the absence of movements of the user's head avoids incorrect detections of movement of the eyes, actually due to movements of the head. Furthermore, the fact that the electrodes 22a-22d are carried by the glasses 10 and are not in contact with the skin of the user's face prevents detection errors due to the slipping of the same on the skin (e.g., when the latter is humid or sweaty).
The detection method 50 requires reduced computational resources to be implemented, and therefore minimizes the electrical consumption required.
Finally, it is clear that modifications and variations may be made to the invention described and illustrated herein without thereby departing from the scope of the present invention, as defined in the attached claims. For example, the embodiments described may be combined with each other to obtain further solutions.
Steps S24-S38 (as well as steps S40a-S40e if any) may be optional. In general, if the first condition is not determined at step S22, a blink is detected, and if the first condition is determined at step S22, a click is detected. For example, if the first condition is not determined at step S22, a blink signal (not shown) is generated which assumes a first value (e.g., 0) indicative of the detection of a blink; if, on the other hand, the first condition is determined at step S22, the blink signal is generated with a second value (e.g., 1) indicative of the detection of a click. For example, the blink signal may be used to control the one or more functionalities of the glasses 10 or of the apparatus, similarly to what has been previously described. Following the generation of the blink signal, steps S24-S38 (as well as steps S40a-S40e if any) may optionally be performed as previously described.
The glasses 10 may also comprise a single control unit coupled to the electrodes 22a-22d. For purely illustrative purposes, this control unit may comprise the main control unit 21 and the sensor control units 15 or it may be the main control unit 21 and the sensor control units 15 may be absent; in these cases, the actions previously described with reference to the sensor control units 15 of the charge variation sensors 20a, 20b are performed by this control unit.
Although the case in which the detection method 50 is performed by processing time windows superimposed on each other (scrolling buffer) has been previously described, the previous description applies in a similar manner also to the case of time windows that are consecutive and not superimposed on each other.
Furthermore, the number of electrostatic charge variation sensors may be greater than what has been previously considered. In particular, the glasses may comprise one or more third electrostatic charge variation sensors (not shown and similar to the second electrostatic charge variation sensors 20c and 20d), each of these comprising a respective pair of electrodes (not shown, for example a fifth and a sixth electrode) for an improved detection of the fifth and sixth activities. For example, the fifth and sixth electrodes are spaced from the electrodes 22a-22d, are radially internal with respect to the first and second electrodes 22a, 22b with respect to the center of the respective lens 14a, 14b, are distant from each other by the second mutual distance D2 and are fixed to the respective lens 14a, 14b so as to face the eye 30 of the user when the latter wears the glasses 10. In particular, the fifth and sixth electrodes are angularly equi-spaced with respect to the third and fourth electrodes 22c and 22d with respect to the center of the respective lens 14a, 14b. For example, the fifth and sixth electrodes are aligned with each other along a second axis (not shown) orthogonal to the first axis, such that they are arranged as a cross with respect to the third and fourth electrodes 22c, 22d where the center of this cross corresponds to the center of the respective lens 14a, 14b (and therefore to a center of the eye 10, e.g. at the position of the pupil when the user's gaze is oriented along a direction orthogonal to the face). In this manner the movements of the eyes 30 in the absence of blink may be detected in a more efficient and accurate manner. Similarly to what has already been described for the third and fourth electrodes 20c and god, for each third electrostatic charge variation sensor the respective sensor control unit 15 is configured to process respective detection signals SR acquired through the fifth and sixth electrodes and to generate a respective further second electrostatic charge variation signal indicative of a difference between the detection signals SR acquired through the fifth and sixth electrodes, and therefore indicative of a difference between the electrostatic charge variations detected through the fifth and sixth electrodes.
Furthermore, the glasses 10 may comprise only one first electrostatic charge variation sensor (hereinafter exemplarily considered to be the first left electrostatic charge variation sensor 20a). In this case, the detected information refers only to one of the eyes 30 of the user (i.e. to the eye 30 having the first left electrostatic charge variation sensor 20a facing thereto). Nevertheless, the information obtained regarding the involuntary activity of one eye (e.g., blink and movements of the eyes in the absence of complete closure of the eyelids) may similarly be considered for the other eye as well, as the involuntary movements of the eyes (both blinks and movements of the eyes in the absence of blink) are generally substantially synchronous and dual. This consideration does not apply instead to voluntary movements of the eyes (e.g., clicks) that are deliberately performed by the user with only one eye.
One embodiment of the detection method 50 corresponding to the case in which only the first left electrostatic charge variation sensor 20a is present is shown in
However, unlike the previously discussed embodiments, in
Furthermore, the determination of the first condition (previously step S20, replaced in
Furthermore, in this embodiment at step S24 the IEBI parameter is determined as the average, over the predefined period, of the blink time distances, each blink time distance being calculated between two consecutive blink patterns of the first electrostatic charge variation signal SQ,1a, each blink pattern defining a respective blink.
While this invention has been described with reference to illustrative embodiments, this description is not intended to be construed in a limiting sense. Various modifications and combinations of the illustrative embodiments, as well as other embodiments of the invention, will be apparent to persons skilled in the art upon reference to the description. It is therefore intended that the appended claims encompass any such modifications or embodiments.
Number | Date | Country | Kind |
---|---|---|---|
102021000027866 | Oct 2021 | IT | national |