The present invention relates to a myoelectric-controlled prosthetic device and a method for calibration and usage of said device.
It is to specified that in the present document the term prosthetic device refers to a device comprising a prosthesis, an orthosis, a partial or integral exoskeleton and relative control means.
In particular, the present invention relates to a myoelectric-controlled prosthetic device to be used with a particularly rapid and intuitive training of the patient, and not subject to physiological variations of the signal levels detected by the myoelectric sensors over the time.
People born with a limb, and who, during their life, lose it in an accident or due to an illness, often keep the capacity to contract the muscles which articulated the limb previously. On the contrary, people born without a limb due to a genetic malformation or other neonatal illnesses or accidents have no feeling of “ghost limb” but can imagine anyway the movements the failing limb could carry out after determined muscle contractions thanks to the experience of the contralateral limb. There exists also a group of people who, owing to neuromuscular disorders, absence of stimulations from the environment or other physiological/environmental reasons, have not matured the expectation of a proprioceptive feedback of the muscular contraction effects on the amputated or missing limb.
According to what known at the state of the art, a myoelectric prosthetic device comprises:
According to embodiments known at the state of the art, the prostheses are directed to all the just described categories of amputees, with three possible approaches described in the following: in case of residual hypotrophic muscles, absence of space for positioning more than one electrode, antagonist muscles innervation loss, only one electrode is applied at one of the residual muscles used to activate the amputated or missing limb, and the prosthesis is actuated by carrying out two movements thanks to the setting of two thresholds on the amplitude of the sEMG signal measured for the unique electrode provided. For example, in case of prosthetic hand, this can open when the first threshold is reached, and close when the second threshold is reached, as it is described for example in Merletti at al Electromyography: Physiology, Engineering, and Non-Invasive Applications, 2004, IEEE Press Series on Biomedical Engineering.
In other cases, two electrodes are applied, one being positioned at agonist muscles and one at antagonist muscles. In this case, an actuation threshold is set for each electrode to control the movement of agonist/antagonist muscles. For example, in case of prosthetic hand the activation of the finger flexor muscles activates the hand closing and the activation of finger extensor muscles activates its opening). Anyway, in this configuration it is needed to set a biunivocal correspondence between electrode and controlled movement. In the prosthetic hand example, the electrode positioned on the flexor muscles controls the closing movement while the electrode positioned on the extensor muscles controls the opening movement.
At the state of the art, for example in U.S. Ser. No. 10/448,857B2 and U.S. Pat. No. 9,566,01662, prostheses are known that comprise a crown of three or more electrodes. In these systems, the electrodes have to be arranged with a precise order since the amplitude of the drawn sEMG signals can be used to pilot multi-parametric systems, such for example systems based on Kiviat diagrams whose axes represent the amplitude of each channel. In these cases, the subject is normally subjected to a difficult training step, during which he learns to carry out constantly the contractions needed for the generation of correct polygons of the Kiviat diagram which enable the prosthesis correct movements.
In the calibration of a prosthesis, it is also needed to calibrate the torque and/or the prosthesis activation speed. The most intuitive case is the one of a robotic hand, in which the activation torque determines the strength with which the hand grabs an object.
According to the calibration methods known at the state of the art, an EMG relation vs strength or muscle torque is imposed, possibly derived from the contemporaneous recording of the torque exerted by the hand joints of a healthy subject, and the electromyographic signals detected by the electrodes installed on the limb of such subject. Such relation between measured strength or torque and electromyographic signals is used to build a calibration curve used then for the prostheses worn by amputees. These relation curves EMG-strength or EMG-torque, when pre-determined by literature studies and applied directly on amputees, are little efficient since this kind of solution does not consider the subject muscle morphology and his anthropometric characteristics and, so, it is likely to not allow the subject to use the prosthesis instinctively and immediately.
Document US2016278947 describes a prosthetic system comprising computing means configured to acquire signals from an angular velocity sensor and to control the prosthesis as a function of these acquired signals.
All the devices known at the state of the art and in particular the methods that such devices use to associate the movements of the prosthetic device with the signals detected by the input sources are limited, since:
Generally, it is needed to guarantee stability to the movement recognition algorithm. Stability means the capacity of an algorithm to recognize the instruction expressed by the subject with low error, without often ending in error situations which could lead to the impossibility to manage/pilot the prosthesis. This is a condition potentially developable by all the systems integrating machine learning algorithms (such for example fuzzy decisional algorithms or neuronal nets, in particular not linear ones). These results are not reached by the calibration methods known at the state of the art.
So, the present invention provides a prosthetic device which overcomes the limits linked to the embodiments known at the state of the art, and in particular which eliminates the need of configuration for the user or technician, which does not require the association of each electrode with a specific movement, which can be used indifferently with any number of electrodes since the number of connected electrodes does not increase the configuration or calibration complexity.
Yet, the present invention provides a prosthetic device able to adapt autonomously and automatically its own functioning to the final user physiologic characteristics, and which is able to adapt its own configuration parameters automatically over the time, so that a correct functioning is kept even while the user physiological conditions vary or while the environmental conditions in which the device is used vary, without any intervention of the technician or final user.
Yet, the invention provides a prosthetic device able to pilot the torque/speed of one or more movements proportionally to the muscle contraction and, in particular, which is able to carry out rapidly and intuitively, also for subjects with only residual muscles, or who have never had in life the feeling to move the limb substituted by the prosthesis, the calibration of the torque and/or activation speed.
The prosthetic device according to the invention comprises:
The device comprises also preferably:
The prosthetic device according to the invention is characterized in that, on said electronic computing means computer programs are loaded configured to carry out the method described in the following. The method according to the invention comprises the steps of:
In order to carry out the supervised calibration procedure (1), the method comprises the next steps.
(100) recording the signals (Ns) relative to each electrode provided in the system with the user in a predetermined condition (rest condition or condition of execution of a predetermined movement) for a predetermined time interval ΔT, thus obtaining N tracks or—pieces— of electric signal in the time domain, one relative to each sensor. It is to be specified that rest condition means a condition in which the subject is asked to remain still, with relaxed muscles, in an inactivity state, thus avoiding contracting muscles, avoiding exerting pressure on the portions of the prosthetic device and avoiding activating possible sensors provided in the system. Said predetermined time interval, according to the kind of connected electrodes and the kind of prosthetic actuator, can be varied, for example, between 1 and 10 seconds. (110) subdividing each of said signals (Ns) in a plurality of time intervals (also called epochs or windows) of predetermined duration, thus obtaining “M” signal pieces for each signal. The duration of each window, according to the kind of signals and the kind of prosthetic actuator, is preferably between 200 ms and 2 seconds. It is also to be specified that the time windows can be partially overlapping or not. The overlapping allows to have a greater number of windows at the same signal duration, thus maximizing the extraction of information contained therein, also in presence of noise;
The above-described procedure can comprise one or more of the following variations:
Possibly after point (130), the point can follow of
For example, a transhumeral amputee has not the forearm muscles that he would normally use to activate the finger flexing, but he could use the flexor muscles of the forearm (one or two heads of the biceps brachii). Similarly, a trans-radial amputee, even if he has still the muscles used for the natural flexion of the fingers, in order to optimize the piloting of the movement relative to the finger flexion could prefer to carry out the muscular contraction associated with the wrist flexion, since the muscular group contracting in this case is mainly concentrated on the inner side of the forearm, thus exposing the detecting electrodes to a lower cross-talk phenomenon. Similarly, in case of opposite movement (in the example the finger extension), a trans-humeral amputee has not the forearm muscles that would normally use to activate the finger extension, but he can use the extensor muscles of the forearm (one or more heads of the triceps brachii). Similarly, a trans-radial amputee, even if he has still the muscles used for the natural finger extension, in order to optimize the piloting of the movement relative to the finger flexion can prefer to carry out the muscular contraction associated with the wrist extension, since the muscular group contracting in this case is mainly concentrated on the outer side of the forearm, thus exposing to a lower cross-talk phenomenon.
The steps (100) to (131) are carried out sequentially for a rest condition (rest) and for a plurality of predetermined movements (Mov_1, Mov_2, . . . , Mov_n), thus obtaining a plurality of vectors of dimensions K×1 defining each, in the K-dimensional space relative to the K acquired features, the position of a point representing each detected movement.
The method provides preferably the execution of the above steps sequentially for a movement and in the following for the movement opposite thereto. For example, in case of trans-radial prosthesis, if the first calibrated movement (Mov_1) is the finger flexion, the second movement (Mov_2) is normally the finger extension.
Moreover, preferably, at the end of the calibration procedure for all the movements intended, the calibration of the rest condition (rest) is carried out again in order to compensate possible alterations of the sEMG signal features upon the introduction of muscular fatigue phenomena. In order to do so, the steps (100) to (131) for the rest condition are carried out, thus obtaining the definition of another point representing the rest condition in K-dimensional space. The point representing the rest condition is then calculated as the average point between the first and the second point calculated for said condition. The rest condition can be also object of detection before and after the detection relative to each movement, by calculating for each of the K features relative to the rest condition the average value of all the values calculated for each detection, in order to define the point representing the rest condition in the K-dimensional space.
According to a preferred embodiment, the prosthetic device according to the invention comprises at least a position sensor, configured to detect the position of said device, and is configured to store for each movement, as well as for the rest condition, a plurality of reference points and to select automatically the suitable reference point to be considered during usage as a function of the position of said prosthetic device detected by said position sensor. Said position sensor is preferably an inertial unit (IMU, inertial measuring unit).
This characteristic satisfies the need to compensate possible alterations of the signals recorded by the sensors due to the position variation of the prosthetic device. For example, in case of a trans-radial prosthesis piloted by one of more EMG sensors, the variation of the arm position can cause the variation of the relative position electrode-muscle even in absence of variation of the relative position electrode-skin. In this case, by using an IMU positioned in the storage (integrated or not in the electrodes, or in the storage itself or in the battery housing, or in other components of the prosthesis), it is possible to recognize the position of the prosthesis and, on the basis thereof, to select automatically and to use the most suitable calibration parameters.
In order to calibrate the functioning of the device in a plurality of positions, the supervised calibration procedure (1) comprises the steps of:
After the execution of the supervised calibration procedure, in order to increase the calibration stability (i.e. in order to reduce the error frequency), the method can provide the execution of the following steps:
It is to be specified that in a first embodiment the distance of point (140) can be calculated as Euclidean distance in a K-dimensional space; in a second embodiment at each dimension of K-dimensional space a specific “weight” can be assigned, and the distance can be calculated as the Euclidean distance in the K-dimensional space in which each addend of the summation of the squares of the distances along each axis is multiplied by its own weight.
(150) comparing said distances calculated at point (140) with a minimum threshold of predetermined distance;
In this way, it is obtained the definition of a series of points representing a plurality of movements distant to each other more than the minimum threshold. This reduces the possibility of error in the next step of device usage, as it will be clearer in the following.
Once the supervised calibration step according to the just described procedure is ended, the method provides the possibility to carry out a test (2) of the recorded parameters, according to the following steps:
(220) individuating, among all the points representing the movements (or the rest condition) stored during the supervised calibration procedure, the one with the lower distance from the point defined at step (210).
(230) providing the user with a feedback indicating the movement individuated at point (220).
It is to be specified that in a first embodiment, the distance of point (220) can be calculated as Euclidean distance in a K-dimensional space; in a second embodiment at each dimension of the K-dimensional space a specific “weight” can be assigned, and the distance can be calculated as the Euclidean distance in K-dimensional space in which each addend of the summation of the squares of the distances along each axis is multiplied by its own weight.
So, the user can consider the test procedure as successful in case the feedback of point (230) indicates the movement he was really carrying out.
In case the recognition of movements is not satisfying, the user can repeat the supervised calibration procedure (1), by repeating it as a whole or only for some movements (or for the “rest” condition). In case, instead, the configuration obtained with the supervised calibration is satisfying, it is validated and used for the next steps.
Clearly, in case a plurality of sets of reference points relative to a plurality of significant positions of the device are provided, the test procedure (2) can be conveniently carried out for each of said significant positions.
It is to be specified that the feedback provided at step (230)— as also the warning of step (160)— can be:
Moreover, preferably, at step (230), the device is configured to provide the user with an indication relative to the distance between the point calculated at step (210) and the point individuated at step (220). The indication can be provided by means of various feedback tools:
After the calibration test procedure (2), in positive case, the method provides to carry out the procedure of usage of the device (3) and the not supervised calibration (4).
The procedure of usage of the device (3) provides the steps of: (300) acquiring, at predetermined time intervals, the signals (Ns) coming from said electrodes for a predetermined duration and storing them in a relative buffer.
Said predetermined time intervals are preferably between 200 ms and 1000 ms; said predetermined duration is preferably between 200 ms and 2000 ms.
(310) subdividing each of said signals (Ns) in a plurality of time intervals (also called epochs or windows) of predetermined duration, thus obtaining “M” signal pieces for each signal. The duration of each window, according to the kind of signals and the kind of prosthetic actuator, is preferably between 200 ms and 2 seconds. It is also to be specified that the time windows can be partially overlapping or not. The overlapping allows to have a greater number of windows at the same signal duration, thus maximizing the extraction of information contained therein, also in presence of noise.
(320) for each one of the M windows of each recorded signal, extracting at least a characteristic parameter (feature) relative to the signal in the time domain, or relative to the signal FFT transform. The features extracted during the using step are the same features extracted in the calibration step at step (120),
The calculation modes adopted, including the discarding of the values distant from the statistical descriptor, are the same used in the supervised calibration step.
(340) calculating the Euclidean distance between the point calculated at step (330) and each point representing each movement stored during the supervised calibration procedure, thus identifying the movement for which such distance is minimum.
(350) carrying out the movement identified at step (340).
It is clear that, generally, the word “movement” can mean also the “rest” condition.
Moreover, in case a plurality of sets of reference points relative to a plurality of significant positions (P1, . . . , Pr) are provided, the procedure of usage of the prosthetic device (3) comprises, before the step (300), the steps of:
The above-described procedure can comprise one or more of the following variants:
Moreover, in order to reduce the errors of rest condition recognition, defined as DMOV the distance of the point individuated at step (330) from the point representing the movement identified at step (340) and as DREST the Euclidean distance of the point individuated at step (330) from the point representing the rest condition, at step (350) the following condition can be added:
Wherein α is a suitable coefficient equal or lower than 1 and greater than 0.
In particular, the more a is low, the lower the system is sensible to the actuation of movements; for too low values of a the system could not recognize any movement. Alfa is preferably between 0.75 and 0.95. It is now described the procedure “not supervised calibration” (4). As yet said, this procedure allows to consider automatically and in absolutely transparent manner for the user, all the variations over the time due to factors such as: variations of the impedance of the electrode-skin contact, physiological variations of the muscle tissues due to fatigue, muscle training or vascularization change, alterations of signals recorded by sensors sensible to environmental conditions. More specifically, this not supervised calibration procedure (4) intervenes automatically to modify, if needed, the reference points in the multidimensional space relative both to the rest condition and to each movement.
The procedure provides the following steps:
Said period is preferably between 60 seconds and 30 minutes.
(410) storing the values of the features relative to the first movement following the timer expiring of point (400) associated with the rest condition, thus defining the point Rest_1 in the K-dimensional space, and starting a new timer of predetermined period.
(420) storing the values of the features relative to the first movement following the timer expiring of point (410) associated with the rest condition, thus defining the point Rest_2 in the K-dimensional space.
(430) calculating the Euclidean distances of the two points defined at steps (410) and (420) from the reference point representing the rest condition (Rest_ref).
(440) In case the distance from the reference point (Rest_ref) of the point (Rest_2) defined at step (420) is greater than the distance from the reference point (Rest_ref) of point (Rest_1) defined at step (410), updating the values of the reference point, on the contrary, not updating said values.
(450) in case at step (440) it is needed to update the values, calculating the value of each ith feature of the new reference point relative to the rest condition (Rext_ref′) as
Rest_ref′(i)=Rest_ref(i)+b·[Rest_2(i)]−Rest_ref(i)]
Wherein b is a coefficient of value between 0 and 1, and preferably between 0.05 and 0.2.
(460) substituting the values of the features relative to the first movement associated with the rest condition individuated at point (410) with the values of the second movement associated with the rest condition individuated at point (420), and starting from point (420) again.
The timer period at point (400) can be modified manually by the user or automatically by the system, in order to compensate possible factors which could alter the features measured during the step of “usage of the prosthetic device”, as for example the variation of the electrode-skin impedance measured directly (through a suitable impedance sensor) or indirectly (through the measure of environmental parameters as temperature and humidity).
The modification of the timer period can be actuated in response to:
For example, in case of a transradial prosthesis, it is possible to associate the control of the torque and/o speed with the finger closing so that it is proportional to the muscular contraction exerted by the subject.
So, for the proportional control it is needed to derive a command value, to be used for the proportional control of the actuator, from the signals detected by the sensors (electromyography sensors, MMG sensors (mechanomyogram), AMG sensors (acoustic myogram)).
Also in this case, as for the calibration procedure, the procedure can be repeated separately for a plurality of different positions of usage. It is to be specified that the calibration procedure to manage the proportional control of an actuator is carried out conveniently after the just described supervised calibration procedure, and so, the records of the signals detected in the time domain are present in the memory, and in particular the signals recorded by the sensors at the execution of the movement are present in the memory for which the proportional control is to be calibrated. The method comprises the steps of:
In other terms, the function used is a function of the following kind, in which there are zero, one or more summations next to the first one.
V%=Σi=1Kω1i·Fi+Σi=1Kωni·Fin+Σi=1KFi·eωn
(1100) setting with value equal to one the values of all the coefficients (ωei, ωni) which are not relative to the direct proportionality;
Substantially a plurality of relations as the following one are written, each one for a different value of instruction for the actuator
V%=Ei=1Kω1i·V%·Fi_max
The relations are then used to estimate, according to technique known per se at the state of the art, the values of the coefficients of direct proportionality (ω1i) which reduce error.
It is to be noted that, for the estimation of the coefficient values, no measured torque value has been used, but only the values deriving from EMG sensors.
(1300) making the user repeat the movement object of the calibration, asking him to increase gradually the torque applied, and providing him with a feedback by means of graphic interface of the applied torque, calculated with the coefficient values estimated at point (1200) and the relation defined at point (1000), in which only the terms relative to the direct proportionality are valued.
(1400) calculating the features of the signals detected by the EMG sensors during step (1300) and estimating the coefficient values of the relation defined at point (1000) according to what described at point (1200) but considering also the exponential terms and/or the ones with proportionality higher than the first one.
(1500) repeating the steps (1300) and (1400) up to when the estimated error goes under a predetermined threshold.
It is to be noted that also in this case no measured torque value has been used, but only the values of EMG signals.
The feedback of point (1300) can be matched and/o substituted with various kinds of proportional feedbacks, as for example:
Number | Date | Country | Kind |
---|---|---|---|
102020000023011 | Sep 2020 | IT | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/IB2021/058004 | 9/2/2021 | WO |