The invention relates to a system for determining values of parameters representing a movement of at least two limbs of an entity suitable for being represented in the form of an articulated chain. The purpose of this movement is to cause a movement of said entity.
An articulated chain is composed of articulations and substantially rectilinear rigid segments.
The parameters representing a movement of such an entity may be orientation (or angular position) and position (or cartesian position) parameters.
The movement of the entity may, for example, for an articulated chain representing the lower limbs of a living or artificial being (for example robotised), be a stepping movement, a running movement, a jumping movement, or a movement on a bicycle. For an articulated chain representing the upper limbs, the movement of the entity may for example be a movement by actuation or simulation of action of oars or paddles on a canoe, kayak or rowing boat.
In the case of a movement of an articulated chain, the movement of the articulated chain is characterised by the orientation and/or the position of a point of the segments of the articulated chain over time, or at least at particular instants.
For example, estimation of the position of the whole of an articulated chain may serve for navigation. One example of a navigation device is the GPS system, standing for “Global Positioning System”, which is widely used at the present time. However, GPS poses problems in buildings or in a dense urban environment because of poor communication with the satellites. However, location has a certain interest inside buildings, for example for monitoring aged persons at their home, firefighters during their interventions, or visitors to an exhibition. Location by inertial sensor (using an accelerometer and/or a gyrometer and/or a magnetometer) may then be an advantageous alternative or complement to the GPS, since this is an autonomous system that is always available and is also less expensive than GPS.
Estimation of orientations may also be used, for example for analysing walking or running, or for medical, games or sports-training applications. Estimation of these orientations is all the more advantageous when the orientations are estimated for all the segments of the articulated chain. It is thus possible to represent movement studied overall (for example in order to animate an avatar or to study the angle of the knee or hip).
Movement capture systems are known that prove to be generally expensive or incapable of estimating both the position and the orientation of all the segments in the articulated chain used for locomotion.
The movement capture systems generally used, for example for cinema, comprise optical devices (systems marketed by Vicon or Codamotion), or magnetic devices (systems marketed by Polhemus). These devices are capable of estimating the movement (position and orientation) of the entire body but they are of high cost. In addition they require equipping the environment in which the movement takes place, and have a small coverage area.
Recently, movement capture systems based on electromechanical microsystems or MEMS, standing for “Micro-Electro-Mechanical System”, have been developed. These devices, referred to as attitude units (marketed for example by Movea, Xsens and Microstrain) are generally composed of a triaxial magnetometer, a triaxial accelerometer and a triaxial gyrometer or a subassembly of these three movement sensors. They make it possible to estimate the orientation of the movable rigid body to which they are fixed, but not the position. In the case of an articulated chain, it is necessary to use one sensor per segment of the articulated chain for which it is sought to estimate the orientation. The installation of the system then proves to be difficult for the user. On the other hand, the coverage area is not reduced.
Systems have been proposed for merging the data of several inertial sensors disposed on the body with a biomechanical and/or articulated chain model.
The American patent application US 2008/0285805 proposes a method for capturing the movement of an object having at least two segments each equipped with at least one movement sensor. The measurements from the movement sensors are collected, each movement sensor comprising at least one triaxial accelerometer and one triaxial gyrometer (3A3G). The orientation and position of each segment is calculated by integrating the measurements delivered by the gyrometers and integrating twice the measurements delivered by the accelerometers, taking account in the calculations of stresses on the segments.
Their solution is general and does not take account of the specificity of the type of activity that uses complementary movements of several limbs. For example, in the case of walking, the movements of two limbs are similar and offset in time and the degrees of freedom of the two limbs are comparable. In addition, only embodiments systematically using at least one 3A3G sensor per limb are described, which is an embodiment having a large number of sensors, of high cost.
Patent application EP1984696 proposes a system for estimating the movement of an articulated chain consisting of N consecutive segments comprising means for measuring the acceleration of a segment of the chain and means for measuring the orientation on each of the segments. The presence of one sensor per segment being necessary, the system is of high cost.
Patent application EP 1 990 138 A1 relates to a method for estimating the movement of an articulated chain consisting of several segments. Each segment is equipped with a sensor and uses measurement models and an observer. Thus a large number of sensors is necessary and the cost and size are high. In addition, the algorithmic complexity is relatively high.
In order to study walking or navigation, devices are known using inertial sensors disposed on the lower limbs, in which the sensors are generally disposed on the foot or on the tibia and sometimes on the thigh.
For navigation, the systems can be grouped into two categories. A first category of systems uses relationships between parameters extracted from the measurements such as variance in acceleration or the length of steps, such as for example described in the American patent application U.S. Pat. No. 7,467,060. These relationships are of limited precision and vary from one user to another.
A second category of systems uses a double integration of the natural acceleration. These systems require using accelerometers. In order to obtain the natural acceleration, it is necessary to compensate for the gravity of the accelerometric measurements and therefore to know the orientation at each instant. These systems therefore also require either a sensor for estimating an angle such as a gyrometer and a magnetometer (WO 2010/046364), or an angular evolution model (US 2002/0002863).
These systems do not use information on the articulated chain and the performance/cost ratio is low.
These various systems have limited precision and high cost.
According to one aspect of the invention, a system is proposed for determining values of parameters representing a movement of at least two limbs of an entity for causing a movement of said entity, said entity being suitable for being represented in the form of an articulated chain, in which said at least two limbs connected together by an articulation each include at least one segment, the system being characterised in that it comprises:
Such a system makes it possible, at low cost, to have the attitude of each of the segments of the articulated chain and the movement of any particular point on this chain.
In one embodiment, said measurement means comprise a magnetometer with at least one measurement axis and/or a gyrometer with at least one measurement axis.
Thus the size and cost are limited.
According to one embodiment, said first determination means comprise a magnetometer with at least one measurement axis and/or a gyrometer with at least one measurement axis fixed to each said other limb.
Thus the orientation of each of the segments may be obtained with improved precision. Moreover, the system adapts more rapidly to changes in speed.
In one embodiment, said first determination means are suitable for calculating the value of said second parameter(s) from the value of said first parameter by means of a time change model linking the values of said first parameter to those of the said second parameter(s).
Thus the number of sensors is minimised (it is not necessary to instrument all the limbs of the articulated chain).
According to one embodiment, said change model comprises an offset in time between the value of said first parameter and the value of said second parameter(s).
Thus the change model is simplified and takes into account a characteristic commonly encountered in the type of movement process.
In one embodiment, said change model is adapted to link the orientation of all the segments of said articulated chain to that of said first segment.
Thus the instrumentation of a single segment of the articulated chain makes it possible to go back to the orientation of all the segments of the articulated chain.
According to one embodiment, said second determination means comprise an accelerometer with at least two measurement axes.
Thus the precision can be improved.
In one embodiment, said articulated chain represents two upper limbs of a living or artificial being.
According to one embodiment, said articulated chain represents two lower limbs of a living or artificial being.
In one embodiment, said articulated chain comprises two segments per lower limb and six degrees of freedom.
Thus the placing of the lower limbs throughout a movement of the living or artificial being can be estimated with a minimised number of degrees of freedom and number of sensors.
According to one embodiment, said articulated chain comprises three segments per lower limb, and eight degrees of freedom.
Thus the sensor can be put on the foot (the least difficult case for the user and economically the most advantageous since the sensor can be in the shoe). If the time-change models linking all the angles are known, it is possible to refer back to all the angles in the articulated chain.
In one embodiment, said measurement means, said first determination means and said second determination means are suitable for being fixed to the articulated chain at a point corresponding to a tibia.
Thus the phases of placing the foot on the ground can easily be determined (angular sensors at the tibia).
In one embodiment, said measurement means, said first determination means and said second determination means are suitable for being fixed to the articulated chain at a point corresponding to the foot or ankle.
It is thus possible to use the zero speed information for limiting the drift due to the integration used for estimating the position.
According to one embodiment, the system comprises first means of detecting the placing on the ground of a foot of the articulated chain.
Thus the phases of the foot at zero speed are easily determined.
In addition, the system can comprise means of estimating the inter-foot distance.
Thus the system requires the instrumentation of only one limb, and the precision of the path of the feet can be improved.
In one embodiment, the system comprises second means for detecting movement cycles or particular instants characteristic of these cycles.
Thus it is possible to adjust the parameters of the time change models and to derive therefrom the movement over the duration of the cycle.
According to another aspect of the invention, a method for determining values of parameters representing a movement of at least two limbs of an entity in order to generate a movement of said entity is also proposed, said entity being suitable for being represented in the form of an articulated chain, comprising at least two segments connected by an articulation, characterised in that it comprises the steps consisting of:
To this end, the subject matter of the invention is a method for determining values of parameters representing a movement by means of a sensor assembly, the movement involving at least two limbs of an entity represented in the form of an articulated chain in which each limb is itself represented by at least one segment and said at least two limbs are connected together by an articulation, the sensor assembly being fixed to the entity and comprising at least one sensor, referred to as the orientation sensor, for a parameter representing an orientation of a first segment of the articulated chain, this method comprising the following steps:
Thus the number of sensors is minimised (it is not necessary to instrument all the limbs of the articulated chain).
Optionally, the time-change model comprising a relationship of time dependency, in particular a relationship of time offset, between the movements of at least two distinct limbs of the entity, said at least one other parameter comprises a parameter representing a movement of another segment of the articulated chain associated with at least part of a limb of the articulated chain other than the limb represented at least by the first segment.
Also optionally, the time-change model comprises a relationship of time dependency of the orientations of all the segments of the articulated chain relative to that of the first segment.
Also optionally, the entity is represented in the form of an articulated chain representing two upper or lower limbs of a living or artificial being:
Another subject matter of the invention is a computer program downloadable from a communication network and/or recorded on a medium readable by computer and/or executable by a processor, characterised in that it comprises instructions for executing the steps of a determination method according to the invention, when said program is executed on a computer.
Another subject matter of the invention is a system for determining values of parameters representing a movement, this movement involving at least two limbs of an entity represented in the form of an articulated chain in which each limb is represented by at least one segment and said at least two limbs are connected together by an articulation, comprising:
Optionally, the orientation sensor comprises at least one of the elements of the assembly consisting of a magnetometer with at least one measurement axis and a gyrometer with at least one measurement axis.
Also optionally, a determination system according to the invention may comprise several orientation sensors, to the extent of at least one orientation sensor per limb of the entity.
Also optionally, a determination system according to the invention may also comprise an acceleration sensor with at least two measurement axes provided with means for fixing to the part of the limb represented by the first segment, for estimating cartesian position values for a point linked to the first segment.
Also optionally, a determination system according to the invention may also comprise at least one of the elements of the assembly consisting of first means of detection of the placing on the ground of a foot of the entity represented by said articulated chain, means for estimating an inter-foot distance and second means for detecting movement cycles or particular instants characteristic of these cycles.
The invention will be better understood from a study of a few embodiments described by way of in no way limitative examples illustrated by the accompanying drawings, in which:
In all the figures, the elements having the same references are similar.
If in a variant it is considered that the articulated chain does not include the feet, and therefore not the ankles, but that the movement conditions are identical (in a plane orthogonal to the rotation axes of the various articulations, these various axes all being colinear), the articulated chain then comprises four segments, three articulations and six degrees of freedom.
During a given time range a point on the articulated chain CH is considered to be fixed. For example, during a part of the walking cycle, the foot is placed on the ground. Thus the position now depends only on the articular angles and the length of the segments, and the number of degrees of freedom is reduced.
In addition, the articular angles are bounded, i.e. lying between a minimum value and a maximum value, the articulations of the human body having articular stops.
The system comprises a module MES_O for measuring the value of a first parameter βr representing an orientation of a first segment of a first limb, in this case the right leg. The system further comprises a first module DET_O for determining the value of a second parameter βl representing an orientation of a second segment belonging to the other limb of the articulated chain CH, the left leg. When the articulated chain comprises more than two limbs, each other limb, different from the first limb provided with the measurement module MES_O, is provided with a respective module DET_O for determining the value of a second parameter representing an orientation of a second segment belonging to this other limb.
The system also comprises a second module DET_P for determining the value of a third parameter representing the change over time of a cartesian position P of a moving point linked to the articulated chain CH. The third parameter representing the change over time of a position P of a moving point linked to the articulated chain CH may be the cartesian position, the speed or the acceleration of the point P.
The measurement module MES_O may comprise a magnetometer with at least one measurement axis and/or a gyrometer with at least one measurement axis. The first determination module DET_O may comprise a magnetometer with at least one measurement axis and/or a gyrometer with at least one measurement axis fixed to a segment of the other limb.
The first determination module DET_O is adapted to determine, by means of a sensor and/or a change model, the value of said second parameters or parameters βl from the value of the first parameter βr, by means of a time-change model EVOL linking the values of the first parameter βr to the values of said second parameter or parameters βl.
The model EVOL may comprise an offset in time between the value of the first parameter βr and the value of said second parameter or parameters βl.
In a variant, the model EVOL may be adapted to link the orientation of all the segments of said articulated chain to that of said first segment. The second determination module DET_P comprises, for example, an accelerometer with at least two measurement axes.
In a variant, the articulated chain CH may represent two upper limbs of a living or artificial being, in particular when the movement of the entity is for example a movement by actuation or simulation of actuation of oars or a paddle on a canoe, kayak or rowing boat. In this case, the thigh, tibia and foot segments are respectively replaced by the arm, forearm and hand segments.
The accelerometer may be adapted to be fixed to the articulated chain CH at a point corresponding to an ankle on a tibia.
The system may also comprise first means DET_PPS for detecting the placing on the ground of a foot in the articulated chain CH. It is then possible to take the position Pr equal to P0 throughout the duration of the step, P0 being the last estimated position before the placing of the foot. The position Pl of the other foot may be estimated by means of an estimation module EST_DIP able to retrieve the inter-foot distance, for example by means of a radio telemetric device. The inter-foot distance, associated with the change model, makes it possible to know where one is in the walking cycle for the second foot. Moreover, there is additional information on the length of the stride when the foot is placed, which improves the precision of the system. The use of a distance measurement system is only one example of use of the detection of the placing of a foot but it is very useful outside this particular embodiment.
The system may comprise second means DET_CY of detecting movement cycles. A cycle for walking corresponds to the succession of a step of each foot. This cycle corresponds to the pseudo-period between the occurrence of two similar events: between two instants at which the heel strikes the ground HS or between two lifts of the toes TO or between two instants where the foot is flat FF. In order to obtain the position and orientation of a point on the articulated chain, several means can be envisaged.
The positions are described by three cartesian coordinates in a reference frame linked to the terrestrial reference frame.
The orientations can be described by three angles or by a quaternion describing the rotations for passing from a reference frame linked to the terrestrial reference frame to a reference frame linked to the limb for which it is wished to measure the orientation.
Parameters representing the position and orientation means all or some of the variables expressing the position or orientation. The choice of these variables depends on the number of degrees of freedom of the articulated chain CH.
For example, in the case of walking forwards or backwards, only three items of information are needed: the two coordinates in the walking plane and the rotation angle about the medio-lateral axis.
One embodiment consists of positioning one inertial sensor per limb.
An inertial sensor, included in the measurement means MES_O, must be able to measure a parameter representing the orientation and position of the element to which it is linked. Thus an accelerometer with at least two measurement axes serves for measuring the position and a gyrometer with at least one measurement axis or a magnetometer with at least one measurement axis serves for measuring the orientation.
Preferentially, an inertial sensor is situated on the tibia at the ankle of each leg. Thus it is avoided having to estimate the angle of the foot. In addition, the proximity with the foot makes it possible to assume a zero speed when the foot is placed, and thus to avoid drift when integrating the natural acceleration issuing from the accelerometer.
The trajectory of each unit, used in the second determination module DET_P, is calculated by detecting the steps performed by the cycle detection module DET_CY and then the detection module DET_PPS, and then the calculation of the orientation at each measurement instant during the duration of the step. This orientation is used to express the measurement of the accelerometer in a reference frame linked to the terrestrial reference frame. This makes it possible to compensate for gravity and to use the assumption of zero speed at the start and end of the step. The natural acceleration thus obtained is then integrated twice. For example, in order to implement the second determination module DET_P, the system described in the patent application WO 2010/046364 will be taken.
Another solution makes it possible to minimise the number of sensors. For example, the path, determined by the second determination module DET_P, and the orientation, determined by the measurement module MES_O, of a point on a first limb is obtained by a first unit. The path of a point on the second limb and/or the orientation of the second limb is obtained by offsetting in time the signal from the first unit (raw signals from the sensors or treated, for position and orientation, obtained after treatment). This is because there often exists in locomotion a time relationship between the movements of the various limbs.
For example, in the case of walking, the movement of the limbs is offset by a half walking cycle. In the case of rowing the movement of the two arms is symmetrical.
Next, parameters of the articulated chain are calculated from the paths and angles.
Given the articulated chain model, it is possible to write the information previously calculated (angles by the modules MES_O and DET_O, and positions by the module DET_P) as functions of the parameters sought (articular angles, global position of the chain CH). The values of the parameters of the articulated chain CH are those that make it possible to verify the information previously calculated (angles and positions). These values can be obtained by any model reversal method (optimisation, Kalman filtering, geometry, etc.).
In one embodiment, the following notations are used:
The coordinates are given in the reference frame (Y, Z), Y being the axis given by the direction of walking and Z being the vertical axis upwards.
(yw,zw): Coordinates of the hip
(yal,zal): Coordinates of the left ankle
(yar,zar): Coordinates of the right ankle
β11: angle of the left thigh with respect to the vertical
β12: angle of the left tibia with respect to the left thigh
β1: angle of the left tibia with respect to the vertical
βr1: angle of the right thigh with respect to the vertical
βr2: angle of the right tibia with respect to the right thigh
βr: angle of the right tibia with respect to the vertical
Ll1: length of the left thigh
Ll2: length of the left tibia
Lr1: length of the right thigh
Lr2: length of the right tibia
Describing the kinematics of the human body is very difficult. For example, a leg has 29 bones and 37 muscles. However, from analysis of the walking of pedestrians, the part of the legs of the pedestrians situated between the waist (the hips) and the ankle is modelled.
As illustrated in
The concern is not with walking to the side or walking to the rear, but walking forwards. Consequently it is assumed that the femur has a single degree of freedom in the sagittal plane defined by the axes (ySF,zSF). Consequently the waist is represented by a single articulation to which the two legs are connected, affording two angles of rotation βr1 and βl1.
Thus the considered biomechanical model has six degrees of freedom and can be represented by four angles βr1, βr2, βl1, βl2 and the cartesian coordinates with two dimensions (yWs,zWs) of the waist of the pedestrian.
βr and βl designate respectively the angles of the right and left tibias with respect to the descending vertical direction in accordance with the following equations:
βr=βr1+βr2
βl=βl1+βl2 (1a)
During walking the angles of the legs vary within a limited range and have a similar form for each step. The knee angles βr2 and βl2 must always be negative.
Hereinafter, β, β1, and β2 are used, for the two legs, to designate respectively {βr, βl}, {βr1, βl1}, and {βr2, βl2}.
In the remainder of this embodiment, it is possible to proceed in two ways:
The first solution is the one that results in the best performance.
Measurements are made with an accelerometer and a magnetometer on each tibia at the ankle.
In a variant, the measurements made on the second tibia may be obtained by making an offset in time of one half-cycle of a step of the measurements of the first tibia.
The path of each of the two feet and the angles β1, βr of the tibias with respect to the vertical at each instant are calculated as for example described in the document WO 2010/046364. In this way six quantities are obtained (four positions and two orientations) G={yal, zal, βl, yar, zar, βr}.
The above may, in a variant, be performed by an accelerometer and a gyrometer, or by an accelerometer, a magnetometer and a gyrometer fixed to the ankle, and a known method for estimating the path.
All the parameters representing the movement of the articulated chain CH can be described by a set PAR of parameters. In
A set PAR* of parameters is sought that best verify the information G. They can be obtained by seeking the parameters that minimise a distance, in the mathematical sense, between G and f(PAR*,L) with G=f(P,L); f representing the function that links the information G estimated previously to the known parameters of the articulated chain CH and to the parameters linked to the movement of this articulated chain CH.
This minimisation may, for example, be done by an optimisation algorithm such as the Levenberg-Marquard algorithm.
The function f is given by the following system of equations:
It is possible to add constraints. A constraint may be that the angle of the knee is always positive: βl2>0 and/or βr2>0>0. Another constraint may be that, when a foot is placed (left or right), its coordinates (yla and zla for the left foot placed and yra and zra for the right foot placed) are constant and equal to the coordinates estimated the first instant the foot is placed.
It is also possible to derive the above equations and to work on the speeds.
A geometric solution can be obtained as illustrated in
For each leg, given the position of the ankle (ya,za), the angle β and the length of the tibia L2, it is possible to determine the position of the knee. For this purpose a segment is traced representing the tibia having a first end of coordinates (ya,za), a length L2 and an orientation β with respect to the vertical. The coordinates of the knee are then given by the coordinates of the other end of the segment. Thus it is then possible to trace a circle having at its centre the position of the knee and at its radius the length of the thigh. The hip is situated at the intersection of the two circles traced for the two legs. The hip cannot be situated below the ground. There is therefore a single solution S. Once the position S of the hip is determined, the segments of the thighs can be traced and the articular angles deduced.
Hereunder follows another embodiment of the invention, for an articulated chain CH representing two lower limbs of a living or artificial being.
A sensor of the measurement module MES_O is used to estimate a parameter representing the angle of at least one of the segments of the articulated chain CH.
An angle or angular velocity sensor is used, for example a magnetometer or a gyrometer with at least one measurement axis. The estimation of the angle or angular velocity using a magnetometer or gyrometer is known and is described hereinafter.
In an advantageous embodiment this sensor is positioned on a tibia. This is because, on a tibia, the change curve of the angles is richer, i.e. it has more singular points useful for the comparisons described in the remainder of the description.
In an advantageous embodiment, a sensor is positioned on each limb. This embodiment makes it possible to achieve the merger of two sensors in order to improve the precision of the system. It makes it possible, among other things, to take into account the changes in speed more precisely.
In an advantageous embodiment, a model EVOL for the change over time of the angle of at least one segment of the articulated chain CH is used. Thus, over time, the angles cannot change randomly but follow a change law that takes account of the movements made during the movement. This change model EVOL is characterised by minima and maxima.
An example of a change model may be a combination, linear or not, of trigonometric functions of time (such as cosine and sine) the frequency and amplitude of which are parameters to be determined. Advantageously, the angle between the thigh and the vertical is defined by a local minimum and a local maximum during each movement cycle of a substantially cyclic movement, for example a walking movement.
The angle between the thigh and the tibia is defined by two local minima and two local maxima defining the various phases of the walking cycle. The time-change curves EVOL of the various angles of the articulated chain CH are linked together. Thus the determination of an angle of a segment of the chain CH makes it possible to deduce the angles that are linked to it. This greatly reduces the number of sensors necessary.
The estimated angle can be compared with the angle of the change model EVOL, in order to determine at what point in the walking cycle it is situated, as well as the parameters of the angle-change model EVOL previously described.
One advantageous example of comparison consists of detecting local minima and/or maxima and/or zero crossings of the angle or angular velocity function. It is thus possible to determine the various phases of the walking as well as the value of the angles at particular instants during the walking: keeping the foot flat or FF, standing for “Foot Flat”, corresponding to the maintenance of the pelvis above the foot; placing the foot on the ground or HS, standing for “Heel Strike”, corresponding to the start of the damping phase that ends when the pelvis passes above the foot; the instant when the foot will leave the ground or TO, standing for “Toe Off”; the instant at which the foot goes forward again, or MS, standing for “Mid Swing”, corresponding to the middle of the flight phase of the foot; the zero angular velocity instant or ZC, standing for “zero crossing”, when the angular velocity passes through zero.
By means of the angle model EVOL and the values of the angles at these points, it is possible to adjust the parameters of the models and to derive therefrom the complete movement of the articulated chain CH between the points.
Among the parameters, it is possible to determine the inclination on the ground on which the user is moving, which is very useful for detecting climbs and descents.
For example, the reference frames are first defined and then the models of the measurements used are determined. The walking phases are then stated and a conventional way of detecting steps is given.
The vectors are measured in a reference frame linked to the sensor. Thus the measurements depend on the orientation of the sensor. As the sensors are attached to a moving body, the measurements are sensitive to instantaneous movements. This requires instantaneous measurements of rotation, since the attitude or orientation of the sensor changes over time.
Three reference frames are defined to clarify the transformations to be made in the remainder of the description. The North frame NF, the local frame BF, standing for “Body Frame”, and the intermediate frame or SF, standing for “Step Frame”, are respectively defined by the axes (xNF, yNF, zNF), (xBF, yBF, zBF), (xSF, ySF, zSF) and the origins ONF, OBF, OSF.
The North frame NF is a reference frame fixed with respect to the terrestrial reference frame during the whole of the movement, for example walking or running. The North frame NF can be considered to be the reference frame: the orientations or angular positions and the positions or cartesian positions are measured in this reference frame. The axes refer to the terrestrial reference frame through the directions East, North and Up. Its origin ONF can correspond to the place where the movement begins or to any other place. The data expressed in this reference frame are denoted with an exponent n.
The local reference frame, or the one linked to the sensor BF, is the reference frame in which the raw data are measured. Its centre is OBF. The data in this reference frame are denoted with an exponent b. Its axes (xBF, yBF, zBF) correspond to the axes of the sensors (in the case of triaxial sensors) at each instant. The coordinates of its origin OBF in the fixed global reference frame NF correspond to the location of the sensor in the fixed global reference frame NF at each instant.
The intermediate frame SF is a reference frame linked to the Earth, but only during an interval of time [t1; t2] (for example during a step). This frame is used with a movement sensor with fewer than six degrees of freedom or 6 DOF. For example, it is used when the movement sensor assembly EC has an invariant rotation axis and/or a flat movement during the interval of time [t1; t2]. xSF is the rotation axis and the sensor assembly EC is subjected to a flat movement in the plane defined by the axes (ySF, zSF). Whenever the plane is vertical, ySF is chosen for the horizontal axis while the axis zSF is the vertical axis. Its origin corresponds to the place of the sensor at the start of the interval of time (for example at t=t1). The data expressed in this reference frame are denoted with an exponent s.
The intermediate frame SF is linked to the fixed global frame NF by a translation that is invariant over time and a rotation invariant over time for the whole of the interval of time [t1; t2]. Here the local frame BF turns about the rotation axis xSF and, in some cases, it is assumed that the translation of OBF, for tε[t1, t2] is given in the plane defined by the axes (ySF, zSF) However, since the data measured are vectors, the interest relates particularly to the rotation between the frames in order to obtain the movement data in the fixed global frame NF.
The orientation of the fixed frame NF in the local frame BF with respect to the fixed frame NF can be expressed by the unit quaternion qbn. The unit quaternion qbn is described with the rotation angle ψ about the rotation axis defined by the unit vector of direction qbn with three dimensions or with a unit vector with four dimensions as follows:
In the present application, all the quaternions are unit quaternions.
It can be noted that qnb is the conjugate of qbn and that:
To do this, a rotation matrix Rnb is defined that makes the vector x, expressed in the frame BF linked to the sensor assembly, change to the same vector expressed in the fixed global frame NF by the following equation:
xn=Rnbxb (3)
The inverse transformation matrix is Rbn=Rnb−1=RnbT. In an equivalent manner, the matrix Rnb is used to make a vector x, expressed in the fixed global frame NF, change to the same vector expressed in the frame BF linked to the sensor assembly.
The rotation matrix Rnb is linked to the quaternion qbn by the following equation:
The model of the measurement supplied by the triaxial sensors used is now given.
Accelerometers measure the sum of all the accelerations applied to the sensor. On the Earth, accelerometers measure, in addition to terrestrial gravity, the external acceleration undergone by the sensor. Natural acceleration is defined by the external acceleration undergone by the sensor, which is equal to the derivative of the speed of the sensor.
Accelerometers have been used in numerous commercial, military and scientific applications, including inertial navigation, vehicle safety systems such as airbags, driving comfort control, stabilisation platforms and vibration monitoring. Commercial versions of triaxial accelerometers with orthogonal axes are widely used for all these applications, for which they are mounted fixedly on a moving body.
An accelerometer of the electromechanical microsystem or MEMS type comprises, among other components, a proof mass that is suspended elastically by one or more suspension springs. The proof mass m moves when the accelerometer undergoes an acceleration, and its movement xm is proportional to the force fm applied to the mass
fm=max=−kxm (5)
in which k is the constant of the spring. The movement of the proof mass m is then converted into an electrical signal having an amplitude parameter that is proportional to the acceleration.
The acceleration measured includes terrestrial gravity as a shift of the acceleration of the body measured, and thus the acceleration of the body measured ab is linked to the natural acceleration apn by the following equation:
ab(t)=Rbn(t)an(t)=Rbn(t)(apn(t)−gn) (6)
in which t represents time, and gn=[0 0−g]T represents the terrestrial gravity vector with g=9.81 m/s2.
The Earth can be considered, as a first approximation, to be a magnetic dipole, with the south magnetic field pole S close to the magnetic north pole of the Earth, and the north magnetic field N close to the magnetic south pole of the Earth. The magnetic field varies from one place to another and differences in the terrestrial magnetic field may be due to the different nature of the rocks. Three magnetometers integrated on three orthogonal axes are used to measure the amplitude and orientation of the magnetic field, or in other words a triaxial magnetometer.
Although magnetic sensors are a little more difficult to use, they supply precise reliable data. In many commercial, military and scientific applications, magnetometers are included in particular for inertial navigation. Magnetometers also detect changes or perturbations in magnetic fields that have been created or modified. From these measurements, it is possible to derive, for example, information on the direction, presence, rotation, angle or presence of electric currents. In the absence of magnetic perturbations, it is assumed that the terrestrial magnetic field bn in three dimensions, expressed in the frame NF invariant over time. For example, in France,
h≈47 μTesla. It is assumed hereinafter that the horizontal component of the terrestrial magnetic field bn is along the axis yNF.
The magnetic field bb measured in the local frame BF is given by the following equation:
bb(t)=Rbn(t)(bn+(t)) (7)
wherein Tn designates the magnetic perturbation.
In the absence of any magnetic perturbation, bn is deemed to be invariant over time: bb depends solely on the orientation of the sensor. However, the terrestrial magnetic field may be perturbed by metal constructions, electronic or electrical apparatus, etc. Consequently the measurements of the magnetometer are less reliable for determining the direction of the terrestrial magnetic field in certain situations, such as in an interior environment, and in particular close to walls or floors or ceilings, which generally comprise metal structures.
Certain properties of walking useful hereinafter are now described. First of all, the various phases of the walking cycle are defined as they are defined in the literature. The use of an optical movement capture system (of the Codamotion type) makes it possible to demonstrate a few particular properties of walking movement. More precisely, the movement of an ankle is represented by a substantially flat movement. This property is used in the present invention. Typical models are also analysed, observed on the measurements of the sensor during a walking cycle.
In this part, the spatio-temporal movement of an ankle is observed by means of a movement sensor comprising an accelerometer, a magnetometer or a gyrometer, mounted fixedly close to the ankle. The measurements supplied by the accelerometer and the variations in the attitude or orientation of the sensor during a step are observed.
During walking, the walking cycle is divided into two phases:
In particular the walking cycle is characterised by several notable characteristic instants:
In addition, the angular speed passes through the zero value on two occasions during a flight phase SW, respectively before and after the instant MS. A zero-crossing instant of the angular velocity and an instant of placing the foot on the ground HS are used hereinafter.
To study the behaviour of the ankle during a walk, using a three-dimensional optical movement capture system, called Coda CX1 by the company Charnwood Dynamics. Its main measuring unit contains three aligned cameras connected fixedly, which follow the position of a certain number of active markers (light-emitting diodes or infrared LEDs) in real time. The output of the system is recorded in a text file containing the coordinates in three dimensions of the active markers PLED(l), where l designates the time index. The spatial precision is around 1 mm. With three markers on a rigid body, it is also possible to follow the orientation of the body. Thus three markers are fixed to the ankle of a pedestrian walking in front of the camera.
The position samples obtained are given in the reference frame of Coda CX1, which is linked to the reference frame NF and is defined by the position of the LEDs at the start of the walking. However, since we do not know this relationship and for reasons of clarity, the position samples are turned in order to correct or rectify the path of the ankle so as to have a horizontal stride and a movement in a vertical plane.
Consequently, v is determined, the unit vector orthogonal to the plane of the movement, with the smallest translation, by a singular value decomposition or SVD, applied to all the three-dimensional position samples PLED=[pLED(l
l1 and l2 are time indices, corresponding respectively to the start and end of a step. The vector v is the straight singular vector, i.e. the eigenvector associated with the smallest singular value or eigenvalue. The vector v is colinear with the axis xSF, and it is assumed that the direction of the stride is colinear with the axis ySF.
The instants when the foot will leave the ground TO and of placing the foot on the ground HS can be determined precisely from the two minima of the angular velocity measured.
To illustrate the variations in the measurements of the accelerometer,
As indicated previously, an accelerometer with two measurement axes can be considered to be a proof mass fixed to springs.
Many step detection methods are known. They are for example based on Fourier transformations, the counting of zero crossings of the output of an accelerometer or the counting of points that exceed a threshold (“In step with INS: navigation for the blind, tracking emergency crews” by Q. Ladetto and B. Merminod, GPS World, Vol. 13 (10), pp. 30-38, 2002).
Even if these methods are simple, they may suffer from significant errors when the users are walking on staircases, along a sloping ground, or when they run or drag their feet. Other precise methods are based on the angular variation of the foot and can estimate each phase of a step (“A magnetometer-based approach for studying human movements” by S. Bonnet and R. Heliot, IEEE Transactions on Biomedical Engineering, Vol. 54 (7), pp. 1353-1355, 2007).
In the present invention, a simple step detection is used. A practical solution for detecting the cycle of walking is to attach the sensor to the ankle of the pedestrian. Then, by means of acceleration or angular velocity measurements, the step is detected in a robust and reliable manner.
During its movement, the foot of a pedestrian follows almost the same form of path for the various steps. In fact, this is true even for different pedestrians who do not have any problem with walking. The acceleration norm ∥an∥ measured is considered and its model is shown in
Even if the trajectory of the ankle is similar for the various steps, the outputs of the sensor depend on the orientation of the sensor during the step. This problem is merely due to the orientation of the sensor on the heel at the start (of the step?). To overcome the problem of the variation in time of the orientation of the sensor on the ankle, it is possible to use simply the amplitude of the measurements of the accelerometer in the reference frame BF (“Robust step detection method for pedestrian navigation systems”, by H.-J. Jang, J. W. Kim, and D. H. Hwang, Electronics Letters, 43 (14), 2007, in accordance with the following equation:
∥ab(t)∥=∥Rbn(t)an(t)∥=∥an(t)∥ (7)
Thus it is possible to use ∥ab(t)∥ since the model is equal to the model ∥an(t)∥ (cf. the model in
One solution is to compare this model with a model. Unfortunately, the model is often a version to a reduced scale in time and amplitude (to be clarified), which requires numerous signal processing operations. One practical solution consists of detecting the extrema values of the model, for example from a threshold crossing.
The aim is not only depth detection but also the determination of the start and end of the step. By means of this determination, it is possible to limit the drifts due to integration by fixing the zero speed at the start and end of the step.
Use is made of a system for monitoring the placing of a leg using only two sensors fixed respectively to the right ankle and to the left ankle of a pedestrian. The sensors may be 3A3M sensors (triaxial accelerometer and triaxial magnetometer), 3A3G sensors (triaxial accelerometer and triaxial gyrometer) or 3A3G3M sensors (triaxial accelerometer and triaxial magnetometer and triaxial gyrometer). The idea is to use the hypothesis of sagittal walking (in a vertical plane oriented in the anteroposterior direction on the median line of the body), in order to reduce the number of degrees of freedom for the model. Furthermore, a kinematic model of the legs is used, similar to those used in robotics. The sagittal plane hypothesis and the biomechanical model of the legs make it possible to solve the problem of estimation of the movement of the lower limbs and of navigation with a reduced number of sensors.
In the remainder of the description, the kinematic modelling of the legs of the pedestrian during walking is described. The parameters that define the model are designed to exploit the relationships between each part of the articulated chain CH of the system and for the walking phases. Next, an example of movement of the legs during walking is described using the simulation parameters. It is also described how to determine the parameters of the model, given the inertial data and the path of the pedestrian allowing spatio-temporal monitoring of the movement of the legs of the pedestrian.
The part of the legs of the pedestrian situated between the waist (the hips) and the ankle is modelled, as in the example described previously in the description of
One way of modelling the variations in the angles of the legs is to interpolate them between the extreme values.
It is now shown how the model proposed above can imitate the walking of a human. The angles are simulated by fixing the parameters of the model so as to respect the critical angles previously shown in
A conventional solution for modelling the change over time of these angles is to use cosine functions. By way of example,
Tgait=TStance+TSwing is the duration of a walking cycle. The angle β2 is simulated using four consecutive cosines, each representing the part of the model of the angles between two consecutive extrema:
THS,ST, TST,TO, TTO,SW and TSW,HS designate the intervals of time separating the extrema corresponding to the indices. The angle β12 is a phase angle between β1 and β2. The angle β12 is explained below in the description, where its dependency with the slope of the walking is demonstrated.
The extrema are given for this simulated walking in the following table, as well as β12. LF and LT, represent the length of the segments representing respectively the femur and tibia.
Finally, assuming that the ankle does not move during a bearing phase ST (assuming the speed of the ankle to be zero), the coordinates (yWs,zWs) can be calculated. This is because, knowing the position of a point on the chain (here the ankle on the ground) and all the articular angles (given by equations (9) and (10) and the position in the walking cycle), it is possible to determine the position of any point on the chain (in this case the hip).
There are two ways of determining the slope during walking from the angles β1 and β2.
The first way of determining the slope (or the angle between the horizontal and the vector of the movement during a step) is based solely on the extrema of β1 that increase with the slope or inclination.
The second way of determining the slope is to use the relationship between the phase angle β12 in equations (10) and the slope of the step. In practice, the phase β12 is almost equivalent to the interval of time between the instant at which the foot moves forward again MS and the minimum value of β2.
In fact, when a pedestrian walks on a given slope, his waist (his hips) follows a path with a constant slope. As the pedestrian must move his foot above the ground, the minimum value β2,SW is reached when the line connecting his waist and his foot in the air becomes orthogonal to the ground.
As indicated previously, the two-dimensional biomechanical model of
Generally, during walking, at least one of the feet touches the ground. It can therefore be assumed that the speed of the ankle is equal to zero during the bearing phase ST. With this assumption and knowing the rotation movement of the legs, it is possible to determine the speed of the waist (hips). Consequently, if all the four angles {βr1, βr2, βl1, βl2} are known, it is then possible to calculate (yWs, zWs) according to the angles of the leg that is in the bearing phase ST. Let (xRKs, yRKs, zRKs), (xLKs, yLKs, zLK2), (xRAs, yRAs, zRAs) and (xLAs, yLAs, zLAs), be the coordinates of the right and left knees and the right and left ankles respectively, where all the coordinates xs are equal to zero. The placing of the legs is constructed and the coordinates are calculated as follows:
Whereas the “β” values were obtained with the measurement means MES_O, the values denoted “{tilde over (β)}” are deduced using the change models EVOL (equations (9) and (10)) and certain hypotheses that link β1 and β2 to β (equation (19) for example).
(δy(l), δz(l)) defines a translation of the waist (hips) so as to verify the bearing phase ST. Thus:
As the two-dimensional cartesian coordinates of the waist of the pedestrian can be calculated once the four angles of the legs have been determined, the problem is reduced to four degrees of freedom.
The placing of the legs with four degrees of freedom can be determined by all the four angles {βr1, βr2, βl1, βl2}. The most simple means of determining these angles is to attach a movement sensor to each segment of the biomechanical model (two femurs and two tibias). However, since we assume that, during walking, the segments turn on an orthogonal invariant axis and move in a plane orthogonal to this axis, it is not necessary to determine the orientation in three dimensions in order to construct the biomechanical model. Thus the minimum solution consists of positioning a magnetometer with two measurement axes 2M in a plane orthogonal to the rotation axis or a gyrometer with one measurement axis 1G parallel to the rotation axis (medio-lateral in the case of normal walking). Another solution consists of dispensing with the orientation of the sensor with respect to the segment by using a triaxial magnetometer 3M or a triaxial gyrometer 3G attached to each part of the legs of the pedestrian (i.e. the four segments). It is also assumed that the movement sensor turns about an axis invariant over time, and it is possible to determine the axis of the rotation. Thus the angle of each segment of the biomechanical model can be determined.
Furthermore, reducing the number of sensors is more practical for daily use. In addition, this reduces the cost and the electrical consumption of the whole of the navigation system. Consequently two movement sensors are used fixed to the left and right ankles, enabling the placing of the legs of the pedestrian to be monitored. This method has been applied in an external environment where the magnetometers fixed to an ankle are not interfered with. Thus it is possible to determine the position from the magnetometer mounted on the ankle. In a monitoring of placing of the legs of the pedestrian indoors, a magnetometer close to the floor is greatly disturbed. Thus another movement sensor may be used for determining the position. For example, a third movement sensor mounted on the waist of the pedestrian can be used to monitor the position of the pedestrian.
Hereinafter the estimation of the placing of the legs of the pedestrian using the two movement sensors mounted on the ankles is focused on.
Determining the placing of the legs for the biomechanical model requires knowledge of the four angles. Consequently it is necessary to deduce four degrees of freedom from βl and βr.
One solution consists of reducing the four degrees of freedom of the model to two degrees of freedom, the number of angles available. For example, the possibility of establishing functions of time for the angles {βr1, βr2, βl1, βl2} using the cosine functions is shown previously. By means of these functions, only one angle is necessary for determining the whole of the placing of the legs, reducing the model to one degree of freedom instead of four. However, this method lacks precision. Furthermore, the slope of the walking changes the time occurrence and the value of the extrema of angles {β1,TO, β1,HS, β2,TO, β2,SW, β2,HS, β2,ST} as described previously, which makes a universal equation very complex to establish.
Another solution consists of using the implicit information included in the angles βl and βr, in addition to the angles themselves. This information makes it possible to predict four angles from the two unknown angles by relying on equations of angles relating to the walking of a pedestrian. It is proposed to interpolate the angles βr1 and βl1 while relying on the critical values and their instants of occurrence, and the zero crossings of the angular velocities dβr/dt and dβl/dt, where Max1 and Max2 are respectively achieved at instant TO where the foot will leave the ground and that instant HS of placing the foot on the ground, while the zero crossings correspond to the maximum value of β.
The angles βr and βl are now determined. It is possible to use a magnetometer with at least two measurement axes 2M or a gyrometer with at least two measurement axes 2G or a gyrometer with one measurement axis G for determining the angles βr and βl. When a triaxial magnetometer 3M or a triaxial gyrometer 3G is used, it is then possible to dispense with the orientation of the sensor with respect to the segment and to determine βl and βr by estimating first the rotation axis of the segment in the reference frame of the sensor. Let all the three singular vectors issuing from the SVD of the matrix of measurements (rows=all the measurements during the step and column=the 3 axes) be noted (v1b, v2b, v3b), where (v1b, v2b) are associated with the plane of the walking and v3b the axis of the rotation substantially invariant over time expressed in the reference frame of the sensor BF. With the measurements ωb of the gyrometer, it is possible to obtain the angular velocity about the axis v3, in accordance with the following equation:
ωv
in which “·” designates the scalar product. Next the angular velocity ωv
β(t)=β(t0)+∫0ωv
t0 is the instant FF of maintaining the foot flat during the bearing phase ST, determined by a detection method for identifying the various phases of walking (EP 2106747), β(t0) is the angle of the tibia with respect to the vertical at instant t=t0.
Likewise, β(t) can be calculated using bb(t) by the following equation:
β(t)=β(t0)+βb (16)
in which:
^ representing the vectorial product.
To determine β(t0), an observation of β at t=t0 is needed:
It is assumed that zBF is aligned with the direction of the tibia. If an accelerometer is used, its measurements at the instant FF of maintaining the foot flat (on a flat ground) is ab(t0)=gb, β(t0) being able to be determined as the angle between zBF and the vertical given by ab(t0)=gb, zBF and ab(t0) both being projected into the plane of the walking.
After having calculated βr and βl using equations (16), (17) and (18), the placing of the legs can be estimated.
It can be remarked that the occurrences of the maximum of β and βl are relatively close (cf
each step κ, βr1κ and βl1κ are interpolated with two cosine functions, for tεIZCκ, with:
The determination of a value of βl for a time sample of each interval is sufficient to find the parameters of the change model of β1. Using only the values βr and βl, it is not possible to determine the angle values at β1. Consequently additional information or hypotheses must be included.
For example, during walking, at t=tZC it can be considered that the angle of the knee β2 is close to zero. However, the exact values can be predicted only with difficulty since they depend a great deal on the behaviour parameters of the walking (for example the slope of the walking, the length of the step, the speed, the user, etc.).
It is possible to estimate {tilde over (β)}1(t) by modelling β1(t) for tε[tZCκ−1, tZCκ] by means of a cosine function passing through the estimated values βl(tZCκ−1) and β1(tZCκ). Thus it is not necessary to have a movement sensor to observe β1 continuously. Furthermore, since β=β1+β2, the angle {tilde over (β)}2 is obtained by the following equation:
{tilde over (β)}2(t)=β(t)−{tilde over (β)}1(t) (19)
Consequently it is possible to calculate all the four angles {{tilde over (β)}r1(t), {tilde over (β)}r2(t), {tilde over (β)}l1(t), {tilde over (β)}l2(t)}.
Finally, the coordinates of the legs are calculated as described previously, where the bearing phase ST is switched between the left and right legs at each placing of the foot on the ground HS.
The coordinates in the reference frame NF linked to the Earth can be calculated, the heading angle φW(l) of the waist (or hips) of the pedestrian being determined from one or all of the magnetometers mounted on the ankles.
Advantageously, it is possible to use two triaxial magnetometers 3M (or 2M in the plane of the walking) or two triaxial gyrometers 3G (or 1G in the axis orthogonal to the plane of the walking) mounted on each ankle of the pedestrian. Next the following steps are applied:
1) the angles βr and βl are determined as described previously by equations (14) and (15) with a gyrometer or (16), (17) and (18) with a magnetometer. These angles are shown in
2) the angular velocities dβr/dt and dβl/dt are calculated (cf
3) the detection of a step for detecting the instants HS and ZC is implemented (cf
4) the angles βr1 and βl1 are modelled by cosine functions relying on the occurrences tZCκ−1 and tZCκ, and on β(tZCκ−1) and β(tZCκ) (cf
5) the angles {tilde over (β)}r2 and {tilde over (β)}l2 are calculated (cf
Number | Date | Country | Kind |
---|---|---|---|
10 59965 | Dec 2010 | FR | national |
10 59967 | Dec 2010 | FR | national |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/FR2011/052844 | 12/1/2011 | WO | 00 | 8/13/2013 |
Publishing Document | Publishing Date | Country | Kind |
---|---|---|---|
WO2012/072961 | 6/7/2012 | WO | A |
Number | Name | Date | Kind |
---|---|---|---|
7467060 | Kulach | Dec 2008 | B2 |
8988438 | Bang | Mar 2015 | B2 |
9060714 | Bajcsy | Jun 2015 | B2 |
20080262772 | Luinge et al. | Oct 2008 | A1 |
20080278497 | Jammes et al. | Nov 2008 | A1 |
20080285805 | Luinge et al. | Nov 2008 | A1 |
20090067678 | Caritu et al. | Mar 2009 | A1 |
Number | Date | Country |
---|---|---|
1 970 005 | Sep 2008 | EP |
1 990 138 | Nov 2008 | EP |
2007 093641 | Aug 2007 | WO |
Entry |
---|
Dejnabadi, H. et al., “Estimation and Visualization of Sagittal Kinematics of Lower Limbs Orientation Using Body-Fixed Sensors,”, IEEE Transactions on Biomedical Engineering, IEEE Service Center, vol. 53, No. 7, pp. 1385 to 1393, (Jul. 1, 2006). |
International Search Report Issued Nov. 7, 2012 in PCT/FR11/52844 Filed Dec. 1, 2011. |
Number | Date | Country | |
---|---|---|---|
20130324890 A1 | Dec 2013 | US |