Patent Application
20200155035
The present invention relates to systems and methods for self-learning of gait characteristic and speed during bipedal locomotion using sensor and GPS technology.
Mitigating the risk of falls is a primary medical concern for the elderly. Predicting falls is one method for mitigating the risk. Scientific evidence suggests that increased gait variability is a key prediction factor of falls among community-living older adults. Hausdorff et al., 2001. Also, the 5-year survival rate of patients above 65 years old can be predicted from gait speed along with age and gender. Studenski et al., 2011. Gait speed is also a good predictor of future risks of falls and hospitalizations. Studenski et al., 2003. Studies have also provided normative gait speed values such as the sufficient speed to cross the street of 1.22 m/s [Langlois et al., 1997] and the comfortable speed of various patients [Bohannon et al, 1997].
The science of predicting falls currently relies on many different factors including past experience (e.g., frequency, context, and severity) with falls, visual impairment, cognitive impairment, and psychology (e.g., depression in the patient), to name a few. Gait analysis is one consideration. Typically, all of these factors are combined to assign a risk of fall category to the elderly. Living community staff (i.e., therapists, nurses, doctors, and the like) can monitor each patient in the community based on the risk category. Stalenhoef et al., “A Risk Model for the Prediction of Recurrent Falls in Community-Dwelling Elderly: a Prospective Cohort Study,” J Clin Epidemiol, 2000.
However, insurance providers and clinical institutions struggle to meet the challenges of caring for and treating the elderly at risk of falls and the fallen and injured elderly. The growth rate in the number of elderly and strict regulations result in fewer doctors and other caregivers and, thus, contribute to the challenges. Additionally, the ageing population, as wells as their families, wish to continue living at home for as long as possible but mobility limitation and falls prevent them from doing so.
However, current systems and methods have only small to moderate effects on gait deficits and focus on treating patients after adverse events when it is too late. This is likely due to a lack of appropriate tools available to the health care providers to understand their patients' behavior and physical abilities post-consultation. The current systems and methods do not meet clinicians' needs for an easy yet reliable functional evaluation of patient status. The current systems and methods are thus unable to provide objective evaluation, prevent falls, and prescribe interventions which would benefit patients. Indeed, current systems generally provide gait measurements specifically for clinical evaluation and assessed in laboratory settings in protocoled conditions. These time-consuming evaluations are thus limited to a very restricted number of patients.
To compensate, some clinicians have turned to basic trackers found on the market, such as pedometers, to understand their patients' behavior in out-of-the-lab conditions. As discussed in more detail below, these tools are not able to provide them with the relevant parameters, such as speed for example, with sufficient accuracy to be meaningful.
The state of the art includes a wide range of movement measuring devices and methods, as discussed below. Generally, these devices and methods either indicate that a fall is underway or do not provide reliable data and, consequently, lack market acceptance past short-term use. Indeed, a report by PricewaterhouseCoopers showed that the average use of FitBit is two weeks due to limitations of reliability. This reports also states that “[c]onsumers recognize enormous potential in the emerging category—but right now, they are skeptical that wearable technology can deliver on that potential.” (http://www.mobihealthnews. com/37543/pwc-1-in-5 -americans-owns-a-wearable-1-in-10-wears-them-daily).
For example, Petelenz, U.S. Pat. No. 6,433,690, describes a method for detecting whether a movement is a fall by recording the acceleration and body position data using an accelerometer. Petelenz attempts to distinguish a fall from other accelerated movements and determine the severity of the fall for the purpose of sending an alert signal. Thus, Petelenz is useful only for indicating a fall has already occurred.
Some existing systems provide long-term gait analysis by providing floor-based sensing. For example, Alwan, U.S. Pat. No. 8,894,576 describes a floor-based sensing system to gather longitudinal data of a person's gait to use in a fall prediction model. Alwan describes a prediction model based on distinguishing normal and abnormal gait. The gait characteristics include step count, pace, normal condition, limp shuffle, falls, average walking velocity, step or stride length. Such systems can be effective but require the subject to walk on the monitored surface and are ineffective elsewhere. Furthermore, such systems can be expensive, requiring specialized flooring systems and may not be easily replaced or repaired.
Azzaro, U.S. Pat. No. 7,612,681, describes collecting range-controlled radar sensor data an applying different techniques to predict the fall risk likelihood by distinguishing among a normal walk, a limp, or a shuffle using a wavelet analysis. Techniques include a Hidden Markov Model and a Bayesian network. Azzora requires potentially expensive motion sensing equipment throughout the living quarters of the subject. Additionally, data is captured only where motion sensing equipment is lacking or outside the living quarters. Thus, systems like Azzaro fail to capture relevant data, including gait data captured in areas less familiar to, and thus riskier for, the subject.
Some existing systems attempt to predict whether a fall is imminent. For example, Kasama, U.S. Pat. No. 9,299,235 describes detecting acceleration in the gravitational acceleration direction of the portable electronic apparatus, determining whether or not the acceleration in the gravitational acceleration direction is a threshold value or less, the threshold value being stored in a determination threshold value table, and raising an alarm for prediction of stumbling of a user when the acceleration in the gravitational acceleration direction is the threshold value or less. Kasama collects three-dimensional acceleration data and calculates step frequency to determine an immediate risk of falling. Kasama does not, however, assist in determining a generalized risk of falling and focus solely on whether a fall is imminent or in progress.
What is needed is a long-term monitoring system useful in generating a generalized risk of falls along with a determination of imminent fall risk so that the risk can be managed before a fall and proper therapy can be applied after a fall. Additionally, there is a need for systems and methods that allow remote caregivers, friends, and loved ones to be involved in fall prevention of the elderly.
Aspects of the invention relate to deriving a generalized risk determinant using low-cost sensors and GPS to accurately measure gait speed.
In one embodiment, advanced gait parameters are measured from global distance and position metrics and detailed kinematics metrics and received by a third-party insurance provider. A discount tier is determined by the third-party insurance provider from the advanced gait, or more generally, locomotion, parameters and customer threshold requirements. In another embodiment, advanced locomotion parameters are measured from global distance and position metrics and detailed kinematics metrics and received by a clinical system outside of a protocolled condition context. The clinical system can generate a gait or locomotion analysis summary. In another embodiment, advanced locomotion parameters are measured from global distance and position metrics and detailed kinematics metrics and received by a remote server. A risk determinant can be generated from the advanced locomotion parameters and a message signal based on the risk determinant can be sent from the remote server.
Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The materials, methods, and examples provided herein are illustrative only and not intended to be limiting.
Various embodiments of the methods, systems and apparatuses of the present disclosure can be implemented by hardware and/or by software or a combination thereof. For example, as hardware, selected steps of methodology according to some embodiments can be implemented as a chip and/or a circuit. As software, selected steps of the methodology (e.g., according to some embodiments of the disclosure) can be implemented as a plurality of software instructions being executed by a computer (e.g., using any suitable operating system). Accordingly, in some embodiments, selected steps of methods, systems and/or apparatuses of the present disclosure can be performed by a processor (e.g., executing an application and/or a plurality of instructions).
Although embodiments of the present disclosure are described with regard to a “computer,” and/or with respect to a “computer network,” it should be noted that optionally any device featuring a processor and the ability to execute one or more instructions is within the scope of the disclosure, such as may be referred to herein as simply a computer or a computational device and which includes (but not limited to) any type of personal computer (PC), a server, a cellular telephone, an IP telephone, a smartphone or other type of mobile computational device, a PDA (personal digital assistant), a thin client, a smartwatch, head mounted display or other wearable that is able to communicate wired or wirelessly with a local or remote device. To this end, any two or more of such devices in communication with each other may comprise a “computer network.”
Embodiments of the disclosure are herein described, by way of example only, with reference to the accompanying drawings. With specific reference now to the drawings in detail, it is stressed that particulars shown are by way of example and for purposes of illustrative discussion of the various embodiments of the present disclosure only and are presented in order to provide what is believed to be a useful and readily understood description of the principles and conceptual aspects of the various embodiments of inventions disclosed therein.
Referring now to
User device 105 features an IMU (inertial measurement unit) 110 for collecting angular velocity and linear acceleration data, in regard to movements of user device 105, thereby collecting such data about movements of the user. IMU 110 is preferably selected for a suitable sensitivity and range, according to the functions described in greater detail below. User device 105 can also feature a barometric pressure sensor. The barometer is used to enrich movement context and improve the accuracy of energy expenditure estimation. User device 105 also comprises a GPS or other type of positioning system module 115 for collecting positioning data, in regard to movements of user device 105, thereby collecting such data about movements of the user. GPS module 115 is preferably selected for a suitable sensitivity and range, according to the functions described in greater detail below and can include one or more positioning data receivers. GPS module 115 provides geolocalization to compute with the highest precision gait speed and walking distance. Besides, GPS data is used as the reference training data to build up a data-driven model for gait speed and walking distance in indoor areas. Once the data driven model is built, GPS usage can be temporized to enhance the wearable device battery life. It should be understood as implied above that other positioning systems can be used in addition to or in place of GPS and that the use of “GPS” herein can refer to the use of one or more such satellite-based navigation systems. In addition, those of skill in the art can appreciate that other types of positioning systems such as local position systems that generally use a type of signaling beacon to determine a position can be used in addition to or in place of a satellite-based navigation system. GPS module 115 can include a receiver for communicating with the appropriate satellite or beacon and other hardware and/or software suitable to allow for the collection of positioning data.
Within the user device 105, optionally the following components are included as a non-limiting implementation example:
An analysis module 120 receives such data from IMU 110 and then classifies the locomotion of the user according to such data and one or more reference model templates. As described in greater detail below, such activity classification preferably features determining a gait, or locomotion, category, such as walking, jogging, running, climbing stairs, walking with a cane, and the like. Activity classification can be performed as described in PCT Application No. PCT/IB2018/059933, which is hereby incorporated by reference as if fully set forth herein.
Self-learning module 130 receives data from IMU 110 and GPS module 115 to update one or more reference model templates in reference model 125 according to the data. User device 105 also includes communications module 145 for sending data to one or more servers 160 for further processing, including reporting functions. The speed calculation model is updated during outdoor activities or activities in which the speed of the user is more readily determined, such as when the user is using a connected treadmill for example. As discussed in further detail below, if GPS data is available, user device 105 can provide more accurate speed and locomotion kinematics data that if only IMU data is available. Furthermore, relevant reference model templates can be updated to more closely match the gait of the user with GPS data, if available. The algorithmic intelligence to update reference model templates is based on advanced machine learning techniques to improve indoor tracking by fusing IMU data with GPS data during outdoor measurements. Analysis module 120, self-learning module 130, and communications module 145 and other modules of device 105 can be further separated into other modules or combined. Furthermore, the modules can be implemented either in software or hardware.
Reference model 125 preferably includes one or more reference model templates representing different types of gaits. In some preferred embodiments, reference model templates can be created from locomotion kinematics sequence (e.g., as a time series) data obtained from one or more people preferably having a cross-section of locomotion characteristics (e.g., gait characteristics). In other embodiments, reference model templates can be created as a synthetic template in which the template is estimated from known or common locomotion characteristics. In other preferred embodiments, reference model can be built over time and include data from the user. In this case, locomotion kinematics sequence data from the user can be collected as part of a calibration process to create a reference model template. Such a calibration process would be a supervised process to ensure accurate classification of locomotion kinematics sequence data and template creation. In some instances, a reference model template can be customized to the gender, age, or physical characteristics of different types of users. Physical characteristics can include height, leg height, weight, and the like. Thus, a reference model template can be matched to a user based on one or more of these characteristics for analysis as described further herein. Reference model 125 can be stored according any well-known data storage or memory technique or device. In preferred embodiments, reference model can include data from one or more anonymous users or a synthetic model. In some instances, reference model can be stored at server 160 and at wearable device 105 and be updated periodically if changes are made to one or the other.
Self-learning module 130 can include hardware or software for updating reference model 125 from data sampled from GPS module 115, IMU 110, or both (see, e.g., the block diagram of
The results of the classification may be displayed on a display 140 of the wearable device 105, which may be integrally formed with or attached to wearable device 105. For example,
Communications module 145 is coupled to a network interface 150 to communicate with server 160 over network 180. Data from the various modules of wearable device 105 or IMU 105 or GPS module 110 may be shared with server 160 through such communication. Server 160 includes a network interface 165 for communications and reporting module 170. Reporting module 170 can be hardware or software and can be further combined with or separated from other components of server 160. Reporting module 170 can generate reporting information about the movements of the user including gait speed over time, average gait speed over a time, locomotion classification, visual representations of locomotion kinematics, and the like. System 100 can include other remote servers that provide reporting services or other processing or data storage services. In some other embodiments, wearable device 105 or other type of user device (e.g., smartphone, tablet, and the like) can include a reporting module, a display for displaying reporting information, or both.
Wearable device 105 is preferably able to perform most, if not all, of the analysis functions independently. However, wearable device 105 is preferably in communication with a remote server 160 through a computer network 180 as shown, for additional services, and optionally to access and/or share additional data. Computer network 180 may optionally comprise the Internet for example. The user is assumed to be holding, wearing or otherwise to be attached to wearable device 105, such that movements of the user are reflected in movements of wearable device 105.
It should be understood that each of the modules and components described above can be combined into fewer modules, components, or devices or further separated into additional modules, components, or devices to perform the functions described herein in a more consolidated or more distributed manner. Furthermore, wearable device can include one or more memory storage device that contains one or more of the modules or computer processor instructions that implement a portion or all of the modules.
Referring now to
The remaining components of wearable device 205 are similar to the other components of wearable device 105, including IMU 210, GPS module 215, communications module 220, display 225, and network interface 230. In preferred embodiments in accordance with system 200, communications module is configured to send data from IMU 210 and GPS module 215 to server 250 for additional processing.
In the example shown, user device 350 includes communications module 355 to transmit data, receive data, or both on behalf of wearable device 305 to and from server 360 via network 180. User device 350 can be configured to provide other services, including but not limited to displaying data or reporting information or to analysis and processing services described herein. For example, in some preferred embodiments, user device 350 can include an analysis module, self-learning module, reporting module, reference model, or some combination thereof to provide analysis or services. IMU 310, GPS module 315, communications module 320, display 330, and network interface 335 in wearable device 305 substantially correspond to the IMU 110 and 210, GPS module 115 and 215, communications module 145 and 220, display 140 and 225, and network interface 150 and 230 from
Referring now to
At step 415, the IMU and GPS signals are conditioned. Such signal conditioning preferably includes performing a dynamic calibration so IMU axes are virtually aligned to the functional movement axes. The calibration is preferably performed as an optimization that minimizes the difference between virtually-rotated-IMU signals and the function axis of body segments. Such a calibration means that the analyzer is able to determine the activity parameters without requiring specific direction of attachment of IMU to the user body.
At step 420, parameters are extracted from IMU signals. Extracted parameters have or reveal biomechanical information on the user's gait which can include duration of movement, velocity, and IMU orientation in 3D space (e.g., position along different axes). Optionally, statistical features are extracted cycle by cycle. The feature extraction is insusceptible to cycle duration or amplitude and mainly dependent on geometric shape of the IMU signal at each cycle.
At step 425, probabilistic classification is performed. In preferred embodiments, a label that indicates the type of activity and a confidence interval on the certainty of chosen label is determined for the locomotion kinematics sequence data. Once the IMU is aligned to the bodily axis through, for example, signal conditioning as discussed above, and movement generic parameters are extracted, then the activity type or locomotion can be classified. In preferred embodiments, activity or locomotion classification is limited to particular types of gait involving legged locomotion of the user. Legged locomotion of the user can include, in some cases, locomotion with assistance (e.g., walking with a cane or walker). It is possible for some embodiments to classify other types of locomotion. IMU signals that indicate some movement other than locomotion are disregarded. Optionally dynamic time warping is applied to the data, to account for temporal effects. For example, the classification may be performed according to multi-class QDA (quadratic discriminant analysis), a technique which is well known in the art. The features used for the covariance matrix of the QDA preferably include, but are not limited to, statistical features such as signal amplitude, auto regressive coefficients that describe each cycle of IMU data (preferably in 6 channels), and signal form features extracted from the dynamic time warping.
At step 430, expert rules are preferably applied, based on the output of probabilistic classifier and temporal sequence of activity in terms of previous activities. According to the application of such rules, the activity type is preferably accepted or modified.
At step 435, a speed is estimated. Preferred embodiments can use a Locally Linear Model Tree (LoLiMoT) speed estimator updater. A LoLiMoT is a subset of fuzzy algorithms for input space decomposition with local linear least squares optimization. The input space is decomposed in an axis-orthogonal manner yielding hyper-rectangles which accommodate fuzzy membership functions. The standard deviation of membership functions is chosen proportionally to the extension of hyper-rectangle. LoLiMoT provides a flexible framework to model intrinsic nonlinearities between input and output spaces. LoLiMoT was used to find the model that maps Kinematics features to running (walking) speed. Other preferred methods can estimate speed based on sensor data. For example, in some embodiments, it is possible to bypass a speed estimation from IMU data and use only the GPS data.
At step 440, the relevant reference model template determined and updated. If reference template data is available, the LoLiMoT estimator, updates the mathematical speed model of the user which is preferably part of a reference model as discussed in connection with
At step 445, locomotion characteristic data for reporting are stored in a gait reporting data store. In some preferred embodiments, a user device can have a data storage or memory component in which the data classification and speed. In some preferred embodiments, locomotion characteristic data is stored on the user device temporarily and periodically or on-command sent to another server for storage and further analysis and/or processing. Locomotion characteristics can include steps, cadence, speed, duration, number of sessions, and the like, as well as metadata associated with the characteristics such as time, date, and location, if available.
At step 450, gait analysis report is created and sent. In some preferred embodiments, reporting is sent to a caregiver (e.g., doctor, therapist, nurse) to diagnose gait problems or to monitor gait change trends in the user. Reporting can take the form of any well-known reporting technique including printing, email, web page, and the like. Reporting can also be sent or made available to the user or others. In some preferred embodiments, reporting can be sent to a user device such as user device 105.
Referring now to
At step 540, gait analysis reporting data is generated. Such data can include temporal locomotion kinematics sequence data, speed data, and the like. Reporting data generated in preferred embodiments is discussed in further detail above. At step 545, gait analysis reporting is sent. This step can be performed in response to a request, for example, as a request for a web page or the presentation of a reporting user interface. In some instances, this step can be performed in response to a periodic or event-based trigger. For, example, periodic reporting data can be sent as an email or through some other messaging or user interface presentation or in response to a threshold related to a user's gait deterioration or development.
At step 610, a reference model template is received and a probabilistic match to the IMU locomotion kinematics sequence data is determined. This step and steps 615 and 620 are repeated for each reference model template. In some preferred embodiments, only relevant reference model templates can be used. A relevant reference model template can be based on criteria such as speed range or templates selected as relevant to the user. At step 615, dynamic time warping is applied to the IMU cycle data. As described herein, for the probabilistic match, some technique known in the art, for example QDA, is used. In preferred embodiments, a confidence level for the probabilistic match is generated. In preferred embodiments, a confidence level or margin of error range preferably is from 10-20% and more preferably 15%. If a match is not found, then at step 630, an instruction is sent indicating that sequence data is unclassified. In this case, in some preferred embodiments, the locomotion kinematics sequence data can still be saved for reporting and the reporting can indicate that is was unclassified. In some preferred embodiments, the locomotion kinematics or activity sequence data can be classified as provisional based on the closest matching reference model template. Later, the locomotion kinematics or activity sequence data can be corrected for noise or a reference model template can be updated such that the locomotion kinematics or activity sequence data matches within the threshold and, thus, be classified.
If a match is found for the locomotion kinematics or activity sequence data, then at step 635, a check is made whether GPS signal data is available. If so, then at step 640, an instruction is sent indicating that the relevant reference model template can be updated and at step 645, an instruction is sent indicating that the locomotion kinematics or activity sequence data can be classified. If GPS signal data is not available, then step 645 is performed, but step 640 is not. The instructions sent can be flags stored in a buffer that are communicated with locomotion kinematics or activity sequence or other data to a segment of an analysis module self-learning module, communications module as described herein or some other module for processing locomotion kinematics or activity sequence data. For example, in an embodiment consistent with system 100, an instruction flag can be received by communications module 145 in preparation for sending data to server 160 or by self-learning module 130 in preparation for updating or skipping an update of a reference module template in reference model 125.
Once locomotion type has been identified, the IMU-based speed calculation model, corresponding to the locomotion, can be updated based on reference GPS data. In fact, this scheme is capable of personalizing speed estimation based on kinematics of user.
In preferred embodiments, the reference model includes templates for walking and running as discussed in connection with
Referring now to
System 1200 also includes GPS module 1255 which generates GPS data as described herein. GPS data is received by adaptive algorithm 1265 to adjust a locomotion reference model template. In system 1200, one of run speed model 1230 or walk speed model 1235 can be updated based on criteria as described herein. In other embodiments, additional models can be included and can be updated. Locomotion time 1240, distance 1245, locomotion speed 1250, and GPS data can be stored for later retrieval for reporting or other purposes.
Any and all references to publications or other documents, including but not limited to, patents, patent applications, articles, webpages, books, etc., presented in the present application, are herein incorporated by reference in their entirety.
Example embodiments of the devices, systems and methods have been described herein. As noted elsewhere, these embodiments have been described for illustrative purposes only and are not limiting. Other embodiments are possible and are covered by the disclosure, which will be apparent from the teachings contained herein. Thus, the breadth and scope of the disclosure should not be limited by any of the above-described embodiments but should be defined only in accordance with claims supported by the present disclosure and their equivalents. Moreover, embodiments of the subject disclosure may include methods, systems and apparatuses which may further include any and all elements from any other disclosed methods, systems, and apparatuses, including any and all elements corresponding to target particle separation, focusing/concentration. In other words, elements from one or another disclosed embodiment may be interchangeable with elements from other disclosed embodiments. In addition, one or more features/elements of disclosed embodiments may be removed and still result in patentable subject matter (and thus, resulting in yet more embodiments of the subject disclosure). Correspondingly, some embodiments of the present disclosure may be patentably distinct from one and/or another reference by specifically lacking one or more elements/features. In other words, claims to certain embodiments may contain negative limitation to specifically exclude one or more elements/features resulting in embodiments which are patentably distinct from the prior art which include such features/elements.
| Number | Date | Country | |
|---|---|---|---|
| 62686287 | Jun 2018 | US |
