Patent Application
20130179112
Personal navigation systems have received attention from both the industrial and academic research communities in recent years. Numerous target applications have been proposed, including localization for members of a team of firefighters, first responders, and soldiers. In these applications, the safety and efficiency of the entire team depends on the availability of accurate position and orientation (pose) estimates of each team member. While the team is operating within the coverage area of GPS satellites, each person's pose can be reliably estimated. However, a far more challenging scenario occurs when the team is inside a building, in an urban canyon, or under a forest canopy. In these cases, GPS-based global localization is not sufficiently accurate or may be completely unavailable, and pose estimation must be accomplished through secondary means. One popular approach is to equip each person with a body-mounted strap-down inertial measurement unit (IMU) typically comprising three accelerometers and three gyroscopes in orthogonal triads, which measure the person's motion. To mitigate the drift errors in strap-down inertial navigation, conventional systems typically include aiding sensors, such as a camera or laser scanner which sense the color, texture, or geometry of the environment. Each person's pose can then be estimated individually by fusing the available sensing information.
In these personal navigation systems, the number of gait templates stored in a motion dictionary is based on the number of motion classes (e.g., walking, running, crawling), the number of frequencies (e.g., slow, medium, fast), and the number of phases (e.g., the number of full steps, such as, from right-foot-down position to the next right-foot-down position). Prior art systems typically require about 30 phases. A personal navigation system, which stores five motion classes at five frequencies for ten people, will store 7,500 gait templates in a motion dictionary.
The present application relates to a method to accurately detect true peaks and true valleys in a real-time incoming signal. The method includes segmenting the real-time incoming signal into short-time intervals; determining an initial estimated frequency by fast Fourier transforming data in the short-time intervals, setting a sliding window width based on the initial estimated frequency, determining at least one peak data element or valley data element based on analysis of the real-time incoming signal within a first sliding window; and determining at least one peak data element or valley data element based on analysis of the real-time incoming signal within a second sliding window. A first portion of the second sliding window overlaps a second portion of the first sliding window.
Understanding that the drawings depict only exemplary embodiments and are not therefore to be considered limiting in scope, the exemplary embodiments will be described with additional specificity and detail through the use of the accompanying drawings, in which:
In accordance with common practice, the various described features are not drawn to scale but are drawn to emphasize specific features relevant to the exemplary embodiments.
In the following detailed description, reference is made to the accompanying drawings that form a part hereof, and in which is shown by way of illustration specific illustrative embodiments. However, it is to be understood that other embodiments may be utilized and that logical, mechanical, and electrical changes may be made. Furthermore, the method presented in the drawing figures and the specification is not to be construed as limiting the order in which the individual acts may be performed. The following detailed description is, therefore, not to be taken in a limiting sense.
In order to reduce the size of gait templates (i.e., the number of phases in gait templates) in a motion dictionary for a personal navigation system, a robust detection of peaks and valleys in a real-time incoming data stream from an inertial measurement unit (IMU) on the user of the personal navigation system is required. As defined herein, robust detection (determination) of peaks and valleys is a correct detection (determination) of the true (global) peaks and the true (global) valleys in the real-time incoming data stream. The terms “true peak” and “true peak data element” are used interchangeably herein. The terms “true valley” and “true valley data element” are used interchangeably herein.
The terms “motion dictionary”, “motion classification dictionary”, and “dictionary” are used interchangeably herein.
As used herein, the terms “gait” and “gait mode” refer to the pattern of movement of the user's limbs while moving from one location to another. Thus, each gait mode is comprised of a pattern of repetitive motions. The cycle of repetitive motion is generally referred to herein as a “step” although it may refer to motions of parts of the body other than the legs, such as the arms or torso in a military crawl. Exemplary gait modes include, but are not limited to, level walking, stair ascending/descending, side-shuffle walking, duck (firefighter) walking, hand-and-knee crawling, military (elbow) crawling, jogging, running, and sprinting. In addition, in some embodiments, a “no-model” hypothesis is stored in the dictionary. The “no-model” hypothesis is a default hypothesis that covers the instances when the executed motion, as represented by the wavelet, does not match any of the stored gait templates.
The terms “frequency” and “gait frequency” refer to how quickly the corresponding motion is repeated. For example, the frequency of level walking refers to how quickly each step is repeated.
The terms “phase” and “gait phase” refer to the shift in the starting position of the repetitive motion. The gait phase in the gait template defines which foot is starting on the ground. In one implementation of this embodiment, a 0 degree phase template is generated from the time when the left foot is on the ground and the right foot is moving in the direction of movement. The 0 degree phase is completed when the left foot is back on the ground. Thus, the 0 degree phase extends from a first true peak (or first true valley) to the third true peak (or third true valley) in the signal received from the IMU. Thus, the 0 degree phase extends for two complete periods (2T). In this case, the 180 degree phase template is generated from the time when the right foot is on the ground. The 180 degree phase template extends from a second true peak (or second true valley) to the fourth true peak (or fourth true valley) in the signal received from the IMU. Thus, the 180 degree phase extends for two complete periods (2T). In another implementation of this embodiment, the 0 degree phase template is generated from the time when the right foot is on the ground and the 180 degree phase template is generated from the time when the left foot is on the ground.
The terms “template” and “gait template” are used interchangeably herein. In one implementation of this embodiment, each template corresponds to one of the six channels from six sensors in the IMU. The gait templates associated with the plurality of channels for a given gait mode are referred to herein as a set of associated gait templates. In one implementation of this embodiment, each set of associated gait templates corresponds to three channels from three gyroscopes in the IMU and the three channels from three accelerometers in the IMU. In another implementation of this embodiment, each set of associated gait templates corresponds to three channels from three gyroscopes in the IMU. In yet another implementation of this embodiment, each set of associated gait templates corresponds to three channels from three accelerometers in the IMU.
As noted above, the prior art methods to classify motion compare the received data to a motion class dictionary, which contains wavelets (gait templates) with thirty phase shifts (e.g., 30 full right-foot-down to right-foot-down steps) for each motion type at each frequency. The thirty phases for each gait and gait frequency require a large amount of processing time and template storage in the motion dictionary.
The methods and apparatuses described here improve the robustness of a motion classifier (e.g., processing unit) in a personal navigation system. Since the robust motion classifier correctly identifies the true peaks and true valleys, the number of phases required in each gait template is reduced from thirty phases to four phases or two phases. Thus, the methods described herein only require either four phases (two phases for a 0 degree phase and two phases for a 180 degree phase) in the gait templates or two phases (two phases for a 0 degree phase or two phases for a 180 degree phase) in the gait templates. The embodiments in which one or more of the gait templates include only two phases for a 0 degree phase or include only two phases for a 180 degree phase are used for those gait modes in which only the peaks or only the valleys in the signal output from the IMU are clearly distinguishable.
A motion classification system with four (or two) phases for each gait and gait frequency requires a reduced amount of template storage in the motion dictionary and a reduced amount of processing time from the prior art motion classification systems that require about thirty phases for each gait and gait frequency. Reducing the size of gait templates from 30 phases to four or two phases also permits use of templates for more users in a single motion dictionary.
The IMU 102 provides the inertial motion data to the processing unit 104. In some embodiments of the personal navigation system 100, the IMU is implemented as a Micro-electro-mechanical system (MEMS) based inertial measurement unit, such as a Honeywell BG1930 MEMS inertial measurement unit.
The processing unit 104 includes segmentation functionality 123, wavelet motion classification functionality 106, Kalman filter functionality 108, and inertial navigation functionality 110. The segmentation functionality 123 periodically segments one selected channel of real-time incoming signal from the IMU 102 into short-time intervals and finds the true peaks and true valleys in each of the short-time intervals. The channel data is segmented to represent a single step for gait modes related to walking or running. The channel data can be segmented based on one or more of a plurality of techniques, such as peak detection, valley detection, or zero crossing.
The wavelet motion classification functionality 106 compares the true peaks and true valleys in each of the short-time intervals for the selected channel to a plurality of gait templates 116 stored in the motion dictionary 117 in the memory 114. The wavelet motion classification functionality 106 also compares the non-segmented data from the other channels (e.g., the other five channels from the six channels of data received from the IMU 102) to the gait templates 116 in the set of associated gait templates. The wavelet motion classification functionality 106 identifies the template which most closely matches the at least one wavelet obtained from the received inertial motion data based on a statistical analysis. Each set of associated gait templates in the gait templates 116 corresponds to a particular user gait, frequency, and phase (either a zero degree phase or a 180 degree phase).
The inertial navigation functionality 110 calculates a navigation solution including heading, speed, position, etc., based on the inertial motion data received from the IMU 102 using techniques known to one of skill in the art.
As shown in
The duration of a sliding window 300 is referred to herein as the width W of the sliding window 300 and is less than the period T of the real-time incoming signal 250. The width W of the sliding window 300 is less than the duration of the short-time interval 240. In the exemplary embodiment shown in
The “inertial motion data received in real-time” is also referred to herein as “real-time incoming signal 250”. The processing unit 104 segments one channel 403 of the one or more channels (e.g., three accelerometer channels 401-403) of inertial motion data received in real-time based on an estimated initial frequency fest of the short-time interval 240. The estimated initial frequency fest is the inverse of the period T of the real-time incoming signal 250 (
During the segmentation process, the processing unit 104 identifies the true peaks 381-387 and true valleys 371-377 in the data received in real-time within the plurality of sliding windows 301-305 (
The segmenting functionality 123 in the processing unit 104 also executes (implements) width logic 120 in memory 114 on the data elements 330 as they are received in real-time within the plurality of sliding windows 301-305 to remove a later-received peak 336 that is offset from an earlier-received peak 382 by less than a pre-selected percentage (e.g., 50%) of a calculated mean difference. Likewise, the segmenting functionality 123 implements the width logic 120 on the data elements 330 as they are received in real-time within the plurality of sliding windows 301-305 to remove a later-received valley 330 that is offset from an earlier-received valley 373 by less than the pre-selected percentage (e.g., 50%) of the calculated mean difference. In this manner, during the segmentation process, the portion of the real time incoming signal 250 shown in
The wavelet motion classification functionality 106 in the processing unit 104 selects one of a plurality of gaits 116 as the user's gait based on the segmentation of the one of the one or more channels and the identification of the true peaks 381-387 and true valleys 371-377 in a short-time interval 240. In the exemplary embodiment shown in
The wavelet motion classification functionality 106 identifies the gait of the user to which the personal navigation system 100 is strapped. The gait for each short-time interval 240 is identified based on the segmented signal 251 (
Similarly, the processing unit 104 executes inertial navigation instructions 111 stored on a memory 114 in implementing the inertial navigation functionality 110. The processing unit 104 updates the navigation solution based, at least in part, on the distance-traveled estimate.
The Kalman filter functionality 108 outputs Kalman filter corrections to the inertial navigation functionality 110. In one implementation, the Kalman filter functionality 108 also pre-processes and/or pre-filters the input to the Kalman filter prior to calculating the corrections. In addition, in some embodiments, corrections to the distance-traveled estimate are provided from the Kalman filter functionality 108 to the wavelet motion classification functionality 106.
In some embodiments, the system 100 also includes one or more aiding sensors 112 which provide additional motion data to the Kalman filter functionality 108. In this embodiment, the Kalman filter functionality 108 calculates corrections to the navigation solution based on motion data received from the aiding sensors 112. In the exemplary embodiment shown in
The processing unit 104 generates the user gait templates 116 in the motion dictionary 116 during a training process. For an indicated user gait mode, the processing unit 104 overlays and time averages the segmented data 251 to create a selected channel gait template for each of the short-time intervals 240, divides the data into two phases correlated with the pose (e.g., right foot down) of the user, and performs a waveform transformation to create the gait template. The processing unit 104 associates the channels for a given gait mode with each other to form the set of associated gait templates.
The processing unit 104 includes or functions with software programs, firmware or other computer readable instructions in a storage medium for carrying out various methods, process tasks, calculations, and control functions, used in the implementing the functionality described above.
It is to be understood that data from the three gyroscopes 48 in three gyroscope channels is similarly analyzed by the segmentation process to identify the true peak for each phase. It is also to be understood that the data from the IMU channels is similarly analyzed by the segmentation process to identify the true valley for each phase. The segmented gyroscope data is part of the set of associated gait templates. The gait templates in the motion dictionary 117 include data channels that are analyzed by the segmentation process during a training process to identify the true peaks and true valley for one of: four phases (two 180° phases and two 0° phases); two 180° phases; or two 0° phases.
The structure and function of the personal navigation system 100 is described with reference to
At block 502, the processing unit 104 segments the real-time incoming signal 250 into short-time intervals 240. In one implementation of this embodiment, the short-time interval 240 is a 3 second interval.
At block 504, the processing unit 104 determines an initial estimated frequency fest by fast Fourier transforming the data (e.g., data elements 330) in the short-time intervals 240. Each short-time interval 240 is sequentially processed as it is received at the processing unit 104. In another implementation of this embodiment, the processing unit 104 determines the estimated frequency fest by fast Fourier transforming the data 330 in the short-time interval 240 with a weighted fast Fourier transform (FFT). The FFT is weighted using a weighting function since the signal may contain multiple frequency components. The lower frequency components correspond to the major peaks and valleys of the real-time incoming signal 250, so the lower frequencies are weighted more than the higher frequencies.
At block 506, the processing unit 104 sets a sliding window width W based on the estimated frequency fest. In the exemplary real-time incoming signal 250 shown in
At block 508, the processing unit 104 determines at least one peak data element 382 or valley data element 373 based on analysis of the real-time incoming signal 250 within a first sliding window 301. The processing unit 104 determines if a selected-data element 330 within the sliding window 301 is a maximum data element, such as maximum or peak data elements 382, 383, 335, and 336 shown in
If the selected-data element 330 is a peak data element (e.g., data element 382 as shown in
In the exemplary real-time incoming signal 250 shown in
The processing unit 104 operates is a similar manner to determine the valleys in a first sliding window 301. The processing unit 104 determines if a selected-data element 330 within the sliding window 301 is a minimum data element, such as minimum or valley data elements 372, 333, 373, 334, and 374 shown in
Each short-time interval 240 extends long enough in time for a plurality of sliding windows 301-305 to be processed sequentially. At block 510, the processing unit 104 determines at least one peak data element 382 or valley data element 373 based on analysis of the real-time incoming signal 250 within a second sliding window 302 (
The processing unit 104 continues to process for peaks and valleys within the plurality of sliding windows 301-305 in the short-time interval 240. Once the data in the short-time interval 240 has been analyzed in this manner, the processing unit 104 does post processing to remove outliers from the detected peak data elements and the detected valley data elements in the short-time interval 240. For example, as shown in
To summarize, there are three phases to identifying the phase of the incoming wave. First, the incoming real-time incoming signal 250 is broken up into short-time intervals 340 (e.g., three second intervals). A weighted fast Fourier transform (FFT) is performed to find the initial frequency fest of the real-time incoming signal 250. Second, the initial estimated frequency fest is used to calculate a sliding window width W. Once the sliding window width W is set, the peak and valley detection is actually performed on 2 cycles. The sliding window is defined as less than ½ of the complete phase (0° or 180° phase) of the complete first step (e.g., right-foot-down to right-foot-down) and is only ‘forward looking’ in time.
Third, a heuristic step for post processing is used to remove any false alarms of peak or valley. The processing unit 104 removes the first peak 381 or the first valley 370 in each short-time interval 240 in order to eliminate peaks lying on the short-time-interval boundaries. The processing unit 104 also implements (executes) the height logic 121 and the width logic 120 to remove outliers.
The processing unit 104 implements height logic on the segmented channel of inertial motion data received in real-time within the short-time interval 240 to remove peaks and valleys (peak data elements and valley data elements) that lie outside a selected range. Specifically, the height logic 121 is executed by the processing unit 104 to calculate a mean deviation and a standard deviation of height for the plurality of peak data elements 335, 382, 336, and 383 and the plurality of valley data elements 372, 333, 373, and 334 detected in the short-time interval 240. Then, the height logic 121 is executed by the processing unit 104 to remove peak data elements 335-336 that have respective height values outside a pre-selected-maximum range 280 and to remove valley data elements 333-334 that have respective height values outside that lie outside a pre-selected-minimum range 281 (
The processing unit 104 implements width logic 120 on the segmented channel of inertial motion data received in real-time within the short-time interval 240 to remove a later-received peak (e.g., data element 336 shown in
After the height logic 121 and the width logic 120 are executed by the processing unit 104, the remaining plurality of peak data elements 381-387 are true peaks 381-387 and remaining plurality of valley data elements 371-377 are true valleys 371-377.
In one implementation of this embodiment, the short-time interval is 3 seconds. In another implementation of this embodiment, the short-time interval is 3 seconds and the method is performed every 1.5 seconds, to produce 1.5 second overlaps between successive data sets. In yet another implementation of this embodiment, the short-time interval is about 10 seconds. This longer short-time interval is useful when the user of the personal navigation system 100 is an elderly person, who moves slowly.
The personal navigation system 100 shown in
The training phase to generate the gait templates for a user requires the user to strap on the person navigation system 100 and to move in accordance with the various gait modes to be stored in the motion dictionary 117. During the training phase, the inertial measurement unit 102 is configured to output a plurality of channels 400 of inertial motion data while a user moves in a particular gait mode, with the gait having a gait frequency. The user (or other technician) sends an input to the person navigation system 100 to indicate which gait mode is being generated. For example, the user indicates the gait template for a fast run is being prepared and then the user runs fast while the processing unit 104 collects and analyzes the data from the IMU 102.
Specifically, the training phase to generate the gait templates for the user wearing the person navigation system 100 is as follows: receive a user input indicating a user gait at the processing unit 104; receive a plurality of channels of real-time signals from an inertial measurement unit (IMU) 102 for the indicated user gait at the processing unit 104 while the user is moving according to the user gait (gait mode); segment the plurality of channels of the real-time signals into short-time intervals 240; identify true peaks and true valleys in the segmented plurality of channels of the real-time signals for the short-time intervals 240; overlay and time average the signals for the indicated user gait for a plurality of short-time intervals 240; divide the data (e.g., the overlayed and time averaged signals) into a 0 degree phase and a 180 degree phase at the processing unit 104; and transform the data to create a 0-degree-phase-gait template for the indicated user gait for the plurality of channels and to create a 180-degree-phase-gait template for the indicated user gait for the plurality of channels; associate the transformed data for the 0-degree-phase-gait template for the plurality of channels to each other to form a O-degree-phase set of associated gait templates correlated with the indicated user gait; and associate the transformed data for the 180-degree-phase-gait template for the plurality of channels to each other in order to form a 180-degree-phase set of associated gait templates correlated with the indicated user gait. In one implementation of this embodiment, the processing unit that generates the gait templates 116 is not the processing unit 104 in the person navigation system 100. In that case, the processing unit that generated the gait templates 116 downloads the gait templates into the motion dictionary 117 prior to use of the person navigation system 100.
At block 602, the processing unit 104 receives a plurality of channels of real-time signals from an inertial measurement unit (IMU) 102 and implements segmentation functionality 123. During the operational phase, one channel (a selected channel) of the plurality of channels from the IMU 102 is fed into the segmentation functionality 123. In one implementation of this embodiment, the processing unit 104 receives six channels of the real-time signals from the inertial measurement unit 102.
At block 604, the selected channel of the plurality of channels of real-time signals received from the inertial measurement unit is segmented into short-time intervals 240. As defined herein, the selected channel is that channel receiving data from the accelerometer and gyroscope that is sensing in the direction approximately parallel to the gravitation force of the earth. For example, if the user is walking (standing vertically), the vertical channel selected for segmentation is provided by an exemplary Z-axis sensor. Later, if the user has gone from walking gait mode to a crawling gait mode, the Z-axis sensor is approximately parallel to the earth's surface. In that case, the selected channel that is selected for segmentation is the channel that receives data from the accelerometer and/or gyroscope that is sensing in the direction most parallel to the gravitation force of the earth, e.g., an X-axis sensor or a Y-axis sensor. Technology to determine which of the accelerometers and gyroscopes are sensitive to the direction approximately parallel to the gravitation force of the earth is known to one skilled in the art.
The selected channel (e.g., channel 403 shown in
At block 606, the processing unit 104 analyzes data in the selected channel within a plurality of overlapping sliding windows 300 within the short-time intervals 240 as described above. During this analysis, the processing unit 104 determines peaks and valleys within each of a plurality of sliding windows as described above with reference to
At block 608, the processing unit 104 identifies true peaks (global peaks) and true valleys (global valleys) in the short-time intervals 240 using the sliding windows 300 of the respective short-time intervals 240 as described above. The processing unit 104 implements post-processing on the determined peaks and the determined valleys within the short-time intervals to eliminate false peaks and false valleys in the respective short-time interval.
At block 610, the processing unit 104 correlates the true peaks and true valleys in respective short-time intervals 240 to gait templates 116 in a motion dictionary 117 for the selected channel that was created during a training process. The gait templates comprising the motion dictionary include four phases or two phases
At block 612, the processing unit 104 correlates non-selected channels of the plurality of channels to the gait templates 116 in the motion dictionary 117. For example, if there are six channels of data from the IMU, the five non-selected channels of the six channels are correlated to gait templates. The processing unit 104 generates a composite score for the six channels based on: 1) correlating the true peaks and true valleys in the selected channel to a gait template in the motion dictionary; and 2) correlating the five non-selected channels to gait templates in the motion dictionary. Typically, the gait templates for six channels are associated with each other in the set of associated gait templates. Thus, when the true peaks and true valleys in the selected channel are matched to a gait template for the selected channel in the dictionary, the five non-selected channels are then compared to the five channels in the set of associated gait templates with the gait template matched for the selected channel. A single selected channel is used to segment the data, but all six IMU channels are compared to the dictionary that was created during a training process. In one implementation of this embodiment, the processing unit 104 periodically implements a nearest-neighbor algorithm to choose a motion type for each short-time interval 240 in the received real-time signals.
In this manner, the phase of the real-time incoming signal 250 is identified and the gait and the gait frequency of the user are identified and a motion classification is quantified by the person navigation system 100 for each of the plurality of short-time intervals 240. In one implementation of this embodiment, the quantifying occurs every half of the period T of the short-time interval.
These instructions are typically stored on any appropriate computer readable medium used for storage of computer readable instructions or data structures. The computer readable medium can be implemented as any available media that can be accessed by a general purpose or special purpose computer or processor, or any programmable logic device. Suitable processor-readable media may include storage or memory media such as magnetic or optical media. For example, storage or memory media may include conventional hard disks, Compact Disk-Read Only Memory (CD-ROM), volatile or non-volatile media such as Random Access Memory (RAM) (including, but not limited to, Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate (DDR) RAM, RAMBUS Dynamic RAM (RDRAM), Static RAM (SRAM), etc.), Read Only Memory (ROM), Electrically Erasable Programmable ROM (EEPROM), and flash memory, etc. Suitable processor-readable media may also include transmission media such as electrical, electromagnetic, or digital signals, conveyed via a communication medium such as a network and/or a wireless link.
Moreover, although the processing unit 104 and memory 114 are shown as separate elements in
Although specific embodiments have been illustrated and described herein, it will be appreciated by those of ordinary skill in the art that any arrangement, which is calculated to achieve the same purpose, may be substituted for the specific embodiments shown. Therefore, it is manifestly intended that this invention be limited only by the claims and the equivalents thereof.
