The described embodiments relate generally to motion detecting. More particularly, the described embodiments relate to a method and apparatus of motion detecting and monitoring of energy expended by an individual.
There is an increasing need for remote monitoring of individuals, animals and inanimate objects in their daily or natural habitats. Many seniors live independently and need to have their safety and wellness tracked. A large percentage of society is fitness conscious, and desire to have, for example, workouts and exercise regimen assessed. Public safety officers, such as police and firemen, encounter hazardous situations on a frequent basis, and need their movements, activities and location to be mapped out precisely.
The value in such knowledge is enormous. Physicians, for example, like to know their patients sleeping patterns so they can treat sleep disorders. A senior living independently wants peace of mind that if he has a fall it will be detected automatically and help summoned immediately. A fitness enthusiast wants to track her daily workout routine, capturing the various types of exercises, intensity, duration and caloric burn. A caregiver wants to know that her father is living an active, healthy lifestyle and taking his daily walks. The police would like to know instantly when someone has been involved in a car collision, and whether the victims are moving or not.
Existing products for the detection of animate and inanimate motions are simplistic in nature, and incapable of interpreting anything more than simple atomic movements, such as jolts, changes in orientation and the like. It is not possible to draw reliable conclusions about human behavior from these simplistic assessments.
It is desirable to have an apparatuses and methods that can accurately monitor activity and energy expended by an individual.
An embodiment includes a method of monitoring energy expended by an individual. The method includes sensing, by a motion sensor, motion of the individual, identifying a plurality of activities performed by the individual over a period of time based on the identified motions, estimating, by a processor, energy expended by the individual for each of the plurality of the plurality of identified activities, and estimating energy expended by the individual by summing the estimated energy expended for each of the plurality of activities.
Another embodiment includes an apparatus for monitoring energy expended by an individual. The apparatus includes at least one acceleration sensing device sensing acceleration of the individual, an artificial neural network receiving the sensed acceleration, accessing stored coefficients, and identifying at least one activity of the individual. A controller is operative to estimate energy expended by the individual for each of the plurality of identified activities, and estimate energy expended by the individual by summing the estimated energy expended for each of the plurality of activities.
Other aspects and advantages of the described embodiments will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the described embodiments.
The monitoring of human physical activities generally falls into three categories: safety, daily lifestyle, and fitness. Physical activity can be defined as body movement that results in energy expenditure. Physical activity is a complex behavior as it can include sports and non-sports activities. Sports are often planned, structured, and repetitive, with the objective of improving or maintaining physical fitness, whereas non-sports activities can be subdivided into different categories such as occupational, leisure-time, and household activities but also personal care and transportation. Clearly, physical activity has an impact on energy expenditure. Additionally, energy expenditure is dependent on body size and body composition of the individual.
The monitoring of physical activity and the resulting expended energy provides a useful method for determining an activity level of the life style of the individual. Estimating the energy expenditure over periods of time can assist the individual with managing caloric expenditure, which can be used to aid the individual with caloric consumption management. For embodiments, the caloric consumption is computed based on the metabolic equivalent (MET)×Mass (Weights in kg) of the person×time duration in minutes, wherein the MET value may be derived with reasonable accuracy based on the activity identified, and some of its derivable attributes from the acceleration data (like walking steps or cadence during biking) and augmented with the supplementary data like the slope of the path traversed during walking or resistance settings of the equipment.
An embodiment of the device 100 includes sensors (such as, accelerometers) that detect motion of the object. One embodiment of the sensors includes accelerometers 112, 114, 116 that can sense, for example, acceleration of the device (and therefore, the individual) in X, Y and Z directional orientations. It is to be understood that other types of motion detection sensors can alternatively be used.
For this embodiment, an analog to digital converter (ADC) 120 digitizes analog accelerometer signals. The digitized signals are received by an artificial neural network (ANN) based discriminator 130, which identifies activities of the individual based on the sensed motion, and stored coefficients 140. A controller 150 computes the calorific burn based on the duration of the activity as well as based on certain additional information like number of steps (in the case of walk).
An embodiment of the ANN based discriminator 130 includes a neural network that includes an input layer, one-or-more hidden layers, and an output layer.
Each node of the input layer is connected to each node of the hidden layer through a link that includes an associated weight. Each node of each layer is only connected to nodes of the next immediate layer. The outputs of the nodes in the output layer provide a decision or recognition in the form of the output vector. Each node determines a weighted sum of the individual inputs that are connected to it from the previous layer. This sum is then fed to an activation function or a sigmoid function (which is within the node, and typically includes an S shaped relation between input and the output). For and embodiment, the activation function generates an output in the range of 0 to 1 from the weighted sum.
The learning capability of the neural network is essentially captured in the inter-connecting weights—which form the coefficients of the neural network. In the case of a neural network of one hidden layer, the learning capability is enabled by adaptation of the coefficients (weights) of an input matrix and an output matrix. During the training process, then ANN is presented with a set of input and output vectors. The coefficients of ANN are adjusted incrementally, so as to match the output vector with the expected output vector.
In execution mode, a trained ANN is able to recognize the output vector from the input vector. This approach is particularly useful when it is not possible to arrive at a firm mathematical relationship between input vectors and output vectors. This is typically the case when the outputs are defined by randomly shaped multiple potentially overlapping clusters within the multidimensional space of input vectors. Additionally, the ANNs may possess an ability to recognize with reasonable accuracy, the correct output corresponding to an unseen input vector.
An embodiment includes filtering the accelerometer signals before attempting to match the signatures. Additionally, the matching process can be made simpler by reducing the possible signature matches.
An embodiment includes identifying a previous human activity context. That is, for example, by knowing that the previous human activity was walking, certain signatures can intelligently be eliminated from the possible matches of the present activity that occurs subsequent to the previous human activity (walking)
An embodiment includes additionally reducing the number of possible signature matches by performing a time-domain analysis on the accelerometer signal. The time-domain analysis can be used to identify a transient or steady-state signature of the accelerometer signal. That is, for example, a walk may have a prominent steady-state signature, whereas a fall may have a prominent transient signature. Identification of the transient or steady-state signature of the accelerometer signal can further reduce or eliminate the number of possible signature matches, and therefore, make the task of matching the accelerometer signature with a signature within the library of signature simpler, and easier to accomplish. More specifically, the required signal processing is simpler, easier, and requires less computing power.
Activity Identification
For an embodiment, identifying a plurality of activities includes identifying a quasi-periodic activity. For embodiments, this identification includes a training mode wherein motion is sensed while the individual performs a known activity. For embodiments, the training mode includes generating ANN coefficients that are specific to a group of individuals sharing a common body profile and/or age group and/or activity level. The quasi-periodic activity can be identified based upon sensing motion of the individual, and the trained coefficient.
For an embodiment, the ANN coefficients are repeatedly tested in a random order. The ANN is provided with a pair of input/output vectors. More specifically, a feature input vector and a corresponding desired output vector (based on the activity tagging) are provided to the ANN. A training algorithm adjusts the coefficients of the ANN in such a manner that the overall difference between actual vector outputs and the desired vector outputs are minimized. This procedure is repeated until the coefficients of ANN converge to the desired training set (a trained set of coefficients). In such a condition, for most of the feature input vectors, the output vector produced by the ANN matches with the desired output.
Embodiments further include an execution mode in which the trained ANN coefficients are used to indentify motions and activities. That is, raw data from, for example, the accelerometer is provided to the statistical processor which generates a feature input vector. The generated feature input vector is provided to the trained ANN, which indentifies the motion and/or activity by matching the feature input vector with a known output vector.
In some situations, the activity cannot be identified by the stored ANN coefficients. Accordingly, for some embodiments, if the activity cannot be identified based on the ANN coefficients, the unidentified activity is tagged as a new activity, and the ANN coefficients are updated to support the identification of that activity in the future. For this change to take place, the training activity may be performed by augmenting a new pair that includes a feature vector and a correspondingly tagged new output vector that correspond to the newly identified activity.
For embodiments, identifying quasi-periodic activity further includes generating an acceleration signature based on sensed acceleration of the individual, and identifying the type of motion of the individual based on discrimination of the acceleration signature using a training set that was generated during the training mode.
For an embodiment, identifying quasi-periodic activity includes generating an acceleration signature based on sensed acceleration of the individual, matching the acceleration signature with at least one of a plurality of stored acceleration signatures, wherein each stored acceleration signatures corresponds with a type of motion, and identifying the type of motion of the object based on the matching of the acceleration signature with a stored acceleration signature. The type of motion can include, for example, at least one of atomic motion, elemental motion and macro-motion. For an embodiment, the stored acceleration signatures are stored in a common library and a specific library, and matching the acceleration signature comprises matching the acceleration signature with stored acceleration signatures of the common library, and then matching the acceleration signature with stored acceleration signatures of the specific library.
For embodiments, identifying a plurality of activities includes identifying at least one non-quasi-periodic activity, including sensing an intensity and direction of the non-quasi-periodic activity.
For embodiments, estimating energy expended by the individual for each of the plurality of identified activities includes estimating the energy expended based on at least one of a quasi-periodicity of the activity, an intensity of the activity and a deviation of an acceleration magnitude from its moving average value.
For embodiments, estimating the energy expended by the individual for each of the plurality of identified activities includes estimating energy expended based on at least one of static orientation of the individual, a change of orientation of the individual, a time taken to change the orientation of the individual, and a number of times the orientation changes within a given amount of time. For embodiments, estimating the energy expended by the individual for each of the plurality of activities includes estimating energy expended based on statistical properties (mean deviation, average, etc.) of an acceleration vector along a plurality of axes of orientation.
Embodiments include calculating a caloric burn of the individual based on the estimated energy expended. Embodiments further include estimating caloric intake of the individual. Further, the caloric burn of the individual can be compared with the caloric intake of the individual, and estimating a weight change per unit time. Embodiments further include estimating a period required to meet a specific weight change target based on a weight change per unit time.
An embodiment includes a quasi-periodic activity through execution of a training mode wherein motion is sensed while the individual performs a known activity. Further, a personal profile is generated for the individual, wherein the profile comprises a plurality of parameters that are personalized to the individual. Embodiments include identifying the quasi-periodic activity based upon sensing motion of the individual, and profile. If the activity cannot be identified based on the profile, the unidentified activity is tagged as a new activity, and the profile is updated to include corresponding parameters that are personalized to the individual.
If the activity could not be identified (step 640) a step 650 includes storing the newly detected activity, a step 660 includes tagging the new activity (feature vector), and a step 670 includes generating new coefficients for the artificial neural network.
If determined to be non-rhythmic, a step 822 is executed that includes determining (detecting) a level of intensity of the activity. This can be accomplished, for example, by sensing the intensity of the sensed acceleration of the individual. If determined to be non-rhythmic, and of high intensity (832), it can be deduced, for example, that the activity is a form of aerobics. If determined to be non-rhythmic, and of medium intensity (834), it can be deduced, for example, that the activity is a form of moderate aerobics, sports or some other identified daily activities. If determined to be non-rhythmic, and of low intensity (836), it can be deduced, for example, that the activity is a form of yoga.
If determined to be quasi-rhythmic, a step 824 is executed that includes determining (detecting) an intensity level of the activity. This can be accomplished, for example, by sensing the intensity of the sensed acceleration of the individual. If determined to be rhythmic, and of high intensity (842), it can be deduced, for example, that the activity is a form of jogging or running If determined to be rhythmic, and of medium intensity (844), it can be deduced, for example, that the activity is a form of walking, spinning or elliptical spinning If determined to be rhythmic, and of low intensity (846), it can be deduced, for example, that the activity is sedentary or resting.
For embodiments, the training mode includes a supervised mode in which training coefficients are generated. The system is trained, for example, with variety of inputs from individuals with different body profile and age groups, while the individual is performing different pre-defined activities, with sensor (such as the previously described accelerometers) connected to different pre-determined body locations.
The raw data generated by the sensors can be recorded and data sets tagged (912) with an activity code, speed/vigor level code, and optionally, with sensor location, and specific details about the person's profile or the details of any fitness equipment settings.
For an embodiment, accelerometer 910 outputs are obtained as raw data which is fed to an instantaneous parameters generator 912. The instantaneous data vectors generated by the instantaneous parameters generator 912 are fed to statistical processor 914. For embodiments, the statistical processor 914 acts on a frame of consecutive N points and provides frame-feature-vectors as outputs.
For an embodiment, the feature vector comprises of a set of numbers that describes some properties of the frame. For example, the common statistical properties like average, standard-deviation, maximum value, form a description of the larger number set of N consecutive points of the frame. Feature set is chosen so as to create maximum differentiation between the different activities the person may perform.
For an embodiment, outputs of an entire training data set are collected together and the overall range of minimum-maximum (min-max range) values for each feature is computed. The min-max range is used to scale each feature within the vector into a fraction, for example, between 0.0 and 1.0. The min value maps to 0.0 while the max value maps to 1.0 and any value in between them maps to a proportional fraction in between. Scaling makes the values more compatible to the expected inputs of the ANN while retaining all the relevant contents of the data.
Training scripts extract the tagging details and feed those as output vectors to fitness activity discriminator. Scaled input feature vectors are also fed to a fitness activity discriminator 930.
The ANN (of the fitness activity discriminator 930) under training mode is repeatedly presented with, in a random sequence, one item at a time, from the training set that comprises an input feature vector, and the corresponding desired output vector. With sufficient iterations, the ANN is trained with entire set of available training data to produce trained ANN coefficients 960.
A static orientation detector 1022 takes acceleration readings of individual axes and low-pass filters those to obtain a smooth static orientation. That is, the static orientation detector 1022 detects the prominent axis as well as the extent of relative orientation of the 3 axes.
A multi-axis dynamic power detector 1032 takes individual acceleration components as input and captures a relative energy content by determining the mean-deviation of most recent consecutive M data points. The multi-axis dynamic power detector 1032 reports the most dynamically active axis over a given duration.
Activity selectors 1020, 1040 provide two levels of decision making to identify an activity. At the first level, activity selector 1020 chooses one activity out of the activities like Run/Jog from time-series-comparator 1010 and other activities detected by Fitness activity discriminator ANN 930, and the multi-axis activity detector 1034. The activity selector 1020 uses a rule base to choose one of the several activities that may be reported simultaneously by different detectors.
At second level, the activity selector-B 1040 maintains a history of K previously detected activities. The activity selector-B 1040 identifies an activity based on the history of previously detected activities, and the currently identified activity. For example, the activity selector-B 1040 can declare the mode value of the series as a simplest algorithm. As a result the activity selector-B 1040 filters the spurious detection of a different activities interspersed in the prominently detected main activity made available as report 1050.
A calorific burn estimator 1045 receives activity details from the activity selector-B 1040 and a profile data and/or activity intensity data entry 1055 and produces an estimate of caloric burn during the activity. The user enters a food/calorie intake profile 1065 and calorie intake estimator 1067 converts the profile into a caloric intake. A fitness target estimator 1060 compares the caloric burn of the individual with the caloric intake of the individual, and estimates a weight change per unit time. It further estimates a period required to meet a specific weight change target based on a weight change per unit time. A fitness target achievement estimator 1070 provides an estimate of the user achievements towards the user's fitness targets.
If the motion activity detection device 1100 does not have any network connections available, the activity detection device 1100 must perform its own activity identification processing and energy consumption estimates. If this is the case, then the processing algorithms may be less complex to reduce processing power, and/or reduce processing speed. Acceleration signals data acquisition is performed in chunk of processing every few mili-seconds by waking up. For all other times the processor rests in low-power mode. Except for the emergency situation, the RF communication is done periodically when the data is in steady state, there is no need to send it to network i.e. when the object is in sedentary there is no need to send data change in the state is communicated to network. Additionally, if no network connections are available, the operation of the activity detection device 1100 may be altered.
The activity detection device 1100 includes a processor in which at least a portion of the analysis and signature matching can processing can be completed. However, if the activity detection device 1100 has one or more networks available to the motion detection activity detection device 1100, the motion detection device can off-load some of the processing to one of the processors 1130, 1150 associated with the networks.
The determination of whether to off-load the processing can be based on both the processing capabilities provided by available networks, and the data rates (bandwidth) provided by each of the available networks.
In another embodiment, the activity detection device 1100 may be connected to processor 1160 using a detachable wired connection, for example, USB. This processor 1160 may extract the motion detection information and optionally carry out part of the processing. Optionally, the wired connection is used for upgrading the firmware or the profile configuration information of the device 1100.
As shown and described, at least some of the embodiments described include method or processes that are operable on a machine, such as, a server or computer. Accordingly, at least some embodiments include a program storage device readable by such a machine, tangibly embodying a program of instructions executable by the machine to perform a method of monitoring energy expended by an individual. At least one embodiment of the method includes sensing motion of the individual, identifying a plurality of activities performed by the individual over a period of time based on the identified motions, estimating energy expended by the individual for each of the plurality of the plurality of identified activities, and estimating energy expended by the individual by summing the estimated energy expended for each of the plurality of activities.
Although specific embodiments have been described and illustrated, the embodiments are not to be limited to the specific forms or arrangements of parts so described and illustrated.
This patent application is a continuation-in-part (CIP) of U.S. patent application Ser. No. 12/560,069, filed on Sep. 15, 2009, which claims priority to US provisional patent application Ser. No. 61/208,344 filed on Feb. 23, 2009 which is incorporated by reference.
Number | Date | Country | |
---|---|---|---|
61208344 | Feb 2009 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 12560069 | Sep 2009 | US |
Child | 13204658 | US |