The invention generally relates to the monitoring of exercise or other forms of physical activity and, more particularly, to the monitoring and rewarding of physical activity.
The detrimental effects of childhood obesity on the health and lifespan of an individual coupled with its far reaching grip on today's youth have caused concern that approaches the level of a nationwide pandemic. A sedentary lifestyle can be a significant contributor to childhood obesity. On the other hand, exercise can help children control their weight, and can help to reduce the risk of illnesses such as high blood pressure, heart disease, and sleep problems. However, many children fail to exercise because they excessively spend time doing stationary activities such as playing video games or watching television.
It is against this background that a need arose to develop the apparatus, system, and method described herein.
One aspect of the invention relates to a non-transitory computer-readable storage medium. In one embodiment, the storage medium includes executable instructions to: (1) receive a measurement of a physical activity from a sensor; (2) process the measurement of the physical activity to derive a valid extent of the physical activity; and (3) control an entertainment device based on the valid extent of the physical activity.
Another aspect of the invention relates to a system for monitoring and rewarding physical activity. In one embodiment, the system includes: (1) a processing unit; and (2) a memory connected to the processing unit and including executable instructions to: (a) receive an identification of valid instances of a physical activity by a user; and (b) control access of the user to an entertainment device based on the valid instances of the physical activity.
Other aspects and embodiments of the invention are also contemplated. The foregoing summary and the following detailed description are not meant to restrict the invention to any particular embodiment but are merely meant to describe some embodiments of the invention.
For a better understanding of the nature and objects of some embodiments of the invention, reference should be made to the following detailed description taken in conjunction with the accompanying drawings.
As shown in
As shown in
In one embodiment, the physical activity monitor 1 can identify or recognize the type of exercise being carried out by a child. For example, the physical activity monitor 1 can be configured to distinguish between different activities such as walking uphill or downhill, walking on a level surface, running, heavy load carrying, and so forth. The physical activity monitor 1 also can be configured to distinguish between different types of environments in which a physical activity is performed, such as the type surface including walking on grass, uneven ground, gravel, sand, carpet, and so forth. The recognition of the type of activity and the type of environment can be carried out in accordance with supervised techniques or unsupervised techniques. For example, certain aspects of an unsupervised technique for exercise recognition is set forth in U.S. Provisional Application Ser. No. 61/448,602 filed on Mar. 2, 2011, the disclosure of which is incorporated herein by reference in its entirety. Such function can be carried out by a module 13 stored or residing in the storage medium 10. Alternatively or in addition, either of, or both, the main host 2 and the web server 4 can perform such function by processing data measured by the physical activity monitor 1.
In one embodiment, the physical activity monitor 1 can calculate or derive parameters indicative of an extent of a physical activity, such as distance traveled, duration of exercise, intensity of exercise (e.g., pace of a walk or run), and calories or energy burned as a result of exercise. The calculation of such parameters can be based on the type of exercise that is performed, the type of environment in which the exercise is performed, or both. For example, the physical activity monitor 1 can use the notion of MET (Metabolic Equivalent of Task) to derive the number of calories burned by a child. The calculation of exercise parameters can be carried out by a module 14 stored or residing in the storage medium 10. Alternatively or in addition, either of, or both, the main host 2 and the web server 4 can perform such function by processing data measured by the physical activity monitor 1.
In one embodiment, the physical activity monitor 1 can perform processing of measurements of physical activity by the set of sensors 7 to reduce or minimize the vulnerability of the system to false positives and cheating. The processing of measurements to mitigate against false positives and cheating can be carried out by modules 11 and 12 stored or residing in the storage medium 10. Alternatively or in addition, either of, or both, the main host 2 and the web server 4 can perform such function by processing data measured and provided by the physical activity monitor 1.
In one embodiment, the physical activity monitor 1 can validate, or determine a valid extent of, measurements of a physical activity made by the set of sensors 7. A measurement of a physical activity can be invalid (or have a valid extent of zero). For example, if some form of cheating has occurred, such as when a first child attaches the physical activity monitor 1 to a second child who performs an exercise, a measurement of the exercise can be deemed invalid. In another example, a measurement of a physical activity, such as a step count, can be deemed invalid because other types of physical activities, such as shaking of a pedometer, can be mistakenly interpreted by the pedometer as steps. Also, a measurement of a physical activity can be substantially valid (or have a positive valid extent). For example, the physical activity monitor 1, through processing of pressure data or acceleration data, can validate or determine a valid (accurate) extent of a step count associated with the pressure data or acceleration data resulting from a walking or running activity.
Referring back to
Based on a valid extent of a physical activity, the main host 2 can issue a command to the power controller 3, which activates (or deactivates) one or more of the entertainment appliances 1 through N in accordance with the command. In this manner, the main host 2, in combination with the power controller 3, can control operation of the entertainment appliances 1 through N as a reward for physical activity, while mitigating against false positives and cheating. In one embodiment, the power controller 3 can be integrated into a home automation system that controls the power outlets 6 in a wireless fashion or via underlying power lines. Based on a physical activity level of a child, the main host 2 can allot a time budget for one or more of the power outlets 6 corresponding to specific appliances. In this embodiment, the power controller 3 can deactivate a corresponding appliance when the time budget expires, so that the child is forced to leave the appliance. In this manner, the main host 2, in combination with the power controller 3, can control a child's access to the functionality of the corresponding appliance. To do so, the main host 2 can issue a command to the power controller 3, and the power controller can transmit a radiofrequency (RF) signal to activate (or deactivate) one or more of the power outlets 6.
Alternatively or in addition, a controller module can be included as a software application that interacts with entertainment applications residing in a child's computer 5, such as video games. If a physical activity by the child has a sufficient valid extent, the child can be rewarded with a stronger avatar for local or web-based games on the child's computer 5, or can be rewarded, based on the child's exercise records, with other types of visual feedback incentives through interaction with the child's computer 5. These additional types of incentives, in addition to control of access to the entertainment appliances 1 through N, can further persuade the child to engage in healthy physical activity.
In one embodiment, a mapping between a valid extent of a physical activity by a child and an allotted time budget (or amount of another type of incentive described above) can be adjusted or otherwise configured. For example, the time budget can linearly increase as a function of the valid extent of the physical activity by the child. Alternatively or in addition, the time budget can increase as a step function. For example, the child can receive no time budget until the valid extent of the physical activity by the child exceeds or reaches some minimum value. Allocation of the time budget can be up to a maximum value that a child can receive per period of time, such as per day. In one embodiment, the mapping between the valid extent of the physical activity by the child and the time budget (amount of another type of incentive described above) can vary across different types of appliances and across different applications, such as video games. In one embodiment, the main host 2 has a recommended mode that calculates a time budget based on characteristics such as age, sex, height, and weight. It is contemplated that adults also can benefit from the system in addition to children.
This section and the following sections describe the processing of measurements of a physical activity to mitigate against false positives and cheating. As previously described, in one embodiment, a measurement of a physical activity, such as a step count for walking or running, can be deemed invalid because other types of physical activities, such as shaking of a pedometer, can be mistakenly interpreted by the pedometer as steps.
In one embodiment, a first sensor can be a pedometer attached to a person's clothing, placed in the person's pocket, or embedded in the person's shoe. The first sensor can provide a first measurement of a physical activity such as walking or running. In some instances, a significant fraction of detected steps can be a result of false positives stemming from intentional or unintentional movements along a vertical axis. To combat this issue, a second sensor of a different type from the first sensor can provide a second measurement of the physical activity. The first sensor and the second sensor can be included in the physical activity monitor 1, or can be separate sensors that communicate with the physical activity monitor 1.
In one embodiment, the second sensor can include a first pressure sensor located in a first area of a shoe insole corresponding to a heel area of a foot, and a second pressure sensor located in a second area of the shoe insole corresponding to a ball area of the foot. As indicated by
Alternatively or in addition, the second sensor can include additional pressure sensors. For example, the second sensor can include three, four, or more pressure sensors located in the heel area, the ball area, and other areas of the foot.
For the example set forth in
The data measured by the pressure sensors in the shoe insole can be valuable when the physical activity monitor 1 uses a typical pedometer to measure a physical activity. In this case, to prevent false positives (e.g., as a result of the pedometer being intentionally or unintentionally shaken), recent pressure data is checked to see if a considerable amount of pressure has been applied to at least one of the heel area and the ball area. In one embodiment, the physical activity monitor 1 validates, or determines a valid extent of, data measured by the pedometer for a physical activity based on data measured by the pressure sensors for the same physical activity. Alternatively or in addition, either of, or both, the main host 2 and the web server 4 can perform such function by processing data measured by the physical activity monitor 1.
To determine a valid extent of a first measurement by the first sensor (such as a pedometer), data from the first pressure sensor and the second pressure sensor are checked for at least a subset of possible (or candidate) steps detected by the pedometer. For example, if output of the pedometer is active (e.g., a possible step is detected by the pedometer), then recent pressure data measured by the first pressure sensor (heel area) and the second pressure sensor (ball area) are checked for a “01” pattern, a “11” pattern, or a “10” pattern (as shown in
In one embodiment, each possible step detected by the first sensor (such as a pedometer) can be checked against data from the second sensor (such as first and second pressure sensors). Alternatively, a subset of possible steps detected by the first sensor can be checked against data from the second sensor. The number of detected steps can be corrected or otherwise adjusted based on a percentage of steps that were detected correctly by the first sensor. For example, if 90% of the detected steps by the first sensor are validated based on the second sensor, then a valid extent of the step data can be 90% of the step data measured by the first sensor. An entertainment appliance, such as an electronic entertainment device, can then be controlled based on this valid extent of the step data.
As previously described, in one embodiment, processing by the physical activity monitor 1 can determine a valid (accurate) extent of a physical activity, such as walking or running. For example, the physical activity monitor 1 can determine a valid step count based on acceleration data. This processing can be facilitated by determining a physical activity template, and by applying the physical activity template to measured acceleration data to count a number of instances of the activity that have occurred, such as a number of steps for a walking or running activity.
In one embodiment, an accelerometer can be included in a sensor that measures a physical activity such as walking or running. The accelerometer can be included in the physical activity monitor 1. A template matching approach can reliably measure a number of steps taken by a person. The template matching approach is based on a template, which represents a typical step cycle. More generally, a template can identify or represent an aspect of a physical activity of a particular type, such as a walking step, a running step, and so forth.
In one embodiment, an acceleration data signal can be divided into multiple data blocks having a particular duration in time, such as data blocks of about 10 seconds.
The template matching approach can then examine whether any physical activity template already exists, such as residing in the storage medium 10. The physical activity template can be derived from measurements of the same type of physical activity during a training period, which can be a time duration of at least about 10 seconds, such as a time duration of about 1 minute. Alternatively or in addition, a first step cycle (e.g., a time duration at the beginning of the measured data signal, such as an initial 10 second period of the measured data signal) can be extracted as a temporary template. Another portion of the measured data signal also can be extracted as the temporary template. In one embodiment, the template can be derived by the physical activity monitor 1. Alternatively or in addition, the template can be derived by either of, or both, the main host 2 and the web server 4.
An instance of a physical activity (or another event) can be detected in a measured data signal when there is a sufficient degree of similarity between the measured data signal and the template. In one embodiment, the template is slid across the entire or a portion of the data signal, and a normalized cross-correlation is calculated between the template and the measured data signal (see
In equation (1), X represents the template, Y represents the measured data signal, k is an index representing a time lag, <X, Y> is the inner product of X and Y, ∥X∥ is the norm of X, ∥Y∥ is the norm of Y, RXY(k) is the cross-correlation of X and Y for arbitrary k, RXX(0) is the auto-correlation of X at zero lag, and RYY(0) is the auto-correlation of Y at zero lag. In one embodiment, the cross-correlation can be derived by the physical activity monitor 1. Alternatively or in addition, the cross-correlation can be derived by either of, or both, the main host 2 and the web server 4.
In one embodiment, a maximum value for the normalized cross-correlation is 1 for absolute identity, which allows a uniform threshold to be set for all data despite varying amplitudes. Peaks in the normalized cross-correlation in
In one embodiment, a physical activity template can be derived based on an average of multiple step cycles, which can be a more representative template than a temporary template. For example, in
Step cycles can be detected based on various techniques, such as (a) finding zeros of a signal; (b) computing the signal's energy; or (c) using the concept of salience used in speech processing. In one embodiment, given that a signal from a sensor, such as an accelerometer, is mixed with noise, the third technique can yield a higher accuracy. The salience of a given data sample can be defined as the length of the longest interval over which the sample is a maximum. The term salience vector denotes a signal containing the salience of each sample in an original, input signal. As a result of feet striking the ground while walking, a start of each walking cycle typically has a large salience. Therefore, cycles can be detected by locating such distinct points.
In one embodiment, the number of steps in a data block, such as a block of acceleration data for walking, can be derived as follows. First, a salience of each sample of the accelerometer data in an input signal r can be found, and a corresponding salience vector, s, can be created. Then, the vector u=(r·s)/max(s) is computed, where “·” represents an element-wise multiplication. This transformation makes the peaks of r more pronounced and attenuates other elements of r. Then, elements in u beyond or reaching a certain threshold are extracted as potential cycle indices. Then, a difference d between these results is computed. Any results differing by a single sample can be discarded. Based on an original signal assumption, an average (or mean) of the difference d can correspond to be an average (or mean) of a cycle length. Then, the difference d is normalized around its mean, and the indices which fall within the threshold are extracted. These are the cycle starting and ending points. The number of steps in the data block is then derived as: (number of extracted indices −1).
If a template is already present, peaks can be detected using the techniques stated above. Before a next data block is processed using the same template, a determination can be made as to whether the template will be updated. A step signal may change dynamically with time; accordingly, the template may not accurately represent a current step signal. In one embodiment, if major peaks in a normalized cross-correlation are lower than 0.55 (or another threshold), a new template can be derived using step cycles in a current data block. Otherwise, peak detection is carried out in the current data block using the existing template.
As previously described, in one embodiment, a measurement of a physical activity by a first sensor, such as a step count for walking or running, can be deemed invalid because a detected identity of a performer of the physical activity (as measured by the first sensor) does not correspond to an identity associated with, or assigned to, the first sensor. In one embodiment, an unique identifier can be assigned to a given physical activity monitor 1 (which can include the first sensor) to associate the physical activity monitor 1 with a given person. If exercise performed by a different person is measured by the physical activity monitor 1, then the system can detect the exercise as cheating.
To detect this type of cheating, the system can determine whether the physical activity monitor 1 is being carried by the person to whom the physical activity monitor 1 is assigned, or by a different person. In one embodiment, such determination is carried out through a histogram comparison to derive a similarity score.
A first histogram, namely a template histogram, can be derived that identifies or represents characteristics of a first instance of a physical activity of a particular type, such as walking or running, where it is known that the first instance of the physical activity is performed by the person to whom the physical activity monitor 1 is assigned. The template histogram can be derived during a training period. In one embodiment, the template histogram can be derived by the physical activity monitor 1. Alternatively or in addition, the template histogram can be determined by either of, or both, the main host 2 and the web server 4.
A second histogram, namely a measured histogram to be validated, can be derived that identifies or represents characteristics of a second instance of the same type of physical activity, where it is desired to determine whether the second instance of the physical activity is performed by the person to whom the physical activity monitor 1 is assigned, or by a different person. In one embodiment, the measured histogram can be derived by the physical activity monitor 1. Alternatively or in addition, the measured histogram can be determined by either of, or both, the main host 2 and the web server 4.
Based on a comparison of the template histogram to the measured histogram, a valid extent of measurements of the second instance of the physical activity can be derived. In particular, the system can determine whether the second instance of the physical activity is performed by the person to whom the physical activity monitor 1 is assigned (if the template histogram is sufficiently similar to the measured histogram), or by a different person. The comparison of the template histogram to the measured histogram can occur during a verification period separate from the training period.
Certain factors can affect the accuracy of a histogram similarity determination for user authentication. First, each histogram can include a number of bins (or resolution) that can correspond to the number of different recognizable outputs that a sensor (such as an accelerometer) can provide. Second, each histogram can include a number of observations (data points) that can correspond to a sampling rate of the sensor times a duration of the physical activity, such as each step.
In one embodiment, the template histogram is derived by measuring a first instance of a physical activity of a known type (such as walking or running) and performed by a known person to whom the physical activity monitor 1 is assigned. A duration of the first instance of the physical activity can be long enough to include at least several cycles of the physical activity, such as multiple steps or strides. For example, the duration of the first instance of the physical activity can be at least about ten seconds, and can be about one minute or more, such as up to about five minutes, about ten minutes, or more.
In one embodiment, the measured histogram is determined by measuring a second instance of the same type of physical activity. A duration of the second instance of the physical activity can be long enough to include at least several cycles of the physical activity, such as multiple steps or strides. For example, the duration of the second instance of the physical activity can be at least about ten seconds, and can be about one minute or more, such as up to about five minutes, about ten minutes, or more. The duration of the second instance of the physical activity can be the same as, or different from, the duration of the first instance of the physical activity.
In one embodiment, an output range of a sensor, such as an accelerometer, is divided into n intervals (typically 100 bins for sampling rates no more than about 100 Hz). Each interval can correspond to a bin of at least one of the template histogram and the measured histogram. With regard to the template histogram, each data point from the first instance of the physical activity can be included in the template histogram. For a sampling rate of about 100 Hz, all data points can be taken into consideration, namely the number of observations is equal to the number of data points.
Various metrics can be used to determine a similarity between the template histogram and the measured histogram. In one embodiment, an absolute distance metric can be used to derive a similarity score between these histograms. The template histogram and the measured histogram can be normalized prior to their comparison, and an absolute distance can be derived as set forth in equation (2):
Here, xi is the probability of a data point residing in bin i of the normalized template histogram, and yi is the probability of a data point residing in bin i of the normalized measured histogram. In one embodiment, this distance value represents a similarity score between two acceleration signals. This metric is both computationally streamlined and effective at measuring similarity between histograms for authenticating the identity of a person performing various types of physical activities.
If a physical activity is performed by the same person to whom the physical activity monitor 1 is assigned, the distance value is typically smaller than a resulting distance value when the physical activity is performed by an impostor. A combined acceleration signal, namely an acceleration signal that combines accelerations along multiple axes (e.g., all three axes), can yield further improvements in authenticating the identity of a performer of the physical activity. In one embodiment, operations involved in comparing two gait samples using the histogram similarity approach are visualized in
An embodiment of the invention relates to a non-transitory computer-readable storage medium having computer code thereon for performing various computer-implemented operations. The term “computer-readable storage medium” is used herein to include any medium that is capable of storing or encoding a sequence of instructions or computer codes for performing the operations described herein. The media and computer code may be those specially designed and constructed for the purposes of the invention, or they may be of the kind well known and available to those having skill in the computer software arts. Examples of computer-readable storage media include, but are not limited to: magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and holographic devices; magneto-optical media such as floptical disks; and hardware devices that are specially configured to store and execute program code, such as application-specific integrated circuits (“ASICs”), programmable logic devices (“PLDs”), and ROM and RAM devices. Examples of computer code include machine code, such as produced by a compiler, and files containing higher-level code that are executed by a computer using an interpreter or a compiler. For example, an embodiment of the invention may be implemented using Java, C++, or other object-oriented programming language and development tools. Additional examples of computer code include encrypted code and compressed code. Moreover, an embodiment of the invention may be downloaded as a computer program product, which may be transferred from a remote computer (e.g., a server computer) to a requesting computer (e.g., a client computer or a different server computer) via a transmission channel. Another embodiment of the invention may be implemented in hardwired circuitry in place of, or in combination with, machine-executable software instructions.
While the invention has been described with reference to the specific embodiments thereof, it should be understood by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the true spirit and scope of the invention as defined by the appended claims. In addition, many modifications may be made to adapt a particular situation, material, composition of matter, method, operation or operations, to the objective, spirit and scope of the invention. All such modifications are intended to be within the scope of the claims appended hereto. In particular, while certain methods may have been described with reference to particular operations performed in a particular order, it will be understood that these operations may be combined, sub-divided, or re-ordered to form an equivalent method without departing from the teachings of the invention. Accordingly, unless specifically indicated herein, the order and grouping of the operations is not a limitation of the invention.
This application claims the benefit of U.S. Provisional Application Ser. No. 61/467,744 filed on Mar. 25, 2011, the disclosure of which is incorporated herein by reference in its entirety.
Number | Date | Country | |
---|---|---|---|
61467744 | Mar 2011 | US |