Patent Application
20110077919
This application claims the benefit under 35 U.S.C. §119(a) of Korean Patent Application No. 10-2009-0092277, filed on Sep. 29, 2009, in the Korean Intellectual Property Office, the entire disclosure of which is incorporated herein by reference for all purposes.
1. Field
The following description relates to a method of recognizing an activity on the basis of a semi-Markov conditional random field (CRF) model.
2. Description of the Related Art
Activity recognition is applied to various fields ranging from daily life to industry, and thus is becoming more important in people's lives. Activity recognition is frequently performed using many different sensors. Among these sensors, an accelerometer has been known as an effective sensor for measuring an activity because of low cost and low power consumption.
Lately, CRF models are used in sequential data modeling, thereby resulting in useful outcomes. These have been disclosed in reference document “Conditional random fields: Probabilistic models for segmenting and labeling sequence data” by John Lafferty, Andrew McCallum and Fernando Pereira.
However, such a conventional CRF cannot model the durations of activities nor the transitions over a long time period between activities.
To solve these problems, various modifications of the CRF have been proposed [Sunita Sarawagi, et al., 2004, and D. L. Vail, et al., 2001]. However, these modifications of the CRF have unrealistic complexity or do not completely solve the problems. For example, the first CRF proposed by John Lafferty, et al., in 2001 cannot model a duration of a state due to the Markov assumption.
Content disclosed by Sunita Sarawagi, et al. in 2004 to overcome this limitation is intended to lessen the Markov property using a semi-CRF. However, when activity recognition is applied, an unknown activity or null activity occurs between two expected activities or target activities, and thus the semi-CRF cannot capture an activity transition over a long time period either.
The following description relates to a solution to conventional problems, which is based on the extension of a semi-Markov conditional random field (CRF) model and has appropriate complexity.
According to an exemplary aspect, there is provided a method of recognizing an activity on the basis of a semi-Markov conditional random field (CRF) model, including: segmenting an input signal measured by an accelerometer to output frame sequences; extracting training feature vectors from the frame sequences; building a codebook containing kernel vectors from the training feature vectors; quantizing vector sequences into discrete symbol sequences; using linear chain semi-Markov CRF model to compute the likelihood of a label given its corresponding symbol sequence.
Additional aspects of the invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the invention.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention, and together with the description serve to explain the aspects of the invention.
The invention is described more fully hereinafter with reference to the accompanying drawings, in which exemplary embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the exemplary embodiments set forth herein. Rather, these exemplary embodiments are provided so that this disclosure is thorough, and will fully convey the scope of the invention to those skilled in the art. In the drawings, the size and relative sizes of layers and regions may be exaggerated for clarity. Like reference numerals in the drawings denote like elements.
Heretofore, activity recognition solutions using a Markov model have not considered the correlation between activities and the durations of the activities and thus have not had high complexity.
An exemplary embodiment of the present invention relates to a semi-Markov CRF model having an algorithm whereby training and inference are simultaneously and rapidly performed to take the correlation between activities and the durations of the activities into consideration.
In other words, an exemplary embodiment of the present invention extends a semi-Markov CRF, thereby capturing an activity transition over a long time period while using the duration modeling performance of a conventional semi-Markov CRF.
To this end, a semi-Markov CRF with discontinuous state time is configured according to an exemplary embodiment of the present invention, and the semi-Markov CRF has a linear chain structure as shown in
In
The beginning time and ending time of an activity are separate from each other and satisfy Expression 1 below.
0<bi≦ei
ei<bi+1 [Expression 1]
A probability P(S|X) of a state sequence S given by an input sequence X is calculated by Expression 2 below.
In Expression 2, Ψ denotes a probability of activity transition from si−1 to si.
Ψ is calculated by Expression 3 below.
QT(y′, y), QD(y, d), QO(y, t1, t2), and QO(IA, t1, t2) in Expression 3 can be calculated by Expression 4 below.
In Expression 4, WD is the weight of duration (D), WT is the weight of activity transition (T), and WO is the weight of observation (O). It is apparent that explicit duration information can be integrated in the model.
Also, in Expression 4, d is a duration variable, t1 and t2 are time variables, IA is a label of unknown activities, my is an average duration, and y is a label value of an expected activity having the average duration my.
As can be seen from the above expressions, an exemplary embodiment of the present invention uses a bell-shaped probability function for duration modeling. The shape of the probability function is shown in
Together with definition of the model, measurement of an increase or decrease of a parameter is performed using Expression 5, Expression 6 and Expression 7 below.
The functions α, λ, γ, β, η and ζ in Expression 5, Expression 6 and Expression 7 can be obtained by Expression 8 below.
When an input signal 31 for training or testing measured by an accelerometer is input to a sliding window 32 (operation 31), the sliding window 32 segments the input signal into frame sequences 33 (operation 32). The sliding window 32 segments the input signal 31 using the Hamming function. The Hamming function is frequently used for filter design, and serves to receive a factor which is a number and segment a signal.
A feature extractor 34 extracts feature vectors from the segmented frame sequences 33 (operation 33). The extracted feature vectors are provided to a vector quantizer 35 (operation 34).
The vector quantizer 35 receives and combines the feature vectors with a kernel vector 38, thereby constructing a discrete input sequence (operation 35). The discrete input sequence is provided to a semi-Markov CRF unit 36. In a training phase, a manual state label set is required when there is an additional input to the semi-Markov CRF unit 36.
On the basis of the discrete input sequence received from the vector quantizer 35, the semi-Markov CRF unit 36 can capture an activity transition by Expression 1 to Expression 8 and output a recognition result.
Meanwhile, the kernel vector 38 input to the vector quantizer 35 together with the feature vector is generated through a separate process, which will be described below with reference to
When a training input signal 41 is input to the sliding window 32, the sliding window 32 generates one set of frames from the input signal and provides the generated frames to the feature extractor 34. The feature extractor 34 extracts feature vectors 42 from the one set of frames.
The extracted feature vectors 42 are provided to a clustering unit 43, and the clustering unit 43 collects the input feature vectors 42 to generate the kernel vector 38.
The generated kernel vector 38 is provided to the vector quantizer 35 of
As a result, in an exemplary embodiment of the present invention, inference from sequential feature vectors and training by kernel vectors are simultaneously and rapidly performed, so that an activity transition recognition result can be output.
The probability P(S|X) of a label state sequence can be calculated by Expression 2 as mentioned above. See Expression 2.
As mentioned above, the function P(S|X) requires the function Zx, which is calculated by the following equation:
A process of calculating the function Zx is illustrated in a flowchart of
Referring to
To this end, first, operations 501, 502, and 504 constitute a loop with the variable t. Also, operations 506, 507, and 510 constitute a loop with a variable d, and operations 508, 512, and 514 constitute a loop with a variable y′. In operations 509 and 513, the functions α, γ and λ are calculated by Expression 8, and in operation 515, the standardized factor Z is calculated.
As apparent from the above description, in activity recognition using an accelerometer according to an exemplary embodiment of the present invention, training and inference are simultaneously performed in a semi-Markov CRF. Thus, an activity transition can be effectively captured for a long duration.
It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention covers the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2009-0092277 | Sep 2009 | KR | national |
