1. Field of the Invention
The present invention relates to a method for indexing feature vector data space, and more particularly, to a method for indexing feature vector data space which efficiently performs indexing within the vector space having high-dimensionality in which feature vectors are not uniformly distributed.
The present application also claims the benefit under 35 U.S.C. § 119(a) of the filing date of Korean Patent Application No. 00-58759 which is incorporated herein by reference.
2. Description of the Related Art
High dimensionality of typical multimedia data descriptors pose challenging problems in designing efficient indexing schemes. Therefore, several new indexing structures have been proposed recently. One of the common assumptions made is that feature vectors in the high dimensional feature space are uniformly distributed within a vector space. However, many media descriptors, such as image texture descriptors, are not uniformly distributed. For example, in the case of a method for using well-known vector approximation (VA) files, the performance of the method depends on the uniform distribution of the feature vectors.
Generally, the method of the related art suffers because its performance abruptly drops when indexing the feature vector data within vector space having high-dimensionality, in which the feature vectors are not uniformly distributed.
To solve the above problems, it is an object of the present invention to provide a method for indexing feature vector data space which performs indexing efficiently within vector space having high-dimensionality, in which the feature vectors are not uniformly distributed.
To achieve the above objective according to the present invention, there is provided a method for indexing feature vector data space including a step of: (a) indexing feature vector space by adaptively approximating feature vectors on the basis of statistical distribution of feature vector data in the feature vector data space.
Step (a) further includes the steps of: (a-1) measuring the statistical distribution of the feature vector data in the feature vector data space; (a-2) estimating marginal distribution of the data using the statical distribution; (a-3) dividing the estimated distribution into a plurality of grids in which a distribution of disposing the data in each grid becomes uniform; and (a-4) indexing the feature vector data space using the divided grids.
Prior to step (a-4), it is preferable to further include a step of updating the grids on the basis of the previous probability distribution function and the updated probability distribution function, when new data is entered.
Also, step (a-4) preferably further includes a step of indexing using vector approximation (VA) files.
In a preferred embodiment, the number of the plurality of grids is determined by the number of bits assigned to the dimension.
Step (a-2) further includes the steps of: (a-2-1) defining the probability distribution function using a weighted sum of the predetermined distribution function; and (a-2-2) obtaining the estimated probability distribution function by estimating the predetermined parameters using the probability distribution function defined in the step (a-2-1).
Step (a-2-2) further includes a step of obtaining an estimated probability distribution function by estimating parameters using all N predetermined data, wherein N is a positive integer, for several iterations on the basis of the expectation-maximization algorithm and using the probability distribution function defined in the step (a-2-1).
Also, preferably, the predetermined distribution function is a Gaussian function.
In a preferred embodiment, the probability distribution function of step (a-2-1) is a one-dimensional signal, p(x), wherein
and wherein p(x|j) is defined as
wherein coefficient P(j) is a mixing parameter that satisfies certain criterion.
In another preferred embodiment, the estimated probability function of step (a-2-2) is obtained by finding Φj, j=1, . . . ,M. which maximizes
wherein parameters v[l], l=1, . . . N, is a given data set.
In a further embodiment, the estimated parameters of step (a-2-2) are updated according to
wherein t is a positive integer representing the number of iteration.
Also, preferably, the estimated parameters set of step (a-2-2) used N data v[l], given as {P(j)N, μjN, (σj2)N}, and the updated parameter set for new data v[N+1] coming in, is calculated using the following equations:
Step (a-2-2) also further includes the steps of: measuring changes of the probability function which is defined as
for each dimension, wherein the previous probability distribution function is {circumflex over (p)}old (x) and the updated probability distribution function is {circumflex over (p)}new(x), and updating an approximation for the dimension if ρ is larger than a predetermined threshold value.
In a preferred embodiment, step (a-3) also includes a step of dividing the probability distribution function into the plurality of grids to make areas covered by each grid equal, wherein the plurality of grids have boundary points defined by c[l], l=0, . . . ,2b, where b is a number of bits allocated, and wherein the boundary points satisfy a criterion,
and wherein the estimated probability distribution function is {circumflex over (p)}(x).
The above objective(s) and advantages of the present invention will become more apparent by describing in detail a preferred embodiment thereof with reference to the attached drawings in which:
Hereinafter, preferred embodiments of the present invention will be described with reference to the appended drawings.
That is, since densely distributed cells can deteriorate the performance of indexing, the approximation of the feature vectors is adaptively formed according to statistical features of the data in the present invention. To do this according to the indexing method as shown in
The approximation formed by the above method reduces the possibility of having densely distributed cells. Therefore, the performance of indexing is enhanced. It should be noted, however, that the marginal distributions of the data can only capture partial information of high dimensional distributions.
Hereinafter, techniques for realizing a method of the present invention will be described in greater detail. First, a probability distribution function is denoted by pj(x) for data on dimension i. Following the assumption that data on each dimension are independent of each other, the algorithm described hereinafter can be applied to each dimension independently.
Also, as previously described, the data distribution is not uniform. In fact, the probability distribution function of data may be irregular or incapable of being modeled by a well-defined function such as, for example, the Gaussian function. To overcome this deficiency, the present invention provides a probability distribution function of one-dimensional data that is modeled using the Gaussian mixture function in order to endure a change of the data distribution.
First, it is assumed that a probability distribution function of a one-dimensional signal, p(x) is defined as follows:
Here, the p(x|j) is defined as follows.
The coefficients P(j) are mixing parameters, which satisfy the criteria 0≦P(j)≦1 and the following formula.
Thus, in this embodiment, the probability distribution function is defined using a weighted sum of the Gaussain function. Then, the task of estimating the probability distribution function is converted to an exercise of parameter estimation for the parameters φj={P(j), μj, σj2}, for j=1, . . . M.
A maximum likelihood based method is used for the parameter estimation using the idea that the optimal estimation of parameters should result in a probability distribution function which most likely would give rise to given data. In this case, we want to find φj, j=1, . . . M to maximize
where v[l], l=1, . . . N, are the given data set.
The above parameters are obtained using an expectation-maximization (EM) algorithm. According to the EM algorithm, N predetermined data are inputted for the estimation, and parameters are estimated iteratively using all the N given data in each iteration.
The following equations are used to update the estimated parameters, where t denotes the iteration number,
However, the above formulas may cause a crash of estimation if the data's distribution has a singular value which can not be grouped with other values as a part of a Gaussian function.
When this is the case, in order to capture a value accurately, μ must be allowed to approach the singular value and the corresponding σ2 must converge to 0. To avoid this singularity problem, a very small value is set as a lower bound for an estimated variance.
In order to explain the efficiency of using the EM algorithm for estimating parameters of the Gaussian mixture function, a histogram illustrating a distribution of feature vector data within the feature vector data space in provided in
As shown in
The parameters may also be updated by on-line estimation using the formulas of equations 5, 6, and 7 if N predetermined data are available. In the case of a large database, N is generally only a small portion of the total number of elements in the database.
In realistic database applications, an estimation is required to be updated at a prescribed point. For example, there may be a case where a larger portion of data is required for a better estimation. Alternatively, when the database is non-static, the probability distribution function must be re-estimated because the statistical characteristics of data change. In any case, a “memory” of the previous estimation is not required to be totally erased.
In view of the parameter estimation, a strategy must be provided for tracking the change of the estimated probability distribution function when a data set is changing. For this purpose, an algorithm is provided in the present invention which can sequentially update the estimation.
Given that {P(j)N, μjN, (σj2)N} is the parameter set estimated using N data v[l], the updated parameter set, when there is new data v[N+1] coming in, is calculated as follows.
μjN+1=μjN+θjN+1(ν[N+1]−μjN (8)
(σj2)N+1=(σj2)N+θjN+1[(ν[N+1]−μjN)2−(σj2)N] (9)
In the formulas 8 and 9,
In order to evaluate the tracking performance using on-line estimation, experimentation was performed on a synthetic data set. The feature vector values of data sets are illustrated in FIG. 4A and include 5,000 elements.
For example, if one wants to estimate the probability distribution function of the data as shown in
Next, nonlinear quantization is applied to segment a probability distribution function into a plurality of grids to make areas covered by each grid equal, wherein the estimated probability distribution function is called {circumflex over (p)}(x). The boundary points are indicated by c[l], l=0, . . . ,2b, where b is the number of bits allocated, wherein the boundary points should satisfy the following criterion:
Using this criterion, it is possible to determine boundary points from one pass scan of the estimated probability distribution function. For example, the boundary points of each dimension are determined by agglomerating all the N data into 2b clusters. In addition to being computationally efficient for determining boundary points, equation 12 also avoids dependency on distance measurements.
According to the above method, a probability distribution function is able to be updated. This feature is very important for maintaining the indexing of a non static database. That is, every time a previous estimation does not fit with the updated estimation, the approximation also needs to be updated.
For this reason, a measure is required to decide when to update the approximation based on the change of probability distribution function estimation. As a result of the parallel scheme of utilizing the probability distribution function to construct the approximation, the measure for updating the approximation can be defined on each dimension.
If the previous probability distribution function is denoted {circumflex over (p)}old(x) and the updated probability distribution function is denoted {circumflex over (p)}new(x), a measurement of change of the probability distribution function can be defined as follows.
The approximation for a dimension is updated when ρ is bigger than a predetermined threshold value.
An experiment was performed for evaluating an image database having 34,698 aerial photograph images. First, 48-dimensional feature vectors describing texture characteristics of each of the images were extracted using a predetermined method for extracting texture features. A probability distribution function was estimated from an entire data set on the basis of the extracted feature vector.
In
In
Comparing graphs 502 and 504 with the graphs 512 and 514, the number of feature vectors, which visit in the first step filtering and the second step filtering using the indexing method of the present invention which forms adaptively VA files, is much larger than the number of feature vectors which visit in the first step filtering and the second step filtering using the conventional indexing method which uses fixed VA files.
The indexing method of the present invention may be written as a program which is performed in a personal computer or a server computer. Program codes and code segments which form the program can be easily derived by computer programmers in the art. Also, the program can be stored in computer readable recording media. The recording media includes magnetic recording media, optical recording media, and carrier wave media.
While a specific embodiment of the invention has been shown and described in detail, it will be understood that the invention may be modified without departing from the spirit of the inventive principles as set forth in the hereafter appended claims.
Number | Date | Country | Kind |
---|---|---|---|
2000-58759 | Oct 2000 | KR | national |
The present application claims the benefit under 35 U.S.C. § 119(e) of the filing date of U.S. Provisional Application No. 60/226,586, filed Aug. 21, 2000 and entitled “A Scabel and Adaptive Index Structure For Similarity Search in High Dimensions”. The contents of this U.S. Provisional Application are incorporated herein by reference.
Number | Name | Date | Kind |
---|---|---|---|
5647058 | Agrawal et al. | Jul 1997 | A |
5713016 | Hill | Jan 1998 | A |
Number | Date | Country |
---|---|---|
7-134774 | May 1995 | JP |
2001-344553 | Dec 2001 | JP |
WO 9748069 | Dec 1997 | WO |
WO 0028441 | May 2000 | WO |
Number | Date | Country | |
---|---|---|---|
20020040367 A1 | Apr 2002 | US |
Number | Date | Country | |
---|---|---|---|
60226586 | Aug 2000 | US |