Patent Application
20080183682
The present invention relates generally to the field of data processing, and more particularly, to methods and apparatus for incremental approximate nearest neighbor searches in large data sets.
Nearest neighbor searching is an important problem in various applications, including e-commence product search, web search, imagine retrieval, data mining, pattern recognition, and data compression. The problem can be formally described as follows. Given a set S of data points, the task is to process these data points so that, given any query data point q, the data point nearest to q (with respect to a certain distance measure) can be reported quickly.
In many applications, users are satisfied with finding an approximate answer that is “close enough” to the exact answer The approximate nearest neighbor can be defined as the follows Consider a set S of data points and query point q. Given E>0, a point p is said to be a (1+ε)-approximate nearest neighbor of q if.
Dist(p,g)<=(1+ε)dist(p*,g)
A number of techniques have been proposed or suggested for determining an approximate neatest neighbor (ANN), especially for high-dimensional data retrieval since exact nearest neighbor retrieval can become very inefficient in such settings. For example, Sunil Arya et al., “An Optimal Algorithm for Approximate Nearest Neighbor Searching in Fixed Dimensions,” J of the ACM, (1994) focuses on improving index structures and better pruning techniques under the assumption that the number of desired ANNs is known in advance.
A need exists for improved methods and apparatus for approximate nearest neighbor searches. A further need exists for methods and apparatus for incremental approximate nearest neighbor searches in large data sets. Yet another need exists for methods and apparatus for incremental approximate nearest neighbor searches that do not require the number of desired ANNs to be known in advance.
Generally, methods and apparatus are provided for incremental approximate nearest neighbor searching. According to one aspect of the invention, an approximate nearest neighbor is incrementally retrieved from a data set having a plurality of objects and cells of objects in a hierarchical organization based on a query object. The present invention maintains an object priority queue to organize the objects based on a distance to the query object and a cell priority queue to organize the cells of objects based on a distance to the query object. The next approximate nearest neighbor is incrementally retrieved based on a state of one or more of the object priority queue and the cell priority queue.
The object priority queue or cell priority queue (or both) can be maintained, for example, based on a requested maximum error or on cells visited. A state of the object and cell priority queues is maintained for future requests. The next item in the object and cell priority queues is determined that is the next approximate nearest neighbor to the query object.
A more complete understanding of the present invention, as well as further features and advantages of the present invention, will be obtained by reference to the following detailed description and drawings.
The present invention provides methods and apparatus for incremental approximate nearest neighbor searches. Generally, given a set S of n objects, a query object q, an error bound s, and a distance metric, dist, that measures the distance between any two objects, the present invention incrementally retrieve the next ε-approximate nearest neighbor (ε-ANN) of q from S. In other words, the present invention finds the (ε-ANN) of q from the set S in the first step. Assume p is returned as the answer In each subsequent iteration, the (8-ANN) of q is found from the remaining points of set S (i.e., set S-{p}). Of course, a naive way to do incremental approximate neatest neighbor search is to run an approximate nearest neighbor search algorithm from scratch repeatedly on the changing data set once every answer is returned and selected from the data set. In the present invention, a more efficient method is disclosed than this naive approach.
Given a data set S and a query point q, let r be the distance of the exact nearest neighbor of q. The ε-approximate nearest neighbors (ε-ANNS) of q are all data points in S whose distance to q is smaller or equal to (1+ε)r. In the example of
The definition of ε-ANN can be extended to ε-k-ANN as follows. Given a data set S and a query point q, let r be the distance of the exact kth nearest neighbor of q The ε-approximate kth nearest neighbors (ε-k-ANN) of q are all data points in S whose distance to q is smaller or equal to (I+ε)r.
Intuitively, these are all the points with an error of ε away from the true k nearest neighbors. In the example of
Each node in the tree 260 is either associated with a data object or a region of space called a cell (i.e., a set of data objects). In this example, nodes 206, 208, 210, 212 and 214 in the tree 260 are associated with the regions represented by the corresponding boxes 250. Note that box 206 represents the whole space that contains all data points and it corresponds to the root of the tree. Nodes 216 and 218 represent two data points in cell 214. It is to be noted that only the innermost cells contain data objects. In this example, each cell is “summarized” through a bounding rectangle (shown as a dashed line). In general, this may be a more general structure or can be omitted entirely. However, if present, this summary information has to be consistent with the nesting structure. For example, in this case, the bounding box of a cell has to be enclosed by the bounding rectangle of its super-cell.
The data structures shown in
Two priority queues, 200 and 202 are also maintained. Queue 200 is a queue Qo for storing data objects that are processed, and queue 202 is a queue Qs for storing the cells that are processed, as discussed further below in conjunction with
In the following discussion, the method 300 is discussed in detail Initially, the two priority queues Qo and Qs are initialized. For each request to get the next ANN, several steps are performed. First, in step 301, variable o is initialized to the next object in Qo and variable s is initialized to the next set in Qs. Next, in step 302, the distance between q and o, d(q,o), is compared with the distance between q and s, d(q,s).
If the firmer, d(q,o), is less than (1+c) times the latter, d(q,s), or if Qs is empty, the execution continues with step 304, otherwise execution continues with step 306. In step 304, o is removed from Qo and returned. In step 306, s is removed from Qs and in step 308, its content is examined. If s contains more sets, the execution continues with step 312, otherwise execution continues with step 310. In step 312, the sets contained in s are inserted into Qs and execution continues with step 300. In step 310, the objects contained in s are inserted into Qo and execution continues with step 300.
Updates to the data points are supported through the underlying data partitioning structure. The present invention does not assume any specific method for inserting, deleting, or changing data points. The present invention works with any structure that partitions the data hierarchically therefore, known structures such as R-trees can be used to allow for dynamic data sets. One advantage of this invention is that it can be used on top of any such hierarchical data structure (whether or not dynamic) and enhances the functionality of the system by providing ANN search methods.
As is known in the art, the methods and apparatus discussed herein may be distributed as an article of manufacture that itself comprises a computer readable medium having computer readable code means embodied thereon. The computer readable program code means is operable, in conjunction with a computer system, to carry out all or some of the steps to perform the methods of create the apparatuses discussed herein. The computer readable medium may be a recordable medium (e.g., floppy disks, hard drives, compact disks, or memory cards) or may be a transmission medium (e.g., a network comprising fiber-optics, the world-wide web, cables, or a wireless channel using time-division multiple access, code-division multiple access, or other radio-frequency channel). Any medium known or developed that can stole information suitable for use with a computer system may be used. The computer-readable code means is any mechanism for allowing a computer to read instructions and data, such as magnetic variations on a magnetic media or height variations on the surface of a compact disk.
The computer systems and servers described herein each contain a memory that will configure associated processors to implement the methods, steps, and functions disclosed herein. The memories could be distributed or local and the processors could be distributed or singular. The memories could be implemented as an electrical, magnetic or optical memory, or any combination of these or other types of storage devices. Moreover, the term “memory” should be construed broadly enough to encompass any information able to be read from or written to an address in the addressable space accessed by an associated processor. With this definition, information on a network is still within a memory because the associated processor can retrieve the information from the network.
It is to be understood that the embodiments and variations shown and described herein are merely illustrative of the principles of this invention and that various modifications may be implemented by those skilled in the art without departing from the scope and spirit of the invention.
This application is a continuation of U.S. patent application Ser. No. 11/217,784, filed Aug. 11, 2005, which is incorporated herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 11217784 | Aug 2005 | US |
| Child | 12058976 | US |
