Not Applicable
Not Applicable
1. Field of Invention
This invention relates to the identification and retrieval of digital data by a computing device.
2. Prior Art
A method for the discovery of digital data, such as images, that are organized for point-wise two dimensional presentation (2DDDs), that are similar to a target 2DDD is invented here. Formulae from algebraic topology [Spanier] are used to compute signatures that characterize equivalence classes of 2DDDs. The method leverages these “equivalence signatures” to find 2DDDs that are similar to target 2DDDs and, separately and alternatively, find 2DDDs that are dissimilar from the target 2DDDs. The most important examples of 2DDD are images, video frames and rasterized graphics data. In this invention, we will refer to such 2DDD simply as images.
The definition of “similarity”, and thus the features and method used to compute it, is idiosyncratic to the retrieval application [O'Connor]. In the case of image retrieval [Gonzalez], methods using entropy, moments, etc. as signatures, have been invented [5,933,823; 5,442,716]. Work in computer graphics has advanced these analytical methods by using an elementary result from topology, the Euler number of polyhedra, as a descriptor of boundary polygons of graphics objects [Foley]. Recently, a method for computing the Euler numbers of binary images using a chip design has been invented [7,027,649]. Another invention [7,246,314], uses closeness to a Gaussian model as a similarity measure for identifying similar videos.
The cost of implementing these methods is typically proportional to the product of the number of 2DDDs in the database with the cost of computing the distance between the target 2DDD and another 2DDD. The latter often [Raghavan] involves the computation of the projection angle between two vectors that represent the features (e.g., histogram of the text elements) of the 2DDDs. For large databases, this process can be both resource and time expensive. A two step method is required wherein, during the retrieval phase, the number of candidates for similarity is significantly reduced in a computationally inexpensive first step and then the traditional features can be applied to the reduced set of candidates.
Intuitively, if two 2DDDs are similar, then they should be deformable into each other without having to remove or glue together portions of 2DDDs. For example, if two images are rescalings and/or rotations of each other, then the images are similar. The field of topology provides a foundation for solving this problem. In particular, we appeal to homotopy invariants that characterize equivalence classes of maps between topological spaces [Bott].
We interpret each 2DDD as a sampling of maps from a two-dimensional presentation space to a n-dimensional topological space and seek homotopy equivalence classes of such maps. Mapping of the Cartesian subspace onto the two-dimensional sphere can be done by a number of methods. Here, we will ascribe the azimuthal and zenith angles (θ1,θ2) of the sphere, S2, as the horizontal and vertical coordinates, (x1, x2), on the image, or vice versa:
where the Li (i=1,2) are the lengths of the respective dimensions. In another embodiment, an embedding into three dimensional Euclidean space is first performed.
We work with 2DDDs that have at least two data planes, {tilde over (σ)}A(θ1,θ2), with maximum and minimum values, {tilde over (σ)}maxA and {tilde over (σ)}minA, respectively. The maximum and minimum values of each of the two planes are used to normalize their data to new minimum and maximum values, σmaxA and σminA respectively, through the expressions:
Additional normalizations of the 2DDD, such as scaling to a fixed width and height and the like, may also be performed.
To leverage the equivalence classes of the Second Homotopy Group [Bott], π2(S2)=Z, we select pairs of the normalized planes of the 2DDD and interpret them as the maps from the world sphere onto the target sphere:
σ:S2→S2
(θ1,θ2)→(σ1(θ1,θ2),σ2(θ1,θ2)) Eqn. 2
An example of such a choice is the two planes of chroma pixel data in a YCC color space representation of images. If objects have been segmented from the 2DDD then the data for these objects are themselves 2DDDs. We refer to each segmented portion of each pair of normalized planes henceforth as a “2DDD section” with its own map, σ. With the definition that two 2DDDs are similar if one can be continuously deformed into the other, the well-established results on equivalence classes of homotopic maps lead to the conclusions: If two 2DDD sections do not have the same value of π2(S2), then they are not similar. If none of the 2DDD sections from parent 2DDDs are similar to each other, then those parent 2DDDs are not similar to each other.
The equivalence signature, ξ[σ], of a 2DDD section is given by the value of π2(S2) for the section computed as [Schwarz]
Consider two 2DDD sections, σ and σ′ such that, at each point, the difference between the values of the maps is infinitesimal:
σ′A(θ1,θ2)−σA(θ1,θ2)=εA(θ1,θ2)
An example of such a difference is the constant resealing and shifts of the data values at each point
σ′A(θ1,θ2)−σA(θ1,θ2)=αAσA(θ1,θ2)+βA Eqn. 5
for some fixed, small, real constants αA and βA. The difference in the values of the equivalence signatures for the two 2DDD sections, in term of the first one, σ, is
Δξ[σ;ε]≡ξ[σ+ε]−ξ[σ] Eqn. 6
Given a choice for the functions εA, a limit on the difference of the values of the equivalence signatures for two 2DDD sections can be set so that the 2DDD sections are still regarded as similar. For example, suppose we are interested in finding images whose color planes differ by no more than p percent at each pixel, then that values αA=p and βA=0 are used in the computation of Δξ[σ;ε]. Retrieval of similarity candidates then proceeds by finding those images with values of ξ[σ], denoted as ξ[σsimilar], for which the following inequalities hold:
|ξ[σtarget]−ξ[σsimilar]|≦|Δξ[σtarget;ε]| Eqn. 7
As an example for the reduction factor for the number of CPU cycles and other resources required to find similar sections of 2DDDs in a corpus, assume for simplicity that and that the equivalences signatures of the 2DDDs in the corpus are uniformly distributed in [ξmax,ξmin]. If for a target 2DDD section, the choice of εA leads to a value for Δξ[σ;ε], the reduction in the number of secondary features to be compared is
In state of the art information retrieval methodologies, the feature vector which is used for each 2DDD would have to be compared to all NA feature vectors computed for the 2DDDs in the corpus. Upon employing the method invented here as a precursor to the feature vector comparison, the number of feature vectors to be compared would be reduced to frNc.
The objects of the current invention include the:
The advantages of the current invention include:
In accordance with the present invention, a method for determining the similarity of sets of data uses the Second Homotopy Group to compute an equivalence signature for each segmented component or section of two-dimensional digital data (2DDD), and further uses the differences of the equivalence signatures of any two sections of 2DDD as the measure of the similarity distance between said 2DDD sections. The output from this method can be used to significantly reduce the computational expense, time and resources required by a subsequent secondary feature comparison.
In the drawings, closely related figures have the same numerically close numbers.
A preferred embodiment of the method of the present invention is illustrated in
A 2DDD is represented as a set of integers (realized in a computing device as a set number of bits). Each 2DDD may be realized as the addition of layers of 2DDD sections. The entire 2DDD, or the resultant from the point-wise addition of all the layers of the 2DDD, is also taken to be a section. Each two-dimensional point in said sections may have a plurality of integer values. For example, some images are composed of a set of layers of segmented objects with each pixel having three color values or one luminance and two color values.
To determine the similarity, or separately and alternatively non-similarity, of one or a plurality of 2DDDs with a plurality of 2DDDs, each 2DDD may be numerically characterized. For example, each section of the 2DDDs of a corpus of 2DDDs may be assigned an equivalence signature that has the property that small changes to the section of the 2DDD, which maintain similarity with the original section of the 2DDD, will not significantly change the equivalence signature.
As specified by Eqn. 3, the equivalence signature for each section of a 2DDD is given by the functional representation of the Second Homotopy Group computed over the data of the 2DDD's section interpreted as a mapping between two, two dimensional spheres. Once an equivalence signature is assigned to a section of a 2DDD, then a plurality of 2DDDs that are deformations of the former 2DDD will have equivalence signatures that are within a bounded range of the equivalence signature of the former 2DDD as given by Eqn. 7. That range is computed based on configurable similarity threshold parameters that specify the maximum shift and ratio of the values of the data at a point in two similar sections of 2DDDs. Consequently, 2DDD sections that are candidates for similarity with a section of a target 2DDD can be identified, in a database, by requiring that the absolute value of the difference between the values of their equivalence signatures and that of the target's section be no more than the maximum allowed difference computed in terms of the target's data and the similarity threshold parameters. If a target 2DDD has NS(T) sections of which NS(T)(X) are similar to the sections of another 2DDD, X, then the degree of similarity of X to the target 2DDD is
The closer the degree of similarity to one, the more similar X is to the target 2DDD. 2DDDs in a database that are not similar to a target 2DDD will have a similarity degree of zero.
Operation—Preferred Embodiment—
In
The Main Memory 1100 typically consists of random access memory (RAM) embodied as integrated circuit chips and is used for temporarily storing the 2DDDs, configuration data, database records and intermediate and final results processed and produced by the instructions implementing the method invented here as well as the instructions implementing the method, the operating system and the functions of other components in the computing device 1000.
Zero or more external audio and/or video interfaces 1200 convert digital and/or analog A/V signals from external A/V sources into digital formats that can be reduced to PCM/YUV values and the like. Video frames of YUV values at each two-dimensional point in the frame are 2DDDs.
Storage sub-system interface 1300 manages the exchange of data between the computing device 1000 and one or more internal and/or one or more external storage devices such as hard drives which function as tangible media for storage of the data processed by the instructions embodying the method of this invention as well as the computer program files containing those instructions, and the instructions of other computer programs directly or indirectly executed by the instructions, embodying the method of this invention.
The bus 1400 embodies a channel over which data is communicated between the components of the computing device 1000.
The processing unit 1500 is typically one or more chips such as a CPU or ASICs, that execute instructions including those instructions embodying the method of this invention.
The network interface 1600 typically consists of one or more wired or wireless hardware devices and software drivers such as NIC cards, 802.11x cards, Bluetooth interfaces and the like, for communication over a network to other computing devices.
The human interface subsystem 1700 typically consists of a graphical input device, a monitor and a keyboard allowing the user to select files that contain 2DDDs that are to be analyzed by the method.
In
An equivalence signature is computed as in, 1500, for a 2DDD under the control of the Analysis Manager. First, the Analysis Manager 1550 instructs the Data Reader 1510 to read the 2DDD and return control to the Analysis Manager 1550 upon completion. Secondly, when control is returned by the Data Reader 1510, the Analysis Manager 1550 instructs the Data Preprocessor 1520 to process the output from the Data Reader 1510 and return control to the Analysis Manager 1550 upon completion. Third, when control is returned by the Data Preprocessor 1520, the Analysis Manager 1550 instructs the Signature Generator 1530 to process the output from the Data Preprocessor 1520 and return control to the Analysis Manager 1550 upon completion. Fourth, when control is returned by the Signature Generator 1530, the Analysis Manager instructs the Signature Database 1560 to record the output from the Signature Generator 1530, said Signature Database may write the output to a file by means of calls to the Operating System 1570, and return control to the Analysis Manager 1550 upon completion. The Analysis Manager 1550 then waits for the next request.
The Data Reader module 1510 reads the 2DDD from its storage medium such as a file on a hard drive interfaced to the bus of the computing device or from a networked storage device or server using TCP/IP or UDP/IP based protocols, and the like.
The Data Preprocessor module 1520 finds the start and end of each section in the 2DDD by finding the start layer markers in the data stream of the 2DDD.
In
The calculations of 102-108 are performed while looping over the remaining sections. When no more sections remain 110, a new record is added to the Signature Database 1560 with fields containing the keys of the record of each section of the 2DDD, as added in the latter step, the meta data about the 2DDD including the path or URL to the file containing the 2DDD, the data and time that the 2DDD was last written, a text description of the data in the 2DDD, the name of the source or author for the 2DDD, the policy for the use of the 2DDD, other signatures or features of the 2DDD, and the like.
In
In a second embodiment, an equivalence signature is computed for a 2DDD as in 1500 through the pipelined steps: Data Reader 1510→Data Preprocessor 1520→Signature Generator 1530→Signature Database 1560 with the Data Reader 1510, Data Preprocessor 1520, Signature Generator 1530, and Signature Database 1560 performing the same function as in the preferred embodiment except that each module calls the succeeded module in the pipeline upon completion of their computation. In this second embodiment, the Analysis Manager is not invoked.
Accordingly, the reader will see that the method invented here introduces novel features of an equivalence signature including that
The present invention has been described by a limited number of embodiments. However, anyone skilled in the art will recognize numerous modifications of the embodiments. It is the intention that the following claims include all modifications that fall within the spirit and scope of the present invention.
This application claims the benefit of PPA Ser. No. 60/883,001, filed Dec. 31, 2006 by the present inventor.
Number | Name | Date | Kind |
---|---|---|---|
5442716 | Otsu et al. | Aug 1995 | A |
5933823 | Cullen et al. | Aug 1999 | A |
7031980 | Logan et al. | Apr 2006 | B2 |
20080140741 | Brooks | Jun 2008 | A1 |
20080162422 | Brooks | Jul 2008 | A1 |
20080215529 | Brooks | Sep 2008 | A1 |
20080215530 | Brooks | Sep 2008 | A1 |
20080215566 | Brooks | Sep 2008 | A1 |
20080215567 | Brooks | Sep 2008 | A1 |
Number | Date | Country | |
---|---|---|---|
20080162421 A1 | Jul 2008 | US |
Number | Date | Country | |
---|---|---|---|
60883001 | Dec 2006 | US |