This invention relates generally to techniques for shape analysis, and more particularly to techniques for automatically classifying two-dimensional (2D) shapes in images.
Classification of 2D shapes regardless of their position, size and orientation is an important problem in computer vision and pattern recognition. Its application is spread out into many fields, such as classification of blood cell, cancer and chromosomes, industrial inspection, target recognition, medical image recognition, scene analysis, and modeling of biological systems. Generally, shape classification is a process of comparing and recognizing shape by analyzing the information of the shape's boundaries. This seems an easy task for a human being, but is quite difficult for computers, particularly after objects are scaled, rotated and/or translated. Thus, the study of shape for the purpose of general object classification, recognition, or retrieval is an active field of current research.
Recent literature has addressed this topic, and various image processing methods have been applied. These methods can be basically classified into two techniques.
The first technique requires the projection of shape instances into a common space and then the implementation of classification on the projection space. For example: Fourier descriptors (E. Persoon, et al. “shape discrimination using Fourier descriptors” IEEE Trans. Syst. Man. Cybern, vol. 7, p 170-179, 1977), invariant moments (F. Zakaria, et. al “Fast algorithm for computation of moment invariants” Pattern Recognition, vol. 20, p 639-643, 1987), autoregressive models (S. R. Dubois, et. al “An autoregressive model approach to two-dimensional shape classification”, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 8 p 55-66, 1986), and principal component analysis (U.S. Patent No. 2002/0164060 A1 entitled METHOD FOR CHARACTERISING SHAPES IN MEDICAL IMAGES).
An advantage of these projection-based methods is that they are independent of translation and rotation. However, a disadvantage is that they exhibit an inherent loss of information since the projection transformation is not a one-to-one correspondence. That is, one point in the projection space may correspond to several shapes whose visual appearance can be quite different. Therefore, shape classification based on such technique may lead to incorrect results.
The second technique comprises locating a set of landmark points along shape boundaries, specifying a distance measure between corresponding landmarks, and performing a distance-based clustering. As such, shape classification is reduced to the general clustering problem for which numerous solutions have been proposed. For example: M. Duta et. al. present a method using Mean Alignment Error (MAE) as a distance to measure the difference of shapes and classify shapes based on MAE. (M. Duta, et. al. “Automatic Construction of 2D Shape Models”, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol 23, no. 5, p 433-446, 2001). U.S. Pat. No. 6,611,630 entitled METHOD AND APPARATUS FOR AUTOMATIC SHAPE CHARACTERIZATION and U.S. Pat. No. 6,009,212 entitled METHOD AND APPARATUS FOR IMAGE REGISTRATION are directed to a method to classify shapes based on a best match probability with an average shape of a characterizing population group. The limitation of this technique is that pair wise correspondence between the landmarks of shapes is difficult to achieve in practice because of the noise and variation among individuals.
Given the drawbacks and limitation of the prior art, there exists a need for a method to find a simple, efficient and highly accurate method for shape classification.
An objective of the present invention is to provide an automated method for 2D shape classification.
According to one aspect of the present invention, this objective is achieved by providing an automated method for classifying 2D shapes. The method includes several steps. A training dataset is created for the shape under study. The training set includes two groups of data: a similar shape group and a dissimilar shape group. A polygonal approximation for each shape in the training dataset is generated, and an average shape from the similar shape group is computed. The shapes in the database are aligned to the average shape and their similarity distances is outputted. The distribution of similarity distances is obtained and the shapes are classified into two clusters based on their distances.
The present invention is viewed as having some advantages. For example, the method characterizes the study shape by an average shape and a threshold related to the similarity distances. The shape classification is efficient, since its computation complexity is controlled under O(mn log(mn)), where m is the number of landmarks on the average shape and n is the number of landmarks on a shape instance. The shape classification is robust, since no pairwise correspondence of landmarks is required during shape alignment, and even the number of landmarks can be different. Further, the orientation correction becomes easier since the rotation angle of a shape instance to the average shape can be provided after shape alignment.
The foregoing and other objects, features, and advantages of the invention will be apparent from the following more particular description of embodiments of the invention, as illustrated in the accompanying drawings. The elements of the drawings are not necessarily to scale relative to each other.
The following is a detailed description of the preferred embodiments of the invention, reference being made to the drawings in which the same reference numerals identify the same elements of structure in each of the several figures.
The present invention discloses a method for automatically classifying 2D shapes in images. A flow chart of a method in accordance with the present invention is generally shown in
In a further embodiment, shown in the flow chart of
At step 14, a polygonal approximation of the boundary is generated to represent each share in the dataset. An advantage of interpreting a shape by a polygon is that it provides a simple description of shape with reduced loss of information. According to an embodiment of the present invention, the polygonal approximation can be generated by connecting a set of signature points and sampled points extracted from shape. The signature points are the points representing the salient features of shape, such as points with high curvature or at specified location. The sampled points are points located between the signature points based on a certain criteria.
One criterion is to find the gravity center of the shape, draw an arbitrary radius from the gravity center to the shape boundary, and then place a set of sampled points along the boundary by moving the radius clockwise with the regular interval of 360/n degrees. The resulting shape can be represented as:
S=(rθ1,rθ2,rθ3,P1,p2 . . . rθi,rθi+1, . . . pj,rθi) θi=(360/n)*I
where rθi, 1<i<n, is a sampled point obtained by the intersection between the i-th radius and the shape boundary. pj is a signature point.
Another criterion is to select sampled points by filling a number of equidistance points (vdi) between signature points. The shape can be given by:
S=(vd1,vd2,vd3,p1,p2 . . . vd1,vd1+1 . . . vdn,)
It is noted that the present invention is not limited to using the above methods to generate the polygonal approximation of shape boundary. Other known algorithms can be used wherein the resultant polygon correctly approximates a shape without losing the salient information.
To measure shape similarity, two shapes are aligned as close as possible. According to an embodiment of the present invention, a method using turning functions as a basis for the similarity measure of shapes is provided. Given the polygonal approximations of two shapes, their distance is computed from the turning functions θ(s) of the two polygons. A turning function, or a cumulative angle function, represents a polygon by measuring the angle of the counter clockwise tangent as a function of the arc-length s from a reference point O on the shape approximation. It tracks the turning that takes place, for example, increasing with left hand turns, and decreasing with right hand turns. To ensure generality, the perimeter length of each polygon can be rescaled to 1. Thus, for a simple closed contour, θ(s) starts at θ(O) (assuming that the reference point O is placed at differential point along the contour) and increases to θ(1)=θ(O)+2π.
There are several properties that make the turning function particularly suited for shape similarity measurement. First, it is piecewise constant for polygons, making computations particularly easy and fast. Secondly, the function θ(s) is invariant under translation and scaling of the polygon according to the definition. Thirdly, rotation of the polygon over an angle θ corresponds to a vertical shift of the function with an amount θ, while changing the location of the reference point O by an amount tε[0,1] along the perimeter of polygon results in a circular shift of the function θ(s).
Let Tp and Sp be polygonal approximations of the template shape T and a shape instance S.
where t represents the position of the reference point along the polygon, and θ corresponds to the rotation of polygon. As shown by Arkin et al (E. M. Arkin, L. P. Chew, D. P. Huttenlocher, K. Kedem, and J. S. Mitcheel, “An efficiently computable Metric for Comparing Polygonal shapes”. IEEE Trans. On Pattern Analysis and Machine Intelligence. Vol. 13 No. 3 1991 pp 209-215), this problem can be solved by:
where θ* is the optimal orientation for any fixed t and is given by:
θ*=∫01θs
From these two equations, two matrices are obtained. One is D2 matrix from Equation 2 and the other is θ matrix from Equation 3. The correct (i.e., best matched) orientation of the shape instance can be determined by searching the minimal L2 distance in D2 matrix and its corresponding element in θc matrix.
There are some advantages of using the turning function. For example, the rotation angle θ* from the best matched turning functions represents the orientation difference between two shapes, and can be used for orientation correction if needed. Further, the computation complexity for aligning two turning functions is O(mn log(mn)), where m is the number of landmarks on the template shape and n is the number of landmarks on a shape instance. This is also the computation complexity of the entire shape classification process, since shape alignment takes most time needed for shape classification comparing to the polygon approximation generation and distance classification as will be discussed below. Thus by choosing an appropriate number of landmarks, the shape classification can be made efficient. Another advantage of using the turning function (for example, rather than the distance between the corresponding landmark points) is that the requirement of one-to-one correspondence is no longer necessary, and even the number of landmarks can be different. This can promote shape classification robustness.
Each shape in the similar shape group contributes to an average shape which characterizes the feature of shape under study.
Next, the aligned turning functions are divided into n equidistance segments along the X-axis, which corresponds the perimeter length of a shape. In each segment, a mean point is determined from the first points on the turning functions. This is shown in
Several cycles of the process may be needed for the average shape to converge. The final average shape is then output as the average shape characterizing the similar shape group (step 54).
When the average shape (step 54/step 11) is obtained, the shapes in the training dataset are aligned to it and their similarity distances D2 are computed (step 12). Combining the distances together forms a distribution of the similarity distances of the training dataset, as shown in
At step 13, the shapes are classified into two groups based on their distances. Since the distance distribution is one dimension, the classification problem can be simplified as determining an appropriate threshold to obtain a “good” discrimination of distances. Known thresholding methods based on image histogram shape analysis can be employed wherein the threshold can minimize the classification error. The distance distribution of the similar shape group can be assumed to the normal distribution. The mean of the distribution is set to zero, which corresponds to the average shape. The variance σ of distribution accounts for the shape variation within the similar shape group. In one particular application, Applicant set the initial threshold (81) as 3*σ, as shown in
According to the present invention, the classification of study shape is dependent upon an average shape together with its classification threshold extracted from the training dataset. As new shapes are input, they are compared against the average shape of the study shape. If a match is found and its distance is less than the threshold, the new shape is classified as a member of the similar shape group. To promote consistency and the accuracy of the classification, the characteristic average shape and classification threshold are refined to reflect the addition of new members, which can be accomplished by updating the average shape and classification threshold (e.g., every time) when the new shapes accumulate to a certain amount.
Note that the disclosed method focuses on one shape type, and the classification result is either YES or NO (eg. 1 or 0). If a more complicated system is needed to classify several different shape types in the database, the disclosed method can be extended by creating a training dataset for each shape type, then studying the average shape and the classification threshold for each type, and finally performing classification by finding a shape type with the minimal similarity distance.
The present invention may be implemented for example in a computer program product. A computer program product may include one or more storage media, for example; magnetic storage media such as magnetic disk (such as a floppy disk) or magnetic tape; optical storage media such as optical disk, optical tape, or machine readable bar code; solid-state electronic storage devices such as random access memory (RAM), or read-only memory (ROM); or any other physical device or media employed to store a computer program having instructions for controlling one or more computers to practice the method according to the present invention.
The system of the invention can include a programmable computer having a microprocessor, computer memory, and a computer program stored in said computer memory for performing the steps of the method. The computer has a memory interface operatively connected to the microprocessor. This can be a port, such as a USB port, over a drive that accepts removable memory, or some other device that allows access to camera memory. The system includes a digital camera that has memory that is compatible with the memory interface. A photographic film camera and scanner can be used in place of the digital camera, if desired. A graphical user interface (GUI) and user input unit, such as a mouse and keyboard can be provided as part of the computer.
The invention has been described in detail with particular reference to a presently preferred embodiment, but it will be understood that variations and modifications can be effected within the spirit and scope of the invention. The presently disclosed embodiments are therefore considered in all respects to be illustrative and not restrictive. The scope of the invention is indicated by the appended claims, and all changes that come within the meaning and range of equivalents thereof are intended to be embraced therein.
Reference is made to, and priority is claimed from, U.S. Provisional Application No. 60/630,270, entitled “METHOD FOR AUTOMATIC SHAPE CLASSIFICATION”, filed on Nov. 23, 2004 in the name of Hui Luo and which is assigned to the assignee of this application, and incorporated herein by reference. Reference is made to commonly assigned application U.S. Ser. No. 10/993,055, entitled “DETECTION AND CORRECTION METHOD FOR RADIOGRAPH ORIENTATION”, filed on Nov. 19, 2004 in the names of Luo et al, and which is assigned to the assignee of this application, and incorporated herein by reference. Reference is made to U.S. Provisional Application No. 60/630,286, entitled “AUTOMATED RADIOGRAPH CLASSIFICATION USING ANATOMY INFORMATION”, filed on Nov. 23, 2004 in the names of Luo et al, and which is assigned to the assignee of this application, and incorporated herein by reference. Reference is made to U.S. Provisional Application No. 60/630,287, entitled “METHOD FOR RECOGNIZING PROJECTION VIEWS OF RADIOGRAPHS”, filed on Nov. 23, 2004 in the names of Luo et al, and which is assigned to the assignee of this application, and incorporated herein by reference. Reference is made to U.S. Provisional Application No. 60/630,326, entitled “METHOD FOR CLASSIFYING RADIOGRAPHS”, filed on Nov. 23, 2004 in the names of Luo et al, and which is assigned to the assignee of this application, and incorporated herein by reference.
Number | Name | Date | Kind |
---|---|---|---|
5295201 | Yokohama | Mar 1994 | A |
6009212 | Miller et al. | Dec 1999 | A |
6181805 | Koike et al. | Jan 2001 | B1 |
6252981 | Guest et al. | Jun 2001 | B1 |
6304677 | Schuster | Oct 2001 | B1 |
6611630 | Miller et al. | Aug 2003 | B1 |
7106903 | Chang et al. | Sep 2006 | B2 |
20020146174 | Shimura | Oct 2002 | A1 |
20020164060 | Paik et al. | Nov 2002 | A1 |
Number | Date | Country |
---|---|---|
0 540 869 | May 1993 | EP |
Number | Date | Country | |
---|---|---|---|
20060110046 A1 | May 2006 | US |
Number | Date | Country | |
---|---|---|---|
60630270 | Nov 2004 | US |