This application claims the benefit of Korean Patent Application No. 2010-0088336, filed on Sep. 9, 2010 in the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference.
1. Field
Embodiments relate to a method and apparatus to generate an object descriptor using curvature gabor filters.
2. Description of the Related Art
The importance of security using face recognition is gradually increasing due to the recent prevalence of terrorism and information theft. Establishing a biometric recognition solution to prevent terrorism is interesting. A representative effective method to counteract terrorism is to reinforce border security and identity verification. The international Civil Aviation Organization (ICAO) has recommended that biometric recognition information be used by a mechanical travel document reader. The Enhanced Border Safety and Visa Entry Reform Act of the U.S. provides for an enhanced introduction level of biometric identifiers and associated software and mandates the use of biometric recognition information in travel documents, passports, and U.S. visas. Biometric passports have been adopted by several nations, such as some European countries, the United States, and Japan. In addition, a new type of biometric passport incorporating a chip in which biometric information of a user is stored has ever been used.
Many agencies, companies, and other types of organizations demand that their employees or visitors use admission cards for identifying individuals today. Accordingly, each employee or visitor may need to always carry a key card or a key pad for use with a card reader when he or she stays in a designated admission allowed area.
However, if the employee or visitor losses or is robbed of the key card or the key pad, serious security problems, such as invasion of an unauthenticated person into a restricted area, may occur. To prevent the security problem, biometric recognition systems to automatically recognize and verify personal identities using human biometric information or behavior characteristics have been developed. These biometric recognition systems have been used in banks, airports, and other high-security facilities, and more simplified and highly reliable biometric recognition systems have also been studied.
Personal features used by the biometric recognition systems include fingerprints, face shape, handprints, hand shape, thermal images, voice, signature, venous shape, typing keystroke dynamics, retina, iris, etc. Face recognition is the most frequently used personal identification technique to verify a personal identify from one or more faces present in a still image or a moving image using a given facial database. Facial image data may greatly vary depending on poses or illumination and therefore, it may be difficult to classify various pieces of pose data of the same person into the same class.
Although various image processing methods to reduce errors in face recognition have been recommended, these methods may result in errors caused by assumption of linear distribution and assumption of Gaussian distribution when attempting to recognize a face.
In particular, a gabor wavelet filter, which has been used in face recognition, is suitable to capture various changes, such as expression change and illumination change in a facial image, but may necessitate complex calculation processes when face recognition is performed using gabor wavelet features. Parameters of the gabor wavelet filter are restrictive. The use of the gabor wavelet filter having these restrictive characteristics may increase occurrence probability of errors in face recognition and deteriorate a recognition rate in face recognition. In particular, when expression change and illumination change in a facial image are significant, the recognition rate may be further deteriorated.
Therefore, it is an aspect of embodiments to increase a recognition rate of even a relatively small image with use of an extended number of curvature gabor filters having controllable curvatures and to reduce the amount of calculation required for face recognition by performing the face recognition using only some of the extended curvature gabor filters which have a great effect on the recognition rate.
Additional aspects of embodiments will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the embodiments.
In accordance with an aspect of an embodiment, a method of generating an object descriptor includes extracting gabor features from an input object image by applying a plurality of curvature gabor filters, generated via combination of a plurality of curvatures and a plurality of Gaussian magnitudes, to the object image, and generating an object descriptor for object recognition by projecting the extracted features onto a predetermined base vector.
The curvatures may be in the range of a value equal to or greater than about 0.0 to a value equal to or less than about 0.2, and the Gaussian magnitudes may be in the range of a value equal to or greater than about 0.5π to a value equal to or less than about 2π.
The curvatures may be represented by c={0.0, 0.05, 0.1, 0.2}, and the Gaussian magnitudes may be represented by σ={0.5π, π, 2π}.
When the plurality of gabor filters are arranged in columns on the basis of the curvatures and in rows on the basis of the Gaussian magnitudes, the gabor features may be extracted using some of the gabor filters which are arranged in a T-shaped form.
The filters arranged in a T-shaped form may include filters having the Gaussian magnitude σ={0.5π, π, 2π} based on the curvature c={0.0}, and filters having the Gaussian magnitude σ={π} based on the curvature c={0.05, 0.1, 0.2}.
The object descriptor generating method may further include extracting gabor features from a training object image by applying the curvature gabor filters to the training object image, and selecting gabor features by performing boosting learning for object image classification with respect to the extracted gabor features, and establishing a gabor feature set including the selected gabor features, and the extraction of the gabor features may include extracting gabor features by applying curvature gabor filters having the established gabor feature set to the input object image.
The object descriptor generating method may further include generating a base vector by performing linear discriminant analysis of the established gabor feature set, and the generation of the object descriptor may include generating an object descriptor for object recognition by projecting the extracted gabor features onto the generated base vector.
In accordance with another aspect of an embodiment, a method of generating an object descriptor includes extracting first gabor features from a training object image by applying curvature gabor filters to the training object image, selecting gabor features by performing boosting learning for object image classification with respect to the extracted gabor features and establishing a gabor feature set including the selected gabor features, generating a base vector by performing linear discriminant analysis of the established gabor feature set, extracting second gabor features from an input object image by applying a plurality of curvature gabor filters, generated via combination of a plurality of curvatures and a plurality of Gaussian magnitudes among the established gabor feature set, to the input object image, and generating an object descriptor for object recognition by projecting the extracted second gabor features onto the generated base vector.
The curvatures may be in the range of a value equal to or greater than about 0.0 to a value equal to or less than about 0.2, and the Gaussian magnitudes may be in the range of a value equal to or greater than about 0.5π to a value equal to or less than about 2π.
The curvatures may be represented by c={0.0, 0.05, 0.1, 0.2}, and the Gaussian magnitudes may be represented by σ={0.5π, π, 2π}.
When the plurality of gabor filters are arranged in columns on the basis of the curvatures and in rows on the basis of the Gaussian magnitudes, the gabor features may be extracted using some of the gabor filters which are arranged in a T-shaped form.
The filters arranged in a T-shaped form may include filters having the Gaussian magnitude σ={0.5π, π, 2π} based on the curvature c={0.0}, and filters having the Gaussian magnitude σ={π} based on the curvature c={0.05, 0.1, 0.2}.
In accordance with a further aspect of an embodiment, an apparatus of generating an object descriptor includes a first feature extractor to extract gabor features from an input object image by applying a plurality of curvature gabor filters, generated via combination of a plurality of curvatures and a plurality of Gaussian magnitudes, to the object image, and an object descriptor generator to generate an object descriptor for object recognition by projecting the extracted features onto a predetermined base vector.
The curvatures may be in the range of a value equal to or greater than about 0.0 to a value equal to or less than about 0.2, and the Gaussian magnitudes may be in the range of a value equal to or greater than about 0.5π to a value equal to or less than about 2π.
The curvatures may be represented by c={0.0, 0.05, 0.1, 0.2}, and the Gaussian magnitudes may be represented by σ={0.5π, π, 2π}.
When the plurality of gabor filters are arranged in columns on the basis of the curvatures and in rows on the basis of the Gaussian magnitudes, the gabor features may be extracted using some of the gabor filters which are arranged in a T-shaped form.
The filters arranged in a T-shaped form may include filters having the Gaussian magnitude σ={0.5π, π, 2π} based on the curvature c={0.0}, and filters having the Gaussian magnitude σ={π} based on the curvature c={0.05, 0.1, 0.2}.
The object descriptor generating apparatus may further include a second feature extractor to extract gabor features from a training object image by applying the curvature gabor filters to the training object image, and a selector to select gabor features by performing boosting learning for object image classification with respect to the gabor features extracted from the second feature extractor and to establish a gabor feature set including the selected gabor features, and the first feature extractor may extract gabor features by applying curvature gabor filters having the established gabor feature set to the input object image.
The object descriptor generating apparatus may further include generating a base vector generator to generate a base vector by performing linear discriminant analysis of the established gabor feature set, and the object descriptor generator may generate an object descriptor for object recognition by projecting the gabor features, extracted from the first feature extractor, onto the generated base vector.
These and/or other aspects of embodiments will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout.
As illustrated in
An object image database 160 stores images for object recognition or identity identification. For example, in the case of face recognition, the object image database 160 may store information about a plurality of facial images having various facial expressions, angles, and brightness values in order to increase face recognition efficiency. An image preprocessor 170 performs predetermined preprocessing of an input object image or an image before it is stored in the database. For example, the image preprocessor 170 removes a background portion from an input image, adjusts the magnitude of the image on the basis of an eye position, and thereafter, modifies the facial image through a pre-processing process for decreasing a dispersion degree of lighting to make the facial image be suitable for generation of a face descriptor. An input image acquisitor 180 acquires an input object image for object recognition. For example, the input image acquisitor 180 acquires an object image or a human facial image for object recognition or identity verification using a camera or a camcorder. The object image database 160, the image preprocessor 170, and the input image acquisitor 180 may be provided separately from the object descriptor generating apparatus 100, or may be integrated with the object descriptor generating apparatus 100 to define a single apparatus.
In the object descriptor generating apparatus 100, the first feature extractor 110 extract features by applying extended curvature gabor filters having various curvatures to an input object image. Here, various extended curvature gabor filters may be used according to curvatures thereof. For example, a set of extended gabor filters and a set of extended curvature gabor filters having different curvatures may be used to extract different features. The first feature extractor 110 may also apply curvature gabor filters to an input object image by receiving optimal curvature gabor filter parameters extracted in a learning stage and filter application positions in the object image. The features and applications of the extended curvature gabor filters will be described below with reference to
The object descriptor generator 120 generates an object descriptor using the gabor features received from the first feature extractor 110. Specifically, the object descriptor generator 120 generates an object descriptor or an object feature vector by projecting the gabor features onto a base vector having a high classification rate. The object descriptor generator 120 may also generate an object descriptor using a base vector generated by the base vector generator 150. The base vector is generated through Linear Discriminant Analysis (LDA) learning of a gabor feature set generated by the selector 140.
The second feature extractor 130 extracts gabor features by applying extended curvature gabor filters to an object image stored in the object image database 160. Although the first and second feature extractors 110 and 130 perform different operations in the present embodiment for convenience of description, the above described functions may be performed by a single feature extractor.
The selector 140 selects efficient gabor features by performing boosting learning on the gabor features extracted by the second feature extractor 130 and establishes a gabor feature set including the selected gabor features. The gabor feature set includes a parameter and its applied position of a curvature gabor filter optimal to a specific object image. The selection of efficient gabor features is performed because the number of gabor filters applied throughout all areas of an image is too numerous. It will be understood that the boosting learning may be replaced with a statistical re-sampling algorithm. The boosting learning will be described below in detail with reference to
The base vector generator 150 generates a base vector by performing LDC learning on the gabor feature set generated by the selector 140. The LDA learning will be described further below.
First, a conventional gabor filter, which is frequently used for object recognition or face recognition, is represented by the following Equation 1.
Here, “kv” denotes the magnitude of the gabor filter and is represented by
and σ=2π. In this case, to provide the gabor filter with an angle according to an (x, y) increase, the following Equation 2 is used.
Here, an angle change according to the (x, y) increase follows
and a total of 8 angles is used. The conventional gabor filter is further represented by the following Equation 3.
The curvature gabor filter according to the present embodiment is formed to provide the gabor filter represented by Equation 3 with a curvature. The curvature gabor filter is further represented by the following Equation 4 as obtained by adding a curvature to Equation 2.
Here, “c” denotes a curvature parameter. If the value of “c” increases, the curvature increases. If the value of “c” decreases, the curvature becomes similar to a straight line. Finally, if the value of “c” is zero, the curvature gabor filter has the same shape as the conventional gabor filter.
The curvature gabor filter, obtained by using Equations 1 and 4, may be defined by the following Equation 5.
Here, values represented by
are used.
Since the extended curvature gabor filters according to an embodiment use angles in the range of 0° to 360°, twice as many as gabor filters are available as compared to the conventional gabor filters. Complexity in calculation, caused by an increase in the number of filters, may be removed by gabor feature selection using boosting learning, which will be described hereinafter. Even in the case of the extended curvature gabor filters, small filters corresponding to v={−2, −1} are added to analyze a relatively small image having a low resolution.
As illustrated in
In an embodiment, an object may be analyzed by both considering linear components and curvilinear components of an object image and respectively applying curvature gabor filters having controllable curvatures to the curvilinear components, with optimal parameters and their positions being found and analyzed by combining the analysis results. The increase in complexity of calculation, caused by using twice as many filters as conventional gabor filters due to analyzing the angle change (i.e. directivity) up to 360°, may be solved by selecting optimal filters through the aforementioned boosting learning.
Referring to
In operation 500, extended curvature gabor filters having various curvatures may be generated. Here, the extended curvature gabor filters may be represented by the above Equation 5. Thus, the curvature of the curvature gabor filter depends on the parameter c.
In operation 502, features may be extracted by applying the extended curvature gabor filters generated in operation 500 to a training object image.
Next, in operation 504, an optimal set of extended curvature gabor filters may be selected by performing the boosting learning with respect to the features extracted in operation 502.
As described above, the number of gabor filters applied to all areas of an object image may be too numerous and thus, it is impossible to use all the gabor filters. When too many features are extracted, it is difficult to generate an optimal LDA base vector by comparing correlations between the gabor filter features, an optimal filter set may be found through adaboost in an embodiment. In this case, a boosting learning method, such as GentleBoost, RealBoost, KLBoost, or JSBoost, may be used. An object image recognition efficiency may be increased by selecting complementary gabor features from respective sub-sets using the boosting learning.
In an embodiment, the boosting learning may be performed under the assumption that one gabor filter, i.e. one parameter and its position are used. Since an adaboost algorithm is known, a detailed description thereof will be omitted herein. Such an adaboost algorithm is as follows:
begin initialize D={x1, y1, . . . , xn, yn}, W1(i)=1/n, i=1, . . . , n
end
The selected gabor filter set M (M={(m, n) I selected features by boosting}) may be represented using the position and parameter by the following Equation 6.
Here, “G” denotes the magnitude of a gabor filter. An applied value of the gabor filter may be divided into a rear number portion and an imaginary number portion. Magnitude information and phase information may be extracted by using the following Equation 7.
The extracted magnitude information and phase information may be used as a feature vector of an object. Since the magnitude may robustly withstand image transformation, the magnitude information may be used. In addition, since the phase information represents complementary features with the magnitude, the phase information may be selectively used with the magnitude information.
In operation 506, a base vector is generated through the LDA learning using the selected gabor filter feature set. That is, a feature vector generated using Equations 6 and 7 is projected onto a base vector optimizing classification. Here, the LDA technique is a technique of linearly projecting data onto a sub-space which reduces within-class scatter and maximizes between-class scatter. The LDA base vector generated in operation 506 may be a representative characteristic value of an object recognition set and may be efficiently used for object recognition of the set. The LDA base vector may be calculated through the following process.
The LDA may use a ratio of a Within-Scatter-Matrix SW to a Between-Scatter-Matrix SB, and may be represented by the following Equation 8.
The calculated scatter matrices may generate a base vector, which minimizes dispersion of the Within-Scatter-Matrix SW and maximizes dispersion of the Between-Scatter-Matrix SB, by using the following Equation 9.
Here, Φopt represents an LDA base vector.
In operation 508, gabor features with respect to an input object image are extracted. More particularly, the gabor features may be extracted by using the parameters and positions of the curvature gabor filters selected in operation 504. In operation 510, the extracted features (feature vectors) are projected onto a base vector. The extracted features may be projected onto the base vector generated in operation 506. In operation 512, an object descriptor may be generated. If it is assumed that the magnitude vector extracted from the above-described Equation 7 is “m”, the vector projection expression, i.e. the following Equation 10 may be used.
y=ΦT(m−mmean) Equation 10
Here, the vector “y” becomes the final feature of the object, and this feature vector may act as a general gabor feature vector or a curvature gabor feature vector because the filter operation may be controlled to act as the general gabor filter or the curvature gabor filter based on the filter parameter c. The projected vector “y” is a final object descriptor.
Referring to
In an embodiment, the similarity calculation is performed using Weighted Normalized Correlation such as the following Equation 11.
The obtained similarity has a value between −4 and 4. An input object is determined to be different from the other object if the similarity value is −4, and is determined to be fully identical to the other object if the similarity value is 4. In addition, a specific threshold may be set and an input object may be recognized as the same as the other object if the similarity value is greater than the specific threshold.
Referring to
The embodiments can be implemented in computing hardware and/or software, such as (in a non-limiting example) any computer that can store, retrieve, process and/or output data and/or communicate with other computers. The computing hardware may include a processor for performing operations. The results produced may be displayed on a display of the computing hardware. For example, the apparatus in
In addition, the above described embodiments may be implemented via a computer readable code in a computer readable recording medium executable by a computer.
The computer readable recording medium includes all kinds of recording devices in which data that may be read by a computer system is stored.
Examples of the computer readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk and an optical data storage device, although a computer readable recording medium is not limited to these examples. The computer readable recording medium may be connected to a computer system via a network and may store and execute the computer readable code in a distributed manner. Programmers skilled in the art will easily reason functional programs, codes, and code segments to realize embodiments.
As is apparent from the above description, according to embodiments, a recognition rate of even a relatively small image may be increased with use of an extended number of curvature gabor filters having controllable curvatures. Further, the amount of calculation required for face recognition may be reduced by performing the face recognition using only some of the extended curvature gabor filters which have a great effect on the recognition rate. Accordingly, it may be possible to achieve a face recognition system to be easily mounted in a portable appliance, such as a mobile phone or a camera.
Although a few embodiments have been shown and described, it would be appreciated by those skilled in the art that changes may be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the claims and their equivalents.
Number | Date | Country | Kind |
---|---|---|---|
10-2010-0088336 | Sep 2010 | KR | national |
Number | Name | Date | Kind |
---|---|---|---|
6466695 | Potzsch et al. | Oct 2002 | B1 |
20070172099 | Park et al. | Jul 2007 | A1 |
20080107311 | Huang et al. | May 2008 | A1 |
20090028444 | Hwang et al. | Jan 2009 | A1 |
20100232657 | Wang | Sep 2010 | A1 |
20110135166 | Wechsler et al. | Jun 2011 | A1 |
20120155718 | Hwang et al. | Jun 2012 | A1 |
20130108123 | Hwang et al. | May 2013 | A1 |
Number | Date | Country |
---|---|---|
2005-44330 | Feb 2005 | JP |
10-2008-0041931 | May 2008 | KR |
Entry |
---|
Su, Yu, et al. “Hierarchical ensemble of global and local classifiers for face recognition.” Image Processing, IEEE Transactions on 18.8 (2009): 1885-1896. |
Andrea F. Abate, Michele Nappi, Daniel Riccio, Gabriele Sabatino, 2D and 3D face recognition: A survey, Pattern Recognition Letters, vol. 28, Issue 14, Oct. 15, 2007, pp. 1885-1906, ISSN 0167-8655. |
Shufu Xie, Shiguang Shan, Xilin Chen, Xin Meng, Wen Gao, Learned local Gabor patterns for face representation and recognition, Signal Processing, vol. 89, Issue 12, Dec. 2009, pp. 2333-2344, ISSN 0165-1684. |
Di Huang; Caifeng Shan; Ardabilian, M.; Yunhong Wang; Liming Chen, “Local Binary Patterns and Its Application to Facial Image Analysis: A Survey,” Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on , vol. 41, No. 6, pp. 765,781, Nov. 2011. |
Arar, N.M.; Hua Gao; Ekenel, H.K.; Akarun, L., “Selection and combination of local Gabor classifiers for robust face verification,” Biometrics: Theory, Applications and Systems (BTAS), 2012 IEEE Fifth International Conference on , vol., no., pp. 297,302, Sep. 23-27, 2012. |
Extended European Search Report issued Mar. 28, 2014 in corresponding European Patent Application No. 11177890.8. |
J.M.H. du Buf “Chapter 9. From semantic texture attributes to a syntactic analysis in Gabor space” 2006; XP-002721492; pp. 139-140. |
Norbert Krüger et al. “Object Recognition with Banana Wavelets”, Apr. 16, 1997; XP-055106740; pp. 61-66. |
Number | Date | Country | |
---|---|---|---|
20120063673 A1 | Mar 2012 | US |