This application claims the benefit of Korean Patent Application No. 10-2005-0106673, filed on Nov. 8, 2005, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein in its entirety by reference.
1. Field of the Invention
An embodiment of the present invention relates to a face recognition method, medium, and system using gender information, and more particularly, to a method, medium, and system determining the gender of a query facial image and recognizing a face using the determined gender.
2. Description of the Related Art
Face recognition techniques include techniques for identifying a user using a given facial database with respect to one or more faces contained in a still image or a moving image. Since facial image data drastically changes depending on the pose or lighting conditions, it is difficult to classify data to take into consideration each pose or each lighting condition for the same person, i.e., the same class. Accordingly, high accuracy classification solution is desired. An example of such a widely used linear classification solution includes Linear Discriminant Analysis (referred to as LDA hereinafter).
Generally, the recognition performance or reliability for a female face is lower than that for a male face. Further, according to a training method, such as the LDA, a training model overfits variations such as expression changes held by samples of a training set. Since female facial images existing in a training set are frequently changing, e.g., due to changes in make-up or the wearing of differing accessories, facial images for the same female person may vary greatly, resulting in within-class scatter matrixes having to be more complicated. In addition, since the typical female face is very similar to an average facial image, compared to the typical male face, and as even different images of different female persons look similar, a between-class scatter matrix does not have a large distribution. Accordingly, the variance between male facial images has a greater influence on a training model than the variance between female facial images.
To overcome these problems, the inventors have found it desirable to separately train models with the separate training samples according to their genders and recognize the samples based on the recognized genders.
An embodiment of the present invention provides a method, medium, and system capable of face recognition by first determining the gender of a person contained in a query image and then selecting a separate training model depending on the determined gender.
Additional aspects and/or advantages of the invention will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the invention.
To achieve at least the above and/or other aspects and advantages, embodiments of the present invention include a method of recognizing a face, the method including classifying genders of at least one respective face in a query facial image and a current target facial image, selecting a training model based on the classifying of the genders, obtaining feature vectors of the query facial image and the current target facial image using the selected training model, measuring a similarity between the feature vectors, and obtaining similarities of a plurality of target facial images and recognizing a person of the query facial image as being a same person as an identified target image having a largest similarity among the obtained similarities.
To achieve at least the above and/or further aspects and advantages, embodiments of the present invention include a system for recognizing a face, the system including a gender classifying unit to classify genders of at least one respective face in a query facial image and a plurality of target facial images and to output a result of the gender classifying in terms of probabilities, a gender reliability judging unit to judge a reliability of a classified gender of the at least one respective face in the query facial image and/or the plurality of target facial images using a respective probability, a model selecting unit to select respective training models based on the gender classifying and the judged reliability, a feature extracting unit to extract feature vectors from the query facial image and the target facial images using the selected training models, and a recognizing unit to compare a feature vector of the query facial image and feature vectors of the target facial images to obtain similarities, and to recognize a person of the query facial image as being a same person as an identified target image having a largest similarity among the obtained similarities.
To achieve at least the above and/or still further aspects and advantages, embodiments of the present invention include at least one medium including computer readable code to control at least one processing element to implement a method including classifying genders of at least one respective face in a query facial image and a current target facial image, selecting a training model based on the classifying of the genders, obtaining feature vectors of the query facial image and the current target facial image using the selected training model, measuring a similarity between the feature vectors, and obtaining similarities of a plurality of target facial images and recognizing a person of the query facial image as being a same person as an identified target image having a largest similarity among the obtained similarities.
These and/or other aspects and advantages of the invention will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. Embodiments are described below to explain the present invention by referring to the figures.
As shown by
Similarly,
The gender of a query facial image may be classified, e.g., by the gender classifier 10, from target facial images, in operation 20. Here, the query facial image may be a facial image for an object to be recognized, and each of the target facial images may be one of a plurality of facial images previously stored in a database (not shown), for example.
The gender classification may be performed according to a classification algorithm according to any one of the conventional classifiers. Examples of the classifiers include neural networks, Bayesian classifiers, linear discriminant analysis (LDA), and support vector machines (SVMs), noting that alternative embodiments are equally available.
The gender classification result may be output as a probability, e.g., according to a probability distribution, and may be judged and output, identifying the query facial image as either a man or woman with reference to a discrimination value, e.g., a probability variable value having a maximum probability in the probability distribution. Here, the probability variable may include pixel vectors that are obtained from the query image or the target image and input to the classifier.
In an embodiment, the model selector 11 may reflect the gender reliability result, e.g., from the gender reliability judging unit 12, for selecting an appropriate face recognition model.
The classified gender of the query image may be judged for it's reliability, e.g., using the gender reliability judging unit 12, based on the gender classification probability, e.g., as output from the gender classifier 10, in operation 21. The classified gender may be judged to be reliable when the gender classification probability is less than a first value, for example, that is, the probability variable may be separated a second value or more from a central value. Here, the first and second values may be determined heuristically.
When the gender of the query image is judged to be reliable, it may be determined whether the genders of the query image and the target image, e.g., classified by the gender classifier 10, are the same, e.g., by the model selector 11, in operation 22. When the genders of the query image and the target image are the same, a global model and a model of the classified gender may be selected, e.g., by the model selector 11, in operation 23.
When the gender of the query image is judged not to be reliable, in operation 21, only the global model may be selected, e.g., by the model selector 10, in operation 24.
Here, the global model and the model for each gender may correspond previously trained models.
The models may be trained in advance via Fisher's LDA based on the target images stored in the database, for example. The target images can be classified into a global facial image group, a male facial image group, and a female facial image group, in order to train the models. Each of the models may be trained with the images contained in the corresponding group.
In addition, the target images may include a plurality of images for each individual, with the images that correspond to each individual making up a single class. Therefore, the number of individuals to be an object of the target image is the number of the classes.
The aforementioned Fisher's LDA will now be described in greater detail with reference to
Here, m represents the number of classes, Ni represents the number of training images contained in an i-th class, and T denotes a transpose.
A within-class scatter matrix Sw, which represents a within-class variance, can be obtained using the below Equation 2, for example.
Here, Xi represents an i-th class.
A matrix Φopt, satisfying the following object function may further be obtained from SB and SW, obtained using the above Equations 1 and 2, according to the following Equation 3, in operation 39, for example.
Here, Φopt, represents a matrix made up of eigenvectors of SBSW−1. The Φopt provides a projection space of k-dimension. A projection space of d-dimension where d<k may be obtained by performing a principal component analysis (PCA) (⊖) on the Φopt.
The projection space of d-dimension becomes a matrix including eigenvectors that correspond to d largest eigenvalues among the eigenvalues of SBSW−1.
Therefore, projection of a vector (x-
y=(ΦoptΘ)T(x-
According to an embodiment of the present invention, training of the models may be separately performed for the global facial image group (g=G), male facial image group (g=M), and female facial image group (g=F).
Between-class scatter matrix SBg and within-class scatter matrix SWg may be expressed by the below Equation 5, for example, depending on each of the models.
The training may be performed to obtain Φoptg satisfying the below Equation 6, for example, for each of the model images.
When the model selector 11 selects a model, the feature extracting unit 12, for example, may extract a feature vector yg for the group, e.g., according to the above Equation 4, using Φoptg for the selected model, in operation 25.
When the model selector 11 selects both the global model and the gender model, the feature vector may be extracted as follows, e.g., using Equation 4, by concatenating the global model with the gender model, according to the below Equation 7.
Here, Wg represents a weight matrix for each gender model, the weight matrix Wg=rI (I is an identity matrix), and r2 represents a ratio of a variance of an entire gender feature to a variance of an entire global feature.
The feature vector of the global model, among the extracted feature vectors, may perform a main role of the face recognition, and the feature vector of the gender model may provide features corresponding to each gender, thereby performing an auxiliary role in the face recognition.
Accordingly, the recognizing unit 14, for example, may calculate the similarity between the extracted feature vectors from the query image and the target image, in operation 26. At this point, when the gender of the query image and the gender of the target image are determined to not be the same, e.g., in the above operation 22, the similarity determination may be set such that the target image has the lowest similarity, in operation 27.
The similarity may be calculated by obtaining a normalized correlation between a feature vector yq of the query image and a feature vector yt of the target image. The normalized correlation S may further be obtained from an inner product of the two feature vectors, as illustrated in the below Equation 8, and have a range [−1, 1], for example.
The recognizing unit 14 may obtain similarity between each of the target images and the query image through the above described process, select a target image having the largest similarity to recognize a querier in the query image as the person of the selected target image, in operation 29.
When the gender of the query image and the gender of the target image are determined to be the same, e.g., during the above process, the recognizing unit 14, for example, may further perform gender-based score normalization when calculating the similarity, in operation 28. An embodiment employs a score vector used for the gender-based score normalization as the similarity between the feature vector of the query image and the feature vector of each target image, for example.
Thus, the gender-based score normalization may be used for adjusting an average and a variance of the similarity depending on the gender, and for reflecting the adjusted average and variance into a currently calculated similarity. That is, target images having the same gender as that of the query image may be selected and normalized, and target images having the other gender may be set to have the lowest similarity and not included in the normalization.
When the number of target images whose gender is the same as that of the target image is Ng, an average mg and a variance σg2 of the similarity of the target images may be determined using the below Equation 9, for example.
Here, gq represents the gender of the query image, and gt represents the gender of the target images.
The similarities of the query image and the target images may be controlled, as illustrated in the below Equation 10, based on the average and variance calculated by Equation 9, for example.
Here, yt
Here, the ROC curve represents a False Acceptance Ratio (FAR) with respect to a False Rejection Ratio (FRR). The FAR means a probability of accepting an unauthorized person as an authorized person, and the FRR means a probability of rejecting an authorized person as an unauthorized person.
Referring to the graph of
Table 1 shows comparisons of recognition performances of LDA, LDA+SN, and the above embodiment of the present invention.
In Table 1, VR represents a verification ratio verifying authorized person as herself/himself, CMC (cumulative match features) represents a recognition ratio recognizing an authorized person as herself/himself. In detail, CMC indicates a measure at which rank a person's face in the query image is presented when the query image is given. That is, when the measure is 100%, at a rank 1, the person's face is determined to be contained in a first-retrieved image. Also, when the measure is 100%, at rank 10, the person's face is determined to be contained in a tenth-retrieved image.
Table 1 reveals that the VR and the recognition ratio, according to an embodiment of the present invention, are higher than those of the conventional art and that the ERR of this embodiment is lower than those of the conventional implementations.
Thus, according to an embodiment of the present invention, since a feature vector can be extracted using a gender model, as well as the global model, a recognition ratio may be enhanced by reflecting the gender feature according to a determined gender, into the face recognition.
In addition, it is possible to prevent confusion caused by an image having a different gender by performing score normalization using gender information. Further, it is possible to perform more accurate normalization by obtaining an average and a variance of the same gender samples.
In addition to the above described embodiments, embodiments of the present invention can also be implemented through computer readable code/instructions in/on a medium, e.g., a computer readable medium, to control at least one processing element to implement any above described embodiment. The medium can correspond to any medium/media permitting the storing and/or transmission of the computer readable code.
The computer readable code can be recorded/transferred on a medium in a variety of ways, with examples of the medium including magnetic storage media (e.g., ROM, floppy disks, hard disks, etc.), optical recording media (e.g., CD-ROMs, or DVDs), and storage/transmission media such as carrier waves, as well as through the Internet, for example. Here, the medium may further be a signal, such as a resultant signal or bitstream, according to embodiments of the present invention. The media may also be a distributed network, so that the computer readable code is stored/transferred and executed in a distributed fashion. Still further, as only an example, the processing element could include a processor or a computer processor, and processing elements may be distributed and/or included in a single device.
Although a few embodiments of the present invention have been shown and described, it would be appreciated by those skilled in the art that changes may be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the claims and their equivalents.
Number | Date | Country | Kind |
---|---|---|---|
10-2005-0106673 | Nov 2005 | KR | national |