1. Field of the Invention
The present invention relates to a face recognition method, particularly reconstructing a 3D morphable face model from a single face image, and recognizing a series of face video frames from a video receiving apparatus by fusing similarity probability of each video frame.
2. Description of the Related Art
Face recognition is one of the most important research areas in computer vision and biometric recognition. For 2D face recognition, the main challenge is the variations of face images for the same individual under different capturing conditions such as angles, light source, etc. To overcome the problem, most developed algorithms rely on a large number of training examples under different conditions for each individual. However, it is difficult to collect 2D face images under a precise condition with the variation in real-world applications.
Recently, 3D face modeling from images is very popular for many applications, such as facial animation and face recognition. Model-based statistical techniques have been widely used for robust face modeling. Most of the previous 3D face reconstruction techniques require more than one 2D face image to achieve satisfactory 3D face modeling. Although another approach for 3D face reconstruction requires only a single image, the problem is restricted by requiring a statistical head model known as a priori. Besides, most of the previous face recognition method cannot differentiate imposters who have not been present before. For the increasingly popular face recognition system, simplifying sampling complexity and training difficulties, and increasing recognition accuracy are problems need to be solved.
To solve aforementioned problem, one objective of the present invention is to propose a face recognition method, which constructs a 3D face model using a single 2D face image, and simulate face image variations under different conditions. The synthesized face images are used to train a face recognition classifier. Besides, the present invention employs a locally linear embedding concept, which not only can recognize face images of legal system users, but also can reject imposters. Furthermore, the present invention temporally integrates single-frame decisions by a probabilistic approach to increase the reliability of face recognition.
To achieve aforementioned objective, one embodiment of the present invention proposes a face recognition method including obtaining an original face image from a database, reconstructing a 3D face model for the original face image using a 3D morphable face model, providing synthesized face images from the 3D face model for training a face recognition classifier, receiving a plurality of video frames of a face, obtaining a similarity probability for each video frame from the face recognition classifier, and fusing the temporal similarity probability for each video frame by accumulation to obtain a face recognition confidence value, and if the face recognition confidence value is larger than a threshold, the face is recognized to be the same as the face in the original face image.
a-
The face recognition method of the present invention reconstructs a 3D face model from a single 2D face image using a 3D morphable model, and synthesizes a large set of 2D face images under different head poses for training the face recognition model. Also, this method temporally integrates single-frame recognition decisions via a probabilistic approach to increase recognition accuracy, and rejects non-system allowed imposters based on a LLE (locally linear embedding) method.
The flowchart of one embodiment of the present invention is shown in , and 3D texture data set, T=(r1, g1, b1, r2, g2, b2, . . . , rN, gN, bN)ε
, where N is the number of sampling points for the curved surface of a 3D face. Then the 3D geometry data set and 3D texture data set of the 3D face is approximated by a linear combination of a plurality of eigenhead basis vectors, shown as follows:
wherein
The embodiment uses 3D scans and image database of BU-3DFE (3D facial expression database) as the data source for producing eigen head basis vectors. Please referring to
The abovementioned 3D face reconstruction is initialized by the feature points in
∥Iinput−Imodel(α,β,Portho,f,R,t2d)∥ (3)
wherein Iinput is the original face image, Portho is an orthographic projection matrix, f is the scaling factor, R is the 3D rotation matrix, and t2d is the translation vector. The minimization of function (3) can be solved by using the Levenberg-Marquart (LM) optimization.
In step S02, possible variations of the image are simulated by the reconstructed 3D face model, and used to train a face recognition classifier for the original face image.
In step S03, a plurality of video frames of a face are received, and a similarity probability is obtained for each video frame from the face recognition classifier. In step S04 the probability of each video frame is fused temporally to obtain a confidence value. If the confidence value is greater than a threshold, the received face video frames are recognized to have the same identity as the individual in the original face image. The method for fusing previous recognition results is shown in expression (4):
wherein ω is a forgetting factor, Pit,SVC is a posterior probability denoting the probability that this user belongs to class i decided by a single video frame at time t. In one embodiment, we merge Pit,SVC with the previously accumulated probability Pit−1 and obtain a final similarity probability Pit accumulated up to time t.
On the other hand, when it comes to an applicable face recognition system, it must have the ability to reject imposters who have not been present before and only accept legal system users. Borrowing the idea from LLE that nonlinear data is locally linear on a manifold, if a to-be recognized image cannot be well-reconstructed linearly via the training images of the same person, the image is likely to be from an imposter. Therefore, a recognized face It at frame t is supposed to be reconstructed well by its K nearest neighbors in the predicted class c. The similarity error εt of an input face It is defined to be:
wherein wjc,t denotes the weights of K nearest neighbors in class c that best approximate said video frame It, and
is the set of K nearest neighbors of the same class. Then a sigmoid function is applied to transform εt to the corresponding rejection probability Prejt with a recursive definition:
wherein εthresholdc is the parameter controlling the rejection rate. It turns out that face It cannot belong to a system user if it has a large rejection probability {circumflex over (P)}rejt. Please referring to
wherein i=1, 2, . . . , n, and βt=1−{circumflex over (P)}rejt is the acceptance rate at time t, and {circumflex over (P)}t=[{circumflex over (P)}1t, {circumflex over (P)}2t, . . . , {circumflex over (P)}rejt]T is the posterior probability distribution. If {circumflex over (P)}it exceeds a threshold Pth, the present invention can decide the identity of the face in video frames. If
(class n+1 for the imposter class), then the to-be recognized face is rejected; otherwise, the face is determined to be of a legal system user and his/her identity is further recognized, i.e.
In conclusion, the present invention reconstructs a face model via a 3D morphable model, and synthesizes variations of a face image as the training database for face recognition classifier. In the process of constructing 3D face model and training face recognition classifier, only a 2D image is needed for the user of each class. This improves from the statistical approach which requires collecting a large amount of face training samples.
The embodiments described above are to demonstrate the technical contents and characteristics of the preset invention to enable the persons skilled in the art to understand, make, and use the present invention. However, it is not intended to limit the scope of the present invention. Therefore, any equivalent modification or variation according to the spirit of the present invention is to be also included within the scope of the present invention.
Number | Date | Country | Kind |
---|---|---|---|
97146813 A | Dec 2008 | TW | national |
Number | Name | Date | Kind |
---|---|---|---|
6556196 | Blanz et al. | Apr 2003 | B1 |
7499574 | Yang et al. | Mar 2009 | B1 |
7536030 | Wang et al. | May 2009 | B2 |
7596247 | Ioffe | Sep 2009 | B2 |
7664962 | Kuhlman | Feb 2010 | B2 |
7728839 | Yang et al. | Jun 2010 | B2 |
7783082 | Koshizen et al. | Aug 2010 | B2 |
8050465 | Ianculescu et al. | Nov 2011 | B2 |
20020106114 | Yan et al. | Aug 2002 | A1 |
20030149803 | Wilson | Aug 2003 | A1 |
20070031028 | Vetter et al. | Feb 2007 | A1 |
20070122001 | Wang et al. | May 2007 | A1 |
20100134487 | Lai et al. | Jun 2010 | A1 |
Number | Date | Country | |
---|---|---|---|
20100135541 A1 | Jun 2010 | US |