Face feature point detecting device and method

Abstract

A face feature point detecting device includes a unit for inputting an image containing a face of a person, a unit for detecting a feature point set candidate comprising plural kinds of feature points, a unit for calculating the corresponding error between the detected feature point set and the corresponding feature point set on the three-dimensional model of the face, and a unit for judging the consistency of the arrangement of the detected feature point set candidate.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the construction of a face feature point detecting device according to a first embodiment of the present invention;

FIG. 2 is a flowchart showing the operation of the first embodiment;

FIG. 3 is a diagram showing projection of feature points on a three-dimensional shape onto an image by a motion matrix;

FIG. 4 is a block diagram showing the construction of a face feature point detecting device according to a second embodiment; and

FIG. 5 is a graph showing an example of a feature point graph according to the second embodiment.

Claims

1. A face feature point detecting device comprising: an image input unit configured to input an image containing a face of a person;a feature point set candidate detecting unit configured to detect feature point set candidates each of which comprises plural kinds of feature points related to the face, from the image;a model information storage unit configured to store three-dimensional model information having information of positions of the feature points of the face on a three-dimensional model of the face;a projecting unit configured to obtain a projected feature point set by projecting a feature point set in the three-dimensional model information onto a two-dimensional area, the feature point set comprising the plural kinds of the feature points on the three-dimensional model of the face;an error calculating unit configured to calculate error between each feature point of the projected feature point set and each feature point of the feature point set candidate; anda selecting unit configured to select a consistent feature point set having feature points whose respective errors satisfy a predetermined condition from the feature point set candidates.

2. The device according to claim 1, wherein the projecting unit calculates a projection matrix for projecting the feature point set in the three-dimensional model information onto the two-dimensional area from the feature point set candidates and the feature point set of the three-dimensional model information, whereby the feature point set in the three-dimensional model information is projected onto the two-dimensional area on basis of the projection matrix.

3. The device according to claim 1, wherein the error calculating unit calculates distance between position of the projected feature point and the position of the feature point of the feature point set candidate, and calculates the error by normalizing the distance concerned on basis of a reference distance calculated from the feature point set candidate.

4. The device according to claim 3, wherein the reference distance is set to distance between right and left pupils contained in the feature point set candidate.

5. The device according to claim 1, wherein the selecting unit calculates maximum error from the errors of the respective feature points contained in the feature point set candidate, and selects the consistent feature point set having the maximum error less than a predetermined threshold value.

6. The device according to claim 1, wherein the feature point set candidate detecting unit detects according to relationship among the plural kinds of feature points, the relationship has plural feature point blocks each concerning to every specific feature point, the plural feature point blocks are linked with one another by a directed graph having unilaterally dependent relationship, only a parent block that is not dependent on the other feature point blocks in the plural feature point blocks independently detects the feature point set candidates from the image, and the feature point set candidates are detected by using the image and information concerning the feature point set candidate belonging to the parent block in the feature point blocks dependent on the parent block.

7. A face feature point detecting method comprising: inputting an image containing a face of a person;detecting from the image, feature point set candidates each of which comprises plural kinds of feature points concerning the face;storing three-dimensional model information having information of positions of the feature points of the face, on a three-dimensional model of the face;projecting a feature point set in the three-dimensional model information onto a two-dimensional area as to obtain a projected feature point set, the feature point set comprising the plural kinds of the feature points on the three-dimensional model of the face;calculating error between each feature point of the projected feature point set and each feature point of the feature point set candidate; andselecting a consistent feature point set having feature points whose respective errors satisfy a predetermined condition from the feature point set candidates.

8. The method according to claim 7, further comprising: calculating a projection matrix for projecting the feature point set in the three-dimensional model information onto the two-dimensional area on basis of the feature point set candidate and the feature point set of the three-dimensional model information; and whereby the feature point set in the three-dimensional model information is projected onto the two-dimensional area on basis of the projection matrix.

9. The method according to claim 7, further comprising: calculating distance between position of the projected feature point and the position of the feature point of the feature point set candidate; andcalculating the error by normalizing the distance concerned on basis of a reference distance calculated from the feature point set candidate.

10. The method according to claim 9, wherein the reference distance is distance between right and left pupils contained in the feature point set candidate.

11. The method according to claim 7, comprising: selecting maximum error from the errors of the respective feature points belonging to each feature point set candidate; and selecting the consistent feature point set that has the maximum error smaller than a predetermined threshold value.

12. The method according to claim 7, wherein the feature point set candidates are detected according to relationship among the plural kinds of feature points, said relationship having plural feature point blocks each concerning to respective specific feature point; the plural feature point blocks are linked by a directed graph having unilaterally dependent relationship;only a parent block that is not dependent on other feature point blocks in the plural feature point blocks independently detects the feature point set candidates from the image, and the feature point set candidates are detected by using the image and information concerning the feature point set candidate belonging to the parent block in a feature point block dependent on the parent block.

13. A program stored in a computer readable medium for detecting face feature point, the program comprising instructions of: inputting an image containing a face area of a person;detecting a feature point set candidate comprising plural kinds of feature points concerning the face, from the image;storing three-dimensional model information having information of positions of the feature points of the face, on a three-dimensional model of the face;projecting a feature point set in the three-dimensional model information onto a two-dimensional area as to obtain a projected feature point set, the feature point set comprising the plural kinds of the feature points on the three-dimensional model of the face;calculating error between each feature point of the projected feature point set and each feature point of the feature point set candidate; andselecting a consistent feature point set having feature points whose respective errors satisfy a predetermined condition, from the feature point set candidates.

14. The program according to claim 13, comprising an instruction of: calculating a projection matrix for projecting the feature point set in the three-dimensional model information onto a two-dimensional area on basis of the feature point set candidate and the feature point set in the three-dimensional model information; and whereby the feature point set in the three-dimensional model information is projected onto the two-dimensional area on basis of the projection matrix.

15. The program according to claim 13, further comprising instructions of: calculating distance between position of the projected feature point and the position of the feature point of the feature point set candidate; andcalculating the error by normalizing the distance on basis of a reference distance calculated from the feature point set candidate.

16. The program according to claim 15, wherein the reference distance is distance between right and left pupils contained in the feature point set candidate.

17. The program according to claim 13, comprising instructions of: selecting maximum error in the errors of the respective feature points belonging to the feature point set candidate; andselecting the consistent feature point set that has the maximum error less than a threshold value.

18. The program according to claim 13, wherein the feature point set candidates are detected according to relationship among the plural kinds of feature points, said relationship having plural feature point blocks each concerning to respective specific feature point; the plural feature blocks are linked by a directed graph having unilaterally dependent relationship;only a parent block that is not dependent on other feature blocks in the plural feature point blocks detects the feature point set candidate from the image, and detection of the feature point set candidate is carried out by using the image and information concerning the feature point set candidate belonging to the parent block in a feature point block dependent on the parent block.