Face detecting apparatus and method

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a means for extracting a photographic subject from still image data, and more particularly to a face detecting apparatus and method.

2. Related Background Art

Digital still cameras and digital video cameras are prevailing rapidly nowadays. Along with this, there are increasing demands for tools such as a tool for processing images photographed with these photographing apparatuses to provide a high image quality.

In the high image quality process, for example, a face detection process is executed for an image, and if there is a human face, a correction amount is determined basing upon the detected face. This method is widely known.

Most of the process time required for the high image quality process is occupied by the face detection process time. This process time has been long desired to be shortened.

Several means for improving a face detection efficiency have been proposed.

For example, Japanese Patent Application Laid-Open No. 2003-163827 (no corresponding U.S. application) proposes an approach by which a photographing apparatus executes a pixel thinning process for a whole image in accordance with a photographing mode and a photographic subject magnification factor, and changes the number of divided blocks to be subjected to the face detection process. With this approach, a face detection precision is changed for each photographed image to improve the process efficiency.

According to the invention described in Japanese Patent Application Laid-Open No. 2003-163827, the face detection process is executed for the whole area of an image. There arises therefore the problem that an area subjected to the face detection process increases greatly as the image data amount increases, and the process time prolongs. Another problem resides in that the detection precision is degraded if the number of thinned pixels is increased in order to shorten the process time.

SUMMARY OF THE INVENTION

The present invention has been made in consideration of the above-described problems and aims to shorten the process time by executing the face detection process more efficiently, while the detection precision is maintained high. Especially, the present invention aims to executing the face detection process efficiently in the case of a portrait photographing with a plurality of faces.

In order to achieve the above object, a face detecting apparatus of the present invention for extracting a face area of a photographic subject from an image on the basis of image data and accessory information associated with the image data comprises: means for determining a plurality of face detection area in an image area of the image in accordance with the accessory information; and means for executing a face detection process in the determined face detection area according to a prescribed priority of order.

According to the present invention, since a face detection area is determined by using accessary information such as photographic conditions, a process time required to detect a face can be shortened. Furthermore, in a system for processing an image in accordance with a face detection result, the process time of the whole system can be shortened because of a shortened face detection process time.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the structure of an image processing system adopting face detecting means of the present invention.

FIG. 2 is a diagram illustrating an image (picture) file format of a digital still camera (DSC).

FIG. 3 is a diagram illustrating that a user of DSC moves a focussing evaluation area.

FIG. 4 is a diagram showing an example of a display screen of a liquid crystal monitor mounted on the back of DSC.

FIG. 5 is a flow chart illustrating the operation of an image processing apparatus.

FIG. 6 is a diagram explaining calculation of a gamma value to be used for correction processing.

FIG. 7 is a flow chart illustrating the detailed operation of setting a face detection area.

FIG. 8 is a diagram showing the relation between a range finding frame and image data.

FIG. 9 is a diagram illustrating an operation of dividing an image in accordance with the number of range finding frames and determining a face detection area by using a range finding position.

FIG. 10 is a diagram illustrating an operation of determining a face detection area by adding an offset value to a range finding frame.

FIG. 11 is a diagram showing the relation between a range finding frame and image data in a system according to a second embodiment.

FIG. 12 is a diagram showing a guide display displayed in an assistance mode on a liquid crystal monitor mounted on the back of DSC in a system according to a third embodiment.

FIG. 13 is a diagram showing an example of a display screen of the liquid crystal monitor in the assistance mode.

FIG. 14 is a flow chart illustrating an operation of setting a face detection area in a system according to a fourth embodiment.

FIG. 15 is a diagram showing a face detection area in the system of the fourth embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

First Embodiment

A face detecting means of the present invention is applied to an image processing system such as shown in FIG. 1.

The image processing system 100 shown in FIG. 1 is constituted of a digital still camera 101 (hereinafter written as DSC), a image processing apparatus 104 and a display unit 115.

DSC 101 is constituted of a photographic condition setup unit 102 for setting photographic conditions and a record unit 103 for recording photographed image data during photographing.

The photographic condition setup unit 102 sets various photographic conditions necessary for photographing. A user can set various conditions such as a photographic scene, a white balance mode, and a flash on/off, by operating buttons on DSC 101.

The record unit 103 generates image data including photographed image data and accessory information of photographic conditions, and records it.

A recording medium may be a memory built in DSC 101 or a detachable external medium.

An image file format to be used for recording image data is in conformity with, for example, Exchangeable Image File Format (Exif). As shown in FIG. 2, the image file format is constituted of a tag region 201 in which the accessory information, i.e., photographic condition data, is recorded and an image (picture) data region 202 in which photographed image data is stored.

The photographic condition data is recorded in the tag region 201 at positions starting from a predetermined offset position from the top, and contains, for example, the contents of TagNo.=0 to 6.

TagNo.=0 is information of a photographic scene type. For a “landscape mode” of DSC, a value representative of the “landscape mode” is stored in TagNo.=0. The photographic scene type is not limited only to this, but a “standard”, “portrait”, “night scenery” or the like may be designated.

TagNo.=1 is information of white balance. For an “automatic white balance” of DSC, a value representative of automatic white balance setup is stored in TagNo.=1. For manual white balance setup, a value representative of manual setup is stored.

TagNo.=2 is information of an exposure mode. For “automatic exposure” of DSC, a value representative of automatically controlled parameter exposure setup is stored in TagNo.=2. For “manual exposure” or “automatic bracket” setup, a corresponding value is stored.

TagNo.=3 is information of a flash. For “flash emission” of DSC, a value representative of flash emission is stored in TagNo.=3. A value representative of detailed information such as “automatic emission mode” and “red-eye reduction mode” of the flash emission may also be stored.

TagNo.=4 is information of a focussing area. Stored in TagNo.=4 are values representative of “range finding frame number”, “used range finding frame” and “photographic subject distance in each range finding frame” respectively used for focussing control of DSC.

Detailed description of focussing control for photographing is omitted because this does not constitute the main characteristics of the present invention. With this focussing control, for example, a blur amount of a photographed image in a predetermined focussing evaluation area is calculated and the lens position is controlled so that the blur amount becomes minimum. This focussing control is described, for example, in Japanese Patent Applicaion Laid-Open No. 2003-163827.

A focussing control method such as shown in FIG. 3 is also known by which a user moves a focussing evaluation position 301 to a desired position and determines the focussing evaluation area in accordance with the focussing evaluation position.

A person or an object like a person is often selected as the main photographic subject in order to determine the focussing evaluation area. When a user determines the focussing evaluation area, there is the tendency that the user determines the focussing evaluation area so as to focus on a person.

FIG. 4 shows an example of a display screen of a liquid crystal monitor 401 mounted on the back of DSC. DSC prepares beforehand a range finding frame 402, and an emphasized range finding frame 403 is actually used for focussing control.

TagNo.=5 is information of a maker name, and character string information representative of the maker name is stored therein. TagNo.=6 is information of a model name of DSC, and character string information representative of the model name is stored therein.

Reverting to FIG. 1, the image processing apparatus 104 is made of, for example, a personal computer, activates predetermined application software, acquires image data from DSC 101, corrects it and outputs it. To this end, the image processing apparatus 101 is constituted of an input unit 105, a data analysis unit 106, a correction processing determination unit 112, a correction processing execution unit 113 and an output unit 114. The functions of the respective units 105 to 114 are realized by activating the application.

The input unit 105 automatically recognizes a connection of DSC 101 to the image processing apparatus 104, and starts reading image data. Another configuration may be used in which image data is read after a user instruction is acknowledged.

The data analysis unit 106 is constituted of an accessory information acquirement unit 107 for acquiring accessory information affixed to image data, a histogram analysis unit 108 for analyzing the feature of an image by generating a histogram from image data, and a face detection processing unit 109 which is the main characteristics of the present invention.

The accessory information acquirement unit 107 acquires photographic condition data during photographing, such as a photographic scene type, a white balance, an exposure mode, a flash and a focussing area. The histogram analysis unit 108 generates a histogram of image data and calculates statistically a feature amount of an image. The feature amount is, for example, an average luminance, a luminance dispersion, and an average chroma, respectively of an image.

The face detection processing unit 109 is constituted of a determination unit 110 for determining an area for which a face detection process is executed, and an execution unit 111 for executing the face detection process in accordance with a result obtained by the determination unit.

The determination unit 110 determines the face detection area in an image area in accordance with focussing area information contained in the accessory information, in order to allow the execution unit 111 to be described later to execute. The detailed description will be given later.

The execution unit 111 executes the face detection process for the setup area. Various methods for the face detection process have been proposed. For example, a face is detected by detecting one eye, generating a candidate of both eyes from the one eye and recognizing a face pattern.

The correction processing determination unit 112 determines an image correction algorithm in accordance with an analysis result obtained by the data analysis unit 106. For example, the correction processing contents and correction procedure are acquired from the accessory information acquirement unit 107 and face detection processing unit 109, and the feature of the luminance distribution of an image is acquired from the histogram analysis unit 108, to thereby calculate correction parameters suitable for each of the correction processing contents. The correction processing contents include a gradation correction process, a contrast correction process, a color balance process, a chroma correction process, a contour emphasis process and the like.

The correction processing execution unit 113 executes an image correction process for image data, in accordance with the image correction algorithm determined by the correction processing determination unit 112.

The output unit 114 converts image data subjected to the correction process by the correction processing execution unit 113 into data capable of being displayed on the display unit, and outputs it to the display unit 115.

The display unit 115 outputs and displays the image input from the output unit 114 in a desired form. Next, with reference to the flow chart of FIG. 5, description will be made on the operation of the image processing apparatus 104 of this embodiment.

At Step 501, when DSC 101 is connected, the image processing apparatus 104 starts reading image (picture) data.

At Step 502, a histogram is generated for the image data read at Step 501 to calculate a feature amount. The feature amount is an average luminance, a luminance dispersion and the like describe earlier.

At Step 503, the accessory information representative of the photographic conditions is extracted from the image data read at Step 501.

The accessory information may be written by using an application or the like after photographing.

At Step 504, a face detection area is set in accordance with the focussing area information in the accessory information acquired at Step 503. If the flow returns to Step 504 because a face cannot be detected at Steps 505 and 506 to be described later, then a face detection area different from the already set face detection area is set again. For example, the first face detection area is set within the focussing area and the second face detection area is set in the area different from the focussing area to start a face detection process from the upper left of the image.

At Step 505, the face detection process is executed for the face detection area set at Step 504. However, if it is judged that a person “exists”, i.e., the face was detected, this Step is terminated to advance to Step 506.

At Step 506, the flow advances to Step 507 if it is judged at Step 505 that a person “exists” or the face detection process is completed for the whole image. In contrast, the flow returns to Step 504 to set again the face detection area, if it is judged that a person “does not exist” or the face detection process is not completed for the whole image.

At Step 507, the correction process contents are selected in accordance with the accessory information during photographing and the face detection result acquired at Steps 503 to 506, and the correction processing amount is determined in accordance with the image feature amounts acquired at Step 502. In selecting the correction processing contents, the gradation correction process, contrast correction process, color balance process, chroma correction process, contour emphasis process or the like is selected in accordance with a combination of the photographic scene type, white balance, exposure mode, flash and the like respectively contained in the accessory information.

The image processing apparatus 104 of the embodiment determines the correction processing contents by recognizing image data as a photographic scene type “person”, if Step 505 judges that a person “exists” and the photographic scene type is “standard”. This is because the correction process suitable for the photographic subject is to be used.

Another configuration may be adopted by which the correction process for person is always used if the face detection results judge that a person “exists”, irrespective of the contents of the photographic scene type.

After the correction processing contents are determined, a correction target value is determined from the accessory information and image feature amounts, to calculate the parameters to be used when the correction is executed.

The correction target value is determined beforehand for each of the processing contents to be determined by a combination of accessory information, and preferably stored as a table in the image processing apparatus. Alternatively, the correction target value may be calculated from the feature amounts by using an equation.

The parameters to be used when the correction is executed are calculated from the correction target value. For example, the gamma value for gradation correction can be calculated from the following equation (1). This equation can be derived by referring to FIG. 6:

γ=(log Y_max−log Y)/(log X_max−log X) (1)

wherein X_maxis a maximum value of an input signal, Y_maxis a maximum value of an output signal, X is an appropriate intermediate level of an input signal and Y is an appropriate intermediate level of an output signal.

At Step 508, the image correction process is executed in accordance with the correction processing contents and parameters determined at Step 508.

Next, with reference to the flow chart of FIG. 7, description will be made on a method of determining a face detection area in accordance with the focussing area information, as an example of the face detection area setup process at Step 504.

At Step 701, the focussing area information is extracted from the accessory information acquired at Step 503 (refer to FIG. 5). In this embodiment, the focussing area information is “range finding frame number” and “used range finding frame”.

At Step 702, range finding frame positions are obtained from the acquired “range finding frame number”, and a face detection area to be actually used for focussing control is determined from the “used range finding frame” to calculate coordinate values.

For example, as shown in FIG. 8 three range finding frame positions are obtained from a “range finding frame number=3”, and a face detection area in an emphasized frame 801 is determined from a “used range finding frame=center” as prior to other frame.

These three range finding positions can be obtained from the maker name and model name in the tag region 201, and it is preferable to store the range finding position information in the image processing apparatus for each pair of maker name and model name of DSC.

Alternatively, the face detection area with a priority of order may be determined from the “used range finding frame” by roughly calculating the range finding positions by dividing image data in accordance with the “range finding frame number”. For example, as shown in FIG. 9 under the conditions of “range finding frame number=9” and “used range finding frame=center”, the image data area is divided in accordance with the range finding frame number (broken lines 901) and the face detection area (hatched area) 902 can be determined from the “used range finding frame” as an area with a priority of order. As to other range finding frames, for example, the range finding frame adjacent to the used range finding frame is selected as the face detection area with subsequent priority of order.

Furthermore, as shown in FIG. 10, in order to obtain a face detection area having a size predetermined by the image processing apparatus 104, the size may be changed by adding or subtracting offset values 1002 to or from a range finding frame size 1001 at up, down, right and left.

It is possible to flexibly deal with an image size during photographing, by determining a ratio of the face detection area to a resolution of photographed image data.

A face detection start position 802, a lateral width 803 and a height 804 are calculated for the position of the face detection area determined by the above-described process.

At Step 703, it is set so that the face detection-area calculated at Step 702 can be subjected to the face detection process.

According to the embodiment, face detection can be performed efficiently by determining the face detection area in accordance with the focussing area information, by utilizing focussing control during photographing with DSC relative to a photographic subject having a high possibility of person.

Efficient face detection can shorten the process time of the whole image correction process.

Namely, according to this embodiment, face detection is performed by means for acquiring focussing area information in an image area by using image data and accessory information of the image data, means for determining a face detection area in the image area in accordance with the focussing area information, and means for executing a face detection process.

Second Embodiment

The second embodiment pertains to a method of setting a face detection area as described in the first embodiment, for the case that the focussing area information has a plurality of “used range finding frames”.

The system configuration of the second embodiment is shown in FIG. 1 same as that of the first embodiment.

In the second embodiment, the determination unit 110 of the face detection processing unit 109 shown in FIG. 1 determines a face detection area in accordance with the “range finding frame number” and “used range finding frames” of the first embodiment and in addition “photographic subject distance in each range finding frame”. It is therefore possible to deal with the case that there are a plurality of “used range finding frames”.

With reference again to FIG. 7, description will be made on the face detection area setting method as the main characteristics of the second embodiment. In the second embodiment, at Step 701 information of the focussing area is acquired from the accessory information acquired at Step 503 shown in FIG. 5. The information of the focussing area includes “range finding frame number”, “used range finding frames” and “photographic subject distance in each range finding frame”.

At Step 702, range finding frame positions are obtained from the “range finding frame number” of the acquired focussing area information. Areas to be actually used for focussing control are determined as face detection candidate areas, in accordance with the “used range finding frames”. According to the characteristics of the second embodiment, the face detection candidate areas are given a priority order in accordance with the “photographic subject distance in each range finding frame”.

For example, as shown in FIG. 11, a range finding frame position 1301 is obtained from “range finding frame number=9”, and areas 1302 having an emphasized frame by “used range finding frames=left, center, right, upper right” are used as the face detection candidate areas. If the “photographic subject distance in each range finding frame” among the used range finding frames is “upper right<right <left<center”, the priority order of the face detection candidate areas is determined from the photographic subject distances.

Namely, in this example, the upper right frame position having the shortest photographic subject distance is first designated as the face detection area. Thereafter, the flow advances to next Step. If the analysis completion (judgement that a person “exists” or analysis completion of the whole image) is not achieved at Step 506, then the face detection area at the “right” range finding frame position is used to advance to next Step. If the analysis completion is not achieved again at Step 506, a similar process is executed in the order of “left” and “center”.

At Step 703, the face detection area information calculated at Step 702 is set so that it can be used in the face detection process.

As described above, according to the second embodiment, face detection can be performed efficiently by determining the face detection area in accordance with the focussing area information, by utilizing the fact that there are a high possibility of focussing control during photographing with DSC relative to a photographic subject person and a high possibility that a person is at a position nearer than that of a landscape.

Namely, according to the second embodiment, face detection is performed by means for acquiring focussing area information of an image by using image data and accessory information of the image data, means for acquiring photographic subject distance information from the focussing area information, means for determining a priority order of the focussing area information, and means for determining a face detection area in accordance with the priority order and executing a face detection process.

Third Embodiment

The third embodiment pertains to a method of setting a face detection area in accordance with photographic condition data other than the focussing area information.

The system configuration of the third embodiment is shown in FIG. 1 same as that of the first embodiment.

According to the configuration of the third embodiment, the photographic condition setup unit 102 of DSC 101 can set a photographic scene type as in the first embodiment and further set a detailed assistance mode.

FIG. 12 shows an example of a display screen of the liquid crystal monitor 1501 mounted on the back of DSC 101. A photographer takes an image by using a guide display 1502 displayed on the liquid crystal monitor 1501.

In the guide display 1502 of the third embodiment, auxiliary lines are displayed in order to confirm horizontal positions when persons in the right screen and a building or the like in a left background are to be photographed.

FIG. 13 shows an image actually photographed. It can be seen that a user photographs by using the guide display 1502.

Photographed image data is recorded in the format same as that of the first embodiment, and assistance mode information is recorded in the tag region 201 (refer to FIG. 2). The assistance mode information includes information of using the assistance mode and the type of the assistance mode.

For example, in the assistance mode shown in FIGS. 12 and 13, a character string “left background” is recorded.

In the image processing apparatus 104 of the third embodiment, the data analysis unit 106 has an accessory information acquirement unit 107 and a face detection processing unit 109 respectively different from those of the first embodiment. The accessory information acquirement unit 107 acquires the assistance mode information, instead of the focussing area information acquired by the first embodiment.

The face detection processing unit 109 has a determination unit 110 different from that of the first embodiment. The determination unit 110 determines a face detection area in an image area in accordance with the assistance mode information contained in the accessory information.

Next, with reference to the flow chart of FIG. 5, description will be made on the characteristic points of the operation of the image processing apparatus 104 of the third embodiment. At Step 503, the accessory information affixed during or after photographing is acquired from the image data read at Step 501. In this case, the assistance mode information is acquired, instead of the focussing area information acquired in the first and second embodiments.

At Step 504, a face detection area is set in accordance with the assistance mode information in the accessory information acquired at Step 503. If the flow returns again to Step 504 because face detection was impossible at Steps 505 and 506, then a face detection area different from the already set face detection area is set again.

Other Steps are similar to those of the first embodiment, and the description thereof is omitted.

Next, with reference to the flow chart of FIG. 14, description will be made on a method of setting a face detection area at Step 504 which is the characteristics of the third embodiment.

At Step 1801, information of the assistance mode is acquired from the accessory information acquired at Step 503 (refer to FIG. 5).

At Step 1802, an area having a high possibility that the face of a person exists is determined as a face detection area, in accordance with the acquired assistance mode information. Thereafter, the coordinate values are calculated.

For example, as shown in FIG. 15, a face detection area (hatched area) 1901 is determined from “assistance mode=left background”.

The face detection area can be calculated from the maker name and model name stored in the tag region (refer to FIG. 2). It is preferable to store the assistance mode information of each maker of DSC and the face detection area corresponding to each assistance mode, in the image processing apparatus.

In the third embodiment, although a rough position (hatched area) 1901 is determined as the face detection area, the face area of a person in the assistance display may be set more strictly as a first face detection area.

At Step 1803, the face detection area information calculated at Step 1802 is set so that it can be used in the face detection process.

As described above, according to the third embodiment, face detection can be performed efficiently by determining the face detection area in accordance with the position information of a person in the guide display, by utilizing the fact that there is a high possibility that the face of a person is photographed based on the guide display in the assistance mode. Namely, according to the third embodiment, face detection is performed by means for acquiring assistance mode information during photographing from image data and accessory information of the image data and means for determining a face detection area in accordance with the assistance mode information and executing a face detection process.

Fourth Embodiment

The fourth embodiment pertains to the case in which the information of a focussing area as in the first embodiment is represented by “coordinate values”.

The system configuration of the fourth embodiment is shown in FIG. 1 same as that of the first embodiment. In the fourth embodiment, DSC 101 uses the “coordinate values” to represent the information of a focussing area which information is one of the photographic conditions.

Namely, coordinate values (X, Y) are stored in TagNo.=4 (FIG. 2) of the accessory information as in the first embodiment, the coordinate values corresponding to the center of the area to be used for focussing control.

As different from the first embodiment, the determination unit 110 of the face detection processing unit 109 in the data analysis unit 106 of the fourth embodiment determines a face detection area in accordance with the “coordinate values”.

In the following, description will be made on a method of setting a face detection area according to the fourth embodiment.

The image processing apparatus acquires the coordinate values of an area to be used for focussing control from the accessory information stored by DSC 101.

Next, a square area is calculated having its center represented by the acquired coordinate values, and used as a face detection area.

The square area is calculated so that it has the size predetermined by the image processing apparatus. Since the size is determined in proportion to a ratio of a photographed image to a resolution. It is therefore possible to flexibly deal with the image size during photographing.

In place of the square area, it is possible to use a circle having a radius R about the acquired coordinate values, or a rectangle area such as the areas 801, 902 and 1001 shown in FIGS. 8 to 10.

With the above-described process, a face detection are is set and a face detection process is executed.

As described above, according to the fourth embodiment, the face detection process can be executed efficiently by determining a face detection area from the coordinate values, in the accessory information, of an area to be used for focussing control.

The main aspects of the present invention are not limited only to detecting a face area. For example, the present invention is also applicable to detecting a characteristic area of an image of a photographic subject such as a specific object, animal or the like instead of the face of a person. By determining a characteristic detection area of an image in accordance with the accessory information of image data, it is possible to detect the characteristic area of an image in a short time and at high precision.

This application claims priority from Japanese Patent Application No. 2004-018245 filed on Jan. 27, 2004, which is hereby incorporated by reference herein.

Face detecting apparatus and method

Information

Publication Number

Date Filed

Date Published

Inventors

Original Assignees

CPC

US Classifications

International Classifications

Abstract

Description

Claims

Priority Claims (1)