Patent Application
20090284799
1. Technical Field
The present invention relates to a technology for analyzing image data and masking a portion of the image on the basis of results of the analysis.
2. Related Art
These days, an image can be easily handled as digital data. A digital still camera, for example, makes it easy to take a picture formed by digital image data of high quality. As the image data is digital, it is easy to open or distribute the captured image through a network.
Meanwhile, as an image can be easily opened or distributed, the image can be opened or distributed contrary to intention of a person photographed in the image in some cases, resulting in his or her privacy being invaded. Thus, a technique is disclosed in JP-A-2008-34963 for protecting privacy by detecting a human face from the photographed image data and performing a masking process by using a mosaic or a black bar beforehand so as to make a person photographed in the image unidentifiable.
In some cases, however, although, in the above method for protecting privacy, a masking process is performed on the detected human face, the human face may not be totally masked as a size of a portion that is hidden is too small, or a portion not to be masked is masked as the size of the hidden portion is too large. That is, in some cases, the above method has a problem in that the person is not properly masked for some unknown reason.
An advantage of some aspects of the invention is that it properly performs a masking process on a person photographed in an image.
In order to address at least some of the above problem, an aspect of the invention is to provide an image processing device configured to analyze image data of an image in which a person is photographed. The image processing device is configured to perform a masking process of a nearly rectangular shape on an eye portion of a face of the person. The image processing device has a controller configured to detect both eyes of the person. The controller is configured to determine an inclination of the nearly rectangular shape on which the masking process is performed with respect to the image on the basis of an inclination of a straight line connecting the detected eyes.
A method for image processing of the invention corresponding to the image processing device described above is provided. In the method for image processing, image data of an image in which a person is photographed is analyzed and a masking process of a nearly rectangular shape is performed on an eye portion of a face of the person. The method for image processing includes detecting both eyes of the person, and determining an inclination of the nearly rectangular shape on which the masking process is performed with respect to the image on the basis of an inclination of a straight line connecting the detected eyes.
According to the image processing device and the method for image processing of the invention, the image processing device reads image data and performs the masking process of a nearly rectangular shape on an eye portion of the person so as to make the face of the person unidentifiable. At this moment, the image processing device determines the inclination of the nearly rectangular shape on which the masking process is performed on the basis of the inclination of a straight line connecting the left and right eyes extracted from the image data.
The face of the person is photographed in the image while being directed straight to the front in one case, inclined to one side in another case, and turned sideways or obliquely in yet another case. Thus, upon detecting the face and performing the masking process, the image processing device may possibly fail to perform the masking process on the eye position. As a result, the eye often protrudes from the mask of the rectangular shape resulting in that the person is identified, or that even a portion not to be hidden is hidden. Meanwhile, if detecting the left and right eye positions of the person and determining the inclination of the rectangular shape on the basis of the inclination of a straight line connecting the eyes, the image processing device can incline the mask of the rectangular shape in accordance with the inclination of the left and right eyes even in a case where the head is inclined to one side or turned sideways and the eye position changes. Thus, the eye never protrudes from the mask of the rectangular shape so that the image processing device can provide an image in which the person is properly hidden.
The image processing device can check the inclination of the straight line connecting the left and right eye areas by using various methods. The image processing device, e.g., can select one of the sides forming an outer fringe of the image and check the inclination of the straight line connecting the left and right eye areas with respect to the selected side. Or, the image processing device can check the inclination of the straight line with respect to a direction in which pixels of the image data are arranged. In order to check the inclination of the straight line, it is enough to check the inclination with respect to a direction determined relatively to the image. The image processing device may check the inclination with respect to any direction as long as it is the inclination with respect to such a direction. In order to determine the inclination of the rectangular shape on which the masking process is performed, it is enough to determine the inclination with respect to a direction determined with respect to the image. The image processing device may determine the inclination with respect to any direction as long as the direction is determined with respect to the image.
In order to perform the masking process, the image processing device can use various methods as long as using such methods can hide the original image. The image processing device, e.g., can change a gray scale value of the image data to a gray scale value of a determined color (e.g., black) so as to paint out the original image. Or, the image processing device can perform a filtering process such as a mosaic process or a shade-off process. The image processing device can hide the original image recorded in the image data by using such methods, and can thereby make the person photographed in the image unidentifiable.
The image processing device of the invention may determine the length of the rectangular shape on which the masking process is performed on the basis of a distance between the detected left and right eyes. Even if the person in the image inclines his or her head to one side or directs his or her face sideways resulting in that the distance between the left and right eyes changes, the image processing device can thereby change the length of the rectangular shape so as to hide the left and right eyes. The image processing device can thereby hide the eyes so as to make the person unidentifiable. The image processing device can prevent the rectangular shape from reaching a portion of the image other than the face of the person. The image processing device can thereby avoid an unnatural image in which the masking process has been performed on the portion other than the face so as to provide an image of good quality.
In a case where only one of the eyes has been detected from the image, the image processing device of the invention can set the inclination of the rectangular shape to be equal to the inclination of the detected one eye.
In a case where only one of the eyes is photographed in the image, it is conceivable that the face of the person is nearly completely, or at least significantly, turned sideways. Thus, the image processing device can make the person unidentifiable by hiding the one eye photographed in the image. The image processing device can set the inclination of the rectangular shape to be equal to the inclination of the detected one eye so as to exactly hide the one eye and to make the person unidentifiable.
In a case where it is difficult to detect the inclination of the eye, the image processing device can use a preset inclination (e.g., “zero degrees”). In such a case, the image processing device can sufficiently hide the one eye, and can make the person unidentifiable.
Upon detecting only one of the eyes of the person from the image, the image processing device of the invention may determine the length of the rectangular shape on the basis of the size of the detected one eye.
The image processing device can sufficiently hide the eye even in the image in which only the one eye is photographed by changing the length of the rectangular shape in accordance with the size of the eye, so as to protect privacy of the person more exactly. The image processing device can prevent the rectangular shape from reaching a portion other than the face of the person, so as to avoid performing the masking process on the portion other than the face and producing an unnatural image. The image processing device can thereby provide an image of good quality.
The invention may be implemented by using a computer that reads a program for implementing the method for image processing described above and performs determined functions. Thus, the invention includes an aspect of the program as described below. That is, the program of the invention is one for implementing a method for image processing by using a computer. In the method for image processing, image data of an image in which a person is photographed is analyzed and a masking process of a nearly rectangular shape is performed on an eye portion of a face of the person. The method includes detecting both eyes of the person, and determining an inclination of the nearly rectangular shape on which the masking process is performed with respect to the image on the basis of an inclination of a straight line connecting the detected eyes.
Upon reading the program and performing the above functions, the computer can properly perform the masking process on the person in the image.
The invention will be described with reference to the accompanying drawings, wherein like numbers reference like elements.
An embodiment of the invention will be explained in the order shown below so as to clarify details of the invention described above.
The controller section 300 has a CPU, a ROM, a RAM and so on. The controller section 300 is configured to perform a determined process on the image data read by the scanner section 100 or read from a recording medium such as a memory card, and then print the image. The controller section 300 is configured, e.g., to analyze the image data so as to detect a person, and to perform a masking process such as painting out a portion of the person's eyes with a rectangular shape. The controller section 300 can thereby print an image in which the person's privacy is protected. In order to print such an image, however, it is important that the masking process be performed on a proper portion of the image. Upon not being performed on the proper portion, the masking process may possibly produce inconvenience such that the face is not masked resulting in unprotected privacy or that a portion not to be masked is masked resulting in an unnatural image. In order to address the above inconvenience, the printer 10 of the embodiment has the image processing device of the invention configured to perform the masking process on the proper portion of the image.
After reading the image data, the image processing device then analyzes the image data and detects the “eyes” of a person photographed in the image so as to perform the masking process (step S102 shown in
When trying to detect eyes by using the pattern matching method and so on, however, the image processing device has to search the whole image for the eyes, resulting in taking some time to detect the eyes. Thus, the image processing device may detect a face before detecting the eyes, and then search the detected face for the eyes. Then, the image processing device detects the eyes by applying the pattern matching method described above to the portion of the face. As an area in which the image processing device searches for the eyes is thereby limited to the portion of the face, the image processing device can detect the eyes more quickly. As a matter of course, the image processing device is not limited to use the above method, and can detect the eyes by using various methods.
After detecting the eye area, the image processing device sets a rectangular area on which the masking process is performed (mask area) in the image. The “printing process adapted for protecting privacy” of the embodiment has a feature for properly setting the mask area in which the image processing device checks an inclination of the face on the basis of the positions of the left and right eye areas and sets the mask area to be inclined in accordance with the inclination of the face. The image processing device can thereby properly perform the masking process for various images in which a person is photographed, which will be described later in detail.
In the “printing process adapted for protecting privacy” of the embodiment, the image processing device draws a straight line to connect the left and right eye areas so as to check the inclination of the face, and obtains an angle of the inclination of the straight line. If the face is inclined, the line connecting the left and right eyes should be inclined accordingly, and the image processing device can thereby grasp the inclination of the face. In order to set the mask area at a more proper inclination, it is important that the inclination of the face be obtained accurately. Thus, in order to properly set the mask area, the image processing device of the embodiment obtains the angle of the inclination of the face by drawing the straight line by using a method shown below.
After drawing the straight line between the left and right eye areas as described above, as shown in
Instead of determining the inclination of the mask area after finding the angle of the straight line connecting the eye areas, the image processing device may directly determine the inclination of the mask area on the basis of the central positions of the two eye areas. As shown in
The image processing device can determine the length of the mask area by using various methods. The image processing device may detect the face area from the image and determine the length of the mask area on the basis of the width of the detected face area. The image processing device can thereby set a mask area of a proper length in accordance with the width of the face. The image processing device can determine the length of the mask area on the basis of the eye area, instead of the face area. The image processing device of the embodiment determines the length of the mask area on the basis of the positions of the eye areas in order to more properly set the length of the eye area.
The image processing device may determine the length of the mask area by referring to a look-up table that determines the length of the mask area, instead of multiplying the distance by the coefficient “K”. That is, the length of the mask area that is suitable for the distance between the eye areas is checked beforehand, and the image processing device stores a look-up table that associates the checked length of the mask area with the distance between the eye areas. Then, the image processing device determines the length of the mask area by referring to the look-up table on the basis of the distance between the eye areas obtained from the image data. As the length of the mask area can thereby be set exactly for each value of the distance between the eye areas, the image processing device can more properly set the length of the mask area. Meanwhile, as it is enough to perform a simple calculation shown in
After setting the mask area as described above, the image processing device actually modifies the image data in the mask area so as to perform the masking process (step S108 shown in
As a matter of course, the image processing device is not limited to using the above method and can perform the masking process by using another method. The image processing device may perform a so-called mosaic process on the image data in the mask area, or may perform a shade-off process by using a shade-off filter such as a Gaussian filter. The image processing device can hide the eyes of the person drawn in the mask area in such cases, and can thereby obtain image data in which the privacy of the person is protected.
After producing image data in which the privacy of the person is protected, the controller section 300 provides the printer section 200 with the image data. Then, the printer section 200 prints the image on the basis of the provided image data (step S110 shown in
According to the “printing process adapted for protecting privacy” of the embodiment, as described above, the image processing device detects the eye areas from the image data and determines the inclination of the mask area on the basis of the positions of the detected eye areas (refer to
Generally speaking, a human face photographed in an image is not only photographed while being directed straight to the front, but is photographed in various states. In some cases, the human face is photographed while being inclined to one side, turned sideways or obliquely. As a face is photographed in various states image by image, as a matter of fact, attempting to detect a face so as to perform the masking process never succeeds in some cases. As a result, a human eye protrudes from the mask area and the person can be identified in some cases.
Meanwhile, according to the “printing process adapted for protecting privacy” of the embodiment, the image processing device grasps the inclination of the face on the basis of the positions of the left and right eye areas and sets the mask area to be inclined in accordance with the inclination of the face. Thus, even if the person in the image inclines her head to one side, the image processing device can set the mask area to be inclined accordingly, and there is no chance that the person can be identified due to one of her eyes that protrudes from the mask area (refer to
According to the “printing process adapted for protecting privacy” of the embodiment, the image processing device can properly perform the masking process not only in a case where a photographed person inclines his or her head to one side, but also in a case where the face is turned sideways or obliquely.
According to the “printing process adapted for protecting privacy” of the embodiment, it is enough to approximately detect the positions of the eye areas, and the image processing device can properly set the mask area without detecting the eye areas with high accuracy. That is, if by approximately detecting the positions of the eye areas, the image processing device can determine the inclination of the mask area on the basis of the detected positions, the image processing device thus need not completely detect the “eyes”. In a case where some hairs are covering an eye or a shadow is cast on an eye, e.g., it is difficult to detect a portion of the eye on which the hairs or the shadow exists, and the image processing device cannot detect such a portion in some cases. In such a case, the image processing device cannot perform the masking process on the portion that cannot be detected (on which the hairs or the shadow exist), and the person may consequently be identified. Meanwhile, according to the embodiment, if detecting a portion of each of the eyes, the image processing device can grasp the inclination of the face on the basis of the above portion and set the mask area to be inclined. Thus, even if there is a portion to be hardly detected (a portion on which the hairs or the shadow exist), the image processing device can properly perform the masking process and properly protect the privacy. Thus, the image processing device thereby need not detect the eyes with high accuracy, and can thereby simplify the detection process and quickly print the image.
According to the “printing process adapted for protecting privacy” of the embodiment, as described above, the image processing device determines the inclination of the mask area on the basis of the positions of the left and right eye areas so as to properly perform the masking process in various images. As the privacy of the person photographed in the image is precisely protected, the image processing device enables the user to easily print the image without being worried about the privacy or the right to refuse to be photographed.
According to the embodiment, the image processing device determines the length of the mask area on the basis of the distance between the eye areas (refer to
As the direction of the human face is various image by image, as described above, the mask area may be too long in some images. In an image in which the face shown in
Meanwhile, the image processing device of the embodiment determines the length of the mask area on the basis of the distance between the left and right eye areas, and thus can set the mask area of a proper length in such an image. That is, if the face is turned sideways, the distance between the left and right eye areas is reduced as much accordingly. The image processing device determines the length of the mask area on the basis of the reduced distance, and thus can set the mask area of the proper length that does not protrude from the face. Thus, the image processing device can avoid performing the masking process on a portion other than the face and producing an unnatural image, and can obtain an image of good quality.
The image processing device of the embodiment described above detects both the left and right eye areas so as to properly set the mask area. The image processing device can properly set the mask area, however, even if detecting only one of the eye areas.
Thus, upon detecting only one of the eye areas, the image processing device sets the mask area while regarding the angle of the inclination of the face as “zero degrees” without checking the angle of the inclination of the face in detail. That is, as shown in
As a matter of course, the image processing device may analyze the image in detail so as to grasp the inclination of the face, and set the mask area to be inclined in accordance with the inclination of the face. Upon analyzing the image data of the eye area and detecting an outline of the eye, e.g., the image processing device can estimate the inclination of the face from the inclination of the eye. Thus, the image processing device may set the mask area to be inclined in accordance with the estimated inclination of the face. The image processing device can thereby more properly set the mask area in accordance with the inclination of the face.
The image processing device can properly set the length of the mask area.
The image processing device has been described above. The invention is not limited to the overall embodiment and modifications described above, and can be implemented in various manners within the scope of the invention.
As described above, the image processing device of the embodiment is included in the printer. The image processing device of the embodiment can be included in a photograph printer such as a so-called minilab device. The image processing of the embodiment can be included in an unmanned photograph printer terminal placed on a street corner or in a public space. Also in such cases, the image processing device can properly perform the masking process on a person and can thereby easily provide an image in which privacy or a right to refuse to be photographed is protected.
The image processing device of the embodiment can be included in a digital still camera. Upon being included in the digital still camera, the image processing device analyzes an image photographed by the user and perform the masking process on a person photographed in the image. As the masking process is thereby properly performed on the person, the user can easily distribute or open the image by uploading it on a website and so on without worrying about the privacy or the right to refuse to be photographed.
As a matter of course, the image processing device of the embodiment can be included not only in the printer device or the digital still camera but in various devices. The image processing device can be included in an image display device such as a liquid crystal display monitor or a CRT display monitor, and in a mobile display device such as a photo viewer or a mobile phone terminal. Also in such cases, the image processing device can properly perform the masking process on a person and can thereby provide an image in which privacy or a right to refuse to be photographed is exactly protected.
The present application claims the priority based on a Japanese Patent Application No. 2008-126982 filed on May 14, 2008, the disclosure of which is hereby incorporated by reference in its entirety.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2008-126982 | May 2008 | JP | national |
