This application claims the benefit of Japanese Patent Application No. 2020-054105, filed on Mar. 25, 2020, the entire disclosure of which is incorporated by reference herein.
The present disclosure relates to an image processing device, an image processing method, and a non-transitory recording medium.
In a face detection function used in a digital camera and the like, even a camera having a high-pixel imaging sensor generally uses an image with a low resolution of a degree of a quarter video graphics array (QVGA, 320×240 pixels) or a video graphics array (VGA, 640×480 pixels) to perform face detection, as in Unexamined Japanese Patent Application Publication No. 2019-12426.
Further, even face authentication and the like for identifying an individual use an image with a low resolution of a degree of VGA. Such use of a low-resolution image in detection and authentication can prevent deterioration in processing speed.
In order to accomplish the above objective, an image processing device according to the present disclosure includes:
a memory storing a program; and
at least one processor configured to execute a program stored in the memory, wherein
the processor
A more complete understanding of this application can be obtained when the following detailed description is considered in conjunction with the following drawings, in which:
An image processing device according to an embodiment of the present disclosure will be hereinafter described in detail with reference to the drawings.
The image processing device according to the embodiment of the present disclosure generates image data for causing a face authentication device of a face authentication system to perform face authentication for use in, for example, security or the like for an office or an event. Note that, the number of persons captured in an image is not particularly limited as long as a face image is prevented from being too small. However, in the following description, the number of persons captured in an image is three, for ease of description.
[Configuration of Face Authentication System]
As illustrated in
An outline of image processing performed in the face authentication system 1 illustrated in
(Configuration of Image Processing Device)
As illustrated in
The controller 20 includes a central processing unit (CPU) and the like, and achieves a function of each section to be described later (an image acquirer 21, an object detector 22, a detection frame setter 23, a detection frame determiner 24, a discriminator 25, a corrector 26, an image processor 27, an image transmitter 28, and an operator 29) by executing a program and the like stored in the storage 30. Further, the controller 20 includes a clock (not illustrated), and can perform acquiring a current date and time, counting elapsed time, and the like.
The storage 30 includes a read-only memory (ROM), a random access memory (RAM), and the like, and all or part of the ROM includes an electrically rewritable memory (a flash memory and the like). The storage 30 functionally includes an object storage 31, a detection frame storage 32, an exclusion range storage 33, and a detection condition storage 34. The ROM stores a program executed by the CPU of the controller 20 and data necessary in advance for execution of the program. The RAM stores data prepared or changed during execution of the program.
The object storage 31 stores a face image being an object detected from an image captured by the imager 40, according to the present embodiment. Further, the object storage 31 stores a minimum detection face Fmin (see
The detection frame storage 32 stores the detection frame 205 set by the detection frame setter 23 and to be described later. Further, the detection frame storage 32 also stores a user-set detection frame 206 voluntarily set by a user. Further, the detection frame storage 32 stores a reference detection frame 200 in advance. Because the image V is divided by the reference detection frame 200, a width and a height of the image V are preferably integer multiples of a width and a height of the reference detection frame 200. The reference detection frame 200 includes a reference detection frame 2001 for first division, a reference detection frame 2002 for second division, . . . , and a reference detection frame 200, for n-th division. A width and a height of the reference detection frame 2001 are equal to a width and a height of the image V. Further, a width and a height of the reference detection frame 200 are the reference detection frame 2001>the reference detection frame 2002>. . . >the reference detection frame 200n−1>the reference detection frame 200n=the minimum detection face Fmin. Note that, in order to prevent increase in processing load, the reference detection frame 200, can also be set to have any size larger than the minimum detection face Fmin, instead of setting the reference detection frame 200n=the minimum detection face Fmin Further, decreasing a value of n in the reference detection frame 200n can prevent increase in processing load. Note that, the smaller a value of n in the reference detection frame 200n (and a detection frame 205n), the closer to a resolution of an original image.
The exclusion range storage 33 stores an exclusion range 210 discriminated and set by the discriminator 25 and to be described later (see
The detection condition storage 34 stores a detection condition Z. The detection condition storage 34 stores, as the detection condition Z, a detection condition Z1 for differentiating a detection frequency for each imaging area, a detection condition Z2 for excluding a range having a predetermined illuminance or more or having a predetermined illuminance or less from a detection target, and the like.
The imager 40 includes an imaging device 41 and a drive device 42.
The imaging device 41 includes a complementary metal oxide semiconductor (CMOS) camera, according to the present embodiment. The imaging device 41 captures the imaging range L at a frame rate of 30 fps to generate the image V. The image V is a Bayer image, and is output with a 12-bit resolution.
The drive device 42 moves, according to an instruction from the operator 29 to be described later, a position of the imaging device 41 to adjust the imaging range L.
The communicator 50 includes a communication device 51 being a module for communicating with the face authentication device 80, external equipment, and the like. The communication device 51 is a wireless module including an antenna when communicating with external equipment. For example, the communication device 51 is a wireless module for performing short-range wireless communication based on Bluetooth (registered trademark). With use of the communicator 50, the image processing device 10 can exchange image data and the like with the face authentication device 80, external equipment, and the like.
The display 60 includes a display device 61 including a liquid-crystal display (LCD) panel.
As the display device 61, a thin-film transistor (TFT) display device, a liquid-crystal display device, an organic EL display device, and the like can be employed. The display device 61 displays the image V, the detection frame 205 to be described later, and the like.
The inputter 70 is a resistive touch panel (an input device 71) provided close to the display 60 or integrally with the display 60. The touch panel may be an infrared ray touch panel, a projected capacitive touch panel, and the like, and the inputter 70 may be a keyboard, a mouse, and the like instead of a touch panel. A user can set the user-set detection frame 206, the user-set exclusion range 211, and the like by using the display 60 through a manual operation via the inputter 70.
Next, a functional configuration of the controller 20 of the image processing device 10 will be described. The controller 20 achieves functions of the image acquirer 21, the object detector 22, the detection frame setter 23, the detection frame determiner 24, the discriminator 25, the corrector 26, the image processor 27, the image transmitter 28, and the operator 29, and performs the object detection processing to be described later and the like.
The image acquirer 21 causes the imager 40 to capture the imaging range L with an exposure condition preset in the image processing device 10 or set by a user, and acquires the image V captured by all pixels in about 33 msec. The image V has a resolution of QVGA. The image acquirer 21 transmits the acquired image V to the object detector 22.
The object detector 22 detects a face image being an object from the image V transmitted from the image acquirer 21, according to the present embodiment. The object detector 22 detects a face image from the image V in about 11 msec by using the detection frame 205 set by the detection frame setter 23 and to be described later. Further, when the user-set detection frame 206 is set, the object detector 22 detects a face image from the image V by using the user-set detection frame 206. The object detector 22 determines whether a face image is detected from the image V by using the detection frame 205. The object detector 22 stores the detected face image in the object storage 31.
The detection frame setter 23 reads a face image in the image V stored in the object storage 31, and sets a width and a height of a smallest face image among the read face images, as a width DFmin_w and a height DFmin_h of a frame-overlapping area of the reference detection frame 200 illustrated by diagonal hatching in
The detection frame determiner 24 determines whether to shrink the detection frame 205n every time a detection operation for a face image over the entire image V by using the detection frame 205 is completed. The detection frame determiner 24 compares a smallest face among faces detected during the detection operation with the minimum detection face Fmin, and determines to shrink the detection frame 205 when the smallest face is larger. When the detection frame determiner 24 determines to shrink the detection frame 205, the detection frame setter 23 sets a width and a height of the smallest face as a width DFmin_w and a height DFmin_h of a frame-overlapping area of a reference detection frame 200n+1 to set a detection frame 205n+1. When the smallest face is equal in size to the minimum detection face Fmin, the detection frame determiner 24 does not determine to shrink the detection frame 205 (the detection frame setter 23 ends an operation of shrinking the detection frame 205).
The discriminator 25, when the detection frame 205 or the user-set detection frame 206 is positioned inside a detected face image 220 already detected by the object detector 22 as illustrated in
The corrector 26 corrects a frequency of face image detection according to setting of a frequency of face image detection for each region in the imageV. A correction method will be described later.
The image processor 27 processes a face image stored in the object storage 31. After the object detection processing to be described later ends, the image processor 27 arranges, according to coordinates on the image V, a face image stored in the object storage 31 on an image map by which the face authentication device 80 can perform face recognition. Alternatively, the image processor 27 associates coordinate data on the image V with a face image.
The image transmitter 28 transmits the acquired image V, the image map, and the like to the face authentication device 80.
The operator 29 transmits, to the drive device 42, an instruction for moving the imaging range L of the imager 40.
The functional configuration of the controller 20 has been described above. Hereinafter, the object detection processing performed by the image processing device 10 will be specifically described by using an example of a case in which a face image acquired from a captured image is
The minimum detection face Fmin (see
The detection frame setter 23 reads the face images of the persons 101 and 102 stored in the object storage 31, and sets a width and a height of the face image of the person 102 being a smallest face image among the read face images, as a width and a height of a frame-overlapping area of the reference detection frame 200 (the diagonally hatched range in
The object detector 22 divides the image V by the detection frame 205 (or the user-set detection frame 206) into regions with a frame-overlapping area having an overlapping width and an overlapping height as illustrated in
The detection frame determiner 24 compares a smallest face among faces detected during the detection operation with the minimum detection face Fmin. Since the face image of the person 103 is detected from the divided region, the detection frame determiner 24 compares the face image of the person 103 with the minimum detection face Fmin, determines that the face image of the person 103 is larger than the minimum detection face Fmin, and determines to shrink the detection frame 205. The detection frame setter 23 calculates a width and a height of the face image of the person 103 being the smallest face image. The detection frame setter 23 adds the width and the height of the frame-overlapping area to a width and a height of the reference detection frame 200 to store the results as the detection frame 205 (or the user-set detection frame 206) in the detection frame storage 32.
The object detector 22 divides the image V by the detection frame 205 (or the user-set detection frame 206) into regions with a frame-overlapping area having an overlapping width and an overlapping height, and then detects a face image in each of the divided regions.
Subsequently, the controller 20 ends the detection after repeating division of the image V and detection of a face image until the width and the height of the frame-overlapping area become as small as the width and the height of the minimum detection face Fmin, and generates a face image map for the entire image V as illustrated in
(Face Authentication Device)
The face authentication device 80 is, for example, a device based on an eigenface using principal component analysis as an algorithm for face recognition. The face authentication device 80 uses image data transmitted from the image processing device 10 to perform face authentication (two-dimensional face authentication).
[Processing Performed by Image Processing Device]
Next, the object detection processing performed by the image processing device 10 will be described by using a flowchart.
(Object Detection Processing)
A flow of the object detection processing performed by the image processing device 10 will be described with reference to
First, the minimum detection face Fmin is set in the image processing device 10 (Step S1). The minimum detection face Fmin can also be voluntarily set by a user through the inputter 70. Further, the imaging range L is also set.
Next, a detection frame 2051 for use in first (n=1) object detection processing is set. Since face image detection in the first object detection processing is performed at once on the entire image V, the detection frame 2051 has a same size as the image V. The detection frame setter 23 sets the detection frame 2051 having a same size as the image V, and stores the detection frame 2051 in the detection frame storage 32 (Step S2).
The image acquirer 21 causes the imager 40 to capture the imaging range L, acquires the captured image V, and transmits the acquired image V to the object detector 22 (Step S3).
After the image processing device 10 acquires the image V, the detection frame setter 23 reads the detection frame 2051 from the detection frame storage 32, and divides the image V by the detection frame 2051. In first division, the entire image V is divided by the detection frame 2051 having the same size as the image V (Step S4).
The discriminator 25 discriminates presence of the detection frame 2051or a user-set detection frame 2061 positioned inside a face image detected in previous division (Step S5). The detection frame 2051 or the user-set detection frame 2061 positioned inside a face image detected in previous division is discriminated as the exclusion range 210 or the user-set exclusion range 211, and is stored in the exclusion range storage 33. Note that, since this time is the first division, a face image detected in previous division is absent (Step S5; No), and the processing proceeds to Step S7. The object detector 22 detects a face image from the image V by using the detection frame 2051 set by the detection frame setter 23 (Step S7). Subsequently, the processing proceeds to Step S8.
In second or subsequent division, when a face image detected in previous division is present and the detection frame 205 positioned inside the face image is present, the detection frame 205 is discriminated as the exclusion range 210 or the user-set exclusion range 211, and is stored in the exclusion range storage 33 (Step S5; Yes). The processing proceeds to Step S6, and the object detector 22 detects a face image from the image V by using the detection frame 2051 set by the detection frame setter 23, excluding the detection frame 2051 being the exclusion range 210. Subsequently, the processing proceeds to Step S8.
In Step S8, the object detector 22 determines whether a face image is detected by the detection frame 2051 in preceding Step S6 or S7.
When no face image is detected (Step S8; No), the detection frame determiner 24 determines whether the minimum detection face Fmin is set as a width and a height of a frame-overlapping area (Step S9). When the minimum detection face Fmin is not set as a width and a height of a frame-overlapping area (Step S9; No), n is incremented by 1 (n=n+1=2), and the detection frame setter 23 adds the minimum detection face Fmin to the reference detection frame 2002 as a width and a height of a frame-overlapping area to set a detection frame 2052 smaller than the detection frame 2051 (Step S10). Subsequently, the processing returns to Step S4, and second division is performed on the image V. When the minimum detection face Fmin is set as a width and a height of a frame-overlapping area (Step S9; Yes), the processing proceeds to Step S15.
When the object detector 22 determines that a face image is detected by the detection frame 2051 in preceding Step S6 or S7 (Step S8; Yes), the processing proceeds to Step S11.
In Step S11, the object detector 22 stores the detected face image in the object storage 31, and the processing proceeds to Step S12.
The detection frame determiner 24 compares a size of the face image of the person 102 being a smallest face image among the detected face images with a size of the set minimum detection face Fmin (Step S12). When a size of the face image of the person 102 is larger than a size of the set minimum detection face Fmin (Step S12; Yes), the detection frame determiner 24 determines to shrink the detection frame 205. n is incremented by 1 (n=n+1=2) (Step S13), and the processing proceeds to Step S14.
The detection frame setter 23 reads a face image in the image V stored in the object storage 31, and sets a width and a height (see
When a size of the face image of the person 102 is equal to a size of the set minimum detection face Fmin (Step S12; No), the detection frame determiner 24 determines not to shrink the detection frame 205, and the processing proceeds to Step S15. When the processing ends (Step S15; Yes), the processing ends, and when the processing does not end (Step S15; No), the processing returns to Step S2.
Through the object detection processing described above, the image processing device 10 performs face detection on the entire image V, then divides the image V, and further searches for a face image in each divided detection frame 205. Thus, a face image smaller than the minimum detection face Fmin can be detected from the image V. Further, upon division of the image V after the face detection on the entire image V, the image V is divided by using a width and a height of a smallest face image among detected face images as a frame-overlapping area. Thus, a situation in which a face image captured in the image V is not successfully detected (a situation in which a not-yet-detected face image cannot be detected, because only a part of the face image is captured in each of the adjacent detection frames 205 and 205 and the whole face image does not fit in any of the detection frames 205) can be prevented. Furthermore, the face detection is performed by each detection frame with a resolution of QVGA. Thus, a small face image can be detected while increase in processing load is prevented.
According to the above-described embodiment, an object being a detection target of the object detector 22 is a face image. However, the object may be a person, a physical object, and the like for person detection and physical object detection (a vehicle and the like).
According to the above-described embodiment, the imaging device 41 captures an image at a frame rate of 30 fps, and the image acquirer 21 fetches the image by all pixels in about 33 msec, as illustrated in
Further, since the region III in
According to the above-described embodiment, the minimum detection face Fmin is set as a width and a height of a frame-overlapping area in Steps S9 and S10 in the object detection processing in
Further, according to the above-described embodiment, the image processing device 10 includes the imager 40. However, an image processing device may not include an imager and may be connected to an external imaging device that is controllable via the communicator 50.
According to the above-described embodiment, the image processing device 10 generates an image for the face authentication device 80 performing two-dimensional face authentication. However, the image processing device 10 may generate an image for a face authentication device performing three-dimensional face authentication.
Each of the functions of the image processing device 10 according to the present disclosure can be implemented also by a computer such as a normal personal computer (PC). Specifically, according to the above-described embodiment, description has been given assuming that a program for exposure correction processing and image processing performed by the image processing device 10 is stored in advance in the ROM of the storage 30. However, a computer that can achieve each of the above-described functions by storing and distributing a program in a non-transitory computer-readable recording medium such as a flexible disk, a compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), and a magneto-optical (MO) disc and reading and installing the program on the computer may be configured.
The foregoing describes some example embodiments for explanatory purposes. Although the foregoing discussion has presented specific embodiments, persons skilled in the art will recognize that changes may be made in form and detail without departing from the broader spirit and scope of the invention. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. This detailed description, therefore, is not to be taken in a limiting sense, and the scope of the invention is defined only by the included claims, along with the full range of equivalents to which such claims are entitled.
Number | Date | Country | Kind |
---|---|---|---|
2020-054105 | Mar 2020 | JP | national |