This case relates to a detection device, a detection method, and a non-transitory computer-readable recording medium storing a detection program.
For palm vein authentication, palm print authentication, or the like, it has been required to accurately detect a hand image from an original image (for example, refer to Patent Document 1). Furthermore, a technique has been known for detecting a hand image from an image binarized by using brightness for an original image captured by performing irradiation with visible light (for example, refer to Patent Document 2).
Examples of the related art include: [Patent Document 1] Japanese Laid-open Patent Publication No. 2018-026012; [Patent Document 2] Japanese Patent No. 6629150; and [Non-Patent Document 1] T. Simon, H. Joo. I. Matthews, and Y. Sheikh. “Hand keypoint detection in single images using multiview bootstrapping.” in CVPR, 2017.
According to an aspect of the embodiments, there is provided a detection device including: a memory; and a processor coupled to the memory, the processor being configured to perform processing including: the processor selecting a color channel, based on a separation degree that is an index calculated from each of a first color channel image and a second color channel image for an original image that includes a background image and a hand image and used to determine separability between the background image and the hand image; and the processor detecting the hand image included in the original image, by using the color channel selected.
The object and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the claims.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention.
detection device, and
In the related art, it is difficult to accurately detect a palm region.
In one aspect, an object of the embodiments is to provide a detection device, a detection method, and a detection program that can improve detection accuracy of a hand image.
Biometric authentication is a technology for performing identity verification using biometric information such as fingerprints, faces, or veins. In the biometric authentication, biometric data for collation obtained by a sensor is compared (collated) with registered biometric data registered in advance in a situation where confirmation is needed, and it is determined whether or not a degree of similarity is equal to or higher than an identity determination threshold, thereby verifying the identity.
In palm vein authentication, palm print authentication, or the like, it is important to accurately detect a hand image from an original image. However, for example, in a case where imaging is performed using a typical camera such as a camera built in a mobile terminal, there is a possibility that it is difficult to separate the hand image from a background image depending on optical characteristics of the camera, an influence of reflection or shadow due to an influence of a light source, a color, a pattern, or the like of background.
Therefore, in the following embodiments, a detection device, a detection method, and a detection program that can improve detection accuracy of a hand image by separating the hand image and the background image regardless of an imaging condition will be described.
The Central Processing Unit (CPU) 101 is a central processing unit. The CPU 101 includes one or more cores. The Random Access Memory (RAM) 102 is a volatile memory that temporarily stores a program to be executed by the CPU 101, data to be processed by the CPU 101, and the like. The storage device 103 is a non-volatile storage device. As the storage device 103, for example, a Read Only Memory (ROM), a Solid State Drive (SSD: Solid State Drive) such as a flash memory, a hard disk to be driven by a hard disk drive, or the like may be used. The storage device 103 stores a detection program. The interface 104 is an interface device with an external device. For example, the interface 104 is an interface device with a Local Area Network (LAN).
The display device 105 is a display device or the like such as a Liquid Crystal Device (LCD). The input device 106 is an input device such as a keyboard or a mouse. The camera 107 is a Metal Oxide Semiconductor (MOS) sensor, a Charged Coupled Device (CCD) sensor, or the like.
The CPU 101 executes the detection program so as to implement the acquisition unit 11, the key point detection unit 12, the region division unit 13, the determination unit 14, the color channel conversion unit 15, the binarization processing unit 16, the candidate extraction unit 17, the separation degree calculation unit 18, the color channel selection unit 19, the hand image detection unit 20, the biometric information extraction unit 21, the collation unit 22, and the storage unit 23. Note that, as the acquisition unit 11, the key point detection unit 12, the region division unit 13, the determination unit 14, the color channel conversion unit 15, the binarization processing unit 16, the candidate extraction unit 17, the separation degree calculation unit 18, the color channel selection unit 19, the hand image detection unit 20, the biometric information extraction unit 21, the collation unit 22, and the storage unit 23, hardware such as a dedicated circuit may be used.
The acquisition unit 11 acquires an original image including a hand image and a background image, from the camera 107 (step S1).
Next, the key point detection unit 12 detects a plurality of key points indicating parts of a hand, in the original image acquired by the acquisition unit 11 (step S2). For example, the key point detection unit 12 detects 21 key points P0 to P20. The key point P0 is a point at a boundary between a palm and a wrist. The key point P1 is a point at a base of a thumb. The key point P2 is a point at a second joint of the thumb. The key point P3 is a point at a first joint of the thumb. The key point P4 is a point at a fingertip of the thumb. The key point P5 is a point at a base of an index finger. The key point P6 is a point at a second joint of the index finger. The key point P7 is a point at a first joint of the index finger. The key point P8 is a point at a fingertip of the index finger. The key point P9 is a point at a base of a middle finger. The key point P10 is a point at a second joint of the middle finger. The key point P11 is a point at a first joint of the middle finger. The key point P12 is a point at a fingertip of the middle finger. The key point P13 is a point at a base of a ring finger. The key point P14 is a point at a second joint of the ring finger. The key point P15 is a point at a first joint of the ring finger. The key point P16 is a point at a fingertip of the ring finger. The key point P17is a point at a base of a little finger. The key point P18 is a point at a second joint of the little finger. The key point P19 is a point at a first joint of the little finger. The key point P20 is a point at a fingertip of the little finger.
These key points P0 to P20 can be detected, for example, from an RGB image. For example, the key points P0 to P20 can be obtained by detecting a Bounding box indicating a region of a hand and then detecting each portion of the hand. For example, the key points P0 to P20 can be detected by using a method in Non-Patent Document 1 “T. Simon. H. Joo. I. Matthews, and Y. Sheikh. “Hand keypoint detection in single images using multiview bootstrapping.” in CVPR. 2017.”. The key points P0 to P20 are illustrated in
Next, the region division unit 13 obtains a plurality of small regions A to D from the original image, using the key points (step S3). The small region is a region smaller than the original image.
Next, the determination unit 14 determines whether or not each small region is contained in the hand, in a case of planar view (step S4). Since the entire region of the small region D is contained in the hand, it is determined as “Yes”. Since the small regions A to C include background, it is determined as “No”.
In a case where it is determined in step S4 as “Yes”, the determination unit 14 determines that the small region determined as “Yes” in step S4 is the region of the hand (step S5).
In a case where it is determined in step S4 as “No”, the color channel conversion unit 15, the binarization processing unit 16, the candidate extraction unit 17, the separation degree calculation unit 18, and the color channel selection unit 19 separate the small region determined as “No” in step S4 into the region of the hand and the background region (step S6). Details will be described later.
Next, the hand image detection unit 20 synthesizes the hand regions in the respective small regions obtained in steps S5 and S6, the hand image detection unit 20 detects the hand image (step S7).
First, the color channel conversion unit 15 converts the small region to be a target into a plurality of color channel images (step S11). The plurality of color channel images includes a first color channel image and a second color channel image that are at least two types of color channels. The color channel includes, for example, red (R), green (G), blue (B), hue (H: Hue), saturation (S: Saturation), brightness (V: value and brightness), a brightness (Y), a color difference (U), a color difference (V), or the like. For example, the color channel conversion unit 15 obtains an image of each color channel (hereinafter, referred to as color channel image), for the small region to be the target. Note that the plurality of color channel images may include a color channel image that includes three or more color channels.
Next, the binarization processing unit 16 executes binarization processing using a predetermined threshold, on each color channel image and obtains a hand candidate and a background candidate (step S12). For example, the hand candidate is represented in white, and the background candidate is represented in black.
Next, the separation degree calculation unit 18 creates each of a color histogram (frequency distribution of pixel value) of the hand candidate and a color histogram of the background candidate, for each color channel image before the binarization processing and calculates a first separation degree (step S13). For example, the binarization processing is executed on an R image, and the hand candidate and the background candidate are obtained. Next, each of a color histogram of the hand candidate and a color histogram of the background candidate is created, for the R image before the binarization processing. From superposition of these histograms, a first separation degree of the R image is calculated.
Next, the separation degree calculation unit 18 executes morphology processing on at least one of the hand candidate and the background candidate, for each color channel image after the binarization processing. The morphology processing is filling a region. The separation degree calculation unit 18 calculates a second separation degree from a region difference between before and after the morphology processing (step S14).
Next, the color channel selection unit 19 selects a color channel that maximizes a sum of the first separation degree and the second separation degree and obtains a result of separation from the palm to the background (step S15). For example, by normalizing the first separation degree and the second separation degree, it is possible to quantify the first separation degree and the second separation degree on the same scale. For example, by expressing the first separation degree and the second separation degree as numerical values from zero to 100, the sum of the first separation degree and the second separation degree can be compared at the same standard for each color channel.
Hereinafter, biometric authentication processing using the detected palm region will be described.
As illustrated in
Next, the hand image detection unit 20 detects a hand image by the hand image detection processing described above (step S22).
Next, the biometric information extraction unit 21 extracts biometric information from the hand image detected in step S22 (step S23). The biometric information is a vein, a palm print, or the like. However, the biometric information is not particularly limited as long as the biometric information is obtained from the hand image.
The storage unit 23 stores the biometric information extracted in step S23 as the registered biometric data, in association with the ID acquired in step S21 (step S24). Through the above processing, the registration processing is completed. For example, by repeating a plurality of times of this registration processing, it is possible to register the biometric information of the plurality of registrants.
As illustrated in
Next, the hand image detection unit 20 detects a hand image by the hand image detection processing described above (step S32).
Next, the biometric information extraction unit 21 extracts the biometric information from the hand image detected in step S32, as biometric data for collation (step S33).
The collation unit 22 performs authentication, by collating the biometric data for collation extracted in step S33 and each registered biometric data stored in the storage unit 23 (step S34). Specifically, it is determined whether or not a degree of similarity between each registered biometric data stored in the storage unit 23 and the biometric data for collation extracted in step S33 is equal to or more than a threshold. Then, if there is the registered biometric data of which the degree of similarity is equal to or more than the threshold, it is determined that the biometric data for collation is biometric information of a person having the ID of the registered biometric data. The collation unit 22 outputs a determination result.
According to the present embodiment, the separation degree is calculated that is an index used to determine the separability between the background image and the hand image, from each of the first color channel image and the second color channel image for the original image including the background image and the hand image. The color channel is selected based on the calculated separation degree. The hand image included in the original image is detected, using the selected color channel. In order to detect the hand image using the color channel with a high separation degree, the hand image and the background image can be accurately separated regardless of an imaging condition. Therefore, the detection accuracy of the hand image is improved.
Note that the separation degree is not particularly limited as long as the separation degree is the index used to determine the separability between the background image and the hand image.
However, by using the first separation degree calculated from the color histogram calculated for each of the background candidate and the hand candidate, the detection accuracy of the separability between the background candidate and the hand candidate is enhanced. Furthermore, by using the second separation degree according to the region difference between before and after the morphology processing executed on at least one of the background candidate and the hand candidate, the detection accuracy of the separability between the background candidate and the hand candidate is enhanced.
Furthermore, by determining the separability between the background candidate and the hand candidate for each of the plurality of small regions obtained from the original image, it is possible to select the color channel for each small region. As a result, the detection accuracy of the hand image from the original image is further improved. By obtaining the small region using the key point regarding the part of the hand, it is possible to reduce an influence of an individual difference of the palm. Therefore, the influence of the individual difference such as a size or a shape difference of the palm can be reduced. A processing amount for detecting the hand image can be reduced, by not selecting the color channel for the small region determined to be positioned in the hand.
Note that, in each example described above, the color channel selection unit 19 is an example of a color channel selection unit that selects the color channel based on the separation degree that is the index calculated from each of the first color channel image and the second color channel image for the original image including the background image and the hand image and is used to determine the separability between the background image and the hand image. The hand image detection unit 20 is an example of a hand image detection unit that detects the hand image included in the original image, by using the color channel selected by the color channel selection unit. The binarization processing unit 16 is an example of a binarization processing unit that binarizes each of the first color channel image and the second color channel image. The candidate extraction unit 17 is an example of a candidate extraction unit that extracts the background candidate and the hand candidate from each of the first color channel image and the second color channel image that have been binarized. The separation degree calculation unit 18 is an example of a frequency distribution calculation unit that calculates the frequency distribution of the pixel value, for each of the background candidate and the hand candidate. Furthermore, the separation degree calculation unit 18 is an example of a morphology processing unit that executes morphology processing on at least one of the background candidate and the hand candidate. The region division unit 13 is an example of a region division unit that divides each of the first color channel image and the second color channel image into the plurality of regions. The determination unit 14 is an example of a determination unit that determines whether or not the plurality of regions includes the background, using the key point.
While the embodiments of the present invention have been described above in detail, the present invention is not limited to such specific embodiments, and various modifications and alterations may be made within the scope of the present invention described in the claims.
All examples and conditional language provided herein are intended for the pedagogical purposes of aiding the reader in understanding the invention and the concepts contributed by the inventor to further the art, and are not to be construed as limitations to such specifically recited examples and conditions, nor does the organization of such examples in the specification relate to a showing of the superiority and inferiority of the invention. Although one or more embodiments of the present invention have been described in detail, it should be understood that the various changes, substitutions, and alterations could be made hereto without departing from the spirit and scope of the invention.
This application is a continuation application of International Application PCT/JP2022/032990 filed on Sep. 1, 2022 and designated the U.S., the entire contents of which are incorporated herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/JP2022/032990 | Sep 2022 | WO |
| Child | 19039541 | US |