Patent Application
20240054752
The present disclosure relates to a technical field of an information processing apparatus, an information processing method and a recording medium that are configured to extract a key point (namely, a feature point) of a target object from an image in which the target object is included, for example.
A Patent Literature 1 discloses one example of an information processing apparatus that is configured to extract a key point (namely, a feature point) of a target object from an image in which the target object is included. Specifically, the Patent Literature 1 discloses a feature point extraction apparatus that extracts, from an image in which a person is included, a feature point of a face of the person that is used for a face authentication of the person.
Additionally, there are Patent Literatures 2 to 6 as a background art document relating to the present disclosure.
It is an example object of the present disclosure to provide an information processing system, an information processing method and a recording medium that aims to an improvement of a technique disclosed in the background art document.
One example aspect of an information processing apparatus includes: a reducing unit that generates a second image by reducing a first image in which a target object is included; a first extracting unit that extracts, as a first key point, a key point of the target object from the second image; a setting unit that sets a target area that designates a part of the first image based on the first key point; and a second extracting unit that extracts, as a second key point, a key point of the target object from a target image of the first image that is included in the target area.
One example aspect of an information processing method includes: generating a second image by reducing a first image in which a target object is included; extracting, as a first key point, a key point of the target object from the second image; setting a target area that designates a part of the first image based on the first key point; and extracting, as a second key point, a key point of the target object from a target image of the first image that is included in the target area.
One example aspect of a recording medium is a recording medium on which a computer program that allows a computer to execute an information processing method is recorded, the information processing method includes: generating a second image by reducing a first image in which a target object is included; extracting, as a first key point, a key point of the target object from the second image; setting a target area that designates a part of the first image based on the first key point; and extracting, as a second key point, a key point of the target object from a target image of the first image that is included in the target area.
Next, an example embodiment of an information processing apparatus, an information processing method and a recording medium will be described with reference to the drawings. In the below described description, the example embodiment of the information processing apparatus, the information processing method and the recording medium will be described by using a key point extraction system SYS.
The key point extraction system SYS is configured to extract a key point KP of a target object from an image in which the target object is included. The below described description explains an example in which the key point extraction system SYS is configured to extract the key point KP of a person that is included in a person image IMG1 from the person image IMG1 in which the person (namely, a living body) that is one example of the target object is included. However, the key point extraction system SYS may be configured to extract the key point KP of any target object that is included in an image from the image in which any target object that is different from the person is included. A living body other than a human (for example, at least one of a mammal such as a dog, a cat and so on, a bird such as a sparrow and so on, a reptile such as a snake and so on, an amphibia such as a frog and so on and a fish such as a goldfish and so on) is one example of the target object. An object that is not the living body is another example of the target object. A robot that imitates the human or an animal is one example of the object that is not the living body.
Moreover, the below described description explains an example in which the key point extraction system SYS is configured to extract the key point KR of an iris of the person that is included in the person image IMG1 from the person image IMG1. This key point extraction system SYS may be used as an iris authentication system described in detail later. The iris authentication system is a system that is configured to authenticate the person by using the iris. However, the key point extraction system SYS may be configured to extract the key point KR of any part of the person that is included in the person image IMG1 from the person image IMG1. A part that is usable for authenticating the person is one example of any part of the person. A face that is usable for a face authentication is another one example of the part that is usable for authenticating the person. A finger on which a fingerprint is formed that is usable for a fingerprint authentication is another one example of the part that is usable for authenticating the person. A hand on which a palm print is formed that is usable for a palm print authentication is another one example of the part that is usable for authenticating the person. A finger in which a pattern of a vein is formed that is usable for a vein authentication is another one example of the part that is usable for authenticating the person.
Next, the key point extraction system SYS will be described in more detail with reference to the drawings.
(1) Key Point Extraction System SYS in First Example Embodiment
Firstly, the key point extraction system SYS in a first example embodiment will be described. Note that the key point extraction system SYS in the first example embodiment is referred to as a “key point extraction system SYSa” in the below described description.
(1-1) Entire Configuration of Key Point Extraction System SYSa
Firstly, with reference to
As illustrated in
The camera 1 is an imaging apparatus that is configured to capture an image of an imaging target range. The camera 1 includes a processing for generating an image in which the imaging target range is included by capturing the imaging target range. Since the key point extraction system SYSa extracts the key point KP of the iris of the person as described above, the person usually exists in the imaging target range. In this case, the camera 1 generates the person image IMG1 in which the person is included by capturing the image of the person. Especially, the camera 1 generates the person image IMG1 in which a face (especially, at least a part of the face including the iris and a part around the iris) of the person is included by capturing the image of the face (especially, at least a part of the face including the iris and a part around the iris) of the person. The camera 1 outputs the generated person image IMG1 to the key point extraction apparatus 2. Specifically, the camera 1 transmits the generated person image IMG1 to the key point extraction apparatus 2 through the communication network 3.
The key point extraction apparatus 2 receives the person image IMG1 transmitted from the camera 1 through the communication network 3. The key point extraction apparatus 2 performs a key point extraction operation for extracting the key point KP of the iris of the person that is included in the person image IMG1 from the received person image IMG1.
(1-2) Configuration of Key Point Extraction Apparatus 2
Next, with reference to
As illustrated in
The arithmetic apparatus 21 includes at least one of a CPU (Central Processing Unit), a GPU (Graphic Processing Unit) and a FPGA (Field Programmable Gate Array), for example. The arithmetic apparatus 21 reads a computer program. For example, the arithmetic apparatus 21 may read a computer program that is stored in the storage apparatus 22. For example, the arithmetic apparatus 21 may read a computer program that is stored in a non-transitory computer-readable recording medium by using a non-illustrated recording medium reading apparatus of the key point extraction apparatus 2. The arithmetic apparatus 21 may obtain (namely, download or read) a computer program from a non-illustrated apparatus that is placed outside the key point extraction apparatus 2 through the communication apparatus 23 (alternatively, other communication apparatus) The arithmetic apparatus 21 executes the read computer program. As a result, a logical functional block for performing an operation (for example, the above described key point extraction operation) that should be performed by the key point extraction apparatus 2 is implemented in the arithmetic apparatus 21. Namely, the arithmetic apparatus 21 is configured to serve as a controller for implementing the logical functional block for performing the operation (in other words, a processing) that should be performed by the key point extraction apparatus 2.
The storage apparatus 22 is configured to store a desired data. For example, the storage apparatus 22 may temporarily store the computer program that is executed by the arithmetic apparatus 21. The storage apparatus 22 may temporarily store a data that is temporarily used by the arithmetic apparatus 21 when the arithmetic apparatus 21 executes the computer program. The storage apparatus 22 may store a data that is stored for a long term by the key point extraction apparatus 2. Note that the storage apparatus 22 may include at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk apparatus, a magneto-optical disc, a SSD (Solid State Drive) and a disk array apparatus. Namely, the storage apparatus 22 may include a non-transitory recording medium.
The communication apparatus 23 is configured to communicate with the camera 1 through the communication network 3. In the first example embodiment, the communication apparatus 23 receives the person image IMG1 from the camera 1 through the communication network 3.
The input apparatus 24 is an apparatus that receives an input of an information from an outside of the key point extraction apparatus 2 to the key point extraction apparatus 2. For example, the input apparatus 24 may include an operational apparatus (for example, at least one of a keyboard, a mouse and a touch panel) that is operable by an operator of the key point extraction apparatus 2. For example, the input apparatus 24 may include a reading apparatus that is configured to read an information recorded as a data in a recording medium that is attachable to the key point extraction apparatus 2.
The output apparatus 25 is an apparatus that outputs an information to an outside of the key point extraction apparatus 2. For example, the output apparatus 25 may output the information as an image. Namely, the output apparatus 25 may include a display apparatus (what we call a display) that is configured to display the image representing the information to be outputted. For example, the output apparatus 25 may output the information as a sound. Namely, the output apparatus 25 may include an audio apparatus (what we call a speaker) that is configured to output the sound. For example, the output apparatus 25 may output the information on a paper. Namely, the output apparatus 25 may include a print apparatus (what we call a printer) that is configured to print a desired information on the paper.
(1-3) Flow of Key Point Extraction Operation performed by Key Point Extraction Apparatus 2
Next, with reference to
As illustrated in
Then, the image reduction unit 211 reduces the person image IMG1 obtained at the step S11 (a step S12). As a result, the image reduction unit 211 generates the reduction image IMG2 that is the reduced person image IMG1 (the step S12).
An operation for reducing the person image IMG1 may mean an operation for compressing the person image IMG1. The operation for reducing the person image IMG1 may mean an operation for reducing a data size of the person image IMG1. The operation for reducing a data size of the person image IMG1 may mean an operation fir reducing the number of pixels of the person image IMG1. For example, the operation for reducing the data size of the person image IMG1 to be ⅕ times may mean the operation fir reducing the number of pixels of the person image IMG1 to be ⅕ times.
The image reduction unit 211 may reduce the person image IMG1 at a desired reduction rate. The desired reduction rate may be a fixed value that is determined in advance. The desired reduction rate may be variable. For example, the key point extraction apparatus 2 may change the reduction rate. For example, the operator of the key point extraction apparatus 2 may change the reduction rate.
The image reduction unit 211 may reduce the person image IMG1 by using any image reduction method for reducing the image. For example, the mage reduction unit 211 may reduce the person image IMG1 by using a bicubic method. For example, the mage reduction unit 211 may reduce the person image IMG1 by using a bilinear method. For example, the mage reduction unit 211 may reduce the person image IMG1 by using a nearest neighbor method.
Then, the key point extraction unit 212 extracts the key point KP of the iris that is included in the reduction image IMG2 from the reduction image IMG2 generated at the step S12 (a step S13).
The key point KP may include a key point KP that corresponds to a point from which a characteristic part of the target object is determinable. Thus, the key point extraction unit 212 may extract, as the key point KP of the iris, a key point KP that corresponds to a point from which a characteristic part of the iris is determinable. For example, as illustrated in
The key point KP may include a key point KP that corresponds to a line from which the characteristic part of the target object is determinable in addition to or instead of the key point KP that corresponds to the point from which the characteristic part of the target object is determinable. Thus, the key point extraction unit 212 may extract, as the key point KP of the iris, a key point KP that corresponds to a line from which the characteristic part of the iris is determinable. For example, the key point extraction unit 212 may extract, as the key point KP of the iris, a key point KP that corresponds to a line from which the inner outline of the iris is determinable (for example, a line that extends along the inner outline of the iris). For example, the key point extraction unit 212 may extract, as the key point KP of the iris, a key point KP that corresponds to a line from which the outer outline of the iris is determinable (for example, a line that extends along the outer outline of the iris). For example, the key point extraction unit 212 may extract, as the key point KP of the iris, a key point KP that corresponds to a line from which the edge of the eyelid is determinable (for example, a line that extends along the edge of the eyelid). For example, the key point extraction unit 212 may extract, as the key point KP of the iris, a key point KP that corresponds to a line that connects the outer corner of the eye and the inner corner of the eye.
At least one of the point and the line that is extracted as the key point KP may be usable to quantitatively determine the target object. Thus, the key point extraction unit 212 may extract, as the key point KP of the iris, a key point KP that corresponds to at least one of the point and the line that is usable to quantitatively determine the iris. For example, the key point extraction unit 212 may extract, as the key point KP of the iris, a key point KP that corresponds to a line that is usable to determine a size of the iris. As one example, when a shape of the iris is a circular shape, the key point extraction unit 212 may extract, as the key point KP of the iris, a key point KP that corresponds to a line that is usable to determine a radius (for example, a diameter) of the iris that is one example of the size of the iris (for example, a line that passes through the center of the iris and both edges of which exist on the outer outline of the iris). As another example, when the shape of the iris is an oval shape, the key point extraction unit 212 may extract, as the key point KP of the iris, a key point KP that corresponds to a line that is usable to determine at least one of a major diameter and a minor diameter of the iris that are examples of the size of the iris. The line that is usable to determine the major diameter of the iris may include a line that indicates a major axis of the iris, for example. The line that is usable to determine the minor diameter of the iris may include a line that indicates a minor axis of the iris, for example. Incidentally, when at least one of the line that indicates the major axis of the iris and the line that indicates the minor axis of the iris is determined, a rotational angle of the iris (namely, a rotational angle of an ellipse formed by the iris) is also determinable. Thus, when the shape of the iris is the oval shape, the key point extraction unit 212 may extract, as the key point KP of the iris, a key point KP that corresponds a line that is usable to determine the rotational angle of the iris (namely, the rotational angle of the ellipse formed by the iris). For example, the key point extraction unit 212 may extract, as the key point KP of the iris, a key point KP that corresponds to a line that is usable to determine a size (for example, a diameter) of the pupil that is located on the center of the iris.
The key point extraction unit 212 may extract the key point KP by using a neural network. Specifically, the key point extraction unit 212 may extract the key point KP by using a neural network that is configured to output the key point KP of the iris included in the reduction image IMG2 (alternatively, any image including the person image IMG1) when the reduction image IMG2 (alternatively, any image including the person image IMG1) is inputted thereto. It is preferable that a parameter of the neural network be learned by using a learning data that includes a learning image for learning the parameter of the neural network and a ground truth label of the key point KP of the iris included in the learning image (namely, a ground truth key point that is the key point KP that should be extracted from the learning image). Note that the neural network is one example of a learnable learning model. Thus, the key point extraction unit 212 may extract the key point KP by using any learning model that is different from the neural network.
An operation for extracting the key point KP may mean an operation for determining a position of the key point KP in the reduction image IMG2 (for example, a position of the key point KP in the reduction image IMG2 along each of a height direction and a width direction). In this case, the key point extraction unit 212 may determine the position of the key point KP in the reduction image IMG2. When the key point KP is extracted by using the neural network as described above, the neural network may output the position of the key point KP in the reduction image IMG2.
Again in
The target area set unit 213 may determine a position in the person image IMG1 that corresponds to the position of the key point KP in the reduction image IMG2 and set the target area at a position in the person image IMG1 that is determined based on the determined position. When a plurality of key points KP are extracted in the reduction image IMG2, the target area set unit 213 may determine a position in the person image IMG1 that corresponds to at least one of the plurality of the key points KP and set the target area at a position in the person image IMG1 that is determined based on the determined position.
In an example illustrated in
Again in
Here, the target area TA is set sometimes so that a part of the target area TA does not overlap with the person image IMG1. For example, when the target area TA that spreads from the position P3 of the center of the iris and the size of which is N times as large as the diameter of the iris is set as described above, there is a possibility that a part of the target area TA does not overlap with the person image IMG1 depending on the position of the iris in the person image IMG1. For example, as illustrated in
Again in
Note that an operation for extracting the key point KP at the step S16 may be same as the operation for extracting the key point KP at the step S13. Namely, the key point extraction unit 214 may extract the key point KP at the step S16, as with the key point extraction unit 212 that extracts the key point KP at the step S13. Thus, a detailed description of the operation for extracting the key point KP at the step S16 is omitted.
The key point extraction unit 214 may determine a position of the key point KP in the target image IMG1_TA (namely, a position of the key point KP in the person image IMG1), as with the key point extraction unit 212 that determines the position of the key point KP in the reduction image IMG2. In this case, the key point extraction unit 214 may determine the position itself (namely, an absolute position) of the key point KP in the person image IMG1. Alternatively, the key point extraction unit 214 may determine the relative position of the key point KP in the person image IMG1 relative to the position of the key point KP in the reduction image IMG2. In this case, the key point extraction unit 214 may be regarded to determine a correction amount for correcting the position of the key point KP in the reduction image IMG2 to the position of the key point KP in the person image IMG1.
Then, the key point KP extracted at the step S16 may be used for a desired information processing (for example, an information processing relating to the above described iris authentication).
(1-4) Technical Effect of Key Point Extraction Apparatus 2
As described above, the key point extraction apparatus 2 in the first example embodiment extracts the key point KP from the target image IMG1_TA of the person image IMG1 that is included in the target area TA. Thus, a time required for the key point extraction apparatus 2 to extract the key point KP from the target image IMG1_TA is shorter than a time required for extracting the key point KP from whole of the person image IMG1. Furthermore, although the key point extraction apparatus 2 extracts the key point KP from whole of the reduction image IMG2 in order to set the target area TA, a time required for extracting the key point KP from whole of the reduction image IMG2 is much shorter than the time required for extracting the key point KP from whole of the person image IMG1. Thus, the key point extraction apparatus 2 is capable of extracting the key point KP relatively rapidly.
Furthermore, since the person image IMG1 is not a reduced image, the target image IMG1_TA is not a reduced image either. Thus, an accuracy of the key point KP extracted from the target image IMG1_TA is higher than that of the key point KP extracted from the reduction image IMG2. Specifically, a difference (for example, a difference of the position) between the key point KP extracted from the target image IMG1_TA and an actual key point KP is smaller than a difference between the key point KP extracted from the reduction image IMG2 and the actual key point KP. Thus, the key point extraction apparatus 2 is capable of extracting the key point KP accurately.
As described above, the key point extraction apparatus 2 is capable of extracting the key point KP relatively rapidly and extracting the key point KP accurately. Namely, the key point extraction apparatus 2 is capable of achieving both of a reduction of the time required for extracting the key point KP and an improvement of the accuracy of the extracted key point KP.
Incidentally, a key point extraction apparatus in a first comparison example that merely extracts the key point KP from the reduction image IMG2 without setting the target area TA has a technical problem that the accuracy of the extracted key point KP is low, although it is capable of extracting the key point KP relatively rapidly. On the other hand, a key point extraction apparatus in a second comparison example that merely extracts the key point KP from whole of the person image IMG1 without setting the target area TA has a technical problem that it is not capable of extracting the key point KP relatively rapidly, although the accuracy of the extracted key point KP is high. Namely, both of the key point extraction apparatus in the first comparison example and the key point extraction apparatus in the second comparison example has a technical problem that they are not capable of achieving both of the reduction of the time required for extracting the key point KP and the improvement of the accuracy of the extracted key point KP. However, the key point extraction apparatus 2 in the first example embodiment is capable of achieving both of the reduction of the time required for extracting the key point KP and the improvement of the accuracy of the extracted key point KP. Therefore, the key point extraction apparatus 2 in the first example embodiment is capable of solving the technical problem of the key point extraction apparatus in the first comparison example and the key point extraction apparatus in the second comparison example.
(1-5) Modified Example of Key Point Extraction Apparatus 2
In the above described description, the key point extraction apparatus 2 includes the key point extraction unit 212 that is configured to extract the key point KP from the reduction image IMG2 and the key point extraction unit 214 that is configured to extract the key point KP from the target image IMG1_TA separately. However, the key point extraction apparatus 2 may include a single key point extraction unit that is configured to extract the key point KP from each of the reduction image IMG2 and the target image IMG1_TA, in addition to or instead of the key point extraction units 212 and 214.
In the above described description, the target area set unit 213 sets the target area TA having a fixed size that is determined in advance. However, the target area set unit 213 may change the size of the target area TA if needed. Namely, the size of the target area TA may be variable. Incidentally, when the shape of the target area TA is the rectangular shape, the size of the target area TA may include at least one of a size of the target area TA in the height direction and a size of the target area TA in the width direction. When the shape of the target area TA is the circular shape, the size of the target area TA may include a radius (for example, a diameter) of the target area TA. When the shape of the target area TA is the oval shape, the size of the target area TA may include at least one of a major diameter and a minor diameter of the target area TA.
Next, a specific example of an operation for changing the size of the target area TA will be described. Note that the target area set unit 213 may not perform all of below described operations when the size of the target area TA is changed. The target area set unit 213 may selectively perform a part of the below described operations when the size of the target area TA is changed.
The target area set unit 213 may change the size of the target area TA based on the key point KP in the reduction image IMG2.
As one example, when the key point KP that is usable to determine the radius (for example, the diameter) of the iris (namely, from which the radius (for example, the diameter) is determinable) is extracted as described above, the target area set unit 213 may change the size of the target area TA so that the size of the target area TA is N times as large as the diameter of the iris. A scaling factor N may be larger than 1, for example. for example, when the scaling factor N is 2 in an example illustrated in
As another example, when the key point KP from which the edge of the eyelid is determinable is extracted as described above, the target area set unit 213 may change the size of the target area TA in the height direction based on the position of the eyelid. Specifically, for example, the target area set unit 213 may change the size of the target area TA so that at least one of two sides of the target area TA that face with each other along the height direction (two sides that face with each other along the vertical direction in
Note that the target area TA may be referred to as a “surrounding area”, because it is typically set to include the iris and an image part around the iris.
In this manner, in the second modified example, the key point extraction apparatus 2 is configured to change the size of the target area TA. Thus, there is a lower possibility that the size of the target area TA set by the key point extraction apparatus 2 is larger than necessary, compared to a case where the size of the target area TA is fixed (namely, is not allowed to be changed). Thus, there is a low possibility that the target image IMG1_TA that is used to extract the key point KP is larger than necessary. Thus, the key point extraction apparatus 2 is capable of extracting the key point KP included in the person image IMG1 relatively rapidly. Furthermore, there is also a lower possibility that the size of the target area TA set by the key point extraction apparatus 2 is smaller than necessary. Thus, there is a low possibility that a situation in which the necessary key point KP cannot be extracted from the target image IMG1_TA is caused due to a part of the iris not being included in the target area TA. Thus, the key point extraction apparatus 2 is capable of extracting the key point KP included in the person image IMG1 properly (typically, without shortage).
Moreover, in the second modified example, the key point extraction apparatus 2 is configured to change the size of the target area TA based on the key point KP extracted from the reduction image IMG2. The key point extraction apparatus 2 is capable of changing the size of the target area TA based on an information relating to the actual iris, because the key point KP extracted from the reduction image IMG2 includes an information relating to the iris. Thus, there is much lower possibility that the size of the target area TA set by the key point extraction apparatus 2 is larger or smaller than necessary, compared to a case where the size of the target area TA is changed without using the key point KP extracted from the reduction image IMG2. Thus, the key point extraction apparatus 2 is capable of extracting the key point KP included in the person image IMG1 relatively rapidly and properly (typically, without shortage).
Moreover, in the second modified example, the key point extraction apparatus 2 is configured to change the size of the target area TA to be a size based on the size of the iris. Thus, there is much lower possibility that the size of the target area TA set by the key point extraction apparatus 2 is larger or smaller than necessary, compared to a case where the size of the target area TA is changed without considering the size of the iris. Thus, the key point extraction apparatus 2 is capable of extracting the key point KP included in the person image IMG1 relatively rapidly and properly (typically, without shortage).
(2) Key Point Extraction System SYS in Second Example Embodiment
Next, the key point extraction system SYS in a second example embodiment will be described. Note that the key point extraction system SYS in the second example embodiment is referred to as a “key point extraction system SYSb” in the below described description.
The key point extraction system SYSb in the second example embodiment is different from the above described key point extraction system SYSa in the first example embodiment in that it includes a key point extraction apparatus 2b instead of the key point extraction apparatus 2. Another feature of the key point extraction system SYSb may be same as another feature of the key point extraction system SYSa. Thus, in the below describe description, the key point extraction apparatus 2b in the second example embodiment will be mainly described. Note that a detailed description of the component that is already described is omitted by assigning the same reference number thereto. Moreover, a detailed description of a process that is already described is omitted by assigning the same step number thereto.
(2-1) Configuration of Key Point Extraction Apparatus 2b
Firstly, with reference to
As illustrated in
As illustrated in
(2-2) Flow of Key Point Extraction Operation performed by Key Point Extraction Apparatus 2b
Next, with reference to
As illustrated in
Then, the key point extracting unit 215 extracts the key point KP of the iris that is included in the person image IMG1 from the person image IMG1 obtained at the step S11 (a step S21). Note that an operation for extracting the key point KP at the step S21 may be same as the operation for extracting the key point KP at at least one of the step S13 and the step S16 in
Then, the image conversion unit 216 performs the image conversion processing that convers the person image IMG1 based on the key point KP extracted at the step S21 (a step S22). As a result, the image conversion unit 216 generates the conversion image IMG3 that is the person image IMG1 on which the image conversion processing has been performed (the step S22).
The image conversion processing at the step S22 may include or may not include a processing for changing a characteristic of the iris included in the person image IMG1. The image conversion processing at the step S22 may include or may not include a processing for normalizing the characteristic of the iris included in the person image IMG1. The characteristic of the iris may include or may not include the size (for example, the diameter) of the iris. The characteristic of the iris may include the position of the iris. The characteristic of the iris may include or may not include a direction (in other words, an angle, an angle between horizontal line and a line that connects the inner conner of the eye and the outer corner of the eye, for example) of the iris.
Alternatively, the image conversion processing at the step S22 may include or may not include a processing for changing a characteristic of the person image IMG1 in addition to or instead of the processing for changing the characteristic of the iris included in the person image IMG1. The image conversion processing at the step S22 may include or may not include a processing for normalizing the characteristic of the person image IMG1.
When a plurality of person images IMG1 are converted to a plurality of conversion images IMG3, respectively, the image conversion processing at the step S22 may include or may not include a processing for reducing a variability between the characteristics of a plurality of irises that are included in the plurality of conversion images IMG3, respectively. Namely, the image conversion processing at the step S22 may include or may not include a processing for allowing the variability between the characteristics of the plurality of irises that are included in the plurality of conversion images IMG3, respectively, is smaller than a variability between the characteristics of the plurality of irises that are included in the plurality of person images IMG1, respectively.
Under the assumption that the plurality of person images IMG1 are converted to the plurality of conversion images IMG3, respectively, the image conversion processing at the step S22 may include or may not include a processing for reducing a variability between the characteristics of the plurality of conversion images IMG3. Namely, the image conversion processing at the step S22 may include or may not include a processing for allowing the variability between the characteristics of the plurality of conversion images IMG3 is smaller than a variability between the characteristics of the plurality of person images IMG1.
Then, the key point extracting unit 217 extracts the key point KP of the iris that is included in the conversion image IMG3 from the conversion image IMG3 generated at the step S22 (a step S23). Note that an operation for extracting the key point KP at the step S23 may be same as the operation for extracting the key point KP at the step S21. Namely, the key point extraction unit 217 may extract the key point KP at the step S23, as with the key point extraction unit 215 that extracts the key point KP at the step S21. Thus, a detailed description of the operation for extracting the key point KP at the step S23 is omitted.
(2-3) Technical Effect of Key Point Extraction Apparatus 2b
As described above, the key point extraction apparatus 2b in the second example embodiment generates the conversion image IMG3 by performing the image conversion processing that converts the person image IMG1 based on the key point KP extracted from the person image IMG1, and extracts the key point KP from the conversion image IMG3. Thus, the key point extraction apparatus 2b is capable of extracting the key point KP from the conversion image IMG3 in which the iris is included in a desired aspect. Thus, the key point extraction apparatus 2b is capable of extracting the key point KP more accurately, compared to a key point extraction apparatus in a third comparison example that extracts the key point KP from the person image IMG1 in which the iris is not necessarily included in the desired aspect.
(2-4) Modified Example of Key Point Extraction Apparatus 2b
The image conversion processing at the step S22 in
In this case, when the plurality of person images IMG1 are converted to the plurality of conversion images IMG3, respectively, the position of the center of the iris included in the conversion image IMG3 is approximately the same among the plurality of conversion images IMG3. Here, when the key point extraction apparatus 2b extracts the key point KP of the iris from each of the plurality of conversion images IMG3 among which the position of the center of the iris varies largely, there is a possibility that the key point extraction apparatus 2b is capable of properly extracting the key point KP from the conversion image IMG3 in which the center of the iris is located at a first position and is not capable of properly extracting the key point KP from the conversion image IMG3 in which the center of the iris is located at a second position that is different from the first position, due to a variability of the position of the center of the iris. However, in the first modified example, the key point extraction apparatus 2b is capable of properly extracting the key point KP from each of the plurality of conversion images IMG3, because the position of the center of the iris is approximately the same among the plurality of conversion images IMG3.
In this case, when the plurality of person images IMG1 are converted to the plurality of conversion images IMG3, respectively, the direction of the iris (here, a direction along which the line that connects the outer corner of the eye and the inner corner of the eye extends) included in the conversion image IMG3 is approximately the same among the plurality of conversion images IMG3. Here, when the key point extraction apparatus 2b extracts the key point KP of the iris from each of the plurality of conversion images IMG3 among which the direction of the iris varies largely, there is a possibility that the key point extraction apparatus 2b is capable of properly extracting the key point KP from the conversion image IMG3 in which the direction of the iris is a first direction (for example, in which the line that connects the outer corner of the eye and the inner corner of the eye is close to the horizontal direction) and is not capable of properly extracting the key point KP from the conversion image IMG3 in which the direction of the iris is a second direction that is different from the first direction (for example, in which the line that connects the outer corner of the eye and the inner corner of the eye is largely different from the horizontal direction), due to a variability of the direction of the iris. However, in the first modified example, the key point extraction apparatus 2b is capable of properly extracting the key point KP from each of the plurality of conversion images IMG3, because the direction of the iris is approximately the same among the plurality of conversion images IMG3.
In this case, when the plurality of person images IMG1 are converted to the plurality of conversion images IMG3, respectively, the size of the iris included in the conversion image IMG3 is approximately the same among the plurality of conversion images IMG3. Here, when the key point extraction apparatus 2b extracts the key point KP of the iris from each of the plurality of conversion images IMG3 among which the size of the iris varies largely, there is a possibility that the key point extraction apparatus 2b is capable of properly extracting the key point KP from the conversion image IMG3 in which the size of the iris is a first size and is not capable of properly extracting the key point KP from the conversion image IMG3 in which the size of the iris is a second size that is different from the first size, due to a variability of the size of the iris. However, in the first modified example, the key point extraction apparatus 2b is capable of properly extracting the key point KP from each of the plurality of conversion images IMG3, because the size of the iris is approximately the same among the plurality of conversion images IMG3.
As described above, the image conversion processing at the step S22 in
In this case, when the plurality of person images IMG1 are converted to the plurality of conversion images IMG3, respectively, a range of the brightness (namely, a range from the maximum brightness to the minimum brightness) of the conversion image IMG3 is approximately the same. For example, a range of the brightness (namely, a range from the maximum brightness to the minimum brightness) of the image part of the conversion image IMG3 that corresponds to the iris is approximately the same. Here, when the key point extraction apparatus 2b extracts the key point KP of the iris from each of the plurality of conversion images IMG3 among which the range of the brightness varies largely, there is a possibility that the key point extraction apparatus 2b is capable of properly extracting the key point KP from the conversion image IMG3 the brightness of which is within a first range and is not capable of properly extracting the key point KP from the conversion image IMG3 the brightness of which is within a second range that is different from the first range, due to a variability of the range of the brightness. However, in the second modified example, the key point extraction apparatus 2b is capable of properly extracting the key point KP from each of the plurality of conversion images IMG3, because the range of the brightness is approximately the same among the plurality of conversion images IMG3.
The key point extraction apparatus 2b may perform an operation for extracting the key point KP from the conversion image IMG3 after generating the conversion image IMG3 and generating (in this case, re-generating) new conversion image IMG3 by performing an image conversion processing that converts the conversion image IMG3 based on the extracted key point KP. Namely, the key point extraction apparatus 2b may perform the operation from the step S21 to the step S22 in
The key point KP that is extracted by the key point extraction unit 215 at the step S21 in
A type of the key point KP that is extracted by the key point extraction unit 215 at the step S21 in
Alternatively, the type of the key point KP that is extracted by the key point extraction unit 215 at the step S21 in
Alternatively, considering that the conversion image IMG3 is generated by performing the image conversion processing on the person image IMG1, the key point KP that is extracted by the key point extraction unit 217 at the step S23 in
In the above described description, the key point extraction apparatus 2b includes the key point extraction unit 215 that is configured to extract the key point KP from the person image IMG1 and the key point extraction unit 217 that is configured to extract the key point KP from the conversion image IMG3 separately. However, the key point extraction apparatus 2 may include a single key point extraction unit that is configured to extract the key point KP from each of the person image IMG1 and the conversion image IMG3, in addition to or instead of the key point extraction units 215 and 217.
(3) Key Point Extraction System SYS in Third Example Embodiment
Next, the key point extraction system SYS in a third example embodiment will be described. Note that the key point extraction system SYS in the third example embodiment is referred to as a “key point extraction system SYSc” in the below described description.
The key point extraction system SYSc in the third example embodiment is different from the above described key point extraction system SYSa in the first example embodiment in that it includes a key point extraction apparatus 2c instead of the key point extraction apparatus 2. Another feature of the key point extraction system SYSc may be same as another feature of the key point extraction system SYSa. Thus, in the below describe description, the key point extraction apparatus 2c in the third example embodiment will be mainly described.
(3-1) Configuration of Key Point Extraction Apparatus 2c
Firstly, with reference to
As illustrated in
As illustrated in
(3-2) Flow of Key Point Extraction Operation performed by Key Point Extraction Apparatus 2c
Next, with reference to
As illustrated in
Then, the key point extraction apparatus 2c performs the key point KP in the first example embodiment by using the conversion image IMG3 instead of the person image IMG1 (the step S12 to the step S16). Specifically, the image reduction unit 211 reduces the conversion image IMG3 generated at the step S22 (the step S12). Then, the key point extraction unit 212 extracts the key point KP from the reduction image IMG2 generated at the step S12 (the step S13). Then, the target area set unit 213 sets the target area TA that designates a part of the conversion image IMG3 based on the key point extracted at the step S13 (the step S14). Then, the target area set unit 213 extracts, as the target image IMG1_TA from the conversion image IMG3, an image part of the conversion image IMG3 that is included in the target area TA (the step S15). Then, the key point extraction unit 214 extracts the key point KP from the target image IMG1_TA extracted at the step S15 (the step S16).
(3-3) Technical Effect of Key Point Extraction Apparatus 2c
As described above, the key point extraction apparatus 2c in the third example embodiment performs the key point extraction operation in the first example embodiment after performing the key point extraction operation in the second example embodiment. Thus, the key point extraction apparatus 2c in the third example embodiment is capable of achieving an effect that is same as the effect achievable by the key point extraction apparatus 2 in the first example embodiment and an effect that is same as the effect achievable by the key point extraction apparatus 2b in the second example embodiment.
(3-4) Modified Example of Key Point Extraction Apparatus 2c
In the above described description, the key point extraction apparatus 2c includes the key point extraction unit 212 that is configured to extract the key point KP from the reduction image IMG2, the key point extraction unit 214 that is configured to extract the key point KP from the target image IMG1_TA and the key point extraction unit 215 that is configured to extract the key point KP from the person image IMG1 separately. However, the key point extraction apparatus 2 may include a single key point extraction unit that is configured to extract the key point KP from at least two of the person image IMG1, the reduction image IMG2 and the target image IMG1_TA, in addition to or instead of at least two of the key point extraction units 212, 214 and 215.
(4) Key Point Extraction System SYS in Fourth Example Embodiment
Next, the key point extraction system SYS in a fourth example embodiment will be described. Note that the key point extraction system SYS in the fourth example embodiment is referred to as a “key point extraction system SYSd” in the below described description.
The key point extraction system SYSd in the fourth example embodiment is different from the above described key point extraction system SYSb in the second example embodiment in that it includes a key point extraction apparatus 2d instead of the key point extraction apparatus 2b. Another feature of the key point extraction system SYSd may be same as another feature of the key point extraction system SYSb. Thus, in the below describe description, the key point extraction apparatus 2d in the fourth example embodiment will be mainly described.
(4-1) Configuration of Key Point Extraction Apparatus 2d
Firstly, with reference to
As illustrated in
Note that a detail of an operation of each of the feature amount extraction unit 218 and the iris authentication unit 219 will be described later in detail, however, a summary thereof will be described briefly here. The feature amount extraction unit 218 extracts a feature amount of the iris based on the key point KP of the iris. The iris authentication unit 219 authenticates the person included in the person image IMG1 based on the feature amount of the iris extracted by the feature amount extraction unit 218.
In this manner, the key point extraction apparatus 2d in the fourth example embodiment is capable of performing the iris authentication based on the extracted key point KP. Thus, the key point extraction apparatus 2d in the fourth example embodiment may be referred to as an iris authentication apparatus. The key point extraction operation that is performed by the key point extraction apparatus 2d in the fourth example embodiment may be referred to as an iris authentication operation. The key point extraction SYSd in the fourth example embodiment may be referred to as an iris authentication system.
(4-2) Flow of Key Point Extraction Operation (Iris Authentication Operation) performed by Key Point Extraction Apparatus 2d
Next, with reference to
As illustrated in
Then, the feature amount extraction unit 218 extracts the feature amount of the iris included in the conversion image IMG3 based on the key point KP of the iris extracted at the step S23 and the conversion image IMG3 (a step S41). Note that the feature amount extraction unit 218 may extract the feature amount by using any method for extracting the feature amount of the iris. For example, the feature amount extraction unit 218 may determine a loop-shaped iris area in which the iris is included in the conversion image IMG3 based on the key point KP of the iris extracted at the step S23. Specifically, the feature amount extraction unit 218 may determine the iris area based on the key point KP1 from which the inner outline of the iris is determinable and the key point KP2 from which the outer outline of the iris is determinable. Furthermore, the feature amount extraction unit 218 may delete an area part that overlaps with the eyelid from the loop-shaped iris area based on the key point KP4 from which the edge of the eyelid is determinable. Then, the feature amount extraction unit 218 may divide the iris area into a plurality of ring-shaped blocks, divide each block into a plurality of sectors, and extract a feature amount of each sector (for example, a feature amount relating to a pattern of the iris included in the plurality of sectors, and a brightness value for example). Incidentally, as can be understandable from the description of the feature amount extraction unit 218, the present disclosure clearly distinguishes a feature point (namely, the key point KP) from the feature amount of the iris.
Then, the iris authentication unit 219 authenticates the person included in the conversion image IMG3 (namely, the person included in the person image IMG1) based on the feature amount of the iris extracted at the step S42 (a step S41). Note that the iris authentication unit 219 may authenticate the person by using any method for authenticating the person based on the feature amount of the iris. For example, the iris authentication unit 219 may perform a matching between the feature amount that is extracted (hereinafter, it is referred to as an “extracted feature amount”) and the feature amounts of the irises of various persons stored (in other words, registered) in an iris database (hereinafter, it is referred to as a “registered feature amount”). Specifically, the iris authentication unit 219 may calculate a matching score that indicates a similarly between the registered feature amount and the extracted feature amount. Then, the iris authentication unit 219 may authenticate the person included in the conversion image IMG3 based on the matching score. For example, when the matching score that indicates the similarly between a certain registered feature amount and the extracted feature amount is smaller than a predetermined threshold value, the iris authentication unit 219 may determine that the person included in the conversion image IMG3 is not a person corresponding to the registered feature amount. On the other hand, for example, when the matching score that indicates the similarly between a certain registered feature amount and the extracted feature amount is larger than the predetermined threshold value, the iris authentication unit 219 may determine that the person included in the conversion image IMG3 is the person corresponding to the registered feature amount.
(4-3) Technical Effect of Key Point Extraction Apparatus 2d
The key point extraction apparatus 2d in the fourth example embodiment is capable of achieving an effect that is same as the effect achievable by the key point extraction apparatus 2b in the second example embodiment. Furthermore, the key point extraction apparatus 2d is capable of authenticating the person included in the person image IMG1 based on the iris of the person.
(4-4) Modified Example of Key Point Extraction Apparatus 2d
In the above described description, the key point extraction apparatus 2d in the fourth example embodiment is different from the key point extraction apparatus 2b in the second example embodiment in that the feature amount extraction unit 218 and the iris authentication unit 219 are implemented in the arithmetic apparatus 21. Namely, in the above described description, the key point extraction apparatus 2b in the second example embodiment includes the feature amount extraction unit 218 and the iris authentication unit 219. However, at least one of the key point extraction apparatus 2 in the first example embodiment and the key point extraction apparatus 2c in the third example embodiment the feature amount extraction unit 218 and the iris authentication unit 219. In this case, the feature amount extraction unit 218 may extract the feature amount of the iris based on the key point KP extracted by the key point extraction unit 214 (namely, the key point KP extracted from the target image IMG1_TA).
The iris database that is used to authenticate the person includes an iris record in which the feature amount of the iris (the registered feature amount) extracted from the image in which a certain person is included is associated with a person identification information for uniquely identifying the person, for example. Especially, the iris database includes a plurality of iris records that correspond to a plurality of persons, respectively, because the feature amounts of the irises of the plurality of persons are stored in the iris database. In this case, the iris database may include the iris record in which the feature amount of the iris extracted from the image that includes the iris of the certain person in a first aspect is associated with the person identification information and the iris record in which the feature amount of the iris extracted from the image that includes the iris of the same person in a second aspect that is different from the first aspect is associated with the person identification information, for example. For example, as described above, the characteristic of the iris may include the size (for example, the diameter) of the iris. In this case, the iris database may include the iris record in which the feature amount of the iris extracted from the image that includes the iris of the person, which is in a state where the size of the iris is a first size, is associated with the person identification information and the iris record in which the feature amount of the iris extracted from the image that includes the iris of the same person, which is in a state where the size of the iris is a second size that is different from the first size, is associated with the person identification information, for example. Even in this case, the iris authentication unit 219 may calculate the matching score by using the iris database that includes the plurality of iris records that correspond to the same person and that correspond to the different characteristics (for example, the sizes) of the iris. Alternatively, the iris authentication unit 219 may extract, from the iris database, a plurality of iris records that correspond to the characteristic (for example, the size) of the iris that is included in the person image IMG1 obtained from the camera 1 and may calculate the matching score by using the plurality of extracted iris records.
(5) Key Point Extraction System SYS in Fifth Example Embodiment
Next, the key point extraction system SYS in a fifth example embodiment will be described. Note that the key point extraction system SYS in the fifth example embodiment is referred to as a “key point extraction system SYSe” in the below described description.
The key point extraction system SYSe in the fifth example embodiment is different from the above described key point extraction system SYSd in the fourth example embodiment in that it includes a key point extraction apparatus 2e instead of the key point extraction apparatus 2d. Another feature of the key point extraction system SYSe may be same as another feature of the key point extraction system SYSd. Thus, in the below describe description, with reference to
As illustrated in
The key point extraction unit 217e extracts the key point KP of the iris included in the person image IMG1 from the person image IMG1. Especially, the key point extraction unit 217e may be configured to extract one type of key point KP more accurately than the key point extraction unit 217. For example, the key point extraction unit 217e may be configured to extract the key point KP from which the center of the pupil is determinable and the key point KP from which the diameter of the pupil is determinable more accurately than the key point extraction unit 217. In this case, the key point extraction unit 217e may extract an image part of the person image IMG1 corresponding to the iris based on the key point KP that is extracted by the key point extraction unit 215 to perform the image conversion processing (the key point KP from which the center of the iris is determinable and the key point KP from which the diameter of the iris is determinable), and may extract one type of key point KP (for example, the key point KP from which the center of the pupil is determinable and the key point KP from which the diameter of the pupil is determinable), which is extractable accurately, from the extracted image part.
The feature amount extraction unit 218e extracts the feature amount of the iris of the person included in the person image IMG1 based on the key point KP of the iris extracted by the key point extraction unit 217e and the person image IMG1.
In the fifth example embodiment, the iris authentication unit 219 authenticates the person included in the person image IMG1 based on the feature amount of the iris extracted by the feature amount extraction unit 218e and the feature amount of the iris extracted by the feature amount extraction unit 218. Specifically, the iris authentication unit 219 may calculate the matching score by using the feature amount of the iris extracted by the feature amount extraction unit 218e and the iris database and calculate the matching score by using the feature amount of the iris extracted by the feature amount extraction unit 218 and the iris database. Then, the iris authentication unit 219 may authenticate the person included in the conversion image IMG3 based on the matching scores.
The key point extraction apparatus 2e in the fifth example embodiment is capable of achieving an effect that is same as the effect achievable by the key point extraction apparatus 2d in the fourth example embodiment.
Note that the feature amount extraction unit 218e extracts the feature amount of the iris based on the key point KP of the iris extracted by the key point extraction unit 217e in the above described description. However, the feature amount extraction unit 218e may extract the feature amount of the iris based on the key point KP of the iris extracted by the key point extraction unit 217e and the key point KP of the iris extracted by the key point extraction unit 215. For example, the feature amount extraction unit 218e may calculate an average value of the key point KP of the iris extracted by the key point extraction unit 217e and the key point KP of the iris extracted by the key point extraction unit 215, and extract the feature amount of the iris based on the calculated average value of the key points KP.
Similarly, the feature amount extraction unit 218 extracts the feature amount of the iris based on the key point KP of the iris extracted by the key point extraction unit 217 in the above described description. However, the feature amount extraction unit 218 may extract the feature amount of the iris based on the key point KP of the iris extracted by the key point extraction unit 217 and the key point KP of the iris extracted by the key point extraction unit 215. For example, the feature amount extraction unit 218 may calculate an average value of the key point KP of the iris extracted by the key point extraction unit 217 and the key point KP of the iris extracted by the key point extraction unit 215, and extract the feature amount of the iris based on the calculated average value of the key points KP. Incidentally, the key point extraction unit 215 extracts the key point KP from the person image IMG1 and the key point extraction unit 217extracts the key point KP from the conversion image IMG3 (namely, the image that is obtained by performing the image conversion processing on the person image IMG1). Thus, there is a possibility that the position of the key point KP of a certain part (for example, the center of the iris) extracted by the key point extraction unit 217 is different from the position of the key point KP of the same part extracted by the key point extraction unit 215 by an amount based on the image conversion processing. Thus, the feature amount extraction unit 218 may perform the coordinate transformation, which is based on the image conversion processing, on the key point KP extracted by at least one of the key point extraction units 215 and 217. Then, the feature amount extraction unit 218 may extract the feature amount of the iris based on the key point KP on which the coordinate transformation is performed.
(6) Supplementary Note
With respect to the example embodiments described above, the following Supplementary Notes will be further disclosed.
[Supplementary Note 1]
An information processing apparatus comprising:
a reducing unit that generates a second image by reducing a first image in which a target object is included;
a first extracting unit that extracts, as a first key point, a key point of the target object from the second image;
a setting unit that sets a target area that designates a part of the first image based on the first key point; and
a second extracting unit that extracts, as a second key point, a key point of the target object from a target image of the first image that is included in the target area.
[Supplementary Note 2]
The information processing apparatus according to Supplementary Note 1, wherein
the setting unit is configured to change a size of the target area.
[Supplementary Note 3]
The information processing apparatus according to Supplementary Note 2, wherein
the setting unit is configured to change the size of the target area based on the first key point.
[Supplementary Note 4]
The information processing apparatus according to Supplementary Note 2 or 3, wherein
the target object includes an iris of a living body,
the setting unit is configured to change the size of the target area to be a size based on a size of the iris indicated by the first key point.
[Supplementary Note 5]
The information processing apparatus according to any one of Supplementary Notes 1 to 4 further comprising:
a third extracting unit that extracts, as a third key point, a key point of the target object from a third image in which the target object is included; and
an image converting unit that generates the first image by performing an image conversion processing that converts the third image based on the third key point.
[Supplementary Note 6]
The information processing apparatus according to Supplementary Note 5, wherein
the target object includes an iris of a living body,
the image conversion processing includes a translation processing that translates the third image so that a center of the iris is closer to a center of the third image.
[Supplementary Note 7]
The information processing apparatus according to Supplementary Note 5 or 6, wherein
the target object includes an outer corner of an eye and an inner corner of the eye of a living body,
the image conversion processing includes a rotation processing that rotates the third image so that a line that connects the outer corner of the eye and the inner corner of the eye is closer to be horizontal in the third image.
[Supplementary Note 8]
The information processing apparatus according to any one of Supplementary Notes 5 to 7, wherein
the target object includes an iris of a living body,
the image conversion processing includes a scaling processing that scales the third image so that a size of the iris in the third image is closer to a predetermined size.
[Supplementary Note 9]
The information processing apparatus according to any one of Supplementary Notes 5 to 8, wherein
the image conversion processing includes a brightness conversion processing that normalizes a brightness of the third image.
[Supplementary Note 10]
An information processing method comprising:
generating a second image by reducing a first image in which a target object is included;
extracting, as a first key point, a key point of the target object from the second image;
setting a target area that designates a part of the first image based on the first key point; and
extracting, as a second key point, a key point of the target object from a target image of the first image that is included in the target area.
[Supplementary Note 11]
A recording medium on which a computer program that allows a computer to execute an information processing method is recorded,
the information processing method comprising:
generating a second image by reducing a first image in which a target object is included;
extracting, as a first key point, a key point of the target object from the second image;
setting a target area that designates a part of the first image based on the first key point; and
extracting, as a second key point, a key point of the target object from a target image of the first image that is included in the target area.
At least a part of the feature of each embodiment described above may be combined with at least other part of the feature of each embodiment described above. A part of the feature of each embodiment described above may not be used. Moreover, the disclosures of all documents (for example, publications) that are cited in the present disclosure described above are incorporated in the present disclosure by reference if it is legally permitted.
The present disclosure is allowed to be changed, if desired, without departing from the essence or spirit of the invention which can be read from the claims and the entire specification, and an information processing apparatus, an information processing method, a computer program and a recording medium, which involve such changes, are also intended to be within the technical scope of the present disclosure.
This application is a Continuation of U.S. application Ser. No. 17/637,183 filed on Feb. 22, 2022, which is a National Stage Entry of PCT/JP2021/010305 filed on Mar. 15, 2021, the contents of all of which are incorporated herein by reference, in their entirety.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 17637183 | Feb 2022 | US |
| Child | 18495279 | US |
