Patent Application
20240020873
This disclosure relates to an image generation system, an image generation method, and a recording medium that generate an image.
A known system of this time generates a face image on the basis of a feature point of a face. For example, Patent Literature 1 discloses a technique/technology of normalizing a face such that both eyes are at a predetermined reference position, and cutting out an image including the normalized face.
As another related technique/technology, for example, Patent Literature 2 discloses a technique/technology of learning a neural network by using a feature extracted from learning data as an adversarial feature. Patent Literature 3 discloses a technique/technology of specifying a person of a face image on the basis of a registered face template and probability distribution sample data. Patent Literature 4 discloses a technique/technology of generating a perturbed face image by rotating a face image.
Patent Literature 1: JP2007-226424A
Patent Literature 2: PCT International Publication No. WO2018/167900
Patent Literature 3: JP2005-208850A
Patent Literature 4: JP2017-182459A
This disclosure improves the related techniques/technologies described above.
An image generation system according to an example aspect of this disclosure includes: a detection unit that detects a position information about a position of a face or a position of a feature point of the face from an image; an acquisition unit that obtains a perturbed position information that takes into account an error that occurs when the position information is detected, for the position information; and a generation unit that generates a new image including the face on the basis of the perturbed position information.
An image generation method according to an example aspect of this disclosure includes: detecting a position information about a position of a face or a position of a feature point of the face from an image; obtaining a perturbed position information that takes into account an error that occurs when the position information is detected, for the position information; and generating a new image including the face on the basis of the perturbed position information.
A recording medium according to an example aspect of this disclosure is a recording medium on which a computer program that allows a computer to execute an image generation method is recorded, the image generation method including: detecting a position information about a position of a face or a position of a feature point of the face from an image; obtaining a perturbed position information that takes into account an error that occurs when the position information is detected, for the position information; and generating a new image including the face on the basis of the perturbed position information.
Hereinafter, with reference to the drawings, an image generation system, an image generation method, and a recording medium according to example embodiments will be described.
An image generation system according to a first example embodiment will be described with reference to
First, with reference to
As illustrated in
The processor 11 reads a computer program. For example, the processor 11 is configured to read a computer program stored by at least one of the RAM 12, the ROM 13 and the storage apparatus 14. Alternatively, the processor 11 may read a computer program stored in a computer readable recording medium by using a not-illustrated recording medium reading apparatus. The processor 11 may obtain (i.e., may read) a computer program from a not-illustrated apparatus disposed outside the image generation system 10, through a network interface. The processor 11 controls the RAM 12, the storage apparatus 14, the input apparatus 15, and the output apparatus 16 by executing the read computer program. Especially in the example embodiment, when the processor 11 executes the read computer program, a functional block for generating a new image from an inputted image is realized or implemented in the processor 11. As the processor 11, one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a FPGA (field-programmable gate array), a DSP (Demand-Side Platform), and an ASIC (Application Specific Integrated Circuit) may be used, or a plurality of them may be used in parallel.
The RAM 12 temporarily stores the computer programs to be executed by the processor 11. The RAM 12 temporarily stores the data that is temporarily used by the processor 11 when the processor 11 executes the computer program. The RAM 12 may be, for example, a D-RAM (Dynamic RAM).
The ROM 13 stores the computer program to be executed by the processor 11. The ROM 13 may otherwise store fixed data. The ROM 13 may be, for example, a P-ROM (Programmable ROM).
The storage apparatus 14 stores the data that is stored for a long term by the image generation system 10. The storage apparatus 14 may operate as a temporary storage apparatus of the processor 11. The storage apparatus 14 may include, for example, at least one of a hard disk apparatus, a magneto-optical disk apparatus, a SSD (Solid State Drive), and a disk array apparatus.
The input apparatus 15 is an apparatus that receives an input instruction from a user of the image generation system 10. The input apparatus 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel.
The output apparatus 16 is an apparatus that outputs information about the image generation system 10 to the outside. For example, the output apparatus 16 may be a display apparatus (e.g., a display) that is configured to display the information about the image generation system 10.
Next, with reference to
As illustrated in
The detection unit 110 is configured to detect a position information about a position of a face, or a position of a feature point of the face, from an inputted image. The feature point of the face is a point representing a feature of the face, and is set to correspond to a particular part, such as eyes, ears, a nose, and a mouth, for example. Which part is set as the feature point may be set as appropriate, for example, by a system manager or the like. The detection unit 110 may detect both the position information about the position of the face and the position information about the position of the feature point of the face. In this case, the detection unit 110 may separately include a part for detecting the position information about the position of the face and a part for detecting the position information about the position of the feature point of the face (e.g., there may be configured two independent detection units 110). Furthermore, the detection unit 110 may be configured to firstly detect the position information about the position of the face and then detect the position information about the position of the feature point of the face on the basis of the detected position information about the position of the face. The position information may be, for example, a coordinate information or a vector information. The detection unit 110 may be configured, for example, as a neural network. A detailed description of a specific method of detecting the position information by the detection unit 110 will be omitted here, because the existing techniques/technologies can be adopted to the method as appropriate. The position information detected by the detection unit 110 is configured to be outputted to the acquisition unit 120.
The acquisition unit 120 is configured to obtain a perturbed position information that takes an error into account for the position information detected by the detection unit 110. The error here is an error that occurs when the position information is detected by the detection unit 110 (i.e., a detection error). In comparison with the position information detected by the detection unit 110, the perturbed position information has a perturbation corresponding to the error. When the error is considered for a plurality of feature points, the error may be added to all the feature points, or the error may be added only to a part of the feature points. When the error is added only to a part of the feature points, the feature point that takes the error into account may be automatically determined from a past history or the like, or may be specified by the user. A specific technique/technology that takes the error into account the position information will be described in detail in another example embodiment described later. The technique/technology that takes the error into account may be common to all the feature points, or may be different for each feature point. The perturbed position information obtained by the acquisition unit 120 is configured to be outputted to the generation unit 130.
The generation unit 130 is configured to generate a new image including the face on the basis of the perturbed position information obtained by the acquisition unit 120. The new image generated by the generation unit 130 is an image having a perturbation corresponding to the error, because it is generated on the basis the perturbed position information. Therefore, there is a difference corresponding to the error between the image inputted to the detection unit 110 and the new image generated by the generation unit 130. A detailed description of a specific method of generating the image from the position information will be omitted here, because the existing techniques/technologies can be adopted to the method as appropriate. The generation unit 130 has a function of outputting the generated new image. The generation unit 130 may be configured to output and display the generated new image on a display unit having a display, for example.
Next, with reference to
As illustrated in
Subsequently, the acquisition unit 120 obtains the perturbed position information that takes the error into account for the position information (step S13). Then, the generation unit 130 generates a new image including the face on the basis of the perturbed position information (step S14). The generation unit 130 outputs the generated new image (step S15).
Next, a technical effect obtained by the image generation system 10 according to the first example embodiment will be described.
As described in
The image generation system 10 according to a second example embodiment will be described with reference to
First, with reference to
As illustrated in
Subsequently, the acquisition unit 120 obtains the perturbed position information that takes the error into account for the position information (the step S13). Then, the generation unit 130 performs a process of normalizing the face on the basis of the perturbed position information (step S21). The generation unit 130 outputs a face normalized image that is an image including the normalized face, as a new image (step S22).
Next, the process of normalizing the face performed by the image generation system 10 according to the second example embodiment will be described, more specifically.
The face normalization is realized by adjusting at least one of the position, size and angle of the face on the basis of the position information. The face normalization is realized by properly adjusting the position, size, and angle of the face such that the feature point of the face, such as eyes, a nose, and a mouth, is at a predetermined position. The face normalization may use an imaging processing technique/technology, such as image enlargement and reduction, rotation, and 2D/3D conversion. Existing techniques/technologies that are not mentioned here may be adopted to the face normalization, as appropriate.
Next, a technical effect obtained by the image generation system 10 according to the second example embodiment will be described.
As described in
In the following example embodiments, as in the second example embodiment, a description will be made by exemplifying the configuration in which the generation unit 130 generates the face normalized image.
The image generation system 10 according to a third example embodiment will be described with reference to
First, with reference to
As illustrated in
The perturbation quantity generation unit 140 is configured to generate a perturbation quantity in accordance with an error parameter (i.e., a value indicating the magnitude of the error). The perturbation quantity generation unit 140 may generate the perturbation quantity, for example, by multiplying the error parameter by a predetermined random number. The random number in this case may be generated by using a random number generator or the like that uses a normal distribution (e.g., a normal distribution with a mean of 0 and a variance of 1). The error parameter may be a value specified by the user. The perturbation quantity generated by the perturbation quantity generation unit 140 is configured to be outputted to a perturbation addition unit 121 of the acquisition unit 120.
The perturbation addition unit 121 is configured to perform a process of adding the perturbation quantity to a parameter (i.e., the position information) indicating the position of the face or the position of the feature point of the face detected by the detection unit 110. The perturbation addition unit 121 may perform a process of adding the perturbation quantity as it is, or may perform a process of adding it after multiplying the perturbation quantity by a predetermined factor. The acquisition unit 120 obtains a result of the addition process by the perturbation addition unit 121, as the perturbed position information.
Next, a flow of operation of the image generation system 10 according to the third example embodiment will be described with reference to
As illustrated in
Subsequently, the perturbation quantity generation unit 140 generates the random number for generating the perturbation quantity (step S31). Then, the perturbation quantity generation unit 140 generates the perturbation quantity from the generated random number and the error parameter (step S32).
Subsequently, the perturbation addition unit 121 adds the perturbation quantity to the position information detected by the detection unit 110, and obtains the perturbed position information (step S33). Then, the generation unit 130 performs the process of normalizing the face on the basis of the perturbed position information (the step S21). The generation unit 130 outputs the face normalized image that is an image including the normalized face, as a new image (the step S22).
Next, a technical effect obtained by the image generation system 10 according to the third example embodiment will be described.
As described in
The image generation system 10 according to a fourth example embodiment will be described with reference to
First, with reference to
As illustrated in
The error evaluation unit 150 is configured to evaluate an error by using test data for error evaluation (i.e., to estimate the error that occurs when the position information is detected). The test data are data having correct answer data about the position information. The error evaluation unit 150 may evaluate the error by a statistical method on the basis of a deviation amount between the position information detected from the test data and the correct answer data. Existing techniques/technologies can be properly adopted to a specific method of evaluating the error, but an example of the specific method may include MAE (Mean Absolute Error). An evaluation result of the error evaluation unit 150 is configured to be outputted to the perturbation quantity generation unit 140 as the error parameter. That is, in this example embodiment, the perturbation quantity is generated on the basis of the error parameter evaluated by the error evaluation unit 150.
Next, a flow of operation of the image generation system 10 according to the fourth example embodiment will be described with reference to
As illustrated in
Subsequently, the perturbation quantity generation unit 140 generates the random number for generating the perturbation quantity (the step S31). Then, the perturbation quantity generation unit 140 generates the perturbation quantity from the generated random number and the error parameter evaluated by the error evaluation unit 150 (step S41). The error evaluation by the error evaluation unit 150 may be performed separately before the start of a series of processing steps illustrated in
Subsequently, the perturbation addition unit 121 adds the perturbation quantity to the position information detected by the detection unit 110, and obtains the perturbed position information (the step S33). Then, the generation unit 130 performs the process of normalizing the face on the basis of the perturbed position information (the step S21). The generation unit 130 outputs the face normalized image that is an image including the normalized face, as a new image (the step S22).
Next, a technical effect obtained by the image generation system 10 according to the fourth example embodiment will be described.
As described in
The third example embodiment and the fourth example embodiment may be combined with each other. For example, both the error parameter specified by the user and the error parameter outputted from the error evaluation unit 150 may be integrated into an integrated error parameter, from which the perturbation quantity may be generated. The integrated error parameter may be a mean value of error parameters, for example. Furthermore, the error parameter specified by the user and the error parameter outputted from the error evaluation unit 150 may be selectively utilized. That is, out of the perturbation quantity generated from the error parameter specified by the user and the perturbation quantity generated from the error parameter outputted from the error evaluation unit 150, the selected one of them may be added to obtain the perturbed position information. In this case, the selection of the perturbation quantity may be performed automatically by the system, or by the user.
The image generation system 10 according to a fifth example embodiment will be described with reference to
First, with reference to
As illustrated in
Next, a flow of operation of the image generation system 10 according to the fifth example embodiment will be described with reference to
As illustrated in
Subsequently, the perturbation quantity generation unit 140 generates the random number for generating the perturbation quantity (the step S31). Then, the perturbation quantity generation unit 140 generates the perturbation quantity from the generated random number and the deviation of the probability distribution (step S42).
Subsequently, the perturbation addition unit 121 adds the perturbation quantity to the position information detected by the detection unit 110 (the step S33), and obtains the perturbed position information. Then, the generation unit 130 performs the process of normalizing the face on the basis of the perturbed position information (the step S21). The generation unit 130 outputs the face normalized image that is an image including the normalized face, as a new image (the step S22).
Next, a technical effect obtained by the image generation system 10 according to the fifth example embodiment will be described.
As described in
The image generation system 10 according to the sixth example embodiment will be described with reference to
First, with reference to
As illustrated in
Each of then position informations detected by the detection unit 110 is configured to be outputted to the acquisition unit 120. In particular, the acquisition unit 120 according to the sixth example embodiment is configured to include a selection unit 122. The selection unit 122 is configured to randomly select one position information from the n position informations detected by the detection unit 110. Since the n position informations are respectively detected at different times, they are informations having errors for each other as a whole. Therefore, the position information randomly selected from the n position informations may be used as the perturbed position information that takes the error into account. The acquisition unit 120 is configured to output the position information selected by the selection unit 122 to the generation unit 130, as the perturbed position information.
Next, a flow of operation of the image generation system 10 according to the sixth example embodiment will be described with reference to
As illustrated in
Subsequently, the selection unit 122 randomly selects one position information from the n position informations (step S52). Then, the generation unit 130 performs the process of normalizing the face on the basis of the randomly selected position information (i.e., the perturbed position information) (the step S21). The generation unit 130 outputs the face normalized image that is an image including the normalized face, as a new image (the step S22).
Next, a technical effect obtained by the image generation system 10 according to the sixth example embodiment will be described.
As described in
The image generation system 10 according to a seventh example embodiment will be described with reference to
First, with reference to
As illustrated in
Each of the position informations detected by the N detection units 110 is configured to be outputted to the acquisition unit 120. In particular, the acquisition unit 120 according to the seventh example embodiment includes the selection unit 122. The selection unit 122 is configured to randomly select one position information from N position informations detected by the N detection unit 110. When N detection units 110 perform n times of detections as in the sixth example embodiment, the selection unit 122 may randomly select one position information from N×n position informations. Since the N position informations are detected by the respective different detection units 110, they are informations having errors for each other as a whole. Therefore, the position information randomly selected from the N position informations detected by the separate detection unit 110 may be used as the perturbed position information that takes the error into account. The acquisition unit 120 is configured to output the position information selected by selection unit 122 to the generation unit 130, as the perturbed position information.
Next, a flow of operation of the image generation system 10 according to the seventh example embodiment will be described with reference to
As illustrated in
Subsequently, the selection unit 122 randomly selects one position information from the N position informations (step S52). Then, the generation unit 130 performs the process of normalizing the face on the basis of the randomly selected position information (i.e., the perturbed position information) (the step S21). The generation unit 130 outputs the face normalized image that is an image including the normalized face, as a new image (the step S22).
Next, a technical effect obtained by the image generation system 10 according to the seventh example embodiment will be described.
As described in
Next, the image generation system 10 according to a modified example of the seventh example embodiment will be described with reference to
First, with reference to
As illustrated in
The probability distribution estimation unit 160 is configured to perform a process of fitting a predetermined probability distribution to the N position informations detected by the N detection units 110. The probability distribution estimation unit 160 may fit a Gaussian distribution to the N position informations, for example. In this case, the position information may be a value that is expressed by an equation of Gaussian model mean+Gaussian model deviation×random number.
A result of the fitting process of the probability distribution estimation unit 160 is configured to be outputted to the acquisition unit 120. In particular, the acquisition unit 120 according to the modified example of the seventh example embodiment includes a sampling unit 123. The sampling unit 123 is configured to sample one position information from the probability distribution that is outputted as the processing result of the probability distribution estimation unit 160. The position information sampled in this way may be used as the perturbed position information that takes the error into account. The acquisition unit 120 is configured to output the position information sampled by the sampling unit 123 to the generation unit 130, as the perturbed position information.
Next, a flow of operation of the image generation system 10 according to the modified example of the seventh example embodiment will be described with reference to
As illustrated in
Subsequently, the probability distribution estimation unit 160 performs the process of fitting the predetermined probability distribution to the N position informations(step S55). Then, the sampling unit 123 samples one position information from the probability distribution that is outputted as the processing result of the probability distribution estimation unit 160 (step S56).
Then, the generation unit 130 performs the process of normalizing the face on the basis of the sampled position information (i.e., the perturbed position information) (the step S21). The generation unit 130 outputs the face normalized image that is an image including the normalized face, as a new image (the step S22).
Next, a technical effect obtained by the image generation system 10 according to the modified example of the seventh example embodiment will be described.
As described in
The image generation system 10 according to an eighth example embodiment will be described with reference to
First, with reference to
As illustrated in
The perturbed image generation unit 170 is configured to generate a plurality of perturbed images by perturbating the inputted image. The perturbed image generation unit 170 is configured to apply the perturbation by such processes as, for example, segmenting, reducing and enlarging, rotating, inverting, and changing a color tone of the image. The number of perturbed images generated by the perturbed image generation unit 170 may be fixed, or may be variable. In the following, a description will be made on the assumption that the perturbed image generation unit 170 generates M perturbed images (M is a natural number). When the perturbed image generation unit 170 generates M perturbed images, one original image and the M perturbed image, i.e., a total of (M+1) images are outputted from the perturbed image generation unit 170.
The (M+1) images outputted from the perturbed image generation unit 170 is configured to be outputted to the detection unit 110. Therefore, (M+1) position informations are outputted (detected?) from the detection unit 110 according to the eighth example embodiment.
Each of the (M+1) position informations detected by the detection unit 110 is configured to be outputted to the acquisition unit 120. In particular, the acquisition unit 120 according to the eighth example embodiment includes the selection unit 122. The selection unit 122 is configured to randomly select one position information from the (M+1) position informations. Since the (M+1) position informations are detected from the respective different images (i.e., the perturbed images), they are informations having errors for each other as a whole. Therefore, the position information randomly selected from the (M+1) position informations may be used as the perturbed position information that takes the error into account. The acquisition unit 120 is configured to output the position information selected by the selection unit 122 to the generation unit 130, as the perturbed position information.
Next, a flow of operation of the image generation system 10 according to the eighth example embodiment will be described with reference to
As illustrated in
Subsequently, the detection unit 110 detects the position informations about the position of the face or the position of the feature point of the face, respectively, from (M+1) images (i.e., one original image+M perturbed images) (step S63). As a result, (M+1) position informations are detected.
Subsequently, the selection unit 122 randomly selects one position information from the (M+1) position informations (step S64). Then, the generation unit 130 performs the process of normalizing the face on the basis of the randomly selected position information (i.e., the perturbed position information) (the step S21). The generation unit 130 outputs the face normalized image that is an image including the normalized face, as a new image (the step S22).
Next, a technical effect obtained by the image generation system 10 according to the eighth example embodiment will be described.
As described in
Next, the image generation system 10 according to a modified example of the eighth example embodiment will be described with reference to
First, with reference to
As illustrated in
The probability distribution estimation unit 160 is configured to perform a process of fitting a predetermined probability distribution to the (M+1) position informations detected from the (M+1) images. The probability distribution estimation unit 160 may fit a Gaussian distribution to the N position informations, for example. In this case, the position information may be a value that is expressed by an equation of Gaussian model mean+Gaussian model deviation×random number.
A result of the fitting process of the probability distribution estimation unit 160 is configured to be outputted to the acquisition unit 120. In particular, the acquisition unit 120 according to the modified example of the eighth example embodiment includes the sampling unit 123. The sampling unit 123 is configured to sample one position information from the probability distribution that is outputted as the processing result of the probability distribution estimation unit 160. The position information sampled in this way may be used as the perturbed position information that takes the error into account. The acquisition unit 120 is configured to output the position information sampled by the sampling unit 123 to the generation unit 130, as the perturbed position information.
Next, a flow of operation of the image generation system 10 according to the modified example of the eighth example embodiment will be described with reference to
As illustrated in
Subsequently, the detection unit 110 detects the position informations about the position of the face or the position of the feature point of the face, respectively, from (M+1) images (i.e., one original image+M perturbed images) (step S63). As a result, (M+1) position informations are detected.
Subsequently, the probability distribution estimation unit 160 performs the process of fitting the predetermined probability distribution to the (M+1) position informations (step S65). Then, the sampling unit 123 samples one position information from the probability distribution that is outputted as the processing result of the probability distribution estimation unit 160 (step S66).
Then, the generation unit 130 performs the process of normalizing the face on the basis of the sampled position information (i.e., the perturbed position information) (the step S21). The generation unit 130 outputs the face normalized image that is an image including the normalized face, as a new image (the step S22).
Next, a technical effect obtained by the image generation system 10 according to the modified example of the eighth example embodiment will be described.
As described in
The image generation system 10 according to a ninth example embodiment will be described with reference to
First, with reference to
As illustrated in
The probability distribution output unit 111 is configured to output an output result of the detection unit 110 in a form of the probability distribution. Therefore, the detection unit 110 according to the ninth example embodiment is capable of outputting the position information about the position of the face or the position of the feature point of the face detected from the image, in the form of the probability distribution. The detection unit 110 according to the ninth example embodiment may be configured as a detector that performs position estimation in the form of the probability distribution. The detector that performs position estimation in the form of the probability distribution, may deterministically estimate the position information in accordance with a rule determined in advance by the user, such as, for example, a median value, a mean value, and a most frequent value.
The sampling unit 123 is configured to sample one position information from the probability distribution outputted from the probability distribution output unit 111. The position information sampled in this way may be used as the perturbed position information that takes the error into account. The acquisition unit 120 is configured to output the position information sampled by the sampling unit 123 to the generation unit 130, as the perturbed position information.
Next, a flow of operation of the image generation system 10 according to the ninth example embodiment will be described with reference to
As illustrated in
Subsequently, the probability distribution output unit 111 outputs a detection result of the detection unit 110 in the form of the probability distribution (step S71). Then, the sampling unit 123 samples one position information from the probability distribution that is outputted as the detection result of the detection unit 110 (step S72).
Then, the generation unit 130 performs the process of normalizing the face on the basis of the sampled position information (i.e., the perturbed position information) (the step S21). The generation unit 130 outputs the face normalized image that is an image including the normalized face, as a new image (the step S22).
Next, a technical effect obtained by the image generation system 10 according to the ninth example embodiment will be described.
As described in
The image generation system according to a tenth example embodiment will be described. The tenth example embodiment describes specific application examples of the image generation systems according to the first to ninth example embodiments, and may be the same as the first to ninth example embodiments in the system configuration and the flow of operation. For this reason, the parts that differ from the first to ninth example embodiments will be described in detail below, and a description of the other overlapping parts will be omitted as appropriate.
The image generation system 10 according to the tenth example embodiment may be used to enhance training data that are used for machine learning. According to the image generation system 10 in this example embodiment, since a different image can be newly generated from one image included in the training data, it is possible to increase the numbers of images included in the training data. By enhancing the training data in this way, it is possible to increase versatility and robustness.
It is conceivable to adopt such processes as, for example, segmenting, reducing and enlarging, and rotating the image, to a method of enhancing the training data, but even if the training data are enhanced in this way, it is hard to increase the robustness of a variation in position in the face normalization, for example.
In the image generation system 10 according to this example embodiment, however, as already described, the perturbation is applied in accordance with the error that occurs when the position information about the position of the face or the position of the feature point of the face is detected. Therefore, it is possible to increase the robustness of a variation in position in the normalization process that uses the position information about the position of the face or the position of the feature point of the face.
The image generation system 10 according to the tenth example embodiment may be used for Virtual Adversarial Training. Specifically, the image generation system 10 may be configured to generate an adversarial example in the Virtual Adversarial Training.
When the adversarial example is generated, an artificial minute noise is added to learning data such that recognition is hardly made by a machine. If, however, it is not considered whether generated data are along a distribution of the learning data, the noisy data that do not actually exist may be generated. It cannot be said that the adversarial example generated in this way contributes to an improvement in learning of a neural network.
In the image generation system 10 according to this example embodiment, however, the perturbation is applied in accordance with the error that occurs when the position information about the position of the face or the position of the feature point of the face is detected. Therefore, by combining the adversarial example for a position of the feature point obtained by the Virtual Adversarial Training with the perturbed position information obtained by the image generation system 10, it is possible to surely enhance an effect of the Virtual Adversarial Training. A method of the combination may use known techniques/technologies of vector/sequence integration, such as mean or projection.
The image generation system 10 according to the tenth example embodiment may also be applied to identification of a person (so-called face authentication). For example, by using a plurality of images generated by the image generation system 10 according to this example embodiment (images to which the perturbation is applied in accordance with the error), it is possible to suppress a reduction in authentication accuracy.
In the person identification, there is a possibility that a normal identification cannot be made depending on a degree of appearance of the face in the image. In this case, for example, there may be circumstances in which a person who is normally to be authenticated is not authenticated, or a person who is not to be authenticated is authenticated.
In the image 10 according to this example embodiment, however, it is possible to nearly generate a different face image from a single face image. Therefore, it is possible to perform the identification of a person by using a plurality of face images (or by selecting an appropriate face image from the plurality of face images). Furthermore, especially in this example embodiment, the perturbation is applied in accordance with the error that occurs when the position information about the position of the face or the position of the feature point of the face is detected. Therefore, it is possible to generate an appropriate image by considering a variation in position of the feature point used for the identification of a person. Consequently, it is possible to effectively increase the accuracy of the authentication by the identification of a person.
In the identification of a person, the perturbation quantity may be calculated to provide the highest identification accuracy when the identification is made under a specific environment. Specifically, it is sufficient to obtain a coefficient that allows the identification accuracy to be maximized when the perturbation quantity is calculated by multiplying the error parameter by a coefficient of 0 to 1.
In the identification of a person, an integrated feature quantity obtained by integrates N feature quantities may be obtained. Specifically, first, N face normalized images are generated from N perturbed position informations by the image generation system 10 according to this example embodiment. Then, face feature quantities are respectively extracted from N face normalized images to generate N feature quantities. Finally, one integrated feature quantity is generated from the N feature quantities by using an arbitrary feature quantity integration method. Existing techniques/technologies, such as, for example, mean and Gaussian estimation, can be properly adopted to the feature quantity integration method.
In the identification of a person, an image that is used for the identification may be selectable. Specifically, the generated face normalized images are displayed on a display, and are presented to a user (i.e., a person who is to be authenticated or a system administrator or manager, etc.). Then, the user may be allowed to select an image to be used for the authentication, from among the presented face normalized images. In this case, the images may be ranked, and the images that are more suitable for the identification may be displayed in higher order. In addition, only a predetermined number of images suitable for the identification may be displayed. In addition to the image display, a sentence like “Are the eyes out of position?” may be displayed to obtain a response from the user.
The image generation system 10 according to the tenth example embodiment may be used to create a montage. When the montage is created, a part of the face is changed little by little, but efficient creation is hardly possible without a proper change. For example, even if a change that is impossible as a human face is given, it is hardly possible to generate an appropriate montage.
In the image generation system 10 according to this example embodiment, however, the perturbation is applied in accordance with the error that occurs when the position information about the position of the face or the position of the feature point of the face is detected. Thus, it is possible to give a proper change that can be realistically expected, to the face image. Therefore, by applying the image generated by the image generation system 10 according to this example embodiment to the montage creation, it is possible to create a montage, more efficiently.
The example embodiments described above may be further described as, but not limited to, the following Supplementary Notes below.
An image generation system described in Supplementary Note 1 is an image generation system including: a detection unit that detects a position information about a position of a face or a position of a feature point of the face from an image; an acquisition unit that obtains a perturbed position information that takes into account an error that occurs when the position information is detected, for the position information; and a generation unit that generates a new image including the face on the basis of the perturbed position information.
An image generation system described in Supplementary Note 2 is the image generation system described in Supplementary Note 1, wherein the generation unit generates, as the new image, a normalized image obtained by adjusting at least one of a position, a size, and an angle of the face on the basis of the perturbed position information.
An image generation system described in Supplementary Note 3 is the image generation system described in Supplementary Note 1 or 2, further including a calculation unit that calculates a perturbation quantity corresponding to the error, wherein the acquisition unit obtains the perturbed position information by adding the perturbation quantity to the position information.
An image generation system described in Supplementary Note 4 is the image generation system described in Supplementary Note 3, wherein the calculation unit calculates the perturbation quantity in accordance with the error that is specified by a user.
An image generation system described in Supplementary Note 5 is the image generation system described in Supplementary Note 3, further including an error evaluation unit that evaluates the error by using an image for a test having correct answer data about the position information, wherein the calculation unit calculates the perturbation quantity in accordance with the evaluated error.
An image generation system described in Supplementary Note 6 is the image generation system described in Supplementary Note 3, wherein the calculation unit calculates the perturbation quantity in accordance with a deviation of a probability distribution of a plurality of position informations.
An image generation system described in Supplementary Note 6 is the image generation system described in Supplementary Note 1 or 2, wherein the acquisition unit obtains the perturbed position information on the basis of a plurality of position informations detected by one detection unit.
An image generation system described in Supplementary Note 8 is the image generation system described in Supplementary Note 7, wherein the acquisition unit obtains the perturbed position information on the basis of a plurality of position informations detected by a plurality of different detection units.
An image generation system described in Supplementary Note 9 is the image generation system described in Supplementary Note 7 or 8, further including a probability distribution estimation unit that estimates a probability distribution from a plurality of position informations, wherein the acquisition unit obtains the perturbed position information by sampling from the estimated probability distribution.
An image generation system described in Supplementary Note 10 is the image generation system described in Supplementary Note 1 or 2, further including a perturbed image generation unit that generates a plurality of perturbed images by perturbing the image, wherein the acquisition unit obtains the perturbed position information on the basis of a plurality of position informations detected for each of the plurality of perturbed images.
An image generation system described in Supplementary Note 11 is the image generation system described in Supplementary Note 1 or 2, wherein the detection unit outputs the position information in a form of a probability distribution, and the acquisition unit obtains the perturbed position information by sampling from the outputted probability distribution.
An image generation method described in Supplementary Note 12 is an image generation method including: detecting a position information about a position of a face or a position of a feature point of the face from an image; obtaining a perturbed position information that takes into account an error that occurs when the position information is detected, for the position information; and generating a new image including the face on the basis of the perturbed position information.
A recording medium described in Supplementary Note 13 is a recording medium on which a computer program that allows a computer to execute an image generation method is recorded, the image generation method including: detecting a position information about a position of a face or a position of a feature point of the face from an image; obtaining a perturbed position information that takes into account an error that occurs when the position information is detected, for the position information; and generating a new image including the face on the basis of the perturbed position information.
This disclosure is not limited to the examples described above and is allowed to be changed, if desired, without departing from the essence or spirit of this disclosure which can be read from the claims and the entire specification. An image generation system, an image generation method, and a recording medium with such changes are also intended to be within the technical scope of this disclosure.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2020/036345 | 9/25/2020 | WO |
