Patent Application
20250209799
The technology disclosed in the present specification (hereinafter, “the present disclosure”) relates to an information processing device, an information processing method, and a computer program that perform processing related to training.
With the evolution of machine learning, recognition, identification, prediction, and the like beyond humans have been realized, and utilization in each field has progressed. On the other hand, fairness in machine learning, that is, unfair determination made by machine learning depending on race, gender, and the like has been a social issue. Causes of such unfairness in machine learning include bias caused by data used for training and bias in training. Therefore, the challenge lies in how to mitigate the bias in data collection and how to learn collected data so as to enable fair determination. However, it is practically difficult to collect complete data ensuring fairness with respect to race and gender. This is because random collection of data causes some biased data distribution, which affects machine learning as data bias.
For example, there has been proposed a system that manages a machine learning model, the system including: a replacement means that performs replacement of a first input value included in an input with a second input value in a case where a first prediction result, which is a result of prediction by a first model using the input including the first input value satisfies a predetermined condition; a storage means that stores a second prediction result, which is a result of prediction by the first model using an input after the replacement has been performed; a reception means that receives feedback for the second prediction result; and a creation means that creates a second model by performing machine learning using training data including the input including the first input value and the second prediction result on the basis of the feedback that has been received (see Patent Document 1). The system is configured to determine whether or not a result of prediction using an input for verification by the second model satisfies the predetermined condition, and replace the first model with the second model in a case where it is determined that the predetermined condition is not satisfied.
An object of the present disclosure is to provide an information processing device, an information processing method, and a computer program that perform processing for training a model in which bias is mitigated.
The present disclosure has been made in view of the above problem, and a first aspect thereof is an information processing device including:
The information processing device according to the first aspect further includes: a training unit which trains the model by using training data to which training data of a minor attribute automatically generated by the generation unit is added; and a model bias determination unit which determines presence or absence of bias of the model that has been trained due to a difference in attributes of input data. Therefore, the information processing device according to the first aspect mitigates bias of training data at the time of training and mitigates the bias of trained model at the time of operating the trained model, thereby securing the fairness of the model.
Furthermore, a second aspect of the present disclosure is an information processing method including:
Furthermore, a third aspect of the present disclosure is a computer program described in a computer-readable format so as to cause a computer to function as:
The computer program according to the third aspect of the present disclosure defines a computer program described in a computer-readable format so as to implement predetermined processing on a computer. In other words, by installing the computer program according to the third aspect of the present disclosure in a computer, the computer can perform a cooperative operation and produce effects similar to those produced by the information processing device according to the first aspect of the present disclosure.
Furthermore, a fourth aspect of the present disclosure is an information processing device including:
The information processing device according to the fourth aspect further includes a notification unit that gives notification of a determination result obtained by the determination unit to an external device, in which a model parameter in which bias is mitigated according to the notification is received and set for the trained model.
Furthermore, a fifth aspect of the present disclosure is an information processing method including:
Furthermore, a sixth aspect of the present disclosure is a computer program described in a computer-readable format so as to cause a computer to function as:
The computer program according to the sixth aspect of the present disclosure defines a computer program described in a computer-readable format so as to implement predetermined processing on a computer, and by installing the computer program in a computer, the computer can perform a cooperative operation and produce effects similar to those produced by the information processing device according to the fourth aspect of the present disclosure
According to the present disclosure, it is possible to provide an information processing device, an information processing method, and a computer program that perform processing for mitigating bias of training data and mitigating bias of a machine learning model.
Note that the effects described in the present specification are merely examples, and the effects to be brought by the present disclosure are not limited thereto. Furthermore, in addition to the effects described above, the present disclosure might further exhibit additional effects in some cases.
Still another object, feature, and advantage of the present disclosure will become clear by further detailed description with reference to an embodiment as described later and the attached drawings.
In the description below, the present disclosure will be explained in the following order, with reference to the drawings.
Fairness in machine learning, that is, unfair determination made by machine learning depending on race, gender, and the like has been a social issue. Causes of such unfairness in machine learning include bias caused by data used for training and bias in training. It can be easily understood that, in a case where a data set used for model training is affected by bias or a gap due to race, ethnicity, gender, or the like, a machine learning model based on the data set is also easily affected. Therefore, the challenge lies in how to mitigate the bias in data collection and how to learn collected data so as to enable fair determination.
For example, essentially, determination should be originally made without depending on a difference in sensitive attributes such as race and gender when various predictions such as personnel acceptance/rejection determination, determination of a re-offending rate, and loan screening are made by artificial intelligence. However, if determination in favor of only some people having a sensitive attribute is made, opportunities of people having other sensitive attributes are unreasonably lost, and unfairness is caused. It is necessary to train a model such that distribution of prediction scores does not change depending on sensitive attributes such as race and gender.
Therefore, in the present disclosure, in a data collection stage, fairness in machine learning is realized by classifying data for each sensitive attribute such as race and gender and supplementing insufficient data. Specifically, collected data is classified for each sensitive attribute, and insufficient data of a sensitive attribute is automatically generated using a generative adversarial network (GAN). Therefore, according to the present disclosure, in the process of data collection, by measuring fairness of collected data and compensating for insufficient data on the basis of a method without human intervention, adjustment can be automatically performed so as to eliminate data bias.
Moreover, in the present disclosure, fairness of the trained model that has been trained using data from which bias has been removed is also checked, and correction processing for mitigating the bias of the machine learning model can also be performed.
Therefore, according to the present disclosure, it is possible to ensure fairness of data by automatically generating data of a minor attribute without human intervention. According to the present disclosure, accuracy of learning does not decrease, unlike a case where data of a major attribute is reduced to remove data bias. Furthermore, according to the present disclosure, it is possible to save labor for collecting data of a minor attribute from the real world, and thus, it is possible to significantly reduce development processes of machine learning. Furthermore, according to the present disclosure, it is possible to set a threshold for bias of data collected for model training and bias determination of a trained model, and thus, it is possible to perform model training and bias determination of a trained model according to a situation.
The illustrated training system 100 includes a training data holding unit 101, a model training unit 102, a model parameter holding unit 103, an inference unit 111, a data input unit 112, and an input data processing unit 113. Among those units, the training data holding unit 101, the model training unit 102, and the model parameter holding unit 103 operate in a training phase of a machine learning model, and the inference unit 111, the data input unit 112, and the input data processing unit 113 operate in an inference phase using the trained model. Although the training system 100 is used by being installed on, for example, an edge device, some or all of the functions of the training system 100 may be constructed on a cloud or an arithmetic device capable of large-scale computation. For example, the model training unit 102 that handles a huge amount of data and needs high calculation ability may be arranged in a server on a cloud, and the inference unit 111 that uses the trained model may be arranged in an edge device such as a digital camera, a multifunctional information terminal such as a smartphone or a tablet, or a personal computer.
Furthermore, in the present embodiment, the training system 100 further includes a training data collection unit 130 that provides training data to be used for training the machine learning model, a data bias determination unit 121 that determines bias of the training data to be used for training the machine learning model in the model training unit 102, and a data supplementation unit 122 that supplements insufficient training data on the basis of the determination result of the data bias.
Furthermore, the training system 100 may further include a model bias determination unit 123 that determines bias when the inference unit 111 makes an inference by using the trained model, and a model bias mitigation unit 124 that executes processing for mitigating bias of the machine learning model on the basis of the model bias determination result.
The training data collection unit 130 collects training data to be used by the model training unit 102 for model training. The training data basically includes a data set (x, y) obtained by combining input data x to be input to a model to be trained and a correct answer label y serving as a correct answer for the input data x. Furthermore, in the present embodiment, in order to determine the data bias for each sensitive attribute s, a data set (x, y, s) further including the sensitive attribute s of the input data x in addition to the input data x and the correct answer label thereof may be used as the training data. For example, in a case where the model training unit 102 trains a machine learning model for image classification, the training data collection unit 130 provides training data including a combination of a captured image and a correct answer label (what the subject of the captured image is) in a case of a digital camera, for example. The training data collection unit 130 collects, for example, training data including images captured by a large number of digital cameras via a wide area network such as the Internet, and accumulates the training data in the training data holding unit 101.
The training data holding unit 101 accumulates training data to be used by the model training unit 102 for model training. Each piece of training data includes a data set obtained by combining input data y input to a model to be trained with a correct answer label y of the correct answer to be inferred by the model, but in the present embodiment, a data set (x, y, s) to which the sensitive attribute s of the input data x is further added may be used as the training data (described above). The training data holding unit 101 accumulates data sets provided from the training data collection unit 130, but may accumulate data sets obtained from another source. In the present embodiment, training data supplied from the data supplementation unit 122 (described later) is also accumulated in the training data holding unit 101. In a case where the data model training unit 102 carries out deep learning, a huge amount of data sets is accumulated in the training data holding unit 101.
The model training unit 102 sequentially reads the training data from the training data holding unit 101, trains the machine learning model to be trained, and updates the machine learning model. The machine learning model includes, for example, a neural network, but may be a model using a type such as support vector regression or Gaussian process regression. The machine learning model including the neural network has a multilayer structure including an input layer that receives data (explanatory variable) such as an image, an output layer that outputs a label (objective variable) serving as an inference result for the input data, and one or a plurality of intermediate layers (or hidden layers) between the input layer and the output layer. Each of the layers includes a plurality of nodes corresponding to neurons. Coupling between the nodes across the layers has a weight, and a value of the data input to the input layer is transformed as the data passes from layer to layer. For example, the model training unit 102 calculates a loss function defined on the basis of an error between a label output from the machine learning model for the input data and a known correct answer label corresponding to the input data, and trains the machine learning model while updating the model parameter (weight coefficient between nodes, or the like) by backpropagation in such a manner that the loss function is minimized. Note that, since the training processing of the machine learning model is enormous in calculation amount, distributed learning using a plurality of graphics processing units (GPUs) or a plurality of calculation nodes may be carried out.
Then, the model training unit 102 stores the model parameter obtained as a training result, in the model parameter holding unit 103. The model parameter is a variable element that defines the model, and is, for example, a coupling weight coefficient or the like to be given between nodes of a neural network model.
The inference unit 111, the data input unit 112, and the input data processing unit 113 implement the inference phase by using the trained model. The data input unit 112 inputs sensor information obtained by a sensor included in the edge device. The input data processing unit 113 performs data processing on data input from the data input unit 112 such that the data has a data format that can be input to the model (for example, a neural network model), and inputs the data to the inference unit 111. The inference unit 111 outputs a label inferred from the input data by using the model in which the model parameter read from the model parameter holding unit 103 is set, that is, the trained model.
The data bias determination unit 121 analyzes the training data for training the machine learning model accumulated in the training data holding unit 101 to determine the bias. Specifically, the data bias determination unit 121 classifies the training data into each sensitive attribute and determines the presence or absence of bias. For data bias determination, the training data may be a data set (x, y, s) further including the sensitive attribute s of the input data in addition to the input data x and the correct answer label y. Furthermore, the sensitive attribute may not be included in the training data, and the sensitive attribute of each training data may be determined in the data bias determination unit 121 or another functional module.
The sensitive attribute herein is, for example, gender, race, or the like. Specifically, gender may be defined as a first sensitive attribute s1, a male may be defined as “s1=1”, a female may be defined as “s1=0”, race may be defined as a second sensitive attribute s2, a white may be defined as “s2=1”, and a black may be defined as “s2=0”. However, gender and race are examples of the sensitive attribute, and what attribute is treated as the sensitive attribute is arbitrary, and the sensitive attribute may be flexibly set according to an application field of the machine learning model.
As a result of classifying the training data into each sensitive attribute, regarding a certain sensitive attribute s1, in a case where the ratio of the number of pieces of training data in which s1=1 (that is, a major attribute) to the number of pieces of training data in which s1=0 (that is, a minor attribute) is equal to or greater than a predetermined threshold, the data bias determination unit 121 outputs a determination result indicating that there is data bias for the sensitive attribute s1 to the data supplementation unit 122 (here, i is a serial number of the sensitive attribute).
Note that the threshold is desirably a value representing a range in which unfairness is socially acceptable. In the present embodiment, the threshold for data bias determination for each sensitive attribute can be set by a user operation via, for example, a user interface (UI). Furthermore, an attribute to be handled as sensitive in the training system 100 may also be able to be specified through a user operation.
The data supplementation unit 122 performs processing of supplementing insufficient training data on the basis of the determination result of the data bias. For example, when receiving a determination result in which regarding the sensitive attribute s1, there is data bias in which the number of pieces of training data satisfying s1=0 is small (that is, the sensitive attribute s1=0 is a minor attribute) from the data bias determination unit 121, the data supplementation unit 122 supplements data of the minor attribute and accumulates the data in the training data holding unit 101. In the present embodiment, the data supplementation unit 122 artificially increases the data of the minor attribute from the training data that has already been acquired (is accumulated in the training data holding unit 101). According to the method of artificially increasing data of the minor attribute, the work is easier than in the case of manual adjustment, and the hurdle for obtaining the consent of the person himself/herself is also lowered. Specifically, the data supplementation unit 122 automatically generates data of the minor attribute by using a generative adversarial network (GAN) algorithm. Of course, the method of supplementing data may be, for example, a method of collecting data via a wide area network such as the Internet, similarly to the training data collection unit 130.
The model bias determination unit 123 analyzes the results of inference performed by the inference unit 111 using the trained model, and determines whether or not fairness of the trained model is ensured, that is, model bias. For example, the model bias determination unit 123 determines whether or not inter-group fairness, that is, estimation of the machine learning model for a group belonging to a certain sensitive attribute is disadvantageous. In a case where, regarding a certain sensitive attribute s1, the percentage of disadvantageous estimation results that are output is higher in the attribute of s1=0 than in the attribute of s1=1, the model bias determination unit 123 determines that there is model bias. Furthermore, the model bias determination unit 123 may also determine fairness between individuals, for example, whether or not the difference in the estimation results of the machine learning model is large for individuals having the same ability. However, in the present specification, fairness between groups will be mainly described.
Note that, in the present embodiment, the threshold for model bias determination can be set by a user operation via a UI, for example.
The model bias mitigation unit 124 executes processing for mitigating the bias of the machine learning model on the basis of the model bias determination result. The model bias mitigation unit 124 may execute a model bias mitigation algorithm such as transfer learning, fine tuning, and incremental learning, and the model training unit 102 may mitigate the model bias for the trained model.
Note that transfer learning is a learning method of applying knowledge learned in another task to learning of another region. For example, in a convolutional neural network (described later) including a feature amount extractor and a discriminator, the result of training the feature amount extractor of the preceding stage with another task is fixed, and only the image classification unit of the subsequent stage is trained with another problem. In contrast, in fine tuning, fine adjustment is also performed in the feature amount extractor. Furthermore, incremental learning is learning processing that is additionally performed at a timing different from learning in the training phase. For example, incremental learning processing is started at a timing when it is detected that there is model bias in the inference phase, but incremental learning may be periodically performed.
First, the training phase will be described. The training data collection unit 130 collects training data to be used by the model training unit 102 for model training and accumulates the training data in the training data holding unit 101 (step S201). The training data collection unit 130 collects, for example, training data including images captured by a large number of digital cameras via a wide area network such as the Internet, but the source from which the training data is collected is not particularly limited.
The training data basically includes a data set (x, y) obtained by combining input data x and a correct answer label y corresponding to the input data x. The training data collection unit 130 may simultaneously acquire the label y at the stage of acquiring the data x, or may label the acquired data x after acquiring the data x. Furthermore, in the latter case, the labeling method is not particularly limited.
Furthermore, in the present embodiment, in order to determine whether there is bias in data x collected for the sensitive attribute in the subsequent data bias determination processing, the training data collection unit 130 may simultaneously acquire the sensitive attribute of each data x at the stage of acquiring the data x. Of course, the sensitive attribute of the acquired data x may be determined after the data x is acquired. For example, the training data collection unit 130 or a functional module, not illustrated, may determine the sensitive attribute of the acquired data x by using artificial intelligence (AI), or may determine the sensitive attribute by using any other algorithm.
Next, the data bias determination unit 121 classifies the training data accumulated in the training data holding unit 101 (alternatively, only data used for training among the data accumulated in the training data holding unit 101) for each sensitive attribute (step S202), and determines whether the data is biased to a specific attribute value for each sensitive attribute, that is, whether there is data bias (step S203).
As a result of classifying the training data into each sensitive attribute, in a case where, regarding a certain sensitive attribute s1, the ratio of the number of pieces of training data in which s1=1 (that is, a major attribute) to the number of pieces of training data in which s1=0 (that is, a minor attribute) is equal to or greater than a predetermined threshold, the data bias determination unit 121 determines that there is data bias for the sensitive attribute s1.
In the present embodiment, the threshold for data bias determination for each sensitive attribute and an attribute to be treated as sensitive can be specified by a user operation via, for example, the UI or the like.
Here, in a case where the data bias determination unit 121 determines that there is data bias (Yes in step S203), the data supplementation unit 122 supplements the data of the minor attribute of the sensitive attribute determined to have data bias (step S204), and accumulates the data in the training data holding unit 101.
In step S204, data of the minor attribute is artificially increased from the training data. According to the method of artificially increasing data of the minor attribute, the work is easier than in the case of manual adjustment, and the hurdle for obtaining the consent of the person himself/herself is also lowered. Specifically, the data supplementation unit 122 automatically generates data of the minor attribute by using the GAN algorithm.
Then, the model training unit 102 trains the machine learning model to be trained by using the training data in which there is no data bias or the data bias is eliminated, and updates the machine learning model (step S205). Hereinafter, a case where the machine learning model includes a convolutional neural network will be described as a main example. Furthermore, the model training unit 102 calculates a loss function defined on the basis of an error from the known correct answer label corresponding to the input data, and performs training by backpropagation so as to minimize the loss function. However, the present disclosure is not limited to a specific learning algorithm. Then, the model training unit 102 stores the model parameter obtained as a training result, in the model parameter holding unit 103.
Next, the inference phase will be described. The data input unit 112 inputs data to be inferred. The data input unit 112 inputs, for example, sensor information obtained by a sensor included in the edge device as data to be inferred. The input data processing unit 113 executes data processing in such a way that the data input from the data input unit 112 has the data format that can be input to the machine learning model to be trained, and inputs the data to the inference unit 111. Then, the inference unit 111 outputs a label inferred from the input data by using the model in which the model parameter read from the model parameter holding unit 103 is set, that is, the trained model (step S211).
Next, the model bias determination unit 123 analyzes the results of inference performed by the inference unit 111 using the trained model, and determines whether or not fairness of the trained model is ensured, that is, model bias (step S212). The model bias determination unit 123 checks whether or not the results of inference by the trained model are biased according to the difference in the sensitive attributes of the input data (for example, whether the disadvantageous inference results are one-sided depending on a difference in race or gender). For example, the model bias determination unit 123 may start the model bias determination processing at every predetermined number of times of inference, or may start the model bias determination processing at a predetermined time interval.
The model bias determination unit 123 determines whether the inference by the trained model for each sensitive attribute is fair. In a case where, regarding a certain sensitive attribute s1, a difference in ratios of the labels inferred between the attribute of s1=1 (that is, the major attribute) and the attribute of s1=0 (that is, the minor attribute) exceeds a threshold, the model bias determination unit 123 determines that the trained model is unfair, that is, there is model bias. In the present embodiment, the threshold for model bias determination can be set by a user operation via a UI, for example.
Then, in a case where the model bias determination unit 123 determines that there is model bias (Yes in step S213), the model bias mitigation unit 124 executes the model bias mitigation algorithm and the model training unit 102 may mitigate the model bias for the trained model (step S214).
After the model bias mitigation processing in step S214 is executed, the processing returns to step S211, and inference using the machine learning model after model bias mitigation and model bias determination are repeatedly performed. Then, in a case where it is determined that there is no model bias (No in step S213), it is determined that a fair machine learning model without bias is obtained, and the processing ends.
The machine learning model includes, for example, a neural network, but may be a model using a type such as support vector regression or Gaussian process regression (described above). In the present specification, an embodiment to which a convolutional neural network (CNN) is applied will be described. In this section C, the configuration of the convolutional neural network will be described.
The feature amount extractor 310 includes four stages of convolutional layers and pooling layers. Assuming that a first-stage convolutional layer C1, a second-stage convolutional layer C2, a third-stage convolutional layer C3, and a fourth-stage convolutional layer C4 are disposed in this order from a side closer to an input image PIC, resolution of processed images is smaller and the number of feature maps (number of channels) is larger in later stages. More specifically, assuming that resolution of the input image PIC is m1×n1, resolution of the first-stage convolutional layer C1 is m2×n2, resolution of the second-stage convolutional layer C2 is m3×n3, resolution of the third-stage convolutional layer C3 is m4×n4, and resolution of the fourth-stage convolutional layer C4 is m5×n5 (m1×n1<m2×n2≤m3×m3≤m4×n4 (m5×n5). Furthermore, the number of feature maps of the first-stage convolutional layer C1 is k1, the number of feature maps of the second-stage convolutional layer C2 is k2, the number of feature maps of the third-stage convolutional layer C3 is k3, and the number of feature maps of the fourth-stage convolutional layer C4 is k4 (k1≤k2≤k3≤k4, but k1 to k4 are not the same). Note that illustration of the pooling layers is omitted in
The discriminator 320 includes a fully connected layer which includes an input layer FC1, one or more hidden layers FC2, and an output layer FC3, and in which all nodes of each layer are connected with all nodes of the subsequent layer. Outputs of the fourth-stage convolutional layer C4 of the feature extractor 310 are arranged one-dimensionally and used as inputs to the fully connected layer. For simplification of the description, when the fully connected layer is simplified as illustrated in
y1 and y2 of the output layer in
As a result of classifying the training data into each sensitive attribute, in a case where, regarding a certain sensitive attribute s1, the ratio of the number of pieces of training data in which s1=1 to the number of pieces of training data in which s1=0 is equal to or greater than a predetermined threshold, it is assumed that there is data bias for the sensitive attribute s1.
For example, in a case where gender is defined as the first sensitive attribute s1, the male is defined as “s1=1”, and the female is defined as “s1=0”, if the model training unit 102 performs training while leaving data bias in which the number of pieces of training data that satisfies “s1=0” is significantly smaller than the number of pieces of training data that satisfies “s1=1” among the pieces of training data accumulated in the training data holding unit 101, there is a possibility that an unfair machine learning model is generated in which inference can be accurately made with respect to the major attribute that satisfies “s1=1” but inference cannot be accurately made with respect to the minor attribute that satisfies “s1=0”.
Similarly, in a case where race is defined as the second sensitive attribute s2, a white is defined as “s2=1”, and a black is defined as “s2=0”, if the model training unit 102 performs training while leaving data bias in which the number of pieces of training data that satisfies “s2=0” is significantly smaller than the number of pieces of training data that satisfies “s2=1” among the pieces of training data accumulated in the training data holding unit 101, there is a possibility that an unfair machine learning model is generated in which inference can be accurately made with respect to the major attribute that satisfies “s2=1” but inference cannot be accurately made with respect to the minor attribute that satisfies “s2=0”.
In order to determine whether data x collected for the sensitive attribute in the data bias determination processing is biased, the training data collection unit 130 may simultaneously acquire an attribute label indicating the sensitive attribute of each data x at the stage of acquiring the data x. Of course, the sensitive attribute of the acquired data x may be determined after the data x is acquired. The training data collection unit 130 or a functional module, not illustrated, may determine the sensitive attribute of the acquired data x by using AI, or may determine the sensitive attribute by using any other algorithm.
For example, in the case of personal information that is highly necessary to be considered, labeling may be performed on the basis of quantitative information obtained by analysis of image data or the like without labeling the sensitive attribute by directly using the personal information. Specifically, the sensitive attribute may be labeled using an individual typology angle (ITA) score of the face image as the input data instead of labeling with the personal information that needs to be considered such as race. The ITA score is known as a value that characterizes the color of the subject's skin measured by a skin contact reflex spectrophotometer or a skin colorimeter.
In a case where the data bias determination unit 121 outputs the determination result indicating that there is data bias, the data supplementation unit 122 artificially increases the data of the minor attribute from the training data that has already been acquired (is accumulated in the training data holding unit 101). According to the method of artificially increasing data of the minor attribute, the work is easier than in the case of manual adjustment, and the hurdle for obtaining the consent of the person himself/herself is also lowered.
Specifically, the data supplementation unit 122 automatically generates data of the minor attribute by using the GAN algorithm. The GAN is an algorithm for deepening learning of input data by causing a generator and a discriminator each including a neural network to compete with each other.
Here, the GAN algorithm will be briefly described with reference to
Specifically, the data supplementation unit 122 artificially generates data of the minor attribute by using StyleGAN2 (see, for example, Non-Patent Document 1) obtained by further improving StyleGAN that has achieved high-resolution image generation using progressive growing, thereby eliminating data imbalance between the attributes.
In supervised learning of a machine learning model, a variable to be obtained (or to be predicted) is an object variable y, and a variable acting on the object variable is an explanatory variable. For example, in a case where pass/fail in a company recruitment test, a re-offending rate of an ex-convict, pass/fail in loan screening, and the like are used as object variables, various attribute information of the subject such as race, ethnicity, gender, age, and religion are used as explanatory variables. The explanatory variables are roughly divided into sensitive attributes s and non-sensitive attributes x other than the sensitive attributes. Generally, race, ethnicity, gender, and the like are considered as sensitive attributes. Of course, the definition of the sensitive attribute may be appropriately changed as necessary, or the sensitive attribute may be specified by a user operation via a UI or the like.
As described above, in order to prevent the fairness of the machine learning model from being threatened due to “data bias” in which training data is biased to a specific sensitive attribute, in the present disclosure, the data bias is mitigated by performing supplementation of training data of a minor attribute (specifically, generation of training data of the minor attribute by the GAN). Nevertheless, “model bias” in which prediction made by the trained model is biased can occur. When a difference in advantage or disadvantage occurs in the prediction result between groups or individuals due to model bias, the fairness of the trained model is not maintained.
Furthermore,
For example, in a case where distribution of prediction scores changes depending on a sensitive attribute such as race or gender in prediction of pass/fail in a company recruitment test, a re-offending rate of an ex-convict, pass/fail in loan screening, or the like, social bias is not excluded from the trained model, and thus the trained model is unfair.
In the training system 100 according to the present embodiment, the model bias determination unit 123 analyzes the results of inference made by the inference unit 111 using the trained model, and determines fairness of the trained model (described above). The model bias determination unit 123 may also determine the fairness between individuals, for example, whether or not the difference in the estimation results of the trained model is large for individuals having the same ability. However, in the present specification, fairness between groups will be mainly described.
For example, the model bias determination unit 123 aggregates the inference results by the trained model, and determines model bias on the basis of whether or not a predefined definition of fairness is satisfied. The model bias determination unit 123 may determine model bias by using one or a plurality of definitions designated by the user through a UI operation or the like from among several types of fairness definitions prepared in advance. Furthermore, the user may be allowed to specify the definition of fairness via a UI operation or the like. For example, the user may specify the definition of the fairness according to the task of the machine learning model.
In the field, for example, definitions of fairness such as demographic parity, equalized odds, and equal opportunity are known.
Demographic parity is a definition of fairness in which the same predicted distribution should be used regardless of the sensitive attribute. For example, in human resources recruitment by a company, as illustrated in
Furthermore, equalized odds is a definition of fairness in which the ratio of a true positive rate (TPR) to a false positive rate (FPR) is the same regardless of the sensitive attribute (alternatively, a difference between the ratios is equal to or less than a predetermined threshold). For example, in prediction as to whether or not an ex-convict will re-commit a crime, as illustrated in
Furthermore, equal opportunity is a definition of fairness in which the ratio of the true positive rate (TPR) is the same regardless of the sensitive attribute (alternatively, a difference between the ratios is equal to or less than a predetermined threshold), and the ratio of the FPR is not taken into consideration unlike equalized odds. For example, in prediction as to whether or not a debtor of a loan discharges a debt, as illustrated in
In addition to the above, there are definitions of fairness such as fairness through awareness in which “similar individuals should obtain similar predicted values” and fairness through unawareness in which “fairness should be achieved even if a sensitive attribute is not explicitly used”.
In the field, for example, AI Fairness 360 (see Non-Patent Document 2) provided by IBM Corporation is known as an open source toolkit that detects and removes bias of a machine learning model. Meanwhile, it has been described that the training system 100 according to the present embodiment has a configuration in which the data supplementation unit 122 artificially generates data of a minor attribute by the GAN and mitigates the data bias, and the model bias mitigation unit 124 mitigates the bias of the trained model. Also in the present disclosure, with reference to AI Fairness 360, bias mitigation of the machine learning model may be performed in each of three stages of preprocessing, during training processing, and after training to realize fairness.
In the preprocessing, bias mitigation of the training data held in the training data holding unit 101 is performed. In addition to a method in which the data supplementation unit 122 artificially generates data of a minor attribute by the GAN and mitigates the data bias, bias mitigation may be performed by reweighting, that is, weighting on training data. The preprocessing may be performed by the data supplying unit 122, or a functional module for preprocessing may be additionally arranged in the training system 100.
During the training processing, in the convolutional neural network (see
After training, the bias of the trained model is mitigated by prediction. Specifically, prediction made by the trained model is changed to be fair by using the reject option classification algorithm. The bias mitigation processing after training may be performed by the model bias mitigation unit 124.
Some or all of the functions the training system 100 are constructed on a cloud or an arithmetic device capable of large-scale computation (described above). In this section H, an operation example will be described in which the model training unit 102 that handles a huge amount of data and needs high calculation ability is arranged in a server on a cloud, and the inference unit 111 that uses the trained model is arranged in an edge device such as a digital camera, a multifunctional information terminal such as a smartphone or a tablet, or a personal computer.
In the server, functional modules including the model training unit 102, the functional modules being related to training processing of the machine learning model are arranged. On the server side, the model training unit 102 performs the training processing of the machine learning model and a model parameter obtained as a result is held in the model parameter holding unit 103. In contrast, there are a large number of edge devices including the inference units 111 on the cloud side. The edge device is a digital camera, a multifunctional information terminal such as a smartphone or a tablet, a personal computer, or the like. Each edge device includes the inference unit 111, and there is also an edge device further including the model bias determination unit 123.
The server provides the learned model parameter to each edge device. The form in which the model parameter is provided is variable. For example, the model parameter may be installed in advance at the time of shipping the edge device product. In addition, the edge device may download the model parameter from the server after product shipment. Furthermore, even after the start of use, the model parameter updated on the server side may be appropriately downloaded to the edge device as necessary. The update of the model parameter on the server side includes an update accompanying model bias mitigation, and it is a matter of course that the update of the model parameter on the server side may also include an update for other purposes.
In each edge device, the model parameter provided from the server is set in the inference unit 111, and prediction processing by the trained model is executed. A task performed by the trained model may be face detection or face identification from a captured image of a camera, and furthermore, may be determination of possibility of employment of an employee, pass/fail in loan screening, re-offending, or the like based on a face image.
The edge device including the model bias determination unit 123 analyzes the prediction result of the inference unit 111, determines whether or not the inference by the trained model for each sensitive attribute is fair, that is, whether or not model bias is present, and uploads the determination result to the server (alternatively, may notify the server only when it is determined that there is model bias). In addition, the edge device that does not include the model bias determination unit 123 may upload the prediction result of the inference unit 111 to the server, and the model bias determination unit 123 on the server side may perform the model bias determination processing.
The model bias mitigation unit 124 on the server side may execute a model bias mitigation algorithm in response to the determination result indicating that there is model bias, and the model training unit 102 performs mitigation of the model bias such as transfer learning, fine tuning, or incremental learning for the trained model. Then, the model parameter of the machine learning model in which model bias has been mitigated is accumulated in the model parameter holding unit 103 and further provided to each edge device.
First, the server provides the learned model parameter to the edge device (TR1401). Alternatively, the edge device acquires the model parameter from the server.
Thereafter, on the edge device side, the model parameter provided from the server is set in the model to be used by the inference unit 111, and prediction processing by the trained model is executed (TR1402). A task performed by the trained model may be face detection or face identification from a captured image of a camera, and furthermore, may be determination of possibility of employment of an employee, pass/fail in loan screening, re-offending, or the like based on a face image.
Then, on the edge device side, it is determined whether or not the inference by the trained model is fair, that is, whether or not there is model bias (TR1403), and the determination result is uploaded to the server (alternatively, the server may be notified only when it is determined that there is model bias) (TR1404). Furthermore, the edge device that does not include the model bias determination unit 123 may upload the prediction result of the trained model to the server, and the model bias determination unit 123 on the server side may perform the model bias determination processing.
On the server side, in response to the determination result indicating that there is model bias from the edge device (alternatively, by analyzing the determination result of the trained model received from the edge device), the model bias is mitigated for the trained model (TR1405). The server mitigates the model bias of the trained model by, for example, transfer learning, fine tuning, incremental learning, or the like.
Then, the server provides the model parameter after mitigating the model bias to the edge device (TR1406). For example, a new model parameter may be distributed from the server to the edge device in the form of product software update.
After the trained model is mounted on a product or a service and released, it is difficult to verify or correct the model. In contrast, according to the transaction as illustrated in
The optical unit 1501 includes, for example, a plurality of optical lenses that condenses light from a subject on a light receiving surface of the sensor unit 1502, a diaphragm mechanism that adjusts the size of an opening with respect to incident light, and a focus mechanism that adjusts the focus of irradiation light on the light receiving surface. The optical unit 1501 may further include a shutter mechanism that adjusts the time during which the light receiving surface is irradiated with light. The diaphragm mechanism, the focus mechanism, and the shutter mechanism included in the optical unit 1501 are configured to be controlled by, for example, the sensor control unit 1503. Note that the optical unit 1501 may be configured integrally with the imaging device 1500 or may be configured separately from the imaging device 1500.
The sensor unit 1502 includes a pixel array in which a plurality of pixels is arranged in a matrix. Each pixel includes a photoelectric conversion element, and a light receiving surface is formed by the pixels arranged in a matrix. The optical unit 1501 forms an image of incident light on the light receiving surface, and each pixel of the sensor unit 1502 individually outputs a pixel signal corresponding to irradiation light. The sensor unit 1502 further includes a drive circuit for driving each pixel included in the pixel array, and a signal processing circuit that performs predetermined signal processing on a signal read from each pixel and outputs the signal as a pixel signal of each pixel. The sensor unit 1502 outputs a pixel signal of each pixel included in a pixel region, as digital image data.
The sensor control unit 1503 includes a microprocessor, for example, controls reading of pixel data from the sensor unit 1502, and outputs image data based on the pixel signal read from each pixel. The pixel data output from the sensor control unit 1503 is passed to the inference unit 1504 and the visual recognition processing unit 1506. Furthermore, the sensor control unit 1503 generates an imaging control signal for controlling imaging in the sensor unit 1502, and supplies the imaging control signal to the sensor unit 1502. The imaging control signal includes information indicating exposure and analog gain at the time of imaging in the sensor unit 1502. The imaging control signal further includes a control signal for performing an imaging operation of the sensor unit 1502, such as a vertical synchronization signal or a horizontal synchronization signal. Furthermore, the sensor control unit 1503 generates control signals to the diaphragm mechanism, the focus mechanism, and the shutter mechanism, and supplies the control signals to the optical unit 1501.
The inference unit 1504 corresponds to the “inference unit 111” illustrated in
The visual recognition processing unit 1506 executes processing for obtaining an image suitable for visual recognition by humans on the pixel data passed from the sensor control unit 1503, and outputs image data including a group of pieces of pixel data, for example. For example, in a case where a color filter is provided for each pixel included in the sensor unit 1502, and each piece of pixel data has color information of any of red (R), green (G), or blue (B), the visual recognition processing unit 1506 executes demosaic processing, white balance processing, and the like. Furthermore, the visual recognition processing unit 1506 can instruct the sensor control unit 1503 to read pixel data necessary for the visual recognition processing from the sensor unit 1502. The visual recognition processing unit 1506 passes the image data obtained by processing the pixel data, to the output control unit 1507. For example, by an image signal processor executing a program stored in advance in a local memory (not illustrated), the above-described function of the visual recognition processing unit 1506 is implemented.
The output control unit 1507 includes, for example, a microprocessor. The output control unit 1507 receives image data as a result of the visual recognition processing from the visual recognition processing unit 1506, and executes processing for displaying the image data on the display unit 1508 or outputting the image data to the outside of the imaging device 1500. The user can visually recognize a display image on the display unit 1508. The display unit 1508 may be built in the imaging device 1500 or may be externally connected to the imaging device 1500.
Furthermore, a result of inference or prediction about the image data is passed from the inference unit 1504 to the output control unit 1507. In a case where the output control unit 1507 has the function of the “model bias determination unit 123” illustrated in
In the configuration example illustrated in
The image sensor 2002 includes, for example, an element such as a CMOS, and captures an image formed on an imaging surface by the lens 2001. The signal processing unit 2003 performs signal processing on RAW data output from the image sensor 2002. The signal processing performed in the signal processing 2003 corresponds to, for example, demosaicing, noise reduction, white balance adjustment, gamma correction, sensor spectral correction, YC conversion, and the like.
The inference unit 2004 corresponds to the “inference unit 111” illustrated in
The control unit 2005 integrally controls an operation of each unit in the in-vehicle camera 2000. The control unit 2005 controls, for example, an imaging operation in the image sensor 2002 and signal processing in the signal processing unit 1503. Furthermore, the control unit 2005 controls driving of the host vehicle through a control system 2010 in the subsequent stage on the basis of the object recognition result in the inference unit 2004 and the prediction result of an accident or the like.
In a case where the control unit 2005 has the function of the “model bias determination unit 123” illustrated in
The vehicle control system 2010 controls operation of the host vehicle, on the basis of an image captured by the image sensor 2002 and developed by the signal processing unit 2003 under the control of the control unit 2005 and on the basis of a recognition result or a prediction result obtained by the inference unit 2004. The vehicle control described herein includes vehicle control for automated driving or ADAS such as, for example, adaptive cruise control (ACC), lane departure warning (LDW), lane keeping assist (LKA), automatic emergency braking (AEB), and blind spot detection (BSD), and further includes drive control of each drive unit such as an active cornering light (ACL), a brake actuator (BRK), and a steering device (STR).
In this Section I, an application example in which the training system 100 according to the present disclosure is applied to name learning will be described. The “name learning” mentioned in the present specification is a task of detecting a face in content and identifying the name of the person with the face by a face recognizer by collation with a database. The machine learning model having learned names can be used as a name display system that displays the name of a person appearing in various contents such as a movie or a TV drama.
Here, accuracy of face recognition and accuracy of name display differ depending on a difference in sensitive attributes such as race (or skin color) and gender, and fairness becomes a problem. For example, if accuracy is lower in black people than in white people and is lower in women than in men (for example, a black woman's name appearing in the content tends to be mistaken), this leads to problems of racism and gender discrimination.
Therefore, in the present disclosure, in the model training stage, training is performed after the bias of the training data is mitigated, thereby ensuring fairness of the model.
First, the collected training data is classified for each sensitive attribute (step S2201), and it is determined whether the data is biased to a specific attribute value for each sensitive attribute, that is, whether there is data bias (step S2202).
Here, in a case where it is determined that there is data bias (Yes in step S2203), the data supplementation unit 122 automatically generates the data of the minor attribute of the sensitive attribute determined to have data bias by the GAN algorithm, mitigates the data bias (step S2204), and the model is trained (step S2205). Furthermore, in a case where it is determined that there is no data bias (No in step S2203), the model is trained without performing data bias mitigation (step S2205).
Furthermore, the model bias is corrected even after the trained model is mounted on a product or a service and released.
On the edge device side, the presence or absence of model bias is verified while the mounted trained model is being used. Then, when the model bias is confirmed, the server is notified (TR2301). In response to this, on the server side, a model parameter in which model bias has been mitigated by relearning, fine tuning, or the like is redistributed to the edge device (TR2302), and the trained model is updated. In this manner, in a case where the problem of the fairness of the trained model becomes apparent in the actual operation, the model bias can be corrected by cooperation between the server and the edge device.
Each time data is input, the edge device makes an inference or a prediction (in this case, face detection in the content and display of the name of each face) is performed) using the mounted trained model (step S2401).
Then, the edge device checks whether or not the result of inference or prediction by the trained model is biased according to the difference in the sensitive attributes of the input data, that is, whether or not there is model bias (step S2402). For example, it is verified whether accuracy of face recognition and accuracy of name display differ depending on a difference in sensitive attributes such as race (or skin color) and gender.
For example, if accuracy is lower in black people than in white people and accuracy of name identification is lower in women than in men (for example, a black woman's name appearing in the content tends to be mistaken), this leads to problems of racism and gender discrimination. In a case where it is determined that there is model bias (Yes in step S2403), the edge device notifies the server of the model bias (step S2404).
On the server side, when notification of model bias is received, the model bias of the trained model is mitigated by relearning, fine tuning, or the like, and the updated model parameter is redistributed to the edge device. Therefore, the edge device sets the model parameter updated on the server side to the model (step S2405), and resumes the operation of the model in a state where the model bias is mitigated. In this manner, in a case where the problem of the fairness of the trained model becomes apparent on the edge device side in the actual operation, the model bias can be corrected by cooperation with the server.
The present disclosure has been described in detail with reference to the specific embodiment. However, it is obvious that those skilled in the art can make modifications and substitutions of the embodiment without departing from the scope of the present disclosure.
The present disclosure can be applied to training of a machine learning model that executes processing such as face detection, face identification, person/object detection, and posture estimation, as well as various types of detection, determination, estimation, and recognition by receiving an image or sound, and bias determination of the trained model. According to the present disclosure, it is possible to train a model by preparing training data from which data bias due to a sensitive attribute such as race or gender has been eliminated, and further to perform bias mitigation processing by performing model bias determination processing on the trained model.
In short, the present disclosure has been described in an illustrative manner, and the contents disclosed in the present specification should not be interpreted in a limited manner. To determine the subject matter of the present disclosure, the claims should be taken into consideration.
Note that the present disclosure may also have the following configurations.
(1) An information processing device including:
(2) The information processing device according to (1), in which
(3) The information processing device according to any one of (1) and (2), in which
(4) The information processing device according to any one of (1) to (3), in which
(5) The information processing device according to (4) further including
(6) The information processing device according to any one of (1) to (5), in which
(7) The information processing device according to any one of (1) to (6) further including
(8) The information processing device according to any one of (1) to (7) further including
(9) The information processing device according to (8), in which
(10) The information processing device according to (9), in which
(11) The information processing device according to (10) further including
(12) An information processing method including:
(13) A computer program described in a computer-readable format so as to cause a computer to function as:
(14) An information processing device including:
(15) The information processing device according to (14) further including
(16) The information processing device according to (15), in which
(17) An information processing method including:
(18) A computer program described in a computer-readable format so as to cause a computer to function as:
| Number | Date | Country | Kind |
|---|---|---|---|
| 2022-052909 | Mar 2022 | JP | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2023/003208 | 2/1/2023 | WO |
