The present disclosure relates to a method and an apparatus for monitoring an AI module that forms part of a processing chain of a partly automated or automated vehicle driving function.
The use of AI modules for handling highly complex situations is inevitable when developing functions for partially automated or automated driving. To sustainably develop and deploy these functions, monitoring the execution of the AI modules is imperative. This applies both to the training phase of the AI modules, where one looks for exceptional cases, so-called corner cases, e.g., typical situations of limited correctness of the module output, and to productive use, where one has to perform redundancy and system monitoring to assess the trustworthiness of AI module decisions.
Currently, redundancy and system monitoring mainly relies on model-intrinsic confidences, e.g., the interpretation of softmax excitations, or on plausibility checks that use analytical methods. The identification of exceptional cases, e.g., corner cases, is based primarily on functional drops, e.g., cases in which an AI module or the corresponding function terminates, or target-actual comparisons.
Model-intrinsic confidences are subject to a training bias and can be highly inaccurate, meaning that adversarial examples, namely small changes in the data signal resulting in a change of model output, lead to misclassifications with high intrinsic confidence.
Furthermore, plausibility checks using analytical methods require high potency plausibility check functions. These functions are either extremely defensive, thus preventing high-frequency use of the AI modules, or they cannot be realistically constructed.
US 2016/0335536 A1 relates to a hierarchical neural network and a classifying learning method, and a discriminating method based on the hierarchical neural network. A hierarchical neural network apparatus therein comprises a weight learning unit for generating loosely coupled parts by forming couplings between partial nodes in the hierarchical network based on a verification matrix of an error-correcting code and for learning weights between the coupled nodes. Furthermore, the apparatus comprises the hierarchical neural network having an input layer, at least one intermediate layer and an output layer, wherein each of these layers has nodes, and a discrimination processor for solving classification problems or regression problems using the hierarchical neural network whose weights between the nodes are updated with the weights the weight learning unit has learned.
US 2016/0071024 A1 relates to a multimodal data analysis device including instructions that are embodied in one or more non-transitory machine-readable storage media, wherein the multimodal data analysis device effects, by means of a computer system having one or more computer devices, that at least two different modalities are used to access a quantity of time-variable instances of multimodal data, wherein each instance of the multimodal data has a different temporal component and, using a deep-learning architecture, algorithmically learns a feature representation of the temporal component of the multimodal data.
The publication by I. J. Goodfellow et. al.: “Generative Adversarial Nets,” NIPS 2014 (https://papers.nips.cc/paper/5423-generative-adversial-nets) describes a method for evaluating generative models using an adversarial process, wherein two models are trained simultaneously, namely a generating model G, which generates data of a data distribution, and a discriminating model D, which determines the probability that a data set to be discriminated is part of training data instead of data of the generating model. In other words, the method comprises two neural networks which perform a zero-sum game. One network, the generator, creates candidates and the second neural network, the discriminator, evaluates said candidates. Typically, the generator maps from a vector of latent variables to the desired result space. The goal of the generator consists in learning to generate results according to a certain distribution.
The discriminator, on the other hand, is trained to distinguish the results of the generator from the data based on the real, specified distribution. Thus, the objective function of the generator consists in producing results that the discriminator cannot distinguish. In this way, the generated distribution is to gradually become aligned with the real distribution.
Aspects of the present disclosure are to provide a method and an apparatus for improved monitoring of an AI module that forms part of a processing chain of a partially automated or automated driving function of a vehicle.
These aspects are achieved by the independent claims found below. Other embodiments of the present disclosure are the subject matter of the subclaims.
In some examples, a method is disclosed for monitoring the input data stream of an AI module that forms part of a processing chain of a partially automated or automated driving function of a vehicle by means of a monitoring module formed by a generative adversarial network including a generator and a discriminator, wherein the method has a training phase and an inference phase, and
The numerical value expressing the distance, which is a real number, is interpreted as the distance of the input data to the training data set, so that the situation underlying the real input data stream of the inference phase can be evaluated using this numerical value.
In other words, if, for example, the numerical value of a distance d is normalized to the interval [0, 1], a numerical value of the distance of d≤0.5 could mean that the input data are “real”, while a numerical value of d>0.5 means that the input data are, so to speak, “false,” i.e. they were not part of the training data set. The latter suggests an environmental situation that has not been trained or learned, such as a corner case. The definition of the value interval is not limited to the above-mentioned interval, rather, other intervals and assignments of the distance d to the respective interval are possible; however, it must be apparent from the distance d whether the currently evaluated situation is typical for the learned training data sets or not, i.e. whether the present case deviates from the learned training data sets.
Preferably, the generator uses a background data source to generate the false training data that are fed to the discriminator. The background data source can be formed, for example, by a suitable random number generator.
In some examples, a training loss with respect to the real training data may be determined from the distance, which training loss is used to train the generator and the discriminator of the monitoring module.
In some examples, an apparatus is disclosed for monitoring the input data stream of an AI module, wherein the apparatus is configured and designed for carrying out the method explained herein, comprising
In some examples, a method is disclosed for monitoring the input data stream and the output data stream of an AI module that forms part of a processing chain of a partially automated or automated driving function of a vehicle comprises three monitoring modules, each formed by a generative adversarial network including a generator and a discriminator, wherein the method has a training phase and an inference phase, and wherein, in the training phase
Therefore, in the inference phase, the first distance is used for evaluating the typicality of the real input data stream, for example, from an environmental sensor means, relative to the training data of the training phase; the second distance is used for evaluating the typicality of the output of the AI module relative to the output of the training; and the third distance is used for evaluating the typicality of the output of the AI module relative to seen ground truth from the training. With regard to the three distances, what was said about the distance in the first method applies here as well. In other words, the distances are a measure of whether the observed data stream is similar to or whether it deviates from the data streams during training. Thus, if the deviation is outside of a specification, it can be concluded that a situation has not been trained and must be responded to accordingly.
In some examples, the generators generate false training data using a respective background data source, and the data are fed to the discriminators of the respective monitoring modules.
In some examples, during the training phase, training losses with respect to the respective training data are determined from each of the three distances, and said training losses are used to train the generators and discriminators of the respective monitoring module.
In some examples, an apparatus is disclosed for monitoring the input data stream and the output data stream of an AI module that forms part of a processing chain of a partially automated or automated driving function of a vehicle, wherein the apparatus is configured and designed to carry out the method explained above, including
Aspects of the present disclosure are explained below with reference to the drawings. In the drawings:
Aspects of the present disclosure are directed to a generative-discriminative situation evaluation that provides a control mechanism for AI modules along the processing chain of an automated driving function.
A control mechanism may be implemented in the form of a monitoring unit for monitoring input and output data of a control module for partially automated or automated driving, which control module is implemented by an AI module.
This situation evaluation, which is to say the control mechanism, measures the data stream flowing into and out of an AI module and measures a distance of the reference data distribution with which the AI module was originally developed and trained.
The situation evaluation makes use of a preferably threefold generative-discriminative approach, which is developed during the development phase of the AI model and will be described in general terms below.
The development and monitored training of AI modules uses an iterative approach in which a module is presented with reference data and the resulting module output is compared to a so-called ground truth. A loss value, the loss, is calculated from the difference between the last two values and the module is adjusted in accordance with this loss.
The control unit for monitoring the AI module preferably includes three independent monitoring modules, the so-called generative-discriminative distance measurement modules, for measuring the distance to the training input, the distance to the training output and the distance to the training ground truth. Each of the three individual modules consists of a generator and a discriminator. The task of the generator is to create the most realistic data possible; namely, input, output, and ground truth. The task of the discriminator, on the other hand, is to distinguish between real data and generated data. Its output is consequently the learning of a measure for distinguishing between typical and atypical input and output data. This measure is then interpreted and used as a distance to the real data.
The discriminator of a distance measurement module is an AI module whose training is implemented during the training of the actual AI module of the driving function. The data used in and resulting from the training, namely input, output or ground truth, are used as training data for the respective discriminator. Additional training data for the discriminator are provided by the generator. The generator, in turn, makes use of a background data source, which is called latent space, and generates false training data therefrom.
It can itself also be an AI module, such as a GAN approach from the machine learning field, but also a simulation or an image search on the Internet.
At the time of execution, which is called inference, the control unit according to the present disclosure uses only the discriminators. At runtime, they then evaluate and monitor the distance of the incoming and outgoing data stream in the actual AI module of the driving function to the reference data set.
In the interest of computing efficiency or in cases where AI modules of the driving function are extremely well trained, it is possible to omit one of the two distance measurement modules for monitoring the output flow.
For the best possible monitoring of the correct functionality of AI modules, relationships between the typicality of a current module output with respect to reference ground truth data, on the one hand, and reference training outputs, on the other hand, should also be considered. This can potentially provide insight into a generalization capability of the AI module.
The distance measurement module, which is responsible for the input data stream of the AI module, can also be operated by itself. In other words, in its simplest embodiment, the control unit comprises only the distance measurement module for the input data stream; but this results in reduced performance.
The comprehensive approach of the control device according to the present disclosure with at least two distance measurement modules allows for monitoring relationships between incoming and outgoing data streams. Since the training of the individual modules can be carried out in parallel with the training of the actual AI module of the driving function without incurring any significant additional technical effort, this option represents a considerable savings potential compared to currently known solutions.
If the AI module KI is an object recognition module, the AI module KI is to recognize and determine, based on the received signals IN, the objects in the environment of the vehicle, so that such objects, their spatial arrangement, and the type of objects, for example, vehicle, pedestrian, or cyclist, are output as the output OUT of the AI module, wherein the type of objects represents a probability statement. This output OUT can then be fed to a scene recognition and scene prediction (not shown) so that, ultimately, an automated driving function (not shown) can be controlled.
The AI module KI may also be a lane recognition module that determines, based on the signals IN of the environment sensors, the lanes of the roadway on which the vehicle is located as its output OUT, whereby, when merging these results with the results of an object recognition, it is possible to determine which object is located in which roadway. The list of uses of AI modules in automated driving provided here should be regarded only as exemplary and not as complete.
In the application of
In addition to the AI module KI, the illustrated control unit ST with AI module KI comprises three generative-discriminative distance measurement modules, namely the module M_IN for determining a distance to a training input, the module M_OUT for determining a distance to a training output, and the module M_GT for determining a distance to a training ground truth, wherein the modules will be explained in detail below.
The distance measurement module M_IN comprises a generator G_IN which generates false input training data that are as realistic as possible, wherein the generator G_IN makes use of a background data source L_IN, the so-called latent space, for generating the false training data. Further, the module M_IN comprises a discriminator G_IN which compares the false training data generated by the generator G_IN with real training data TD, and outputs a distance Dist_1N as an output of the distance measurement module M_IN, wherein the distance Dist_1N represents the distance of the false training data to the real training data, i.e. it is a measure of the expected affiliation of the current false data unit relative to the quantity of generated false data. A function called training loss TL_IN is determined based on the distance Dist_1N and the corresponding data, and said function is used for training the distance measurement module M_IN with the generator G_IN and the discriminator D_IN.
Since it is not sufficient to consider only input data, for example based on the environmental sensor means, for evaluating the behavior and functionality of an AI module KI of a partially automated or automated driving function, such as, e.g., a parking assistant or the like, the output and the so-called ground truth data of the AI module KI are also monitored.
Thus, the training data TD are also fed to the AI module KI that generates an output OUT therefrom, which, for example, is responsible for controlling a driving function. This output OUT of the AI module KI is fed to the discriminator D_OUT of a second generative-discriminative distance measurement module M_OUT. The second module M_OUT comprises a generator G_OUT which uses another background data source L_OUT for generating false training data, which are fed to the discriminator D_OUT. The discriminator D_OUT generates a distance Dist_OUT that is based on the real output data OUT of the AI module KI and the false training data generated by the generator G_OUT, wherein the distance Dist_OUT represents the distance of the false training data to the real output OUT of the AI module KI, i.e. it is a measure of the expected affiliation of the current false data unit relative to the quantity of generated false data. A function called training loss TL_OUT with respect to the output OUT of the AI module KI is determined on the basis of the distance Dist_OUT and the corresponding data, and this function is used for training the distance measurement module M_OUT with the generator G_OUT and the discriminator D_OUT.
To train the AI module KI itself, the output OUT is combined with ground truth data GT resulting in a loss function TL which can be used for training the AI module, wherein TL stands for “training loss”.
The ground truth data GT are fed to the discriminator D_GT of a third generative-discriminative distance measurement module M_GT. The third module M_GT comprises a generator G_GT which uses a third background data source L_GT for generating false training data which are fed to the discriminator D_GT. The discriminator D_GT generates a distance Dist_GT on the basis of the real ground truth data GT and the false training data generated by the generator G_GT, wherein the distance Dist_GT represents the distance of the false training data to real ground truth data GT, i.e. it is a measure of the expected affiliation of the current false data unit relative to the quantity of generated false data. A loss function TL_GT with respect to the ground truth data GT is determined based on the distance Dist_GT and the corresponding data, and this loss function is used for training the distance measurement module M_GT with the generator G_GT and the discriminator D_GT.
The input signals, for example from environmental sensor means, are fed as real input IN both to the AI module KI for processing and to the discriminator D_IN of the module M_IN, which is responsible for evaluating the input signals. The discriminator D_IN determines based on the input signal IN a first distance Dist_1N with respect to the input signals IN. The module output OUT is determined by the AI module KI on the basis of the input signal, and the module output OUT is further processed within the processing chain of the driving function, as symbolized by the arrow, and it is fed both to the discriminator D_OUT of the second distance measurement module M_OUT with respect to the output signal OUT and to the third distance measurement module M_GT with respect to the ground truth GT. In this way, distances Dist_OUT with respect to the output signal OUT of the AI module KI and Dist_GT with respect to the ground truth GT as shown in
The distances Dist_1N, Dist_OUT and Dist_GT generated by the three distance measurement modules M_IN, M_OUT and M_GT monitor the data streams IN and OUT flowing into and out of the AI module KI and therefore provide information about the behavior of the AI module, in particular, if the data stream input IN is a non-trained situation, so that the output OUT of the AI module also does not correspond to a trained situation, which is reflected in the distances Dist_OUT and Dist_GT of the two discriminators D_OUT and D_GT. In this way, it is possible, for example, during operation of the control unit for monitoring an AI module KI of a driving function, to determine so-called corner cases, i.e. borderline cases, which were not taken into account in the simulation, i.e. the production of corresponding training data, since, as is well known, it is hardly possible to take into account all possible situations with respect to a partially automated or automated driving function.
The monitoring unit ÜW monitors the input data stream IN into the AI module KI and the output data stream OUT from the AI module KI, wherein the monitoring unit ÜW is formed by the three discriminators D_IN, D_OUT and D_GT of the distance measurement modules M_IN, M_OUT and M_GT described in
For each of the three distances Dist_1N, Dist_OUT and Dist_GT measured by the discriminators D_IN, D_OUT, D_GT, a respective threshold S_IN, S_OUT and S_GT can now be defined with the following specifications:
Dist_1N≤S_IN: Input data stream IN “real” in the sense of known, i.e. trained,
Dist_1N>S_IN: Input data stream IN “unknown” in the sense of not trained,
Dist_OUT≤S_OUT: Output data stream OUT “real”,
Dist_OUT>S_OUT: Output data stream OUT “unknown”,
Dist_GT≤OUT: Output data stream OUT “real”,
Dist_GT>OUT: Output data stream OUT “unknown”.
Number | Date | Country | Kind |
---|---|---|---|
10 2019 206 720.4 | May 2019 | DE | national |
The present application claims priority to International Patent App. No. PCT/EP2020/060593, to Schlicht et al., titled “Monitoring of An AI Module of a Vehicle Driving Function”, filed Apr. 15, 2020, which claims priority to German Patent App. No 10 2019 206 720.4, filed May 9, 2019, the contents of each being incorporated by reference in their entirety herein.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/EP2020/060593 | 4/15/2020 | WO |