Patent Application
20240160195
The following relates to an assistance apparatus and a method for adaptive data valuation.
Nowadays, in industrial manufacturing operation monitoring and quality monitoring are performed by data-driven applications, like anomaly detection or classification of quality, based on machine learning models. There exists a high diversity of manufacturing scenarios, e.g., different machine types, numerical programs (NC), manufacturing processes and production sites to name a few. Furthermore, the product portfolio of most manufacturers is constantly changing, and so is the recorded data during the product manufacturing. This creates a so called cold-start situation, i.e., each time a new manufacturing scenario, e.g., new machine, product or manufacturing process, is encountered, all data driven applications that are related to this scenario have to be developed from scratch.
The cold-start problem faced when developing data-driven applications for specific manufacturing scenario is not solved in industry. The applications are developed beginning by a usually time- and cost-intensive collection of big amounts of data. Subsequently, the whole collected dataset or random samples from it is annotated by usually scarce and costly domain experts. Finally, a machine learning model is trained with the annotated data. If the desired performance is not reached, more data has to be collected and randomly chosen samples have to be annotated. All the datapoints or randomly chosen subsets, even duplicates and non-useful data examples, are labeled due to the absence a method that assigns value to unlabeled datapoints.
To avoid this tedious process of developing data-driven applications, some problems are addressed with less accurate methods. For example, in the context of anomaly detection in industrial manufacturing, the quality monitoring is mostly performed by humans, either machinists or domain experts. However, due to the high number of produced products/workpieces assessing the quality of all of them is impossible. Hence, only randomly chosen batches are controlled, which does not guarantee a high coverage of the quality inspection.
EP 3705962 A1 discloses a method that leverages labeled data from different manufacturing situations to develop a machine learning model, e.g., for anomaly detection or for other classification applications, that can quickly adapt to newly encountered tasks. Data recorded while performing other previous manufacturing processes are used to train a highly adaptive model. So, if labeled data from several manufacturing scenarios are available, a model can be trained that is highly adaptable to a variety of unseen manufacturing scenarios. This would relax the cold-start situation as only few data samples are required from the new unseen manufacturing scenario and, hence, accelerate the development of data-driven applications for the new unseen, i.e., supplemented manufacturing scenario.
These methods, i.e., learning with few data samples, have two drawbacks. First, in most cases only few labeled data are available, which are sampled randomly. The few data samples of a new unseen manufacturing scenario used to customize the adaptive learning model or even more used to customize an untrained learning model lead to a moderate performance of the model. Second, after having customized the model to the new task, the learning model does not improve anymore since it is not further trained. For example, the performance of a classifying learning model would drop massively if a distribution of the data recorded during manufacturing would change, e.g., due to tooling overuse.
The article of KUNKUN PANG ET AL: “Meta-Learning Transferable Active Learning Policies by Deep Reinforcement Learning”, ARXIV.org, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY14853, 13 Jun. 2018 (2018 Jun. 13), XP080890046 discloses modelling an active learning algorithm as a deep neural network that inputs a base learner state and an unlabeled point set and predicts the best point to annotate next. Training this active query policy network with reinforcement learning, produces a non-myopic policy for a given dataset. A multi-task dataset-embedding approach is described that allows dataset-agnostic active learners to be trained.
An aspect relates to a monitoring apparatus and method that enables a faster customizing of a learning model to a new unseen manufacturing process providing higher performance, and which can continuously be improved.
A first aspect concerns a monitoring apparatus for quality monitoring of a manufacturing process supplemented to a set of predefined manufacturing processes of industrial manufacturing scenario, comprising at least one processor configured to perform:
The monitoring apparatus provides a valuation of the unlabeled data samples and concludes thereof a selection of data samples which are most likely relevant for the supplemented manufacturing process. The valuation is performed by the context embedding unit and the value estimation unit which assigns a value to each unlabeled data sample that is recorded during the supplemented manufacturing process. If the data sample is predicted to be valuable, e.g., is very different from the data samples usually recorded, it is saved and shown to the domain experts for labelling. The resulting labeled data samples are then used to further train the prediction unit for that manufacturing process. This pre-selection of valuable data samples from a large amount of unlabeled data samples reduces the effort of labelling and leads to a high performance of the trained version of the prediction unit in short time.
Labeling or labeled data is used as synonym for annotating or annotated data in this description. Data samples are sensor data recorded over time by sensors measuring physical parameters, e.g., torque, rotation speed, or image data from the work piece captured by a camera during the manufacturing process.
In an embodiment the monitoring apparatus is configured to adapt the value estimation unit based on a validation loss indicator calculated by the trained version of the prediction unit when processed with at least one labeled validation data sample validation loss indicator as input.
This enables a continuous improvement of the value estimation unit. The validation loss indicator can be used by the value estimator to determine how trained the prediction model is. This is based on the intuition that, there are probably data samples that are more useful at the beginning of training and others that are more valuable towards the end.
In an embodiment the monitoring apparatus is configured to adapt the context embedding unit based on the loss indicator and the adapted value estimation unit.
This enables a continuous improvement of the context embedding unit.
In an embodiment the monitoring apparatus is configured such that the context embedding unit and/or the value estimation unit are pre-trained based on at least one set of unlabeled training data samples of each of the predefined manufacturing processes.
This pre-trained context embedding unit and/or the value estimation unit constitutes a task-agnostic data valuation model. More precisely, the pre-trained context embedding unit and/or the value estimation unit leverages labeled data from previously seen, i.e., predefined manufacturing scenarios to yield an active learning system that can generalize to unseen supplemented manufacturing processes, i.e., estimate the value of their data unlabeled examples.
In a further embodiment the monitoring apparatus is configured such that the pre-trained context embedding unit and/or the pre-trained value estimation unit are trained before applying to the supplemented manufacturing process by
The resulting optimized pre-trained value estimation unit and context embedding unit represent a task-agnostic data valuation model that leverages data samples from previously seen, i.e., predefined manufacturing scenarios to yield an active learning system that can generalize to unseen, supplemented manufacturing scenarios, i.e., estimate the value of their unlabeled data samples.
In an embodiment the monitoring apparatus is configured such that the context embedding unit, the value estimation unit and prediction unit are configured as a learning model.
Machine learning involves computers learning from data provided so that they carry out certain tasks. Machine learning algorithms build a learning model based on collected data samples, known as “training data”, in order to make predictions or decisions without being explicitly programmed to do so. The learning models can be structures as artificial neural network or in short neural network. Here each of the context embedding unit, the value estimation unit and the prediction units are separate learning models optimized or trained by data samples either from the predefined or the supplemented manufacturing process.
In a further embodiment the monitoring apparatus is configured such that the context embedding unit is configured as a recurrent neural network aggregating the information of all the data samples in the set into a vector that encodes the manufacturing process.
A recurrent neural network is a class of artificial neural networks where connections between nodes form a directed graph along a temporal sequence. This allows it to exhibit temporal dynamic behaviour. It shows best performance identifying an association between all data sample of the manufacturing processes.
In a further embodiment the monitoring apparatus is configured such optimizing of the value estimation unit and the context embedding unit is performed by stochastic gradient descent rule.
According to a further embodiment the monitoring apparatus is configured such that the initial version of a prediction unit is a randomly initialized prediction unit or a pre-trained prediction unit.
Using randomly initialized prediction units provides no or low requirements with respect to training data samples but requires more iterations to provide high performance in the monitoring result compared to the pre-trained prediction unit.
According to a further embodiment the monitoring apparatus is configured such the labelling unit comprises a user interface to receive labels from a human user and/or comprises a machine-machine-interface to receive labels from a labelling machine.
This allows domain experts or operation personnel of the manufacturing process to input labels for the requested data samples. Further on, labels can be input by a labelling machine providing labels semi or fully automatically. The monitoring apparatus can be configured to support both, human based and machine-based label input.
According to a further embodiment the monitoring apparatus is configured such the manufacturing processes are milling processes and the data of the supplemented manufacturing process are sensor data representing the milling process, especially a torques of the various axes in a milling machine, control deviations of the torque, or image data of the milled workpiece.
A second aspect concerns a method for quality monitoring of a supplemented manufacturing process to a set of predefined manufacturing processes of industrial manufacturing scenario, comprising:
The expression “a set of” input data samples is to be understood as high number of data samples, at least more than one sample. In an embodiment, the method assigns a value to each unlabeled data sample that is recorded during manufacturing. If a data sample is predicted to be valuable, e.g., is very different from the data samples usually recorded, it is saved and forwarded e.g., to the domain experts for labelling. The labeled samples are then used to train the prediction model for that manufacturing scenario.
A third aspect concerns a computer program product (non-transitory computer readable storage medium having instructions, which when executed by a processor, perform actions) directly loadable into the internal memory of a digital computer, comprising software code portions for performing the steps as described before, when said product is run on said digital computer.
Some of the embodiments will be described in detail, with reference to the following figures, wherein like designations denote like members, wherein:
It is noted that in the following detailed description of embodiments, the accompanying drawings are only schematic, and the illustrated elements are not necessarily shown to scale. Rather, the drawings are intended to illustrate functions and the co-operation of components. Here, it is to be understood that any connection or coupling of functional blocks, devices, components or other physical or functional elements could also be implemented by an indirect connection or coupling, e.g., via one or more intermediate elements. A connection or a coupling of elements or components or nodes can for example be implemented by a wire-based, a wireless connection and/or a combination of a wire-based and a wireless connection. Functional units can be implemented by dedicated hardware, e.g., processor, firmware or by software, and/or by a combination of dedicated hardware and firmware and software. It is further noted that each functional unit described for an apparatus can perform a functional step of the related method.
Quality monitoring addresses an anomaly detection of a monitored manufacturing process or a classification solution in general regarding the quality of a monitored manufactured product. The anomaly detection or classification solution is provided by inputting data samples into a unit configured as learning model which is specifically trained for the monitored process in industrial manufacturing, e.g., in an automation plant.
The monitoring apparatus 10 comprises one or several processors configured as a data provision unit 11, a context embedding unit 12, a value estimation unit 13, a labelling unit 14 and a prediction unit 15. The data provision unit 11 obtains a set of unlabeled input data samples recorded at the supplemented manufacturing process executed by a manufacturing device 18. The value estimation unit 13, a labelling unit 14 and a prediction unit 15 are configured to provide a machine learning model, e.g., by an artificial neural network. The value estimation unit 13 is configured as learning model, e.g., a neural network structured as a multi-layer perceptron. The context embedding unit 12 is configured as learning model, as a recurrent neural network, e.g., structured as a long short-term memory.
The labelling unit 14 is configured to receive unlabeled input data samples and to select a subset of data samples out of the set of unlabeled input data samples. It is further configured to output a signal requesting a label associated to each data sample of the subset of data samples being input by a human or a machine. The labelling unit 14 comprises a user interface to receive labels from a human user and/or comprises a machine-machine-interface to receive labels from a labelling machine.
In industrial manufacturing, input data samples of a supplemented manufacturing process are usually recorded using a reasonable set of sensors, including cameras, which measures for example the torques of the various axes in a milling machine, control deviations of the torque, or image data of the milled workpiece.
In an embodiment, the method steps performed by the monitoring apparatus 10 are described in more detail by the schematic information flow diagram of
A set of unlabeled input data samples Ds and a set of labeled validation data samples Dval recorded at the supplemented manufacturing process are obtained for each supplemented manufacturing process by the data provision unit 11. As a first step an initial version of a prediction unit 15 is generated. The initial version of a prediction unit 15 can be a randomly initialized learning model, i.e., parameters of the prediction learning model are initialized with random numbers. Optionally, the initial version of a prediction unit 15 can be pre-trained, e.g., by data samples of the predefined manufacturing processes. A loss indicator L of the initial version of the prediction unit 15 is calculated based on the validation data samples Dval.
An embedding indicator Es indicating an association between the data samples of the supplemented manufacturing process is aggregated by feeding the set of unlabeled input data samples Ds into a context embedding unit 12. The context embedding unit 12 aggregates the information of all the data samples in the set into a vector that encodes the manufacturing process.
The same set of unlabeled input data samples Ds, the loss indicator L and the embedding indicator Es is input into the value estimation unit 13. The value estimation unit 13 aggregates a value indicator Vs for each data sample of the set of unlabeled input data samples Ds.
The labelling unit 14 selects a subset SDs of data samples of the set of unlabeled input data samples Ds depending on the value indicator Vs and outputs a labelling request. On receipt of labels As for the subset SDs of data samples Ds, the labelling unit 14 forwards the labelled subset of input data samples SDs, As to the prediction unit 15. A current version 16, here the initial version, of the prediction unit 15 is trained based on the labelled subset SDs, in several iterations resulting in a trained version of the prediction unit 17. The trained version of the prediction unit 17 outputs a monitoring result Cs indicating the quality of the supplemented manufacturing to control the manufacturing process for the set of unlabeled input data samples Ds.
The context embedding unit 12 and the adapted value estimation unit 13 can be further adapted and optimized based on the loss indicator Ls. This leads to an improvement of each of the context embedding unit 12, the valuation estimation unit 13 and the prediction unit 15 with each obtained and processed set of input data samples.
The steps of the monitoring method using provided in pseudo code are as follows:
In an embodiment, the context embedding unit 12 and/or the value estimation unit 13 are pre-trained based on unlabeled input data samples of a set of predefined manufacturing processes. Pre-trained means, that the context embedding unit 12 and/or the value estimation unit 13 are trained before the set of data samples Ds of the supplemented manufacturing process are processed in the monitoring apparatus 10.
The data samples Ds have to be annotated and pre-processed with the same pre-processing steps applied to the data samples used to train the value estimation unit 13 and context embedding unit 12.
Throughout this description the wording “training a unit” means “training the learning model of the respective unit”.
The data value estimation unit 23 and the context embedding unit 22 are trained jointly with an iterative optimization procedure. The training is based on a set of data samples Di for each of n predefined manufacturing processes Pi.
For each predefined manufacturing process Pi, a set of unlabeled input data samples Dtri is obtained and fed into the context embedding unit 22. The context embedding unit 22 is configured, e.g., to be a recurrent neural network in order to aggregate the information of all the data samples Dtri in the set into an embedding indicator Ei, which is a vector that encodes the manufacturing process Pi. The value estimation unit 23 gets the same set of input data samples Dtri as input, along with the embedding indicator Ei from the context embedding unit 22 and a loss indicator L of the current version of the prediction unit 26 generated based on the set of validation data samples Dvi. The loss indicator L can be used by the value estimation unit 23 to determine how trained the prediction unit 23 is. This is based on the intuition that, there are probably data samples that are more useful at the beginning of training and others that are more valuable towards the end.
The value estimation unit 23 outputs a value indicator Vi for each of the unlabeled data samples in the set, which are then used as sampling probabilities to select a subset SDtri of data samples to be labelled. The subset SDtri of data samples comprises usually less data samples then the set of input data samples Dtri. The selected data samples are then labelled SDtri, Ai and used to train the current version of the prediction unit 26 with respect to a resulting classifier of the considered manufacturing process for a few iterations resulting in a trained version of the prediction unit 27. The labelling is performed as described for the processing a supplemented manufacturing process, see
The finetuned trained version of the prediction unit 27 is then evaluated on the set of validation data samples Dvi. A difference indicator Erri, which is a difference between the resulting validation loss Li and a moving average of previous validation losses Li,ma, is used as an optimization objective, i.e., the current validation loss Li should be lower that the moving average Li,ma. Computed loss gradients, i.e., a gradient of the loss Li with respect to some parameters, e.g., parameters of the data valuation unit 23, are backpropagated to the context embedding unit 22 through the value estimation unit 23 and the finetuning updates of the classifier, i.e., the trained version of the prediction unit 27.
This is done for each predefined manufacturing sample used in the current iteration and average loss gradients are used to update the parameters of the value estimation unit 23 and context embedding unit 22.
By training the value estimation unit 23 and context embedding unit 22 on different tasks, i.e., manufacturing processes they progress towards getting task-agnostic, i.e., they learn patterns that are applicable to several, also unseen tasks from that task distribution (in our case, new but similar manufacturing machines, processes or products.
An example for an implementation of the pre-training of the value estimation unit 23 and context embedding unit 22 is illustrated in pseudocode below.
In contrast to approaches, where few and randomly sampled data samples are used to adapt an adaptive learning model, the current approach trains a model, i.e., the monitoring apparatus, that selects the most useful data samples to train a prediction model to the new task (e.g., new manufacturing scenario). This provides a performance advantage of the initial classifier, i.e., the monitoring result at the beginning of manufacturing.
In an embodiment, the method proposed identifies interesting data examples, e.g., less frequent occurring data samples or borderline data samples, when they are recorded and asks to label them. These are then used to further train the prediction model. This provides a performance advantage during manufacturing.
It is to be understood that the above description of examples is intended to be illustrative and that the illustrated components are susceptible to various modifications. For example, the illustrated concepts could be applied for different technical systems and especially for different sub-types of the respective technical system with only minor adaptions.
Although the present invention has been disclosed in the form of embodiments and variations thereon, it will be understood that numerous additional modifications and variations could be made thereto without departing from the scope of the invention.
For the sake of clarity, it is to be understood that the use of “a” or “an” throughout this application does not exclude a plurality, and “comprising” does not exclude other steps or elements.
| Number | Date | Country | Kind |
|---|---|---|---|
| 21163106.4 | Mar 2021 | EP | regional |
This application claims priority to PCT Application No. PCT/EP2022/055365, having a filing date of Mar. 3, 2022, which claims priority to EP Application No. 21163106.4, having a filing date of Mar. 17, 2021, the entire contents all of which are hereby incorporated by reference.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/EP2022/055365 | 3/3/2022 | WO |
