This disclosure relates generally to predictive maintenance, and more particularly to machine learning systems with sequence modeling for predictive monitoring of industrial machines and/or product parts.
An industrial manufacturing process may include a number of workstations with industrial machines, which are employed in a particular order to produce a particular product. For example, such industrial manufacturing processes are typically used in assembly plants. Unfortunately, there may be instances in which one or more industrial machines may fail to perform at satisfactory levels or may fail completely. Such machine failures may result in low grade products, incomplete products, and/or disruptions in the industrial manufacturing process, as well as major losses in resources, time, etc.
The following is a summary of certain embodiments described in detail below. The described aspects are presented merely to provide the reader with a brief summary of these certain embodiments and the description of these aspects is not intended to limit the scope of this disclosure. Indeed, this disclosure may encompass a variety of aspects that may not be explicitly set forth below.
According to at least one aspect, a computer-implemented method relates to predictive measurement monitoring. The method includes establishing a station sequence that includes a plurality of stations that a given part traverses. The method includes receiving, via a first neural network, observed measurement data regarding attributes of the given part as obtained by one or more sensors at each station of a station subsequence of the station sequence. The method includes generating, via the first neural network, a set of parameter data based on the observed measurement data. The set of parameter data being associated with a latent variable subsequence. The latent variable subsequence corresponds to the station subsequence. The method includes receiving, via a second neural network, history measurement data of another part that was processed before the given part. The history measurement data relates to attributes of the another part that are taken with respect to each station of the plurality of stations of the station sequence. The method includes generating, via the second neural network, next parameter data based on the history measurement data while using the set of parameter data. The next parameter data is associated with a next latent variable that follows the latent variable subsequence. The next latent variable corresponding to a next station that follows the station subsequence in the station sequence. The method includes generating, via the second neural network, predicted measurement data of the given part at the next station based on the next latent variable and the next parameter data.
According to at least one aspect, a system includes a processor and a memory. The memory is in data communication with the processor. The memory has computer readable data including instructions stored thereon that, when executed by the processor, cause the processor to perform a method for predictive measurement monitoring. The method includes establishing a station sequence that includes a plurality of stations that a given part traverses. The method includes receiving, via a first neural network, observed measurement data regarding attributes of the given part as obtained by one or more sensors at each station of a station subsequence of the station sequence. The method includes generating, via the first neural network, a set of parameter data based on the observed measurement data. The set of parameter data being associated with a latent variable subsequence. The latent variable subsequence corresponds to the station subsequence. The method includes receiving, via a second neural network, history measurement data of another part that was processed before the given part. The history measurement data relates to attributes of the another part that are taken with respect to each station of the plurality of stations of the station sequence. The method includes generating, via the second neural network, next parameter data based on the history measurement data while using the set of parameter data. The next parameter data is associated with a next latent variable that follows the latent variable subsequence. The next latent variable corresponding to a next station that follows the station subsequence in the station sequence. The method includes generating, via the second neural network, predicted measurement data of the given part at the next station based on the next latent variable and the next parameter data.
According to at least one aspect, a non-transitory computer readable medium having computer readable data including instructions stored thereon that, when executed by a processor, cause the processor to perform a method for predictive measurement monitoring. The method includes establishing a station sequence that includes a plurality of stations that a given part traverses. The method includes receiving, via a first neural network, observed measurement data regarding attributes of the given part as obtained by one or more sensors at each station of a station subsequence of the station sequence. The method includes generating, via the first neural network, a set of parameter data based on the observed measurement data. The set of parameter data being associated with a latent variable subsequence. The latent variable subsequence corresponds to the station subsequence. The method includes receiving, via a second neural network, history measurement data of another part that was processed before the given part. The history measurement data relates to attributes of the another part that are taken with respect to each station of the plurality of stations of the station sequence. The method includes generating, via the second neural network, next parameter data based on the history measurement data while using the set of parameter data. The next parameter data is associated with a next latent variable that follows the latent variable subsequence. The next latent variable corresponding to a next station that follows the station subsequence in the station sequence. The method includes generating, via the second neural network, predicted measurement data of the given part at the next station based on the next latent variable and the next parameter data.
These and other features, aspects, and advantages of the present invention are discussed in the following detailed description in accordance with the accompanying drawings throughout which like characters represent similar or like parts.
The embodiments described herein, which have been shown and described by way of example, and many of their advantages will be understood by the foregoing description, and it will be apparent that various changes can be made in the form, construction, and arrangement of the components without departing from the disclosed subject matter or without sacrificing one or more of its advantages. Indeed, the described forms of these embodiments are merely explanatory. These embodiments are susceptible to various modifications and alternative forms, and the following claims are intended to encompass and include such changes and not be limited to the particular forms disclosed, but rather to cover all modifications, equivalents, and alternatives falling with the spirit and scope of this disclosure.
The system 100 includes at least a processing system 110 with at least one processing device. For example, the processing system 110 includes at least an electronic processor, a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), any suitable processing technology, or any number and combination thereof. The processing system 110 is operable to provide the functionality as described herein.
The system 100 includes a memory system 120, which is operatively connected to the processing system 110. In an example embodiment, the memory system 120 includes at least one non-transitory computer readable storage medium, which is configured to store and provide access to various data to enable at least the processing system 110 to perform the operations and functionality, as disclosed herein. In an example embodiment, the memory system 120 comprises a single memory device or a plurality of memory devices. The memory system 120 can include electrical, electronic, magnetic, optical, semiconductor, electromagnetic, or any suitable storage technology that is operable with the system 100. For instance, in an example embodiment, the memory system 120 includes random access memory (RAM), read only memory (ROM), flash memory, a disk drive, a memory card, an optical storage device, a magnetic storage device, a memory module, any suitable type of memory device, or any number and combination thereof. With respect to the processing system 110 and/or other components of the system 100, the memory system 120 is local, remote, or a combination thereof (e.g., partly local and partly remote). For example, the memory system 120 can include at least a cloud-based storage system (e.g. cloud-based database system), which is remote from the processing system 110 and/or other components of the system 100.
The memory system 120 includes at least a predictive measurement program 130, a machine learning system 140, machine learning data 150, and other relevant data 160, which are stored thereon. The predictive measurement program 130 includes computer readable data with instructions, which, when executed by the processing system 110, is configured to train and/or employ the machine learning system 140 to learn to generate future measurement data, which also may be referred to as predicted measurement data. The computer readable data can include instructions, code, routines, various related data, any software technology, or any number and combination thereof. In an example embodiment, the machine learning system 140 includes a deep Markov-based model. Also, the machine learning data 150 includes various data relating to the machine learning system 140. The machine learning data 150 includes various data associated with training and/or employing the machine learning system 140. For instance, the machine learning data 150 may include training data, various parameter data, various loss data, etc. Meanwhile, the other relevant data 160 provides various data (e.g. operating system, etc.), which enables the system 100 to perform the functions as discussed herein.
The system 100 is configured to include one or more sensor systems 170. The sensor system 170 includes one or more sensors. For example, the sensor system 170 may include an image sensor, a camera, a radar sensor, a light detection and ranging (LIDAR) sensor, a structured light sensor, a thermal sensor, a depth sensor, an ultrasonic sensor, an infrared sensor, a motion sensor, an audio sensor (e.g., microphone), a weight sensor, a pressure sensor, any applicable sensor, or any number and combination thereof. The sensor system 170 is operable to communicate with one or more other components (e.g., processing system 110 and memory system 120) of the system 100. For example, upon receiving sensor data from a sensor system 170, the sensor system 170 and/or the processing system 110 may generate sensor-fusion data. If needed, the processing system 110 may perform one or more data preparation operations to the sensor data and/or sensor-fusion data to provide input data (e.g., observed measurement data) of suitable form (e.g., numerical data) for the machine learning system 140. The sensor system 170 is local, remote, or a combination thereof (e.g., partly local and partly remote). The sensor system 170 may include one or more sensors at one or more of the stations of a given station sequence that a given part traverses. Additionally or alternatively, there may be one or more sensor systems 170 at each station of the station sequence that a given part traverses. Upon receiving the sensor data, the processing system 110 is configured to process this sensor data in connection with the predictive measurement program 130, the machine learning system 140, the machine learning data 150, the other relevant data 160, or any number and combination thereof.
In addition, the system 100 may include at least one other system component. For example, as shown in
In
As shown in
For the machine learning system 140, the system 100 arranges various data as a collection of part-view trajectories/paths (τk). Each path τk is a sequence of sparse multimodal structural measurements collected at a particular station over time for a specific part alongside the history measurements at that station, i.e., τ=((x1, h1), . . . , (xt
The measurements and/or the measurement data may be a binary value, a strength value, a time series value (e.g., a measurement of the response to pressure), floating precision number, number string, integer, Boolean, aggregation of statistics, or the like which provides attribute information of the part. The measurement data may be based on raw sensor data from one or more sensors at a station, sensor-fusion data from sensors at a station, or any number and combination thereof. The raw sensor data may include image data, video data, audio data, text data, alphanumeric data, or any number and combination thereof. The processing system 110 is configured to perform one or more data preparation operations on the raw sensor data to provide measurement data as input to the machine learning system 140.
The machine learning system 140 is to learn a probability distribution of future measurements given past measurements to be then used for prediction/estimation of the future measurement values. Assuming first-order Markovian dependency among part-view measurements, the joint probability of measurements, given history measurements at each station. ((x1,h1), . . . , (xt,ht), . . . , (xt
Also, to learn the probability distribution over the measurement variables, the system 100 defines a low-dimensional representation of the measurement data that depends on the history measurement data and observed measurement data of the given part. The representations to be learned are defined as a set of latent variables zt. More specifically, in
To predict a future measurement for a particular part at a given station, the machine learning system 140 uses the part's measurements at previous stations as well as the history of measurements that have been performed by one or more other parts at least at that given station. More specifically, the machine learning system 140 handles input data relating to a set of part-view sequences. A part-view sequence is representative of a part traversing along a trajectory of stations ∈{1, . . . , K}. The input data includes a list of D dimensional measurements [x1, . . . , xt
For the next time point prediction, the processing system 110 evaluates the probability distribution expressed below in equation 3. In equation 3, the three distributions on the right-hand side are parameterized and learned by neural networks, and the integral is estimated by Monte Carlo samples. Each of the three distributions on the right-hand side of equation 3 is a Gaussian distribution, where its mean and variance are parameterized by neural networks. The posterior q(|x1:t) is parameterized by neural network 300. The neural network 300 comprises a long short-term memory (LSTM) network, a temporal convolutional network (TCN), or any applicable machine learning network, which takes in previous measurements and outputs the posterior mean and variance of the latent variables. The prior p(zt+1|, ht+1) is parameterized by neural network 400. The neural network 400 takes in history measurements and previous latent values and outputs the prior mean and variance of latent variable zt+1. The predictive distribution p(xt+1|zt+1) is also parameterized by neural network 402, which takes in corresponding latent variable zt+1 and predicts the next time point measurement xt+1. Also, in equation 3, each integral is an expectation that may be replaced with appropriate sampling and summation.
Referring to
As an illustrative example, referring to
As described in this disclosure, the system 100 provides several advantages and benefits. For example, the system 100 models manufacturing sensor data and provides valuable insight into a manufacturing process. Also, the machine learning system 140 is a robust predictive model, which is based on sensor time series data for forecasting and which may alleviate the need for performing expensive and time-consuming measurements at a given instance. The machine learning system 140 is advantageous in being configured to incorporate long-range temporal dependencies.
The machine learning system 140 is configured to generate predicted measurement data, which may be used in various downstream tasks, e.g., predictive monitoring, predictive maintenance, etc. Such predictive measurements of the manufactured part along the manufacturing line can reduce costs associated with scrapping a component or a part. If a measurement of a component or a part can be estimated within the manufacturing line (e.g., at or between every manufacturing station), this can lead to a more precise determination of when a failure or misstep in manufacturing takes place. The observed measurement data and the predicted measurement data provide users with the insight, for example, to scrap a component or a part at any time in the manufacturing process before such action becomes more expensive to do so. Also, depending on the measurement data taken at a particular station and/or predicted by the machine learning system 140 for that station, the system 100 may determine if there are any issues or potential issues that need to be addressed with respect to one or more of the industrial machines at that station, one or more sensors taking measurements at that station, one or more parts traversing through that station, or any number and combination thereof, thereby providing users with the insight to take various actions (e.g., preventative actions, maintenance actions, etc.) that benefit manufacturing processes.
That is, the above description is intended to be illustrative, and not restrictive, and provided in the context of a particular application and its requirements. Those skilled in the art can appreciate from the foregoing description that the present invention may be implemented in a variety of forms, and that the various embodiments may be implemented alone or in combination. Therefore, while the embodiments of the present invention have been described in connection with particular examples thereof, the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the described embodiments, and the true scope of the embodiments and/or methods of the present invention are not limited to the embodiments shown and described, since various modifications will become apparent to the skilled practitioner upon a study of the drawings, specification, and following claims. Additionally or alternatively, components and functionality may be separated or combined differently than in the manner of the various described embodiments, and may be described using different terminology. These and other variations, modifications, additions, and improvements may fall within the scope of the disclosure as defined in the claims that follow.