This application relates to manufacturing technology. More particularly, this application relates to a graph-driven approach to production process monitoring in manufacturing.
Production monitoring is essential in manufacturing for optimizing productivity and efficiency of a production process. Today's production process monitoring technology is based on time series analysis of the data generated by the production system. For example, in a manufacturing execution system (MES)/supervisory control and data acquisition (SCADA) system, all production-relevant data (e.g., process values) are stored in a process historian database. Production monitoring applications are programmed in MES or SCADA systems using the data from the historian database.
In order to achieve efficient production monitoring, heavy programming and configuration are necessary, and multiple systems, such as Control Layer system, human machine interface (HMI) and MES, may be involved. Production monitoring is included as only one of several modules within a production code used to operate and control a production system. When a change in production monitoring is necessary (e.g., updating the production monitoring process to handle new process data related to modifications in the manufacturing machinery), HMI coding of the production code must be reprogrammed and redeployed. For example, HMI coding involves manual generation of multiple alarm screens in response to various events (e.g., process values exceeding allowable thresholds). In such cases, when new potential events are discovered, reprogramming of the HMI coding for a new alarm screen requires operation of the production system to be halted to allow the production code to be reprogrammed. Likewise, other aspects of production monitoring are based on a program of complex rules and logic imposed on process variables to detect faults, errors, and alarms at runtime, which lead to production interruptions when reprogramming is to be performed for updating to handle new configurations and events. A new production monitoring approach is needed that can operate during runtime without the need for reprogramming interruptions.
Productivity of a manufacturing system is improved by a data-driven monitoring process. The data-driven approach enables cloud-based processing, with benefits including offline optimization of the production process, fault detection, predictive maintenance, and improvement of productivity key performance indicators (KPI) (e.g., overall equipment effectiveness (OEE) KPI).
In Ir a system aspect, production process monitoring using graph learning is performed using a processor and a memory having modules stored thereon for execution by the processor. An ontology engine is configured to generate a production ontology based on data received from an engineering design process, the ontology including selection of production machines, definition of work products and workstations, and workflow design. A graph engine is configured to instantiate a production graph based on the production ontology, the production graph comprising graph nodes representing components of the production ontology, including production machines and work products, and graph edges to represent relationships between the components. The graph engine reads production process data from control systems of the production environment and populates the production graph with production process data in real time to generate a time series of production graphs. A human machine interface displays the time series of production graphs to reflect updated status of the production process including location of work products among the workstations. An analytics engine is configured to execute machine learning algorithms to extract trends from the time series of production graphs and generate one or more predictions of the production process based on the extracted trends. The analytics engine indicates the one or more predictions in the production graph.
Non-limiting and non-exhaustive embodiments of the present embodiments are described with reference to the following FIGURES, wherein like reference numerals refer to like elements throughout the drawings unless otherwise specified.
System and method are disclosed for process monitoring based on graph model data of the production process. The graph model is decoupled from the actual control and human machine interface (HMI) programs, and therefore the engineering effort is greatly reduced. An ontology is built using engineering data such as process diagrams, layouts, and configurations. An initial production process monitoring graph is instantiated from the ontology. During production, a time series of production process monitoring graphs are populated with production data. Compared to process monitoring state-of-the-art algorithms that use time series data only, the disclosed embodiments extract information from the structure in the graph derived from actual engineering data to derive insights from the process monitoring in a more efficient manner. This approach to process monitoring can be done in real-time and can provide improved visualization of the process monitoring along with prognostic and diagnostic information. As a result, the novel graph-driven approach for production monitoring disclosed herein improves the production process and minimizes the need for re-engineering of the monitoring algorithms.
In an embodiment, graphs are used to model factory assets (e.g., machines and products) and production processes (e.g., workflows). In a production monitoring graph, machines and products are represented by nodes, and their interactions (e.g., a machine operating on a work piece) are represented by edges. Production data such as process values are encoded as node and edge properties. In an embodiment, a production monitoring graph is built from engineering data, and populated from existing control systems such as PLC, HMI, and SCADA. The properties (process/tag values) of the nodes can be linked to the existing systems using Open communication interfaces (e.g., OPC UA, S7 communication). In an aspect, the graph evolves over time. As production progresses in time, the nodes and relationships change alongside the production. For example, as a product is treated by different machines, this can be modeled as the creation and removal of edges in the graph. The graph can be reprogrammed, engineered and deployed without interference to the production.
To illustrate examples for the evolving nature of a time series of production process monitoring graphs,
In an embodiment, machine learning techniques are applied over dynamic graph data to discover new insights about the production. New insights can be learned over the regular production operation. One advantage from the disclosed embodiments is the discovery of new insights by the application of machine learning over the dynamic production monitoring graph. Targets for the machine learning algorithms may include production key performance indicators (KPIs) (e.g., throughput, OEE, downtime). New insights may include one or more of the following: (1) best production flow with least errors or worst production flows with most errors, (2) new connections/links between assets, (3) correlations between upstream and downstream components in a production line, (4) utilization of machines, (5) trend of movement of the product.
For example, a control program in a programmable logic controller (PLC) runs production logic that controls movement of a work product tray via conveyor and robotic grasp and transfer operations to different machine workstations where various production and assembly operations may be executed. In an arrangement such as shown in
In an embodiment, the prediction is rendered graphically as a visualization cue for the user on the human machine interface (HMI) used for the production in the automation system. For example, with reference to the time series production graph shown in
Further advantages are afforded the operator by the graph-based production monitoring system, such as conveying process value information with the production process graph as a display on the HMI. For example, in an embodiment, the graph engine is configured to reveal process values in response to a human operator selecting a graph node on the HMI (e.g., selection may be by clicking on, touch of a touch screen, or hover over a graph node to reveal the process values). Alternatively, the graph engine can be configured to display one or more process values a graph component by default as an overlay on the respective graph node.
During engineering 1310, the production engineering data generated from the selection of machines 1311, definition of products 1312, production plan design 1313, and the workflow design 1314 is collected and used to generate a production ontology 1321. The production ontology is specific to a factory, and it provides the types of machines, products, their relationships, and their hierarchies. This ontology is used to instantiate a production graph 1322.
During production monitoring 1320, data from production processes 1328 is read from the control systems 1324 and it is used to populate the dynamic production graph 1323. An analytics engine runs analytics algorithms 1326 on the production graph to generate insights about the factory, machines, production process, and the product. In addition, analytics algorithms 1326 analyze historical production graphs stored in graph database 1330 that were generated for similar processes, machines and/or factories and transfers knowledge from the historical graphs to the new production graph. In an embodiment, analytics algorithms 1326 yield analytics results that include predictions (e.g., prognostics of production machines) and trends (e.g., which machines and process sequence variations yield the most efficient production of product). The results of the analytics can be fed back to the production graph or displayed 1325 on a user HMI for human interaction. In an aspect, the production graph is stored 1327 as a graph database 1330.
A technical advantage provided by the disclosed embodiments is delivering accurate predictions for production process even with a small amount of information about the production process. Test results for the disclosed graph-based production process monitoring compared against a random prediction yield a performance improvement of 5:1 to 110:1 for observed links in a new graph ranging from 10% to 70%, respectively. The performance relative to random predictions increases super-linearly, hence the disclosed embodiments capture more graph structure with an increasing amount of data.
Additional advantages include the following. Machine learning algorithms for production monitoring based on time series require more data and take longer to execute compared to algorithms that use graph data. Graphs have inherent structure that can be exploited to accelerate the machine learning. Graphs are a good interface for humans to understand the production process. Nodes and relationships are intuitive when compared to time series data alone. Technical features applied herein include modeling of production data using graphs, generation of graph ontologies that are the basis for the production graph using engineering data from machines, product, configuration, and workflows, graph learning, and graph interface for HMI.
The processors 1520 may include one or more central processing units (CPUs), graphical processing units (GPUs), or any other processor known in the art. More generally, a processor as described herein is a device for executing machine-readable instructions stored on a computer readable medium, for performing tasks and may comprise any one or combination of, hardware and firmware. A processor may also comprise memory storing machine-readable instructions executable for performing tasks. A processor acts upon information by manipulating, analyzing, modifying, converting or transmitting information for use by an executable procedure or an information device, and/or by routing the information to an output device. A processor may use or comprise the capabilities of a computer, controller or microprocessor, for example, and be conditioned using executable instructions to perform special purpose functions not performed by a general purpose computer. A processor may include any type of suitable processing unit including, but not limited to, a central processing unit, a microprocessor, a Reduced Instruction Set Computer (RISC) microprocessor, a Complex Instruction Set Computer (CISC) microprocessor, a microcontroller, an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), a System-on-a-Chip (SoC), a digital signal processor (DSP), and so forth. Further, the processor(s) 1520 may have any suitable microarchitecture design that includes any number of constituent components such as, for example, registers, multiplexers, arithmetic logic units, cache controllers for controlling read/write operations to cache memory, branch predictors, or the like. The microarchitecture design of the processor may be capable of supporting any of a variety of instruction sets. A processor may be coupled (electrically and/or as comprising executable components) with any other processor enabling interaction and/or communication there-between. A user interface processor or generator is a known element comprising electronic circuitry or software or a combination of both for generating display images or portions thereof. A user interface comprises one or more display images enabling user interaction with a processor or other device.
The system bus 1521 may include at least one of a system bus, a memory bus, an address bus, or a message bus, and may permit exchange of information (e.g., data (including computer-executable code), signaling, etc.) between various components of the computer system 1510. The system bus 1521 may include, without limitation, a memory bus or a memory controller, a peripheral bus, an accelerated graphics port, and so forth. The system bus 1521 may be associated with any suitable bus architecture including, without limitation, an Industry Standard Architecture (ISA), a Micro Channel Architecture (MCA), an Enhanced ISA (EISA), a Video Electronics Standards Association (VESA) architecture, an Accelerated Graphics Port (AGP) architecture, a Peripheral Component Interconnects (PCI) architecture, a PCI-Express architecture, a Personal Computer Memory Card International Association (PCMCIA) architecture, a Universal Serial Bus (USB) architecture, and so forth.
Continuing with reference to
The operating system 1538 may be loaded into the memory 1530 and may provide an interface between other application software executing on the computer system 1510 and hardware resources of the computer system 1510. More specifically, the operating system 1538 may include a set of computer-executable instructions for managing hardware resources of the computer system 1510 and for providing common services to other application programs (e.g., managing memory allocation among various application programs). In certain example embodiments, the operating system 1538 may control execution of one or more of the program modules depicted as being stored in the data storage 1540. The operating system 1538 may include any operating system now known or which may be developed in the future including, but not limited to, any server operating system, any mainframe operating system, or any other proprietary or non-proprietary operating system.
The computer system 1510 may also include a disk/media controller 1543 coupled to the system bus 1521 to control one or more storage devices for storing information and instructions, such as a magnetic hard disk 1541 and/or a removable media drive 1542 (e.g., floppy disk drive, compact disc drive, tape drive, flash drive, and/or solid state drive). Storage devices 1540 may be added to the computer system 1510 using an appropriate device interface (e.g., a small computer system interface (SCSI), integrated device electronics (IDE), Universal Serial Bus (USB), or FireWire). Storage devices 1541, 1542 may be external to the computer system 1510.
The computer system 1510 may include a user input interface 1560 for communication with a graphical user interface (GUI) 1561, which may comprise one or more input devices, such as a keyboard, touchscreen, tablet and/or a pointing device, for interacting with a computer user and providing information to the processors 1520. In an aspect, the GUI 1561 relates to an HMI for displaying graphs, graph data, and/or analytics results as earlier described.
The computer system 1510 may perform a portion or all of the processing steps of embodiments of the invention in response to the processors 1520 executing one or more sequences of one or more instructions contained in a memory, such as the system memory 1530. Such instructions may be read into the system memory 1530 from another computer readable medium of storage 1540, such as the magnetic hard disk 1541 or the removable media drive 1542. The magnetic hard disk 1541 and/or removable media drive 1542 may contain one or more data stores and data files used by embodiments of the present disclosure. The data store 1540 may include, but are not limited to, databases (e.g., relational, object-oriented, etc.), file systems, flat files, distributed data stores in which data is stored on more than one node of a computer network, peer-to-peer network data stores, or the like. Data store contents and data files may be encrypted to improve security. The processors 1520 may also be employed in a multi-processing arrangement to execute the one or more sequences of instructions contained in system memory 1530. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions. Thus, embodiments are not limited to any specific combination of hardware circuitry and software.
As stated above, the computer system 1510 may include at least one computer readable medium or memory for holding instructions programmed according to embodiments of the invention and for containing data structures, tables, records, or other data described herein. The term “computer readable medium” as used herein refers to any medium that participates in providing instructions to the processors 1520 for execution. A computer readable medium may take many forms including, but not limited to, non-transitory, non-volatile media, volatile media, and transmission media. Non-limiting examples of non-volatile media include optical disks, solid state drives, magnetic disks, and magneto-optical disks, such as magnetic hard disk 1541 or removable media drive 1542. Non-limiting examples of volatile media include dynamic memory, such as system memory 1530. Non-limiting examples of transmission media include coaxial cables, copper wire, and fiber optics, including the wires that make up the system bus 1521. Transmission media may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.
Computer readable medium instructions for carrying out operations of the present disclosure may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.
Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, may be implemented by computer readable medium instructions.
The computing environment 1500 may further include the computer system 1510 operating in a networked environment using logical connections to one or more remote computers, such as remote computing device 1573. The network interface 1570 may enable communication, for example, with other remote devices 1573 or systems and/or the storage devices 1541, 1542 via the network 1571. Remote computing device 1573 may be a personal computer (laptop or desktop), a mobile device, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to computer system 1510. When used in a networking environment, computer system 1510 may include modem 1572 for establishing communications over a network 1571, such as the Internet. Modem 1572 may be connected to system bus 1521 via user network interface 1570, or via another appropriate mechanism.
Network 1571 may be any network or system generally known in the art, including the Internet, an intranet, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a direct connection or series of connections, a cellular telephone network, or any other network or medium capable of facilitating communication between computer system 1510 and other computers (e.g., remote computing device 1573). The network 1571 may be wired, wireless or a combination thereof. Wired connections may be implemented using Ethernet, Universal Serial Bus (USB), RJ-6, or any other wired connection generally known in the art. Wireless connections may be implemented using Wi-Fi, WiMAX, and Bluetooth, infrared, cellular networks, satellite or any other wireless connection methodology generally known in the art. Additionally, several networks may work alone or in communication with each other to facilitate communication in the network 1571.
It should be appreciated that the program modules, applications, computer-executable instructions, code, or the like depicted in
Although specific embodiments of the disclosure have been described, one of ordinary skill in the art will recognize that numerous other modifications and alternative embodiments are within the scope of the disclosure. For example, any of the functionality and/or processing capabilities described with respect to a particular device or component may be performed by any other device or component. Further, while various illustrative implementations and architectures have been described in accordance with embodiments of the disclosure, one of ordinary skill in the art will appreciate that numerous other modifications to the illustrative implementations and architectures described herein are also within the scope of this disclosure. In addition, it should be appreciated that any operation, element, component, data, or the like described herein as being based on another operation, element, component, data, or the like can be additionally based on one or more other operations, elements, components, data, or the like. Accordingly, the phrase “based on,” or variants thereof, should be interpreted as “based at least in part on.”
The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the Figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/US2021/042455 | 7/21/2021 | WO |