Patent Application
20250165305
The present disclosure relates to industrial technologies. Various embodiments of the teachings herein include workflow generation methods and/or systems, computer-readable storage media, and computer program products.
A workflow is a description of a series of operation processes. The concept of a workflow is widely used in fields such as automated systems, artificial intelligence and robots. For example, a workflow of a product sorting line in an automated system can be simply described as starting, taking photos, classifying, and moving products to target positions. A model deployment workflow in the field of artificial intelligence can be described as data collection, data annotation, model training, and model deployment.
However, at present, these workflows only have text descriptions. If users want to execute such workflows, the users need to follow the text descriptions and may use multiple engineering tools. However, these tools are almost unrelated to each other and provide completely different user operation behaviors, which is not only a challenge to the users, but also greatly increases the costs, reduces efficiency and limits flexibility due to a long development cycle. For example, in the field of artificial intelligence, users need to use a tool for data collection, perform data annotation manually or by using other tools, write python scripts for model training, and use deployment tools for deployment.
Therefore, those skilled in the art are still working on finding other workflow solutions.
Teachings of the present application include workflow generation methods, systems, computer-readable storage media, and computer program products to quickly and conveniently implement workflow creation. For example, some embodiments include a workflow generation method including: receiving a behavior tree construction operation that is performed on a graphical user interface (GUI) by a user, where the behavior tree construction operation includes an addition operation and a line connection operation which are performed on behavior tree nodes that include function block nodes, one behavior tree is used for representing one workflow of service operations to be executed by resources in one workcell, and the function block nodes are used for implementing the service operations in the workflow; instantiating the behavior tree nodes and establishing connection relationships among the instantiated behavior tree nodes in response to the behavior tree construction operation, where some or all of the instantiated function block nodes are associated with resources for executing corresponding service operations; and generating a behavior tree corresponding to one workflow on the basis of the connection relationships among the instantiated behavior tree nodes.
As another example, some embodiments include a workflow generation system comprising: a node library, in which behavior tree nodes for constructing a behavior tree are arranged, where the behavior tree nodes include: function block nodes, one behavior tree is used for representing one workflow of operations to be executed by resources in one workcell, and the function block nodes are used for implementing the service operations in the workflow; a graphical interface module, configured to provide a GUI for a user to perform a behavior tree construction operation, where the behavior tree construction operation includes an addition operation and a line connection operation which are performed on the behavior tree nodes that include the function block nodes; and an editing and processing module, configured to instantiate the behavior tree nodes and establish connection relationships among the instantiated behavior tree nodes in response to the behavior tree construction operation, where some or all of the instantiated function block nodes are associated with resources for executing corresponding service operations; and generate a behavior tree corresponding to one workflow on the basis of the connection relationships among the instantiated behavior tree nodes.
As another example, some embodiments include a workflow generation system comprising: at least one memory, configured to store computer-readable codes; and at least one processor, configured to call the computer-readable codes to execute one or more of the methods described herein.
As another example, some embodiments include an IT domain and OT domain fusion system including an IT device and a workflow generation system as described in herein, where the workflow is an OT domain workflow; the workcell is a workcell in an OT domain; the resources are OT resources; and the workflow generation system further includes: an OT domain microservice generator, configured to generate a microservice on the basis of the behavior tree so that the IT device triggers the runtime of the workcell to execute the OT domain workflow by calling the microservice.
As another example, some embodiments include a computer-readable medium storing computer-readable instructions, and the computer-readable instructions, when executed by a processor, cause the processor to execute one or more of the methods described herein.
As another example, some embodiments include a computer program product stored on a tangible computer-readable medium. The product includes computer-readable instructions, and the computer-readable instructions, when executed, cause at least one processor to execute one or more of the methods described herein.
The service operations that can be executed by various types of corresponding resources are encapsulated into corresponding function block nodes, so that a behavior tree can be used for representing the operation processes of a workflow. Due to the reusability of nodes, decoupling of a specific service and an engineering platform is achieved. Moreover, organizing the nodes in the form of a behavior tree can generate intuitive workflow operation processes from development to implementation, thereby reducing the complexity of workflow construction and achieving quick and convenient workflow creation.
In some embodiments, the behavior tree is analyzed, and the OT domain workflow corresponding to the behavior tree is deployed to the runtime of the workcell so that the resources in the workcell execute the operations according to the workflow. In this way, the purpose of controlling the operations of the workcell on the basis of the workflow is achieved.
In some embodiments, the workflow is an OT domain workflow; the workcell is a workcell in an OT domain; and the device is an OT device. In this way, the workflow creation in the OT domain is achieved.
In some embodiments, a microservice is generated on the basis of the OT domain workflow so that the IT device in the IT domain triggers the runtime of a main controller of the workcell to execute the OT domain workflow by calling the microservice. In this way, the IT device can call the microservice generated on the basis of the OT domain workflow, thereby triggering the execution of the OT workflow to achieve the fusion of the IT domain and the OT domain.
The discussion of the various embodiments described herein is merely intended to make a person skilled in the art better understand and implement the subject described in this specification, and is not intended to limit the protection scope of the claims, the applicability, or examples. Changes may be made to the functions and arrangements of the discussed elements without departing from the protection scope of the content of the embodiments of the present application. Various processes or components may be omitted, replaced, or added in each example according to requirements. For example, the described method may be executed according to a sequence different from the sequence described herein, and steps may be added, omitted, or combined. In addition, features described in some examples may also be combined in other examples.
As used in this specification, the term “include” and variants thereof represent open terms, and mean “include but is not limited to”. The term “on the basis of” represents “at least partially on the basis of”. The terms “one embodiment” and “an embodiment” represent “at least one embodiment”. The term “another embodiment” represents “at least one another embodiment”. The terms “first”, “second”, and the like may represent different objects or the same object. Other definitions may be included explicitly or implicitly in the following. Unless otherwise clearly specified, the definition of one term is consistent in the entire specification.
Step S11A: A behavior tree construction operation performed on the basis of preset behavior tree nodes on a GUI by a user is received. One behavior tree is used for representing one workflow, and the workflow is used for defining the operation to be executed by one workcell. For example, the behavior tree may represent distributed processes in the workcell. During a specific implementation, the workflow here may be further divided into a main workflow and a subworkflow. The main workflow is used for limiting the start, end, and other flow controls that trigger the entire workcell process. The main workflow is an inlet of the entire process, which is linked to at least one subworkflow. Subworkflows are usually master workflows, and each subworkflow corresponds to a subprocess for implementing a specific service operation.
The workcell may be a combination of resources such as systems or devices capable of implementing a relatively complete and independent control process and operation. Workflow creation is performed on the basis of the workcell as a basic unit, which more conforms to the characteristics of industrial control, can improve the integration level of development, and can reduce the complexity of development. For example, in the field of the industrial technology, the workcell may be defined according to actual industrial scenarios. For example, a process may be defined as corresponding to a workcell, or a work station in a process may be defined as a workcell, or a work position in a work station may also be defined as corresponding to a workcell and the like, and different workcells have different process flows.
The behavior tree nodes may include: flow control nodes and function block nodes. The details are respectively described below.
The flow control nodes are used for implementing logical control in workflows, and are usually independent of specific service operations in the workcells. Through the flow control nodes, users can create various workflows according to their own needs. During a specific implementation, the flow control nodes may include: main control nodes, logical control nodes, condition nodes, and the like. In some embodiments, the flow control nodes may also include: one or any combination of main control nodes, logical control nodes and condition nodes. The details are respectively described briefly below.
The main control nodes may include: some or all of a start node, an end node, a goto node, an anchor node, a stop node and an abort node. The main control nodes in this embodiment are not standard behavior tree elements. However, in this embodiment, the main control nodes may be configured to control main processes of a workflow and may be linked to a state machine of the workflow. The start node is mandatory. In addition, one of the end node and the goto node may also be mandatory. The main control nodes are mainly used for defining the start and end of the workflow. In addition, other element nodes may control the state machine (such as abort and stop) or annotate critical process steps (such as anchor nodes) that may be skipped.
The logical control nodes may include: some or all of a sequence (Se) node, a reactive sequence (RSe) node, a parallel (Pa) node, an in-process QC (IPQ) node, a priority (Pr) (fallback) node, a reactive priority (RPr) (fallback) node, an if-then-else (ITE) node, and a switch (Sw) node. The logical control nodes can define how to execute branches in a behavior tree, and are configured to implement branch logical control in the workflow, and the like. Each of the nodes is described briefly below:
In general, the Se node and the Pa node may drive most of logics in a workflow.
Condition nodes are usually basic logical elements in a behavior tree that check expressions, and are used for executing condition judgment and returning judgment results. Success or failure is returned according to whether the conditions are met. The condition node never returns a running state.
In addition, in other implementations, the condition nodes may also be included in function block nodes. In some embodiments, the condition nodes may also be separately used as a type of nodes.
Function block nodes are used for executing commands and implementing service operations in workflows. In general, if the operation is completed correctly, success is returned; if the operation fails, failure is returned; and when the operation is in progress, running is returned. The function block nodes include logical nodes, and may also include some specific types of function block nodes, such as some or all of manual nodes, dynamic nodes, delay nodes, and empty (idle) nodes.
The dynamic node is used for dynamically injecting a node instance at a runtime. The manual node represents a manual step that stops at the current node before obtaining a confirmation signal and exits after obtaining the confirmation signal. The delay node represents that the current node exits after a specified delay time. The empty (idle) node represents that no operation is executed and is a placeholder which may be replaced with any function block node. Each logical node may correspond to one operation template, and each operation template defines at least one type of resources in advance, such as the operations that can be executed by (cooperative robot or PLC devices). For example, the operations may include: actions, methods or skills.
In some embodiments, each operation template may be composed of an interface part and an implementation part. The implementation part may be an application (such as a containerized application) that includes function codes and runtime dependencies. This application may run independently and is publicly available through a specific network communication interface. The interface part may be logical nodes presented in graphical elements, which, like other behavior tree nodes, may be dragged, connected and configured in a GUI. During a specific implementation, each logical node may have a parameter panel for configuring parameters of the logical node, such as input and output parameters. Of course, these input and output parameters may also be preset with default values.
Each logical node may be configured and executed separately. When the logical node in the behavior tree is executed, the input configured by a user for the logical node will be read and transmitted to the implementation part of the operation template, that is, the corresponding application. After a specific operation such as model conversion is completed, the operation result, such as the converted model, will be converted back to the output of the logical node.
The interface part and implementation part of each operation template may be stored separately. For example, the node library may only store the interface part, namely the logical nodes, and the implementation part may be stored in a node service module. In this embodiment, the node service module may be referred to as a runtime. The node service module may be located in a server or may also be located locally.
In some embodiments, the above logical nodes follow an information model interacting with the runtime of the main controller. In this way, standardization of communication between OT domain workflows and main controllers of various OT devices is implemented.
In addition, considering that the service operation corresponding to a function block node may be executed by different main bodies, such as a specific physical device, personnel or other virtualized resources, for the convenience of description, these physical devices, personnel and virtualized resources are collectively referred to as resources herein, and these resources usually refer to the resources that can execute workflows on site as various operation main bodies.
In some embodiments, the resource that serves as an operation execution main body may be used as a common configuration parameter for a function block node, and corresponding resource configurations may be made for the required function block node when a behavior tree is created. In some embodiments, when a function block node is created, the resource that serves as an operation execution main body is configured for the function block node, so that when a behavior tree is created, there is no need to make resource configurations for the required function block node.
In some embodiments, for the convenience of management of resources, these resources may also be represented in the form of resource nodes, and these resource nodes may be stored in the form of a resource knowledge graph. The resource knowledge graph includes: the resource nodes and connecting lines representing relationships among the resource nodes. For example,
Each function block node may be associated with a resource node, so that the function block node may be instantiated as an operation of the corresponding resource. For example, a logical node may be associated with a device resource node to be instantiated as an operation of the corresponding device. During a specific implementation, a header may be set for a function block node, and the resource associated with the function block node will be displayed in the header.
There are various ways to implement specific association processes. For example, each resource node may be associated with a corresponding function block node in advance, so that when a behavior tree is created, the function block node associated with the corresponding resource may be directly pulled without the need for temporary configuration. For example,
As shown in
In some embodiments, function block nodes may also not be associated with resource nodes in advance, and the required function block node is associated with the corresponding resource node when a behavior tree is created.
In some embodiments, there may be both function block nodes associated with resource nodes in advance (referred to as dedicated function block nodes) and function block nodes not associated with resource nodes (referred to as general function block nodes). Although the general function block nodes are not associated with on-site resources, it does not affect the analog simulation of the workflow corresponding to the behavior tree that includes the function block nodes. For example, if a cooperative robot is not purchased yet, in order to verify the feasibility, corresponding general function block nodes may be used for simulation in advance. When it is determined that it has feasibility, procurement may be initiated.
In addition, this embodiment may further include the following decorator nodes:
Decorator nodes are mainly used for decorating function block nodes driven by logical control nodes. For example, decorator nodes can be used for determining whether branches or even individual nodes in a behavior tree can be executed. Decorator nodes may include: some or all of a repeat (Rp) node, a retry (Rt) node, a one-shot (OS) node, a timeout (TO) node, a timer (Tm) node, an inverter (Iv) node, a force run (FR) node, a force OK (FO) node, a force failed (FF) node, and a guard (G) node. Each of the decorator nodes is described briefly below:
In some embodiments, decorator nodes may also be included in flow control nodes. The flow control nodes may include: all or some of main control nodes, logical control nodes and decorator nodes. Each behavior tree node may be listed in the form of an icon on the GUI, and users can determine the nodes required for creating a workflow by selecting and dragging icons to be added to a canvas. Further, necessary parameter configuration, such as resource configuration and/or input and output parameter configuration, may also be performed on nodes. If one workcell needs to perform more than one operation defined in the required workflow, the behavior tree corresponding to the workflow may include multiple function block nodes, corresponding flow control nodes may be set according to the sequence and relationships of operations, and the dragged nodes are arranged and connected correspondingly to finally generate a behavior tree corresponding to the workflow.
That is, the behavior tree construction operation includes an addition operation and a line connection operation of behavior tree nodes. Further, the behavior tree construction operation may include an operation of associating resources for the added function block nodes. In addition, the behavior tree construction operation may further include: a configuration operation for input and output parameters of behavior tree nodes.
Step S12A: A behavior tree corresponding to one workflow is generated in response to the behavior tree construction operation, and some or all of the function block nodes in the behavior tree are associated with resources for executing corresponding service operations. In general, for workflows that need to be implemented by on-site resources, all function block nodes in the behavior tree need to be associated with resources for executing corresponding service operations; and for action flows that do not need to be implemented by on-site resources temporarily, such as workflows for analog simulation, it is not necessary for all function block nodes to be associated with resources for executing corresponding service operations. For example, if a cooperative robot is not purchased yet, to verify the feasibility, corresponding general function block nodes may be used for simulation in advance. When it is determined that it has feasibility, procurement may be initiated.
The behavior tree nodes may be instantiated and connection relationships among the instantiated behavior tree nodes may be established in response to the behavior tree construction operation. Some or all of the instantiated function block nodes are associated with resources for executing corresponding service operations. By executing this step, the added function block node may be instantiated d as an operation of the corresponding resource. Then, a behavior tree corresponding to one workflow is generated on the basis of the connection relationships among the instantiated behavior tree nodes.
In some embodiments, the above behavior tree nodes may be stored in the node library. In addition, for similar application scenarios, to avoid the waste of manpower, time and the like in the process of repeatedly constructing behavior trees, the behavior tree (or uninstantiated behavior tree framework) of the corresponding workflow or corresponding subworkflow that is constructed, preferably debugged or successfully executed by users is stored as a workflow node or a subworkflow node. Correspondingly, the node library may further include workflow (WF) nodes and subworkflow (SWF) nodes. When users need to construct a similar behavior tree or construct a behavior tree that includes the workflow or the subworkflow, the corresponding workflow nodes or subworkflow nodes may be selected and configured as necessary to obtain a behavior tree for implementing the required workflow.
In the above construction process of the behavior tree representing the workflow, the function block nodes may be understood as being presented in the form of a label graph, and the construction of the behavior tree on the basis of the function block label graph requires the participation of flow control nodes and even decorator nodes. In addition, an embodiment of the present invention further provides a behavior tree construction method on the basis of a function block type graph. In this method, function block nodes are presented in the form of a type graph. In practical applications, the two behavior tree construction methods may coexist. Moreover, when a behavior tree is constructed by one of the methods, the other method is also constructed synchronously. For example, if two function block nodes are connected in sequence on the basis of a function block type graph, when a behavior tree on the basis of the function block label graph is synchronously constructed, a sequence node will be automatically added, and the two function block nodes will be added from top to bottom to the sequence node.
Step S11B: An addition operation and a line connection operation of function block nodes performed on the basis of a function block type graph on a GUI by a user are received. In the present application, the type graph and the label graph may be understood as two presentation manners of the same function block node, and are both used for implementing a corresponding service operation.
In some embodiments, to avoid a selection error and improve the sensitivity of the line connection between function block nodes, there is a sensitive area 206 in a set range of the link input port 203, which is referred to as a first sensitive area here. When selection and line connection operations performed by a user are received in the first sensitive area, line connection end points are located on the link input port 203. Similarly, there may also be a sensitive area (not shown in the figure) in a set range of the link output port 204, which is referred to as a second sensitive area here. When selection and line connection operations performed by the user are received in the second sensitive area, line connection end points are located on the link output port 204.
In addition, in response to the line connection operation between the link output port 204 and the link input port 203 between the two function block nodes F1 and F2, an instruction label 207 for indicating an execution sequence of each function block node may be further generated for the interconnected function block nodes F1 and F2, and the instruction label 207 is identified on the function block type graph of the function block nodes F1 and F2, such as 1 and 2 shown in the figure. In addition, the instruction label 205 may also be used as an index for a jump instruction and as a chapter index for an instructional document.
As shown in
In some embodiments, similar to the link input/output port, in order to avoid a selection error and improve the sensitivity of the line connection between function block nodes, there may also be a sensitive area (not shown in the figure) in a set range of the data input port 210, which is referred to as a third sensitive area here. When selection and line connection operations performed by the user are received in the third sensitive area, line connection end points are located on the data input port 210. Similarly, there may also be a sensitive area (not shown in the figure) in a set range of the data output port 211, which is referred to as a fourth sensitive area here. When selection and line connection operations performed by the user are received in the fourth sensitive area, line connection end points are located on the data output port 211.
Step S12B: In response to the addition operation and the line connection operation of the function block nodes, a behavior tree corresponding to one workflow is constructed. In response to the addition operation and the line connection operation of the function block nodes, a behavior tree including flow control nodes and function block nodes on the basis of a function block label graph in S12A may be synchronously constructed. Similarly, in step S12A, when a behavior tree including flow control nodes and function block nodes on the basis of a function block label graph is constructed, a behavior tree on the basis of the function block type graph in S12B may also be synchronously constructed. Moreover, the construction interface of the two behavior trees may be switched according to the selection of the user. For example, according to the selection operation of the user on the behavior tree on the basis of the function block type graph or the behavior tree on the basis of the function block label graph, the behavior tree on the basis of the function block type graph or the behavior tree on the basis of the function block label graph is switched and displayed.
In some embodiments, whether the behavior tree is created on the basis of the function block nodes in the form of the label graph or on the basis of the function block nodes in the form of the type graph, at least one data block may be further added and connected to each function block node in at least one function block node in the behavior tree. Each data block is used for presenting the corresponding data in the service operation of the function block node connected to the data block. The types of data blocks may include some or all of data pairs, data tables, images, videos and charts.
As shown in
The data block in this embodiment is a low code data block, which differs from other SCADA and dashboard systems in that: the low code data block in this embodiment is also a low code element, which may be used as a part of a behavior tree, and all attributes thereof may be managed in the low code logic. A data source in the low code data block is derived from the data layer, and data may be obtained through an interface provided by the data layer in a runtime or cloud execution engine. The data source may be a time series database, an RDBMS or a NoSQL. The low code data block is a flexible, scalable and adaptable system, which may be applied to any function block node that may be associated with a physical device. In short, the data block for implementing data monitoring may be considered as a visual interface of the data layer.
In some embodiments, whether the behavior tree is created on the basis of the function block nodes in the form of the label graph or on the basis of the function block nodes in the form of the type graph, at least one data block may be further added and connected to each function block node in at least one function block node in the behavior tree. Each data block is used for presenting the corresponding data in the service operation of the function block node connected to the data block. The types of data blocks may include some or all of data pairs, data tables, images, videos and charts.
As shown in
The data block in this embodiment is a low code data block, which differs from other SCADA and dashboard systems in that: the low code data block in this embodiment is also a low code element, which may be used as a part of a behavior tree, and all attributes thereof may be managed in the low code logic.
A data source in the low code data block is derived from the data layer, and data may be obtained through an interface provided by the data layer in a runtime or cloud execution engine. The data source may be a time series database, an RDBMS or an NOSQL. The low code data block is a flexible, scalable and adaptable system, which may be applied to any function block node that may be associated with a physical device. In short, the data block for implementing data monitoring may be considered as a visual interface of the data layer.
In some embodiments, the method may further include the following step S13 as shown by the dashed lines in
Step S13: The behavior tree is analyzed, and the workflow corresponding to the behavior tree is deployed to a runtime of the corresponding workcell so that the runtime executes the workflow to control the resources in the workcell to execute the service operations according to the workflow. For the behavior tree connected with data blocks, the runtime may further provide the corresponding data obtained during the execution of the service operation to the corresponding data block for displaying. During a specific implementation, the runtime may provide the corresponding data obtained during the execution of the service operation to the corresponding data block for displaying directly or through a third-party device.
In some embodiments, the workcell may include a main controller, and at this time, the runtime may be located on the main controller of the workcell. Correspondingly, device resources in the resources may be connected to the main controller, and the main controller controls the connected device resources to execute corresponding service operations according to the workflow of the runtime; and human resources and the like in the resources directly execute corresponding operations according to workflow prompts of the runtime. The behavior tree may be stored in a Markup language such as an Extensible Markup Language (XML), and may be validated by an XML Schema (XSD) prototype to verify that the XML format of the behavior tree is correct. After the behavior tree is analyzed, a workflow represented in the form of “node link assembly” may be obtained, and then, the workflow is compiled and downloaded to the runtime of the main controller of the corresponding workcell.
In addition, according to the definition of Gartner, the OT domain usually refers to an operational technology (OT), which combines hardware and software, and changes of processes or events in enterprises may be detected or triggered by directly monitoring and/or controlling physical devices (known as OT devices). The OT uses a computer to monitor or change the physical state of an industrial control system (ICS), and the like. The ICS is a facility, system or device implemented on the basis of a computer and is configured to remotely monitor and/or control critical industrial processes to implement physical functions. The term “OT” is used for distinguishing the ICS from the traditional information technology (IT) system in terms of technical implementations and functions.
At present, there are many IT low code development tools or platforms on the market, where some tools are designed for experienced IT engineers in the usage scenarios of the Internet of things, and OT engineers and junior IT engineers are difficult to understand the paradigm thereof; and some tools are more suitable for usage scenarios of IT-domain low code development and may not be well adapted to the OT domain.
The above workflow generation method in this embodiment may be used for the OT domain and serves as an OT-domain low code development method. In some embodiments, the workflow generation method shown in
The fusion of the IT domain and the OT domain is becoming increasingly important in the process of enterprise digital transformation. In order to fuse the IT domain and the OT domain into an ITOT system, currently, an urgent problem to be solved is how enterprises can collect OT domain data and control OT domain processes in a manner that is easy to understand, rather than a manner of IT domain programming.
The above workflow generation methods may be used for the ITOT system, and serves as an OT-domain low code development method that may be fused with the IT domain. Similarly, the workflow generation method shown in
In some embodiments, to implement the fusion of the IT domain and the OT domain, on the basis of the workflow generation method shown in
When a microservice is generated on the basis of the behavior tree, an API for the microservice may be generated on the basis of the behavior tree. The processing procedure in the API includes each operation in the OT domain workflow, input parameters of the API are parameters obtained from an input port of the OT domain workflow, and output parameters of the API are parameters outputted by an output port of the OT domain workflow.
To enable the IT domain to call the microservice, it is necessary for the IT domain to obtain information of the microservice. Specific implementation manners include but are not limited to the following two:
In the manner 1, OT domain code developers can notify IT domain code developers of the name and IP address of each generated microservice. In this way, the IT domain code developers can directly write the information of each microservice into codes during the development process, thereby enabling IT devices to call the microservice. The manner 1 is more suitable for scenarios with fewer microservices.
In the manner 2, registration and discovery mechanisms may be used. Each microservice may be registered on the knowledge middle platform, so that an IT-domain code development tool may enable IT devices to discover the connected microservice through the knowledge middle platform. During a specific implementation, an IT-domain code development tool may be configured to perform code development to enable IT domain devices to discover the connected microservice through the knowledge middle platform. The apparatus that completes the registration of the microservice may be an OT domain microservice generator or a third-party apparatus. The third-party apparatus may be considered as a part of the OT-domain low code development platform, or implemented in the knowledge middle platform. The manner 2 is more suitable for scenarios with more microservices.
IT devices may include but are not limited to: a manufacturing operation management (MOM) system, a manufacturing execution system (MES), an enterprise resource planning (ERP) system, an enterprise service bus (ESB) system, a product lifecycle management (PLM) system, and the like.
An IT-domain code development tool may be programmed to enable an IT device to call the microservice through a knowledge middle platform, so as to trigger the runtime of the main controller of the workcell to execute the OT domain workflow, thereby implementing the control of OT domain processes by the IT-domain code development platform, that is, implementing the fusion of the IT domain and the OT domain. Here, the microservice is automatically generated by the OT domain microservice generator on the basis of the OT domain behavior tree without the need for the IT-domain code development tool to understand the details of the OT domain workflow, and only an identifier (such as name) and an IP address of the microservice need to be obtained without the need for IT domain developers to understand OT domain devices and control processes, thereby being easy to implement and understand.
The applicable fields of the examples include but are not limited to: industrial automation, logistics, laboratory, maritime, smart grid, electric vehicle infrastructure, electric vehicle, building automation, smart city, water treatment, garbage recycling, smart farm, and the like.
The workflow generation systems can be configured to implement one or more of the workflow generation methods described herein. The details not disclosed in detail in regard to the system can refer to the corresponding descriptions in the method embodiments, which will not be repeated here.
Behavior tree nodes for constructing a behavior tree are arranged in the node library 110. The behavior tree nodes may include: flow control nodes and function block nodes. One behavior tree is used for representing one workflow, and the workflow is used for defining the operation to be executed by one workcell. The flow control nodes are used for implementing logical control in the workflow. The function block nodes are used for implementing service operations in the workflow. The function block nodes may include: logical nodes, where each logical node corresponds to one operation template, each operation template defines at least one type of resources in advance, such as the operations that can be executed by a device, and the operations include: actions, methods or skills. In addition, the node library 110 may further include various types of data blocks, and each data block is used for presenting the corresponding data in the service operation of the function block node connected to the data block. The types of the data blocks include: some or all of the data types such as data pairs, data tables, images, videos and charts.
In some embodiments, the resources are represented in the form of resource nodes, and all the resource nodes are associated and stored in the form of a resource knowledge graph. The resource knowledge graph includes: the resource nodes and connecting lines representing relationships among the resource nodes. Correspondingly, as shown by the dashed line in the figure, the system further includes: a resource library 150, configured to store the resources in the form of a resource knowledge graph, where each resource can execute at least one service operation.
In some embodiments, the flow control nodes may include: some or all of main control nodes, logical control nodes and condition nodes. The main control nodes may include: some or all of a start node, an end node, a goto node, an anchor node, a stop node and an abort node. The logical control nodes include: some or all of an Se node, an RSe node, a Pa node, an IPQ node, a Pr node, an RPr node, an ITE node and an Sw node.
In some embodiments, the function block nodes may further include: some or all of a manual node, a dynamic node, a delay node and an empty (idle) node.
In some embodiments, the behavior tree nodes further include: a decorator node which may include: some or all of an Rp node, an Rt node, an OS node, a TO node, a Tm node, an Iv node, an FR node, an FO node, an FF node and a G node.
In some embodiments, some or all of the function block nodes in the node library 110 are respectively bound with resources that execute the service operations corresponding to the function block nodes.
In some embodiments, the function block nodes in the node library 110 may be presented in the form of label graphs as shown in
The graphical interface module 120 is configured to provide a GUI for users to perform a behavior tree construction operation on the basis of the behavior tree nodes in the node library. In addition, addition and connection operations of data blocks may also be performed on the GUI.
In some embodiments, the GUI may include a first GUI for constructing a behavior tree on the basis of flow control nodes and function block nodes in the form of a function block label graph, and a second GUI for constructing a behavior tree on the basis of function block nodes in the form of a function block type graph, and the two GUIs may be switched and displayed according to the selection of users. The behavior tree construction operation may include an addition operation and a line connection operation of function block nodes. On the second GUI, the addition operation of the function block nodes may include: a dragging operation for the function block main body; and the line connection operation of the function block nodes may include: a line connection operation between a link output port and a link input port between two function block nodes, and a line connection operation between a data output port and a data input port between the corresponding data of two function block nodes.
Each behavior tree node may be listed in the form of an icon on the GUI.
The editing and processing module 130 is configured to generate a behavior tree corresponding to one workflow in response to the behavior tree construction operation. In addition, the editing and processing module 130 is also configured to add and connect at least one data block to each function block node in at least one function block node in the behavior tree in response to the addition and connection operations of the data blocks.
In some embodiments, the editing and processing module 130 can instantiate the behavior tree nodes and establish connection relationships among the instantiated behavior tree nodes in response to the behavior tree construction operation; and generate a behavior tree corresponding to one workflow on the basis of the connection relationships among the instantiated behavior tree nodes. Some or all of the instantiated function block nodes associated with resources for executing corresponding service operations. For example, through this operation, logical nodes are instantiated as operations of corresponding resources such as devices.
Each behavior tree node and each data block may be listed in the form of an icon on the GUI, and users can determine the nodes required for creating a workflow by selecting and dragging icons to be added to a canvas. Further, necessary parameter configuration, such as resource configuration and/or input and output parameter configuration, may also be performed on nodes.
If one workcell needs to perform more than one service operation defined in the required workflow, the behavior tree corresponding to the workflow may include multiple logical nodes, the execution sequence of the logical nodes can be determined according to the sequence and relationships of operations, and the dragged nodes are arranged and connected correspondingly to finally generate a behavior tree corresponding to the workflow.
In some embodiments, the behavior tree construction operation may include: an operation of adding function block nodes and an operation of associating resources with the added function block nodes.
Corresponding to the method shown in
In some embodiments, the workcell may include a main controller, and at this time, the runtime may be located on the main controller of the workcell. Correspondingly, device resources in the resources may be connected to the main controller, and the main controller controls the connected device resources to execute corresponding operations according to the workflow of the runtime; and human resources and the like in the resources may directly execute corresponding operations according to workflow prompts of the runtime.
As described above, currently, there is no low code development tool and platform suitable for an OT domain. In the platform 100 shown in
Specifically, as shown in
Further, as shown in
The composition of the OT-domain low code development platform 100 shown in
Here, the microservice 40 is automatically generated by the OT domain microservice generator 20 on the basis of the OT domain behavior tree without the need for the IT-domain code development tool 301 to understand the details of the OT domain workflow, and only an identifier (such as name) and an IP address of the microservice 40 need to be obtained without the need for IT domain developers to understand OT domain devices and control processes, thereby being easy to implement and understand.
To enable the IT domain to call the microservice 40, it is necessary for the IT domain to obtain information of the microservice 40. Corresponding to the method shown in
In the manner 1, OT domain code developers can notify IT domain code developers of the name and IP address of each generated microservice 40. In this way, the IT domain code developers can directly write the information of each microservice 40 into codes during the development process, thereby enabling IT devices to call the microservice 40. The manner 1 is more suitable for scenarios with fewer microservices.
In the manner 2, registration and discovery mechanisms may be used. Each microservice 40 may be registered on the knowledge middle platform 200, so that the IT-domain code development tool 301 may use code development to enable IT domain devices to discover the connected microservice 40 through the knowledge middle platform 200. apparatus that completes the registration of the microservice 40 may be the OT domain microservice generator 20 or a third-party apparatus 50 as shown in
In some embodiments, the OT domain microservice generator 20 can generate an API for the microservice 40 on the basis of the OT domain behavior tree, where the processing procedure in the API may include the operation of each function block in the OT domain workflow, input parameters of the API are parameters obtained from an input port of the OT domain workflow, and output parameters of the API are parameters outputted by an output port of the OT domain workflow.
An application scenario of the OT-domain low code development platform 100 provided in an embodiment of the present application in the field of industrial automation is shown in
As shown in
At least one memory 51 is configured to store computer programs. At least one memory 51 may include a computer-readable medium, such as a random access memory (RAM). In addition, at least one memory 51 may also store an operating system, and the like. The operating system includes but is not limited to: Android operating system, Symbian operating system, Windows operating system, Linux operating system, and the like. The above computer storage program may include the following program modules: a node library 110, a graphical interface module 120, an editing and processing module 130 and an analyzing and deploying module 140, and optionally may also include an OT domain microservice generator 20, a runtime 30 and a third-party apparatus 50.
At least one processor 52 is configured to call the computer program stored in at least one memory 51 to execute the workflow generation method described in the embodiment of the present application. At least one processor 52 may be a microprocessor, an application specific integrated circuit (ASIC), a digital signal processor (DSP), a central processing unit (CPU), a graphics processing unit (GPU), a state machine, or the like. Data may be received and transmitted through the communication port.
At least one display 53 is configured to display a GUI.
In some embodiments, at least one processor 52 is configured to call the computer program stored in at least one memory 51, so that the system executes the operation in the workflow generation method described in any of the above implementations.
In addition, a communication interface is configured to implement communication with other devices, such as communication with the knowledge middle platform 200.
The OT-domain low code development tool 10 for implementing the graphical interface module 120, the editing and processing module 130 and the analyzing and deploying module 140 may be a lightweight web-based application, which may be implemented on industrial sites (such as edge devices or local servers), or may be implemented on the cloud side (public clouds such as AWS or private clouds such as OpenStack). A visual engineering paradigm is derived from a function block typed diagram (FBTD). The OT domain microservice generator 20 may use modern translation programming languages to generate a standard API such as RESTful or RPC. The runtime 30 may simply implement the OT domain workflow and provide openness on the basis of an ecosystem of an open source community (such as Python). The runtime 30 may be deployed on an embedded IoT device such as a single board computer (SBC).
In some embodiments, there is an apparatus with an architecture different from that shown in
In some embodiments, there is an IT domain and OT domain fusion system, namely an ITOT system, which may include an IT device and a workflow generation system in any implementation of the present application. In addition, the system may further include: an IT-domain code development platform 300 shown in
In some embodiments, there is a computer-readable storage medium. The computer-readable storage medium stores computer-readable codes, and the computer-readable codes, when executed by a processor, cause the processor to execute one or more of the workflow construction methods described herein. In some embodiments, there is a computer program product. The computer program product is stored on a tangible computer-readable medium and includes computer-readable instructions, and the computer-readable instructions, when executed, cause at least one processor to execute one or more of the workflow generation methods described herein.
In some embodiments, a storage medium stores computer-readable codes for implementing functions of any implementation in the embodiment described herein, and a computer (or a CPU or an MPU) of the system or the apparatus is enabled to read and execute the computer-readable codes stored in the storage medium. In addition, the operating system and the like operated on the computer may complete some or all of the actual operations through instructions on the basis of the computer-readable codes. The computer-readable codes read from the storage medium may also be written to the memory arranged in an expansion board inserted into the computer, or written to the memory in an expansion unit connected to the computer, and then, instructions on the basis of the computer-readable codes cause the CPU and the like installed on the expansion board or the expansion unit to execute some or all of the actual operations, thereby implementing the functions of any of the above implementations. In this embodiment, examples of the computer-readable medium include but are not limited to a floppy disk, a CD-ROM, a magnetic disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, or DVD+RW), a memory chip, an ROM, an RAM, an ASIC, a configured processor, an all-optical medium, all magnetic tapes or other magnetic media, or any other medium from which a computer processor can read instructions. In addition, various other forms of computer-readable media may transmit or carry instructions to a computer, including routers, private or public networks, or other wired and wireless transmission devices or channels. For example, computer-readable instructions may be downloaded from a server computer or cloud through a communication network. Instructions may include codes of any computer programming language, including C, C++, C language, Visual Basic, java and JavaScript.
It should be noted that not all steps and modules in the above processes and system structure drawings are necessary, and some steps or modules may be omitted according to actual requirements. An execution sequence of the steps is not fixed and may be adjusted according to requirements. The system structure described in the above embodiments may be a physical structure or a logical structure. That is, some modules may be implemented by the same physical entity, or some modules may be implemented by a plurality of physical entities or may be implemented by some components in a plurality of independent devices together.
The above descriptions are merely example embodiments of the teachings herein, but are not intended to limit the scope of the present disclosure. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present disclosure shall fall within the protection scope thereof.
This application is a U.S. National Stage Application of International Application No. PCT/CN2022/075059 filed Jan. 29, 2022, which designates the United States of America, the contents of which are hereby incorporated by reference in their entirety.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/CN2022/075059 | 1/29/2022 | WO |
