Patent Application
20230281486
This application relates to computer aided design (CAD). More particularly, this application relates to enhancement of CAD with an artificial intelligence advisor for automatic contextual recognition and clustering of design data by functionality and for verification of compliance with standards and policies.
When designing industrial systems in the course of design projects, engineers rely on various engineering tools in the CAD domain including various software packages and applications tailored for one or more disciplines or domains of an industry, such as electrical, mechanical, automation, etc. Typically, an engineering tool presents the project design as a 2D or 3D rendering, with various components of a system design configured in a connected arrangement according to the design. Other presentations may include tables or lists of components that are presently part of the project design. During the course of project design via such engineering tools, recognizing and classifying components (e.g., parts and devices) of a system design project is a must in order to understand how the various system parts operate individually and cooperatively within in the project. For example, a design project may relate to modeling a layout of robotic and conveyor components for an automation factory intended to serve a production line. A complex factory may involve a multitude of component types, each to serve a designated function that collaboratively achieve an objective. Tracking of a component's function and interactive role in the overall system is important as design decisions are being made, as there are contributary relationships among the many components. There is also a need to comply with an increasing number of standards and policies as components are added to a design project. Today, engineers must rely on manual standard and policy compliance check, which requires considerable time and effort and is prone to error. Failure of a designed system to comply with effective regulations can have severe consequences.
Application of an Artificial Intelligence (AI) feature can be helpful to provide recommendations that can guide an engineer during the CAD-based industrial design process. While an AI system can learn to recognize components, the extent of recognition is conventionally restricted to classification of a component by object type since components are self-describing entities (e.g., motor, function block, data block). What today's AI systems lack in the industrial design setting is the ability to readily recognize functionality of the component for contextual classification. Conventional AI systems also fail to provide an automated compliance verification service for industrial system design for assistance with enforcement of standards and policies.
A software solution for computer aided design applications used for design projects is disclosed which incorporates an industrial AI-based assistant feature to classify system components using contextualization and to advise the user with recommendations for the design components according to functionality while operating an engineering software tool. The software solution also performs automated standard and policy compliance verification for engineering validation of the design project.
In an aspect, a method for computer aided design includes training a machine learning-based model using training data obtained from previous design projects to construct a trained machine learning-based model that classifies functionality of components for a current design project. An engineering software tool constructs a graphical design of an industrial system for a design project, the graphical design having a plurality of components. A knowledge graph is constructed for the current project based on data associated with the graphical design, the knowledge graph having nodes and edges representing an ontology for a set of elements and element relationships respectively, with the set of elements including the plurality of components. An artificial intelligence (AI) module runs integrated with the engineering tool during a current project, the AI module using the trained machine learning-based model to classify functionality of project components. The AI module identifies a functionality for each knowledge graph node based on the classifier model, and clusters knowledge graph nodes according to identified functionality. A cluster diagram may be generated and displayed having distinct functionality clusters on a portion of the display as an AI-based assistant feature based on the clustering to provide a visual aid to the user with functionality classifications in the graphical design. Validation of the engineering design data is performed by an inference engine using rules based inference.
Non-limiting and non-exhaustive embodiments of the present embodiments are described with reference to the following FIGURES, wherein like reference numerals refer to like elements throughout the drawings unless otherwise specified.
Methods and systems are disclosed for enhancing an engineering tool by incorporating an AI-based assistant feature that learns contextual classification of components in a computer aided design project and presents functionality information to a user via a graphical user interface (GUI) concurrently with operating an engineering tool that models an industrial system for the design project. The functionality may be displayed as clusters of functionally classified elements from a knowledge graph of the current system design. Additional assistance may be presented in the form of recommendations to the user relative to design elements, such as a notification that elements may be missing, or for determining technical parameters for the design. An AI module and/or inference engine may analyze the knowledge graph, construct functional clusters that reflect elements related to one another by a common functionality, and formulate the recommendations to the user during the design process. Such assistance solves a technical problem facing an engineer who opens a design project in an engineering tool, several years after the project was created by a different engineer, and days and weeks may be required to gain an understanding of just a single subsystem model, let alone many more subsystem models that need to be classified for the entire design project. The AI-based assistant feature can respond to detected potential policy or standard violations for compliance verification in the design project with a notification to a system user.
In an embodiment, engineering data is monitored during the course of multiple design projects, and the engineering data is organized into project specific knowledge graphs 150 as semantic data. In an aspect, a knowledge graph 150 may be generated for each engineering discipline operating a design project and/or a knowledge graph 150 may be created for the system design encompassing all active disciplines. Knowledge graphs 150 are generated using a knowledge graph algorithm that processes an ontology of data exported from an engineering application 112 (e.g., Siemens TIA Portal, ePlan, Process Simulate, etc.). The ontology governs, according to a particular ontology standard, what types of elements of a system and the relationships between the elements are present (e.g., motor control, logic function block, associated sensor signals). The ontology also describes properties of the elements and the element relationships, and may organize the element types into hierarchies, such as super-types and sub-types. A knowledge graph 150 represents the ontology as nodes and edges that correspond to a set of elements of the ontology and element relationships, respectively. An archive of knowledge graphs 150 may be accumulated over the course of many engineering design projects and stored as historical data 155. In an aspect, the ontologies used to create the knowledge graphs may be stored as indexed tables in historical data 155.
AI module 140 is configured to apply a classification technique (e.g., machine learning classification, natural language processing, pattern matching, data-flow analysis) to find interconnected components, classify the component functionality and tag the component accordingly. From this functionality classification, AI module 140 can provide functionality based recommendations to the user through the AI-based assistant. In an embodiment, the AI module 140 includes a machine learning-based network trained by supervised or unsupervised training techniques to generate a machine learning model that can recognize the different parts of the project based on the connections and the properties of elements in the knowledge graph 150 for the current project. A data-driven approach may involve a machine learning model that is trained on labeled historical data 155. Such a model can be trained using graph clustering, graph/node classification or other classification models. Once trained, AI module 140 is used to analyze knowledge graphs 150 to classify components in a design project with contextualization according to functionality. Functionality context information can be presented to a user in the AI-based assistant feature at GUI 151 during a design project. In an aspect, AI module 124 interfaces as a client with a server-based AI module 140 over network 130 to conserve local memory, operating as a cloud-based AI scheme. In some embodiments, local AI module 124 may be implemented as an independent machine learning-based network capable of locally performing the tasks as described for the AI module 140 either entirely or in part as a shared role for the AI-based assistant feature.
Inference engine 122 is used to implement a rule-based approach, in which functionalities are described as graph queries over the ontology of the knowledge graph 150 (e.g., using SPARQL or Gremlin language), and the descriptions are stored in rules database 132. Rules may be defined by a user with assistance from a rule editor software application that, based on the ontology, allows for easy rule composition. Once defined, the rules are stored in rule database 132. Alternatively, the rules may be integrated into a knowledge graph ontology by a knowledge graph algorithm. An example of a defined rule may relate to defining functionality context based on a particular arrangement, configuration, and/or connected relationship of two or more components. A rule-based approach can also incorporate results from data mining techniques (e.g., natural language processing, graph clustering, data-flow analysis, etc.), whereby text from tab or label elements of historical knowledge graph data 155 is extracted and interpreted for functionality context and formulated into rule-based queries and stored in the rules database 132. In an embodiment, once the rules are defined and stored in the rules database 132 according to any of the above techniques, inference engine 122 analyzes new project data inputs by finding the relevant rules and applying rule-based inference on extracted structure features of structured data records in knowledge graph 150 for the current project to determine functionality clusters.
Inference engine 122 may also provide engineering validation feedback to the AI-based assistant displayed on GUI 151. In an embodiment, rules database 132 stores policies, standards and/or regulations for engineering design components. During configuration for a current design project, mapping engine 123 may be used to extract relevant policies, standards and/or regulations information, received from the user or extracted from the rules database 132 and/or documents of information sources, and map the extracted information to design elements of the historical data 155 to define rules-based relationships to project ontology, then stores the rules in rule database 132. During a validation for a running design project, inference engine 122 extracts relevant rules from the rules databased and applies the rules to new project data observed in knowledge graph 150 of the current project, seeking any discrepancy as a potential violation of policy, standards and/or regulations. Discrepancies may be reported to the user through the AI-based assistant feature.
During a project classification phase, following a completed project configuration, the project design may be queried from the AI-based assistant 235, such as if the project is reopened and a user wishes submit a demand for classification information about the design, such as an overview of a particular system or subsystem (e.g., what functions are served by devices in subsystem X?). To perform project classification, AI module 210 may execute component classifications according to functionality, based on the state of the knowledge graph 250 as constructed during configuration phase, while an engineering tool is concurrently in use, such as one of engineering applications 212 (e.g., ePlan, Process Simulate, TIA Portal Hardware Configuration, TIA Portal PLC Software, etc.). To execute the classifications, industrial AI module 210 extracts connected information from knowledge graph 250, and applies a classification technique (e.g., classification, natural language processing, pattern matching, data-flow analysis, etc.) by which interconnected components are identified and classified according to functionality. Using this information, the classification results 215 are fed to the user interface 230, which may include a graphical representation of the components by clustered functionality, described below with reference to
In a separate process that may be implemented alternatively or additionally in the project classification phase, inference engine 220 analyzes knowledge graph 250 against the rules information 231. Using inference analysis, inference engine 220 recognizes clusters within the knowledge graph 250 and applies functionality rules to elements of the graph. For example, code or other software elements (e.g., comments, file structure or hierarchy, code organization, or the like) can be analyzed by the inference engine 220 according to the defined rules information 231 to identify functionality. In another embodiment, data mining can be applied to the knowledge graph 250 to extract the state of the project using text recognition algorithms (e.g., NLP) on textual nodes such as tags and labels. From the extracted meaning of the text and recognized structure patterns, inference engine 220 infers functional context from a comparison to the defined rules 232. The classification results 225 are sent to the user interface 230 for presentation in the AI-based assistant 235. A graphical representation of which is described below with reference to
Once the project components are classified by functionality, either by the data-driven approach or the rule-based approach, one technical improvement includes the ability to easily operate within the design project according to clustered functionality. For example, should it be discovered that a new requirement, constraint, or regulation has arisen, any necessary modifications can be made with confidence that all components for the target classification are captured (e.g., a new crane regulation can be implemented comprehensively and quickly for all components within the crane functionality cluster). In contrast, using conventional CAD based tools, there is no easy way to pinpoint all design components connected by a common functionality within the industrial system.
In an aspect, the AI module 210 and/or inference engine 220 may fill gaps, such as missing blocks in the design project, by recognizing context in the knowledge graph. For example, as an engineer works the design project in an engineering tool, such as a TIA Portal program, the AI modules 210 may scan the knowledge graph and recognize context contributed by the hardware simulation and prompt the engineer that there appears to be tags omitted in the design project related to the robot cluster, which likely should be added in the TIA Portal. For example, as the engineer selects design elements to be associated with the robot design, perhaps entering parameters for each new element, and then moves to another component for the project, the AI module 210 or inference engine 220 may detect that the design aspects for the robot are incomplete and send a notification to the user at the TIA portal interface via AI-based assistant 235 that elements for Sig. A and Sig. B need to be connected to sensors in accordance with the functional clustering learned from knowledge graph contribution by the hardware simulator program. This aspect of the AI-based assistant feature 235 ensures that the engineer does not overlook design elements, and can thereby fill the gaps.
In an embodiment, the engineering data validation involves inference engine 220 validates standards, policies as represented in the rules database 232 against the new engineering data represented in the knowledge graph 250. An example of such an inference engine can be a SPARQL endpoint in case standards, policies are expressed as SPARQL queries. Using a pre-trained knowledge representation of policies and standards represented as semantic data, the inference engine 220 compares the current state of the project to the semantic data for detecting the potential violations. In response to a detected discrepancy, the inference engine 220 sends a validation message 415 to user interface 230, which may be displayed in the AI-based assistant feature 235 on a display of the computing device to include an indication of the potential violation, a recommendation for how to fix the violation, or a combination of both. Alternatively, inference engine 220 can be receive a direct query 414 from user interface 230 relating to regulation compliance for a system component or a subsystem of components and respond with information regarding any detected discrepancy or violation, or an indication of compliance if no discrepancy or violation is detected. In the event of modification to the design resulting from the validation result 415, the system configuration may repeat to ensure proper mapping of relevant rules in knowledge graph 250 is maintained.
As an illustrative example of rule based engineering validation, using the knowledge graph 250, the industrial AI module 210 views the system design with contextual perspective, such as identifying a motor belonging to a conveyor cluster that has known safety constraints, while the motor component is not presently designed with any parameters to address such a constraint (e.g., conveyor speed, forward/reverse advancement limits, etc.). This identified discrepancy, identified by functionality clustering, may then be reported to the user while engaged in the engineering application as the target motor is being added to the design for the project. For example, a message may appear in the AI-based assistant feature 235 on the user interface 230: “motor 125B drives conveyor 23 and is missing speed and advancement parameters”. Examples of encoding a rule to discover discrepancies with standard or policy compliance may include: “Mandatory parameter is missing: speed”; “Mandatory parameter is missing: advancement”; Mandatory parameter(s) missing: speed, advancement.
To provide a comprehensive modelling of the engineering domain, data pertaining to standards, regulations and policies are modelled as semantic data in knowledge graph 250, shown as regulation data graph 552. This semantic data can be expressed as constraints over the ontology. In an embodiment, the knowledge graph 250 is constructed using RDF syntax (triples), which can express standards, regulations or policies as SHACL (shapes constraint language) or SPIN constraints over the ontology (e.g., a naming format convention for knowledge graph nodes that must be followed). Alternatively, the constraints can be directly expressed as graph queries over the knowledge graph 250, for example as SPARQL or Gremlin queries.
The processors 620 may include one or more central processing units (CPUs), graphical processing units (GPUs), or any other processor known in the art. More generally, a processor as described herein is a device for executing machine-readable instructions stored on a computer readable medium, for performing tasks and may comprise any one or combination of, hardware and firmware. A processor may also comprise memory storing machine-readable instructions executable for performing tasks. A processor acts upon information by manipulating, analyzing, modifying, converting or transmitting information for use by an executable procedure or an information device, and/or by routing the information to an output device. A processor may use or comprise the capabilities of a computer, controller or microprocessor, for example, and be conditioned using executable instructions to perform special purpose functions not performed by a general purpose computer. A processor may include any type of suitable processing unit including, but not limited to, a central processing unit, a microprocessor, a Reduced Instruction Set Computer (RISC) microprocessor, a Complex Instruction Set Computer (CISC) microprocessor, a microcontroller, an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), a System-on-a-Chip (SoC), a digital signal processor (DSP), and so forth. Further, the processor(s) 620 may have any suitable microarchitecture design that includes any number of constituent components such as, for example, registers, multiplexers, arithmetic logic units, cache controllers for controlling read/write operations to cache memory, branch predictors, or the like. The microarchitecture design of the processor may be capable of supporting any of a variety of instruction sets. A processor may be coupled (electrically and/or as comprising executable components) with any other processor enabling interaction and/or communication there-between. A user interface processor or generator is a known element comprising electronic circuitry or software or a combination of both for generating display images or portions thereof. A user interface comprises one or more display images enabling user interaction with a processor or other device.
The system bus 621 may include at least one of a system bus, a memory bus, an address bus, or a message bus, and may permit exchange of information (e.g., data (including computer-executable code), signaling, etc.) between various components of the computer system 610. The system bus 621 may include, without limitation, a memory bus or a memory controller, a peripheral bus, an accelerated graphics port, and so forth. The system bus 621 may be associated with any suitable bus architecture including, without limitation, an Industry Standard Architecture (ISA), a Micro Channel Architecture (MCA), an Enhanced ISA (EISA), a Video Electronics Standards Association (VESA) architecture, an Accelerated Graphics Port (AGP) architecture, a Peripheral Component Interconnects (PCI) architecture, a PCI-Express architecture, a Personal Computer Memory Card International Association (PCMCIA) architecture, a Universal Serial Bus (USB) architecture, and so forth.
Continuing with reference to
The operating system 634 may be loaded into the memory 630 and may provide an interface between other application software executing on the computer system 610 and hardware resources of the computer system 610. More specifically, the operating system 634 may include a set of computer-executable instructions for managing hardware resources of the computer system 610 and for providing common services to other application programs (e.g., managing memory allocation among various application programs). In certain example embodiments, the operating system 634 may control execution of one or more of the program modules depicted as being stored in the data storage 640. The operating system 634 may include any operating system now known or which may be developed in the future including, but not limited to, any server operating system, any mainframe operating system, or any other proprietary or non-proprietary operating system.
The computer system 610 may also include a disk/media controller 643 coupled to the system bus 621 to control one or more storage devices for storing information and instructions, such as a magnetic hard disk 641 and/or a removable media drive 642 (e.g., floppy disk drive, compact disc drive, tape drive, flash drive, and/or solid state drive). Storage devices 640 may be added to the computer system 610 using an appropriate device interface (e.g., a small computer system interface (SCSI), integrated device electronics (IDE), Universal Serial Bus (USB), or FireWire). Storage devices 641, 642 may be external to the computer system 610.
The computer system 610 may include a user input/output interface 660 for translating signals to and from input/output devices 661, which may comprise one or more input devices, such as a keyboard, touchscreen, tablet and/or a pointing device, and output devices such as a display (e.g., on which GUI 151 may be displayed), for interacting with a computer user and providing information to the processors 620.
The computer system 610 may perform a portion or all of the processing steps of embodiments of the invention in response to the processors 620 executing one or more sequences of one or more instructions contained in a memory, such as the system memory 630. Such instructions may be read into the system memory 630 from another computer readable medium of storage 640, such as the magnetic hard disk 641 or the removable media drive 642. The magnetic hard disk 641 and/or removable media drive 642 may contain one or more data stores and data files used by embodiments of the present disclosure. The data store 640 may include, but are not limited to, databases (e.g., relational, object-oriented, etc.), file systems, flat files, distributed data stores in which data is stored on more than one node of a computer network, peer-to-peer network data stores, or the like. Data store contents and data files may be encrypted to improve security. The processors 620 may also be employed in a multi-processing arrangement to execute the one or more sequences of instructions contained in system memory 630. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions. Thus, embodiments are not limited to any specific combination of hardware circuitry and software.
As stated above, the computer system 610 may include at least one computer readable medium or memory for holding instructions programmed according to embodiments of the invention and for containing data structures, tables, records, or other data described herein. The term “computer readable medium” as used herein refers to any medium that participates in providing instructions to the processors 620 for execution. A computer readable medium may take many forms including, but not limited to, non-transitory, non-volatile media, volatile media, and transmission media. Non-limiting examples of non-volatile media include optical disks, solid state drives, magnetic disks, and magneto-optical disks, such as magnetic hard disk 641 or removable media drive 642. Non-limiting examples of volatile media include dynamic memory, such as system memory 630. Non-limiting examples of transmission media include coaxial cables, copper wire, and fiber optics, including the wires that make up the system bus 621. Transmission media may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.
Computer readable medium instructions for carrying out operations of the present disclosure may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.
Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, may be implemented by computer readable medium instructions.
The computing environment 600 may further include the computer system 610 operating in a networked environment using logical connections to one or more remote computers, such as remote computing device 673. The network interface 670 may enable communication, for example, with other remote devices 673 or systems and/or the storage devices 641, 642 via the network 671. Remote computing device 673 may be a personal computer (laptop or desktop), a mobile device, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to computer system 610. When used in a networking environment, computer system 610 may include modem 672 for establishing communications over a network 671, such as the Internet. Modem 672 may be connected to system bus 621 via user network interface 670, or via another appropriate mechanism.
Network 671 may be any network or system generally known in the art, including the Internet, an intranet, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a direct connection or series of connections, a cellular telephone network, or any other network or medium capable of facilitating communication between computer system 610 and other computers (e.g., remote computing device 673). The network 671 may be wired, wireless or a combination thereof. Wired connections may be implemented using Ethernet, Universal Serial Bus (USB), RJ-6, or any other wired connection generally known in the art. Wireless connections may be implemented using Wi-Fi, WiMAX, and Bluetooth, infrared, cellular networks, satellite or any other wireless connection methodology generally known in the art. Additionally, several networks may work alone or in communication with each other to facilitate communication in the network 671.
It should be appreciated that the program modules, applications, computer-executable instructions, code, or the like depicted in
It should further be appreciated that the computer system 610 may include alternate and/or additional hardware, software, or firmware components beyond those described or depicted without departing from the scope of the disclosure. More particularly, it should be appreciated that software, firmware, or hardware components depicted as forming part of the computer system 610 are merely illustrative and that some components may not be present or additional components may be provided in various embodiments. While various illustrative program modules have been depicted and described as software modules stored in system memory 630, it should be appreciated that functionality described as being supported by the program modules may be enabled by any combination of hardware, software, and/or firmware. It should further be appreciated that each of the above-mentioned modules may, in various embodiments, represent a logical partitioning of supported functionality. This logical partitioning is depicted for ease of explanation of the functionality and may not be representative of the structure of software, hardware, and/or firmware for implementing the functionality. Accordingly, it should be appreciated that functionality described as being provided by a particular module may, in various embodiments, be provided at least in part by one or more other modules. Further, one or more depicted modules may not be present in certain embodiments, while in other embodiments, additional modules not depicted may be present and may support at least a portion of the described functionality and/or additional functionality. Moreover, while certain modules may be depicted and described as sub-modules of another module, in certain embodiments, such modules may be provided as independent modules or as sub-modules of other modules.
Although specific embodiments of the disclosure have been described, one of ordinary skill in the art will recognize that numerous other modifications and alternative embodiments are within the scope of the disclosure. For example, any of the functionality and/or processing capabilities described with respect to a particular device or component may be performed by any other device or component. Further, while various illustrative implementations and architectures have been described in accordance with embodiments of the disclosure, one of ordinary skill in the art will appreciate that numerous other modifications to the illustrative implementations and architectures described herein are also within the scope of this disclosure. In addition, it should be appreciated that any operation, element, component, data, or the like described herein as being based on another operation, element, component, data, or the like can be additionally based on one or more other operations, elements, components, data, or the like. Accordingly, the phrase “based on,” or variants thereof, should be interpreted as “based at least in part on.”
The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the Figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/US2020/045893 | 8/12/2020 | WO |
