Patent Application
20230050387
The present disclosure relates generally to engineering autonomous systems, and in particular, to a technique for imposing constraints in a skill-based autonomous system.
The requirement to manage rapid innovation cycles, complex customization requirements, and growing cost pressures in a global and highly competitive landscape presents a growing challenge to traditional industrial automation systems. This challenge is motivating a trend for manufacturers to gradually transition from automation to autonomy. In contrast to automation, autonomy gives each asset on the factory floor the decision-making and self-controlling abilities to act independently in the event of local issues.
The industrial use cases for autonomous systems on a factory floor are expected to be wide-spread and cover a large range of application scenarios. In some use cases, this may involve the need to reduce or even remove human involvement. In other scenarios, autonomous machines may augment factory workers' physical and intellectual abilities. This development is a core enabling technology for flexible manufacturing operations as envisioned in the context of Industry 4.0.
It is envisioned that engineering tools for autonomous systems would need to cope with novel programming paradigms challenging the state of the art in industrial automation systems.
Briefly, aspects of the present disclosure are directed to techniques for imposing constraints in engineering autonomous systems, in a skill-based programming paradigm.
According to one aspect of the present disclosure, a computer-implemented method is provided. The method comprises creating a plurality of basic skill functions for a controllable physical device of an autonomous system. Each basic skill function comprises a functional description for using the controllable physical device to interact with a physical environment to perform a skill objective. The method further comprises selecting one or more basic skill functions, from the plurality of basic skill functions, to configure the controllable physical device to perform a defined task. The method further comprises determining a decorator skill function specifying at least one constraint. The decorator skill function is configured to impose, at run-time, the at least one constraint, on the one or more basic skill functions. The method further comprises generating executable code by applying the decorator skill function to the one or more basic skill functions. The method further comprises actuating the controllable physical device using the executable code.
Other aspects of the present disclosure implement features of the above-described method in computing systems and computer program products.
Additional technical features and benefits may be realized through the techniques of the present disclosure. Embodiments and aspects of the disclosure are described in detail herein and are considered a part of the claimed subject matter. For a better understanding, refer to the detailed description and to the drawings.
The foregoing and other aspects of the present disclosure are best understood from the following detailed description when read in connection with the accompanying drawings. To easily identify the discussion of any element or act, the most significant digit or digits in a reference number refer to the figure number in which the element or act is first introduced.
Aspects of the present disclosure described below relate to engineering an autonomous system in a skill-based programming paradigm. In conventional automation, an automated robot is typically programmed to perform a single, repetitive task, such as positioning a car panel in exactly the same place on each vehicle. In this case, an engineer is usually involved in programming an entire task from start to finish, typically utilizing low-level code to generate individual commands. In the presently described autonomous system, an autonomous device, such as a robot, is programmed at a higher level of abstraction using skills instead of individual commands.
For programming in the skill-based paradigm, one starts from the standpoint of graphical editing and builds on top. In this case, an engineer would generally know what they want the robot to do and the attributes of how the job should be accomplished but is less likely to know how to accomplish the task or know how various implementation choices will interact with each other. So, a large part of the engineer's job is selecting and arranging the skills required for a defined task.
The present inventors recognize that, by abstracting specific robot commands into skills, an engineer may lose knowledge of the behavior of the robot for a specific input. Specific machine motion patterns may be deliberately less transparent to engineers, who do not design low-level robot tasks, e.g. path planning or collision avoidance. Instead, engineers of autonomous systems would primarily focus on high-level system and application properties, e.g., goals and skill objectives. This poses the challenge in encoding modifiable constraints in an engineering tool used to program autonomous devices.
Embodiments of the present disclosure address at least the afore-mentioned technical challenges and provide a technique for imposing constraints in a skill-based autonomous system. A non-limiting example application of the present disclosure includes imposing safety constraints in an autonomous system. In an autonomous environment, it is desirable that safety is intrinsic and built into systems implicitly. The present technique would ensure that every action executed by an autonomous device, such as a robot, takes safety constraints into account, without modifying the programmed skills.
Turning now to
Computing system 100 may be described in the general context of computer executable instructions, such as program modules, being executed by a computing system. Generally, program modules may include routines, programs, objects, components, logic, data structures, and so on that perform particular tasks or implement particular abstract data types. Computing system 100 may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computing system storage media including memory storage devices.
As shown in
The computing system 100 comprises an I/O adapter 112 (input/output adapter) and a communications adapter 114 coupled to the system bus 104. The I/O adapter 112 may be a small computer system interface (SCSI) adapter that communicates with a hard disk 116 and/or any other similar component. The I/O adapter 112 and the hard disk 116 are collectively referred to herein as a mass storage 118.
Software 120 for execution on the computing system 100 may be stored in the mass storage 118. The mass storage 118 is an example of a tangible storage medium readable by the processors 102, where the software 120 is stored as instructions for execution by the processors 102 to cause the computing system 100 to operate, such as is described herein below with respect to the various Figures. Examples of computer program product and the execution of such instruction is discussed herein in more detail. The communications adapter 114 interconnects the system bus 104 with a network 122, which may be an outside network, enabling the computing system 100 to communicate with other such systems. In one embodiment, a portion of the system memory 106 and the mass storage 118 collectively store an operating system, which may be any appropriate operating system, to coordinate the functions of the various components shown in
Additional input/output devices are shown as connected to the system bus 104 via a display adapter 124 and an interface adapter 126. In one embodiment, the I/O adapter 112, the communications adapter 114, the display adapter 124 and the interface adapter 126 may be connected to one or more I/O buses that are connected to the system bus 104 via an intermediate bus bridge (not shown). A display 128 (e.g., a screen or a display monitor) is connected to the system bus 104 by the display adapter 124, which may include a graphics controller to improve the performance of graphics intensive applications and a video controller. A keyboard 130, a mouse 132, a speaker 134, among other input/output devices, can be interconnected to the system bus 104 via the interface adapter 126, which may include, for example, a Super I/O chip integrating multiple device adapters into a single integrated circuit. Suitable I/O buses for connecting peripheral devices such as hard disk controllers, network adapters, and graphics adapters typically include common protocols, such as the Peripheral Component Interconnect (PCI). Thus, as configured in
In some embodiments, the communications adapter 114 can transmit data using any suitable interface or protocol, such as the internet small computing system interface, among others. The network 122 may be a cellular network, a radio network, a wide area network (WAN), a local area network (LAN), or the Internet, among others. An external computing device may connect to the computing system 100 through the network 122. In some examples, an external computing device may be an external webserver or a cloud computing node.
It is to be understood that the block diagram of
The engineering tool 200 may be designed to allow an engineer to program a robot to perform a defined task 204 by selecting one or more of the available basic skill functions 202. In one example embodiment, the engineering tool 200 may comprise a graphical user interface configured to allow an engineer to simply drag and drop basic skill functions 202 from a skill menu, and program the robot to perform the task 204 by setting appropriate task parameters.
Referring to
Referring back to
In one embodiment, as illustrated hereinafter referring to
Continuing with reference to
The at least one constraint may be imposed in a time-variant manner, or in an uninterrupted manner, at run-time. In one embodiment, the decorator skill function is configured to impose the at least one constraint at run-time responsive to a predefined trigger. Furthermore, the decorator skill function may be configured to remove the at least one constraint at run-time when the predefined trigger is removed. In the example illustrated in
It should be appreciated that aspects of the present disclosure are not limited, in implementation, to robots, but may extend to other types of autonomous devices. For example, in one embodiment, such an autonomous device may comprise an autonomous vehicle. In this case, a basic skill function may comprise, for example, performing a specific maneuver, on which a safety (or other) aspect may be imposed by way of a decorator skill function as described herein. It should also be appreciated that while a decorator skill function may be configured to impose a constraint (at run-time) on each of the basic skill functions, a decorator skill function may not be always necessary to define a task, and may not be applied to tasks that do not require constraints.
Furthermore, aspects of the present disclosure are not limited to safety and may be extended to superimpose other overarching aspects on basic skill functions. In one embodiment, the decorator skill function may comprise a hardware decorator skill function. In a hardware decorator skill function, the constraints may be specified based on a type of computing platform used to execute the code. For example, a hardware decorator skill function may specify constraints that reflect the ability to execute certain functionalities on an edge computing device versus a cloud computing platform, or may reflect computing resource allocation, such as adjusting the number of CPUs/GPUs made available to execute the code. In one embodiment, the decorator skill function may comprise a communications decorator skill function. In a communications decorator skill function, the constraints may be specified based on a type of communications architecture used for communication between entities of the autonomous system. This is applicable, for example, in autonomous systems comprising multiple devices (such as robots) communicating with each other. In this case, the constraints may specify, for example, communication ports and/or communication protocols used by the devices. In one embodiment, the engineering tool may comprise multiple decorator skill functions, such as safety, hardware, communications, etc., each configured to impose one or more constraints at run-time to the basic skill functions, to modify the behavior of an autonomous device, without affecting the basic skill functions.
Using a decorator skill function allows an engineer to separate the high-level skill objectives of a program or app (e.g. pick and place objects) from overarching aspects such as safe execution and architecture, device hardware configuration and communications architecture. This allows, for instance, modifying execution times of certain program components or skill functions, such as when having a human close to the robot or changing a robot model or add/remove safety constraints to one with different safety characteristics, without modifying the overall functionality captured in the program or app.
The technique disclosed herein may lead to modular architecture, lightweight software, and user-friendliness. This is anticipated to significantly impact current trends such as skill-based programming of autonomous systems. Furthermore, robot user interface, menus and options may look completely different by simply adding an aspect (such as safety, hardware configuration, communications architecture, etc.) to a given program.
Aspects of the present disclosure may include a system, a method, and/or a computer program product at any possible technical detail level of integration. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present disclosure.
The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.
Computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing/processing device.
Computer readable program instructions for carrying out operations of the present disclosure may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuitry, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++, or the like, and procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instruction by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.
Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.
These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
An executable code, as used herein, comprises code or machine readable instructions for conditioning the processor to implement predetermined functions, such as those of an operating system, a context data acquisition system or other information processing system, for example, in response to user command or input. An executable procedure is a segment of code or machine readable instruction, sub-routine, or other distinct section of code or portion of an executable application for performing one or more particular processes. These processes may include receiving input data and/or parameters, performing operations on received input data and/or performing functions in response to received input parameters, and providing resulting output data and/or parameters.
A graphical user interface (GUI), as used herein, comprises one or more display images, generated by a display processor and enabling user interaction with a processor or other device and associated data acquisition and processing functions. The GUI also includes an executable procedure or executable application. The executable procedure or executable application conditions the display processor to generate signals representing the GUI display images. These signals are supplied to a display device which displays the image for viewing by the user. The processor, under control of an executable procedure or executable application, manipulates the GUI display images in response to signals received from the input devices. In this way, the user may interact with the display image using the input devices, enabling user interaction with the processor or other device.
The functions and process steps herein may be performed automatically, wholly or partially in response to user command. An activity (including a step) performed automatically is performed in response to one or more executable instructions or device operation without user direct initiation of the activity.
The system and processes of the figures are not exclusive. Other systems, processes and menus may be derived in accordance with the principles of the disclosure to accomplish the same objectives. Although this disclosure has been described with reference to particular embodiments, it is to be understood that the embodiments and variations shown and described herein are for illustration purposes only. Modifications to the current design may be implemented by those skilled in the art, without departing from the scope of the disclosure.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/US2020/017702 | 2/11/2020 | WO |
