The present disclosure relates generally to methods, systems, and apparatuses related to encoding and transferring scene and task dependent learning information into transferable neural network layers. The disclosed techniques may be applied to, for example, to enhance logistics and task execution in factory settings.
Recent advances in machine learning have sparked a plethora of research and practical development, leading to widespread application in various technical domains spanning from web portals to industrial automation. The latter domain in particular has proven both very promising and challenging in its adoption of novel learning algorithms, such as seen for instance in classification tasks for smart grasping of various objects with common physical hardware. While machine learning and Artificial Intelligence (AI) techniques in the broader sense bring the opportunity to revolutionize industrial automation—in particular with a view towards mass customization in manufacturing lines, as pushed by the prominent vision of Industry 4.0—this domain also highlights shortcomings of current state-of-the-art learning algorithms to be addressed by experts to enable pervasive application. Being able to rapidly and reliably close the gap between potential and realization of AI techniques in industrial automation may well become one of the key distinguishing factors in the manufacturing automation market of the coming decades.
One aspect slowing down the adoption of current state-of-the-art learning algorithms is the transfer of learned policies and decision structures to different agents or even agents deployed to different environments. Our current invention addresses the latter issue of devices being exposed to a variety of distinct environments and associated tasks. In particular, if the number of possible environments and tasks is large, training a single algorithm to capture all possible scenarios the agent may be exposed to can become prohibitive. When using neural networks, for instance, the possible neuron connections increase exponentially as nodes are added to a given network. Computation time depends on the numbers of nodes and their connections; increases may have drastic consequences for processing times. This dimensionality problem is highlighted especially when learning algorithms run on embedded devices, as opposed to general computing PCs equipped with GPUs. At the same time, the industry is moving in the direction of edge automation with more intelligence close to the process. On these edge devices, the computational resources available are rather limited and there are several constrains in terms of memory, speed and footprint. Therefore, deploying big generic networks to AI-enabled edge devices is typically not feasible.
Embodiments of the present invention address and overcome one or more of the above shortcomings and drawbacks, by providing methods, systems, and apparatuses related to encoding and transferring scene and task dependent learning information into transferable neural network layers.
According to some embodiments, a system includes a robot device that comprises a non-transitory computer readable medium and a robot controller. The non-transitory computer readable medium stores one or more machine-specific modules comprising base neural network layers. The robot controller receives a task-specific module comprising information corresponding to one or more task-specific neural network layers enabling performance of a task. The robot controller collects one or more values from an operating environment, and uses the values as input to a neural network comprising the base neural network layers and the task-specific neural network layers to generate an output value. The robot controller may then perform the task based on the output value.
According to other embodiments, a system comprises an object that comprises a non-transitory computer readable medium and networking components. The non-transitory computer readable medium stores a plurality of task-specific modules. Each task-specific module comprises information corresponding task-specific neural network layer enabling performance of a task involving the object. The networking component transmits one or more of the task-specific modules to a robot device upon request. The robot device combines the transmitted task-specific modules with a machine-specific module to form a complete neural network for performing assigned tasks involving the object.
According to other embodiments, a method includes a robot device storing one or more machine-specific modules comprising base neural network layers. The robot device receives a task-specific module comprising information corresponding to one or more task-specific neural network layers enabling performance of a task, and the robot device collects one or more values from an operating environment. The robot device then uses the values as input to a neural network comprising the base neural network layers and the task-specific neural network layers to generate an output value. The robot device may then perform the task based on the output value.
Additional features and advantages of the invention will be made apparent from the following detailed description of illustrative embodiments that proceeds with reference to the accompanying drawings.
The foregoing and other aspects of the present invention are best understood from the following detailed description when read in connection with the accompanying drawings. For the purpose of illustrating the invention, there are shown in the drawings embodiments that are presently preferred, it being understood, however, that the invention is not limited to the specific instrumentalities disclosed. Included in the drawings are the following Figures:
Systems, methods, and apparatuses are described herein which relate generally to encoding and transferring scene and task dependent learning information into transferable neural network layers. The techniques described herein are based on the understanding that, even when a robot is exposed to different environments and tasks, many relevant properties are device-specific and remain constant. In the following, we outline a technical solution enabling agents pre-trained with device-specific functionalities to inherit additional, modular extensions to their decision structures from the infrastructure they are exposed to, in conjunction with selected tasks to be executed by the respective agents. The solution described herein evolves around flexible modular architectures, where a neural network layer or group of layers can be associated to a specific task or a skill. Task specific layers can be stored on passive elements, while generic ones can be stored at mobile or active components.
With the techniques described herein, neural network policies are decomposed into “task-specific” and “machine-specific” modules, where the task-specific modules are shared across machines required to perform given tasks, and the machine-specific modules are shared across all tasks on identical machines. The task-specific and machine-specific modules described herein can be transmitted to: (1) learn how a specific task, such as picking an object, should be performed; (2) combine machine-specific modules for a better task performance. In this way, grasping task information can be shared between machines (e.g., 2 robot devices equipped with a camera focusing at different points of view).
The term “module,” as used herein refers to an individual neural network layer or a group of neural network layers. For each module, the information that needs to be stored and exchanged is composed by the weights and the network shape, such as the numbers of nodes per layer. This information could be managed, for example, via Extensible Markup Language (XML) files or user-defined binary files. Alternatively, if the end user uses existing machine-learning software, the modules can be stored in its own I/O format for storing and loading a trained network. The important factor is that transmitter and receiver know in advance the format of the information to be shared in between devices and objects.
Continuing with reference to
The exact task may be specified in the Identifier 155 itself. For example, a specific “grasping” identifier may be provided by the Box 135. Alternatively, the Identifier 155 may only identify the Box 135 itself, and the Robot Device 120 specifies the task it wishes to complete. For example, in addition to providing the Identifier 155 to the Task Planning Computer 150, the Robot Device 120 may also send a desired command (e.g., “grasp”). In either event, the information provided by the Robot Device 120 may be used directly as a way of identifying the Task-Specific Module 110 on a storage medium accessible to the Task Planning Computer 150. For example, in one embodiment, the Task Planning Computer 150 includes a database that is indexed based on an object identifier and/or task identifier.
In some embodiments, the Task Planning Computer 150 schedules tasks for execution on the Robot Device 120. For example, the Task Planning Computer 150 may assign a task that involves retrieving the Box 135 and transporting it to another location in the operating environment. In this case, instructions for the task are pushed to the Robot Device 120 so that it can execute accordingly. In addition to these instructions, the Task-Specific Module 110 and any other task-specific modules necessary to complete the assigned task can be pushed to the Robot Device 120. Thus, the Robot Device 120 will have this information readily available when needed.
In some embodiments, task-specific 110 and machine-specific 105 modules can be mixed and matched to execute new tasks or collections of tasks and machine combinations or, in case of particularly difficult combinations, jump-start the learning process of a novel task from a good initialization. For instance, an object can store different task-specific modules depending on the collection of anticipated robot actions with respect to the object, as shown conceptually in
The techniques described herein are critical to enable AI applications on the edge. Machines can interact with their environment autonomously without the need to create general-purpose AI components, which has not proven successful in the past. Virtually all the success of AI applications in recent years has been in vertical applications that are domain specific. The modular approach of the present disclosure enables modular learning of autonomous systems, reducing both memory and computing requirements.
Moreover, in applications where training data is scarce, the disclosed modular approach facilitates the deployment of AI applications, making it easy to transfer any acquire knowledge by any agent. In particular, decomposition of learned policies in task- and machine-specific components enables training of smaller, manageable actions instead of universal robot feedback policies.
Furthermore, the techniques described herein can be conceptually understood as splitting the neural networks into a fixed and a flexible part. The fixed part is machine-specific, and may be deployed to inference-optimized parts of hardware accelerators. The flexible, task-specific parts of the neural network are used to train new classes and adapt current classifiers on the fly. These parts may be deployed to flexible parts of a hardware accelerator, such as SHAVE, GPU, or CPUs.
As shown in
The computer system 410 also includes a system memory 430 coupled to the bus 421 for storing information and instructions to be executed by processors 420. The system memory 430 may include computer readable storage media in the form of volatile and/or nonvolatile memory, such as read only memory (ROM) 431 and/or random access memory (RAM) 432. The system memory RAM 432 may include other dynamic storage device(s) (e.g., dynamic RAM, static RAM, and synchronous DRAM). The system memory ROM 431 may include other static storage device(s) (e.g., programmable ROM, erasable PROM, and electrically erasable PROM). In addition, the system memory 430 may be used for storing temporary variables or other intermediate information during the execution of instructions by the processors 420. A basic input/output system (BIOS) 433 contains the basic routines that help to transfer information between elements within computer system 410, such as during start-up, may be stored in ROM 431. RAM 432 may contain data and/or program modules that are immediately accessible to and/or presently being operated on by the processors 420. System memory 430 may additionally include, for example, operating system 434, application programs 435, task-specific modules 436 and program data 437. The application programs 435 may include, for example, one or more executable applications that enable retrieval of one or more of the task-specific modules 436 in response to a request received from the Robot Device 480.
The computer system 410 also includes a disk controller 440 coupled to the bus 421 to control one or more storage devices for storing information and instructions, such as a hard disk 441 and a removable media drive 442 (e.g., compact disc drive, solid state drive, etc.). The storage devices may be added to the computer system 410 using an appropriate device interface (e.g., a small computer system interface (SCSI), integrated device electronics (IDE), Universal Serial Bus (USB), or FireWire).
The computer system 410 may perform a portion or all of the processing steps of embodiments of the invention in response to the processors 420 executing one or more sequences of one or more instructions contained in a memory, such as the system memory 430. Such instructions may be read into the system memory 430 from another computer readable medium, such as a hard disk 441 or a removable media drive 442. The hard disk 441 may contain one or more datastores and data files used by embodiments of the present invention. For example, in some embodiments, the hard disk 441 may be used to store task-specific modules as an alternative or supplement to the RAM 432. Datastore contents and data files may be encrypted to improve security. The processors 420 may also be employed in a multi-processing arrangement to execute the one or more sequences of instructions contained in system memory 430. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions. Thus, embodiments are not limited to any specific combination of hardware circuitry and software.
As stated above, the computer system 410 may include at least one computer readable medium or memory for holding instructions programmed according to embodiments of the invention and for containing data structures, tables, records, or other data described herein. The term “computer readable medium” as used herein refers to any medium that participates in providing instructions to the processor 420 for execution. A computer readable medium may take many forms including, but not limited to, non-volatile media, volatile media, and transmission media. Non-limiting examples of non-volatile media include optical disks, solid state drives, magnetic disks, and magneto-optical disks, such as hard disk 441 or removable media drive 442. Non-limiting examples of volatile media include dynamic memory, such as system memory 430. Non-limiting examples of transmission media include coaxial cables, copper wire, and fiber optics, including the wires that make up the bus 421. Transmission media may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.
When used in a networking environment, computer system 410 may include modem 472 for establishing communications with a Robot Device 480 or other remote computing system over a network 471, such as the Internet. Modem 472 may be connected to bus 421 via user network interface 470, or via another appropriate mechanism. It should be noted that, although the Robot Device 480 is illustrated as being connected to the computer system 410 over the network 471 in the example presented in
Network 471 may be any network or system generally known in the art, including the Internet, an intranet, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a direct connection or series of connections, a cellular telephone network, or any other network or medium capable of facilitating communication between computer system 410 and other computers (e.g., Robot Device 480). The network 471 may be wired, wireless or a combination thereof. Wired connections may be implemented using Ethernet, Universal Serial Bus (USB), RJ-11 or any other wired connection generally known in the art. Wireless connections may be implemented using Wi-Fi, WiMAX, and Bluetooth, infrared, cellular networks, satellite or any other wireless connection methodology generally known in the art. Additionally, several networks may work alone or in communication with each other to facilitate communication in the network 471.
The general architecture of the computer system 410 may be used to implement the internal computing system of the Robot Device 480. In some embodiments, the various components of the computer system 410 described above can be used in a simplified form. For example, the Robot Device 480 may use a single processor and a relatively small amount of system memory 430. Additionally, components such as the hard disk 441 and removable media drive 442 may be omitted. Furthermore the Robot Device 480 may store additional data such as machine-specific modules to enable its performance of the techniques described herein.
The embodiments of the present disclosure may be implemented with any combination of hardware and software. In addition, the embodiments of the present disclosure may be included in an article of manufacture (e.g., one or more computer program products) having, for example, computer-readable, non-transitory media. The media has embodied therein, for instance, computer readable program code for providing and facilitating the mechanisms of the embodiments of the present disclosure. The article of manufacture can be included as part of a computer system or sold separately.
While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.
Unless stated otherwise as apparent from the following discussion, it will be appreciated that terms such as “applying,” “generating,” “identifying,” “determining,” “processing,” “computing,” “selecting,” or the like may refer to the actions and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (e.g., electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices. Embodiments of the methods described herein may be implemented using computer software. If written in a programming language conforming to a recognized standard, sequences of instructions designed to implement the methods can be compiled for execution on a variety of hardware platforms and for interface to a variety of operating systems. In addition, embodiments of the present invention are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement embodiments of the present invention.
An executable application, as used herein, comprises code or machine readable instructions for conditioning the processor to implement predetermined functions, such as those of an operating system, a context data acquisition system or other information processing system, for example, in response to user command or input. An executable procedure is a segment of code or machine readable instruction, sub-routine, or other distinct section of code or portion of an executable application for performing one or more particular processes. These processes may include receiving input data and/or parameters, performing operations on received input data and/or performing functions in response to received input parameters, and providing resulting output data and/or parameters.
The functions and process steps herein may be performed automatically or wholly or partially in response to user command. An activity (including a step) performed automatically is performed in response to one or more executable instructions or device operation without user direct initiation of the activity.
The system and processes of the figures are not exclusive. Other systems, processes and menus may be derived in accordance with the principles of the invention to accomplish the same objectives. Although this invention has been described with reference to particular embodiments, it is to be understood that the embodiments and variations shown and described herein are for illustration purposes only. Modifications to the current design may be implemented by those skilled in the art, without departing from the scope of the invention. As described herein, the various systems, subsystems, agents, managers and processes can be implemented using hardware components, software components, and/or combinations thereof. No claim element herein is to be construed under the provisions of 35 U.S.C. 112(f) the element is expressly recited using the phrase “means for.”
Number | Date | Country | Kind |
---|---|---|---|
19156887.2 | Feb 2019 | EP | regional |