Patent Application
20230113622
The present application claims priority from Japanese patent application JP 2021-165454 filed on Oct. 7, 2021, the content of which is hereby incorporated by reference into this application.
The present invention relates to control of a robot that performs work including gripping an object and moving the object.
A technique disclosed in JP-A-2021-84159 (Patent Literature 1) is known as a technique of controlling a robot that performs work including gripping an object and moving the object. Patent Literature 1 discloses a control device “including a state information generation unit that generates and updates state information on a robot and a gripped object, and a control information generation unit that generates control information for controlling the robot based on a base trajectory generated in advance and the state information, the base trajectory being capable of causing the robot to move the gripped object from a start point to an end point”.
Control is performed based on a current state according to the technique disclosed in Patent Literature 1. It can be expected that robot control will be performed with higher accuracy by taking a future state into consideration. An object of the invention is to implement a system and a method capable of controlling a robot with higher accuracy.
A representative example of the invention disclosed in the present application is as follows. That is, there is provided a computer for controlling a robot that performs work including gripping an object and moving the object. The computer includes: an arithmetic device; a storage device connected to the arithmetic device; and an interface connected to the arithmetic device. The storage device stores trajectory information on a trajectory of the robot when the object is gripped and moved during the work of the robot. The arithmetic device is configured to acquire operating state information including a value indicating an operating state of the robot during the work and work state information including a value for grasping a state of the object gripped by the robot, generate estimated work state information including a value for grasping a future state of the object gripped by the robot based on the trajectory information, the operating state information, and the work state information, and generate control information for controlling the robot based on the trajectory information, the operating state information, the work state information, and the estimated work state information.
According to the invention, the robot can be controlled with high accuracy. Problems, configurations, and effects other than those described above will be apparent from the following description of embodiments.
Hereinafter, embodiments of the invention will be described with reference to the drawings. However, the invention should not be construed as being limited to the description of the embodiments described below. Those skilled in the art will easily understand that a specific configuration of the invention can be changed without departing from the spirit and scope of the invention.
In the configuration of the invention described below, the same or similar configurations or functions are denoted by the same reference numerals, and redundant description will be omitted.
The notations such as “first”, “second”, and “third” in the present specification and the like are used to identify components, and do not necessarily limit the number or order.
Positions, sizes, shapes, ranges, and the like of the components shown in the drawings and the like may not represent actual positions, sizes, shapes, ranges, and the like in order to facilitate understanding of the invention. Therefore, the invention is not limited to the positions, sizes, shapes, ranges, and the like disclosed in the drawings and the like.
The system includes a robot 100 and a computer 101. The robot 100 and the computer 101 are connected directly or via a network.
Based on control information output from the computer 101, the robot 100 grips an object (workpiece) and moves the object from a start point to an end point. The robot 100 includes a work device group 110, a controller 111, and a measurement device 112.
The work device group 110 is a group of devices that grip an object and move the object, and includes, for example, a hand, links, and a drive motor.
The controller 111 controls the work device group 110 based on the control information received from the computer 101. For example, the controller 111 moves the hand by driving the drive motor that functions as a joint and connects the links according to the control information. The controller 111 outputs, to the computer 101, operating state information including an angle, an angular velocity, and an angular acceleration of the joint, a torque and a current value of the drive motor, and the like.
The measurement device 112 measures a value for grasping a state of the object due to work of the robot 100. The value output from the measurement device 112 is also referred to as work state information. The measurement device 112 is, for example, an acceleration sensor, a force sensor, a camera, a contact sensor, or a current sensor. The robot 100 may include a plurality of measurement devices 112 of different types for measurement targets. The invention is not limited to installation positions and the number of the measurement devices 112.
The robot 100 may collectively transmit the operating state information and the work state information as one piece of information.
The computer 101 generates information on a trajectory that is a movement path of the robot 100 (trajectory information), and generates control information based on the trajectory information, the operating state information, and the work state information. The control information includes, for example, the following values.
The computer 101 includes an arithmetic device 120, a storage device 121, a communication device 122, an input device 123, and an output device 124. The hardware elements are connected via an internal bus.
The storage device 121 is a device that stores a program to be executed by the arithmetic device 120 and information, and is, for example, a memory. The storage device 121 stores robot configuration information 140, equipment configuration information 141, trajectory information 142, model information 143, and work history information 144. The storage device 121 is also used as a work area.
The robot configuration information 140 is information on a configuration of the robot 100. A data structure of the robot configuration information 140 will be described with reference to
The equipment configuration information 141 is information on equipment on which the robot 100 performs work. A data structure of the equipment configuration information 141 will be described with reference to
The trajectory information 142 is information on the trajectory that is the movement path of the robot 100. A data structure of the trajectory information 142 will be described with reference to
The model information 143 is information on a model (first model) for estimating an operating state and a model (second model) for generating the control information. The first model and the second model are functions, tables, neural networks, or the like. In the present embodiment, each model is a neural network. Each model is generated by machine learning using a simulator.
The work history information 144 is information for managing the operating state during the work and a history of a work state. Specifically, the operating state information and the work state information are accumulated in the work history information 144.
The arithmetic device 120 is a device that controls the entire computer 101, and is, for example, a processor. The arithmetic device 120 executes the program stored in the storage device 121. The arithmetic device 120 operates as a functional unit (module) that implements a specific function by executing processing according to the program. In the following description, when processing is described with a functional unit as a subject, it indicates that the arithmetic device 120 is executing a program that implements the functional unit. The arithmetic device 120 according to the present embodiment functions as a trajectory information generation unit 130, a work state estimation unit 131, and a control information generation unit 132.
The trajectory information generation unit 130 determines a trajectory of the hand based on input information such as a work content and a position of the object. The trajectory information generation unit 130 determines a trajectory of a tool based on the robot configuration information 140, the equipment configuration information 141, and the trajectory of the hand. The trajectory information generation unit 130 generates the trajectory information 142 indicating the determined trajectory. Since a method for determining the trajectory is a known technique, detailed description thereof will be omitted.
The work state estimation unit 131 estimates a future work state based on the trajectory information 142, the operating state information and the work state information accumulated in the work history information 144, and the first model.
The control information generation unit 132 generates the control information based on the trajectory information 142, the operating state information and the work state information accumulated in the work history information 144, an estimation result of the work state, and the second model.
For the functional units in the computer 101, a plurality of functional units may be integrated into one functional unit, or one functional unit may be divided into a plurality of functional units for each function.
The communication device 122 is a device for communicating with an external device, and is, for example, a network interface card (NIC).
The input device 123 is a device for inputting data, commands, and the like to the computer 101, and is, for example, a keyboard, a mouse, or a touch panel.
The output device 124 is a device for outputting a calculation result or the like of the computer 101, and is, for example, a display, a projector, or a printer.
The robot configuration information 140 stores entries including an ID 201, a classification 202, an item 203, and a content 204.
The ID 201 is a field for storing identification information of an entry. The classification 202 is a field for storing classification of elements constituting the robot 100. The item 203 is a field for storing a management item of each element. The content 204 is a field for storing a content of the management item. The content 204 stores a file, a numerical value, a character string, or the like.
For a link, a shape of the link is managed. For a joint connecting links, the links to be connected, a type of the joint, and a restriction on movement of the joint are managed. For the joint, an item to be managed as the restriction on the movement of the joint differs depending on the type of the joint.
The equipment configuration information 141 stores entries including an ID 301, an equipment name 302, an attachment target 303, a relative position 304, and a relative posture 305.
The ID 301 is a field for storing identification information of an entry. The equipment name 302 is a field for storing a name of equipment. The attachment target 303 is a field for storing a name of a target to be attached to the equipment. The relative position 304 is a field for storing information (for example, coordinates) indicating a relative installation position of the attachment target with respect to the equipment. The relative posture 305 is a field for storing information (for example, coordinates) indicating a relative posture of the attachment target with respect to the equipment.
An entry with the ID 301 of “0” indicates that positions and postures of other pieces of equipment are determined with reference to an assembly cell A. An entry with the ID 301 of “1” stores a relative position and a relative posture of a Robo A installed to the assembly cell A.
The trajectory information 142 stores a table 400 indicating a trajectory of the hand and a table 410 indicating a trajectory of the tool.
The table 400 stores entries including a passing point 401, a position 402, a posture 403, and a Time 404.
The passing point 401 is a field for storing identification information of a passing point on the trajectory. The position 402 is a group of fields for storing values indicating coordinates of the hand at the passing point. Each field of the position 402 stores a value in a Cartesian coordinate system. The posture 403 is a group of fields for storing values indicating a posture of the hand at the passing point. Each field of the posture 403 stores a value defined by a quaternion. The Time 404 is a field for storing a time at which the hand that started moving from a start point reaches the passing point.
The table 410 stores entries including a passing point 411, a posture 412, and a Time 413.
The passing point 411 is the same field as the passing point 401. The posture 412 is a group of fields for storing values indicating postures of joints at a passing point. Each field of the posture 412 stores an angle of the joint. The Time 413 is the same field as the Time 404.
The trajectory information 142 includes a set of the table 400 and the table 410 for each work content. When a main body of the robot 100 moves, the trajectory information 142 may include a table related to a trajectory of the main body of the robot 100.
The control information generation unit 132 starts loop processing of work (step S101). The loop processing of the work is repeatedly executed until a series of work such as gripping an object and moving the object along a trajectory path is completed.
The control information generation unit 132 acquires operating state information and work state information from the robot 100 (step S102), and stores the operating state information and the work state information in the work history information 144.
The operating state information is accumulated in the work history information 144 in a format as shown in
The control information generation unit 132 determines whether processing is possible (step S103). Specifically, the control information generation unit 132 determines whether the necessary number of pieces of operating state information and work state information for the processing are accumulated. In the present embodiment, it is determined whether the operating state information and the work state information for five steps are accumulated. The number of pieces of operating state information and work state information is, for example, a value obtained by dividing a measurement frequency of the measurement device 112 by an analysis frequency resolution.
When the processing is not possible, the control information generation unit 132 returns to step S102 and executes the same processing.
When the processing is possible, the control information generation unit 132 generates estimated work state information by instructing the work state estimation unit 131 to estimate a work state (step S104).
For example, as shown in
It is assumed that the restriction included in the robot configuration information 140 is taken into the first model according to the present embodiment.
The control information generation unit 132 generates control information by inputting, to the neural network that is the second model, the trajectory information 142, the operating state information and the work state information accumulated in the work history information 144, and the estimated work state information, and outputs the control information to the robot 100 (step S105).
When the work is not completed, the control information generation unit 132 returns to step S102 and executes the same processing. When the work is completed, the control information generation unit 132 ends the loop processing (step S106), and ends a series of processing.
At a start point of a trajectory, step S105 is executed without executing the processing of step S104. In this case, the control information generation unit 132 inputs, to the second model, for example, the trajectory information 142, and the operating state information and the work state information accumulated in the work history information 144, and generates the control information. The control information at the start point may be generated by another method.
According to the first embodiment, the computer 101 can generate the control information for implementing safe and reliable work by the robot by taking into consideration a current state of the object, a current state of the robot, and a future state of the object.
In the first embodiment, one computer 101 controls the robot 100, and a computer system including a plurality of computers 101 may perform the same control. In this case, functional units may be distributed and arranged in the plurality of computers 101.
In the related art, a measurement value measured by a sensor is used to grasp states of a robot and an object. Since the measurement value includes noise caused by an environment, the robot, or the object, it is necessary to remove the noise in order to accurately control the robot. In this regarding, a technique disclosed in JP-A-2019-942 (Patent Literature 2) is known.
In a method disclosed in Patent Literature 2, it is necessary to perform sampling by operating a robot in an unloaded state, which increases the number of steps before system operation. In Patent Literature 2, a value necessary for control is calculated by subtracting a value measured by a sensor in the unloaded state from a value measured by the sensor in a loaded state, but a motion component is removed together with noise according to this method.
In a second embodiment, a system will be described in which noise is removed from a measurement value and a robot is controlled using the measurement value without increasing the number of steps. Hereinafter, the second embodiment will be described focusing on differences from the first embodiment.
The configuration of the system according to the second embodiment is the same as that of the first embodiment. A configuration of the robot 100 according to the second embodiment is the same as that of the first embodiment. A hardware configuration of the computer 101 according to the second embodiment is the same as that of the first embodiment. Information stored in the computer 101 according to the second embodiment is the same as that of the first embodiment. In the second embodiment, a functional configuration of the computer 101 is partially different from that of the first embodiment. The arithmetic device 120 according to the second embodiment further functions as a removed frequency determination unit 133 and a noise removal unit 134.
The removed frequency determination unit 133 determines a frequency to be removed as noise based on a shape of a trajectory. The noise removal unit 134 removes noise from a measurement value included in work state information.
Before starting the processing, the computer 101 displays a screen 1100 as shown in
The selection field 1103 is a field for selecting equipment to be used for work. When the read button 1104 is operated, a value related to the equipment designated in the selection field 1103 is read from the equipment configuration information 141.
The selection field 1105 is a field for selecting work to be performed by the robot 100. When the read button 1106 is operated, the tables 400, 410 corresponding to the work are read, and a trajectory is displayed in the display field 1101.
The selection field 1107 is a field for selecting a type of a measurement value used to generate control information. When the read button 1108 is operated, a value related to a sensor that measures the measurement value is read from the equipment configuration information 141.
The selection field 1113 is a field for selecting a first model. The screen 1100 may include a field for selecting a second model.
When the execute button 1114 is operated, the computer 101 starts the processing described below.
The control information generation unit 132 generates a removed frequency pattern by instructing the removed frequency determination unit 133 to generate the removed frequency pattern (step S151). Specifically, the following processing is executed.
(S151-1) The removed frequency determination unit 133 acquires the trajectory information 142, divides the trajectory into partial trajectories, and generates a set having the divided partial trajectories as elements. The removed frequency determination unit 133 generates a plurality of sets by changing a division method. The trajectory itself is also generated as one set. Further, the removed frequency determination unit 133 generates a group having the sets as elements. The removed frequency determination unit 133 assigns an identification number as identification information of the group.
(S151-2) The removed frequency determination unit 133 selects one set from the group.
(S151-3) The removed frequency determination unit 133 selects a partial trajectory from the selected set.
(S151-4) The removed frequency determination unit 133 selects a coordinate axis to be analyzed. At this time, the removed frequency determination unit 133 adds an entry to noise analysis information 1000.
The noise analysis information 1000 stores entries including an ID 1001, a partial path ID 1002, a coordinate axis 1003, and a frequency 1004. The ID 1001 is a field for storing identification information of an entry (group). The partial path ID 1002 is a field for storing identification information of a partial path. The partial path ID 1002 stores a set of a passing point serving as a start point of the partial path and a passing point serving as an end point of the partial path. The coordinate axis 1003 is a field for storing a coordinate axis to be analyzed. The frequency 1004 is a group of fields for storing spectrum values of frequency components. The number of frequency components can be set as desired. A total value of the spectrum values of the respective frequency components is 1.
The removed frequency determination unit 133 sets identification information of a group in the ID 1001 of the added entry, sets identification information of a partial path in the partial path ID 1002, and sets a coordinate axis in the coordinate axis 1003.
(S151-5) The removed frequency determination unit 133 converts a shape of a partial trajectory on the selected coordinate axis into a frequency component using a known technique such as fast Fourier transformation (FFT). The removed frequency determination unit 133 sets a spectrum value of the frequency component in the frequency 1004 of the entry added to the noise analysis information 1000.
(S151-6) The removed frequency determination unit 133 determines whether the processing is completed for all the coordinate axes, namely an X axis, a Y axis, and a Z axis. When the processing is not completed for all the coordinate axes, namely the X axis, the Y axis, and the Z axis, the removed frequency determination unit 133 returns to S151-4 and executes the same processing.
(S151-7) When the processing is completed for all the coordinate axes, namely the X axis, the Y axis, and the Z axis, the removed frequency determination unit 133 determines whether the processing is completed for all the partial trajectories of the selected set. When the processing is not completed for all the partial trajectories of the selected set, the removed frequency determination unit 133 returns to S151-3 and executes the same processing.
(S151-8) When the processing is completed for all the partial trajectories of the selected set, the removed frequency determination unit 133 determines a frequency to be removed for each combination of a partial trajectory and a coordinate axis. In the present embodiment, a frequency corresponding to a spectrum value larger than a threshold is determined as the frequency to be removed. The removed frequency determination unit 133 stores information on the frequency to be removed for each combination of the partial trajectory and the coordinate axis in a work area as a removal pattern. The identification information of the group is assigned to the removal pattern.
(S151-9) The removed frequency determination unit 133 determines whether the processing is completed for all the sets. When the processing is not completed for all the sets, the removed frequency determination unit 133 returns to S151-2 and executes the same processing. When the processing is completed for all the sets, the removed frequency determination unit 133 displays removal pattern information in the removal pattern display field 1109. The removal pattern is displayed in the removal pattern display field 1109 in a table format. The table stores entries including an ID 1110, a removal pattern 1111, and a selection 1112. The ID 1110 is a field for displaying identification information of a removal pattern. The removal pattern 1111 is a field for displaying the removal pattern. The selection 1112 is a field for selecting the removal pattern. A user selects a removal pattern to be used by inputting a check in the selection 1112. The removed frequency determination unit 133 displays the removal pattern superimposed on a trajectory in the display field 1102.
When selection of the removal pattern is received from the user, the control information generation unit 132 ends the processing of step S151.
The processing from step S101 to step S103 is the same as that in the first embodiment. When it is determined in step S103 that the processing is possible, the control information generation unit 132 instructs the noise removal unit 134 to remove noise, thereby removing the noise from the work state information (step S152). Specifically, the following processing is executed.
(S152-1) The noise removal unit 134 reads the removal pattern selected by the user.
(S152-2) The noise removal unit 134 identifies a trajectory of each time step based on the trajectory information 142 and the operating state information, acquires a frequency corresponding to the identified trajectory from the removal pattern, and subtracts the frequency from the work state information. As a result, for example, noise removal as shown in
The processing from step S104 to step S106 is the same as that in the first embodiment. The second embodiment is different from the first embodiment in that the work state information from which the noise is removed is used in steps S104 and S105.
During the work, the control information generation unit 132 may display display fields 1121, 1122, 1123, 1124 indicating a noise removal result and the estimated work state information on a left side of the screen 1100 as shown in
According to the second embodiment, the noise can be removed from the measurement value without increasing the number of steps and while remaining a motion component.
In the second embodiment, one computer 101 controls the robot 100, and a computer system including a plurality of computers 101 may perform the same control. In this case, functional units may be distributed and arranged in the plurality of computers 101.
The invention is not limited to the embodiments described above, and includes various modifications. For example, the embodiments described above have been described in detail for easy understanding of the invention, and the invention is not necessarily limited to those including all the configurations described above. A part of the configuration of each embodiment can be added to, deleted from, or replaced with another configuration.
In addition, a part or all of the above configurations, functions, processing units, processing methods, and the like may be implemented by hardware through, for example, design using an integrated circuit. Further, the invention can also be implemented by a program code of software that implements the functions of the embodiments. In this case, a storage medium recording the program code is provided on a computer, and a processor in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium implements the functions of the above embodiments, and the program code itself and the storage medium storing the program code constitute the invention. Examples of the storage medium for supplying such a program code include a flexible disk, a CD-ROM, a DVD-ROM, a hard disk, a solid state drive (SSD), an optical disk, a magneto-optical disk, a CD-R, a magnetic tape, a nonvolatile memory card, and a ROM.
The program code that implements the functions described in the present embodiment can be implemented in a wide range of programs or script languages such as assembler, C/C++, perl, Shell, PHP, Python, and Java (registered trademark).
Further, the program code of the software that implements the functions of the embodiments may be distributed via a network to be stored in a storage unit such as a hard disk or a memory of a computer or a storage medium such as a CD-RW or a CD-R, and a processor in the computer may read and execute the program code stored in the storage unit or the storage medium.
In the embodiments described above, control lines and information lines that are considered to be necessary for description are shown, and not all control lines and information lines are necessarily shown on a product. All configurations may be interconnected.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2021-165454 | Oct 2021 | JP | national |
