The present invention relates to the field of robots, especially to a method and system for trajectory or path learning of a robot by human demonstration, e.g. for learning of a welding operation by human demonstration. More specifically, the invention provides a robot learning system and a method for controlling a robot involving a probe sensor to be mounted on the robot during a learning session.
Robots are suited for repeating tasks with a high precision. Thus, robots are suited for performing working processes, e.g. welding, laser cutting, spraying etc., where the same sequence of motion for following a trajectory or path must be precisely for each object to be processed. I.e. performing a sequence of leading a process tool through the same trajectory, both with respect to position in space of the tip of the process tool, and with respect to orientation of the process tool.
Different methods exist for trajectory learning of robots. Some methods are based on programming, whereas other methods are based on a human user guiding the robot through the trajectory. Such methods are suited for robot learning in case of robots which are supposed to carry out simple tasks without high demands on precision of the trajectory to be followed. Thus, human demonstration methods are often complicated for high precision tasks, such as following a complicated shape of an object, e.g. for welding a traced. For such tasks, it is often necessary to move the robot in small steps to ensure for each step that the process tool tip is in the right position, and that the orientation of the process tool is also correct for the process to be performed.
Thus, such sequence of manually recording single steps, until the complete trajectory has been followed, is complicated and very time consuming, especially in case of complicated shapes to be followed. Further, since the trajectory has been recorded in single steps, there may be problems in the translation into a continuous sequence to be carried out by the robot, due to its limitations with respect to maximum speed and accelerations etc. This has to be done by off-line processing.
U.S. Pat. No. 6,212,443 describes a robot learning system with one single combined position and orientation controller in the form of a handle which is operated by the user's one hand. This handle is fixed to a force sensor which senses the operator's movements and controls the robot accordingly, thereby allowing the user to control position and orientation of a welding torch positioned on the tool center point of the robot. The user holds in her/his other hand a teaching apparatus which has a servo power switch to be hold down by the user, due to safety reason.
DE 20 2008 014481 U1 describes a robot controller with a control device separate from the robot, wherein the control device has two joysticks for controlling position of a robot arm.
EP 2 342 608 describes a method for programming an industrial robot. The industrial robot is moved manually in particular along a space curve. A distance between the industrial robot and an object is determined, and the drives of the industrial robot are controlled such that the industrial robot can be moved manually only at a maximum speed, which is dependent on the distance determined from the object.
WO 2015/090324 A1 describes a system for switching between control points of a robotic system involving an industrial robot including a robot arm with a number of joints and provided with a tool interest point movable in a plurality of degrees of freedom.
Thus, according to the above description, it is an object of the present invention to provide an efficient robot learning system and method which allows fast robot learning of complicated shaped trajectories.
In a first aspect, the invention provides a robot learning system for trajectory learning of an associated robot comprising a robot arm between a base and a tool center point, by demonstration from a user, the system comprising
Such robot learning system is advantageous, since the probe sensor can assists the user in controlling movement of the robot for following a desired trajectory. The probe sensor serves as a dummy tool in the real-time learning session, where controlling of the tool center point (TCP) of the robot becomes easier, if the probe tip can measure distance and orientation relative to a complex shaped surface of an object be followed. Especially, it has been found that the system is suited for the user to demonstrate complex shaped trajectories to the robot in a continuous manner, where not only position in space, but also orientation of the robot during the trajectory tracking is important, e.g. for teaching a welding operation to the robot.
Thus, data can be logged in a continuous manner during the user operating the robot, and this provides an easy trajectory learning by demonstration, even in case of complex trajectories or paths to be tracked which can be followed at a reasonable pace by the user without interruptions. Especially, the probe may have a tip arranged to continuously touch the surface of the object forming the trajectory to be followed, and preferably the probe sensor tip is resiliently or displaceably mounted in relation to the TCP, such that the tip can be forced in continuous contact with the surface of the object forming the trajectory to be followed. Hereby, the user can control the robot in a less precisely manner, and still obtain that the probe sensor tip can be maintained in contact with the object surface throughout the real-time learning session. With the relevant distance and orientation data from the probe sensor continuously provided to the robot controller during the real-time learning session the necessary information is provided about the position of the probe sensor tip and the TCP during the learning, that the robot controller can calculate position and orientation of the TCP to be followed, in case a tool with a known shape and length is used on the TCP.
Especially, the robot controller can use distance and orientation data from the probe sensor as feedback to continuously adjust movement of the TCP of the robot with the purpose of seeking to maintain a neutral or center position of the probe sensor irrespective of control input from the user via the user interface. This can help the user in controlling the robot even in case of complex trajectories without forcing the probe sensor into extreme positions.
E.g. the distance and orientation data from the probe sensor may be continuously logged during the real-time learning session, or the data from the probe sensor may be used only to continuously adjust TCP position in space and orientation during the real-time learning session, and these adjusted data can then be then logged and stored.
In the following, preferred features and embodiments of the invention will be described.
The probe sensor may comprise a longitudinally displaceable rod connected to the tip at one end and connected to a base of the probe sensor at the opposite end, so as to allow sensing of a distance between the base of the probe sensor and the tip by means of a distance sensor, such as a distance sensor comprising a linear potentiometer. E.g. the probe sensor may comprise a spring loaded rod arranged to be compressed, and it is preferably arranged with the distance sensor arranged to sense a parameter indicative of its length or distance between the tip of the rod and the TCP or at least the base of the probe sensor arranged for mounting on the TCP. This allows the task of controlling motion sequence of the robot to follow a trajectory to be easier, since the position precision is relaxed, as the tip of the rod to follow the trajectory is flexibly arranged in relation to the TCP. Hereby, especially a trace, e.g. a welding trace, can very easily followed by the tip of the rod, without the user having to maintain a precise distance between tip and TCP, as would otherwise be required in case of a tool with a fixed length mounted on the TCP. Especially, the longitudinally displaceable rod and the base of the probe sensor may be connected at a joint, so as to allow multidirectional movement of the longitudinally displaceable rod in relation to the base of the probe sensor, wherein a first angle sensor, such as a potentiometer, is arranged to sense an angle between the base of the probe sensor and the displaceable rod. More specifically, the first angle sensor may be arranged to sense an angle in a first direction between the base of the probe sensor and the longitudinally displaceable rod, and wherein a second angle sensor, such as a potentiometer, is arranged to sense an angle in a second direction between the base of the probe sensor and the longitudinally displaceable rod, wherein said first and second directions are different, such as said first and second directions being perpendicular. With such joint connection between base of the probe sensor and the longitudinally displaceable rod, a freedom of orientation of the TCP is possible without the probe sensor tip is forced out of contact with a surface to be followed. This allows the user to control orientation of the TCP of the robot with less precision.
Preferably, the probe sensor has a neutral or center position of its tip relative to its base, and especially, the tip of the probe sensor may be resiliently connected to its base, so that that the tip will return to the neutral or center position after being forced away from the neutral or center position. Such probe sensor embodiments can help to assist the user in controlling the robot to follow the trajectory, since the precision demand is relaxed, because the probe tip and TCP are allowed to move in relation to each other, within a certain limit, without forcing the probe tip out of contact with the surface to follow. Therefore, the user can control the robot movements during the real-time learning session in a less precise manner, thereby facilitating the real-time learning task.
The tip of the probe sensor may have a spherical shape, e.g. in the form of a small metal ball, so as to allow tracking of a rather narrow trajectory formed by two or more coinciding surfaces.
The processor or robot controller is preferably programmed to continuously calculate a transformation of the robot coordinates in response to data representing said signal or data from the probe sensor during the real-time learning session. Especially, the processor or robot controller may be programmed to control the robot in response to a combination of input from the user interface and data representing the signal from the probe sensor, during the real-time learning session. More specifically, the robot controller may be programmed to move, such as continuously move, the robot in response to feedback from the probe sensor, during the real-time learning session, so as to minimize a deviation between an actual position of the tip of the probe sensor and a neutral or center position of the tip of the probe sensor. Preferably, the robot controller is programmed to continuously move the robot so as to help the user in continuously controlling the robot to ensure that the tip of the probe sensor is in contact with a surface of an object to be followed. Hereby, the robot controller will assist the user in continuously obtaining the probe sensor in its neutral or center position during the real-time learning session, thereby facilitating the trajectory following task.
The user interface may comprise several control elements and be designed in various ways. In some specific embodiment, the user interface comprises a first control element comprising a first joystick arranged for operation by the user's one hand for control of position in space of the tool center point of the robot, and comprising a second control element element comprising a second joystick for tilting or rotating the tool center point of the robot, wherein the second joystick is arranged for simultaneous operation by the user's second hand. Specifically, the first joystick may be positioned at or near the tool center point of the robot, and the second joystick may be positioned at another position on the robot. Such user interface using joysticks for both position and orientation control provides an intuitive control of the robot. The position control joystick is preferably mounted at or near the TCP of the robot in combination with the orientation joystick being mounted on the robot arm, at a distance from the position joystick for comfortable operation, e.g. a distance comparable with the width of the user's shoulder or within the range of the user's shoulder width+/−30 cm. In such embodiment, the user has intuitive control of both position and orientation of the TCP, and thus the task of demonstrating a desired trajectory in real-time becomes a rather easy task for the user.
The processor or robot controller is preferably arranged to control the robot in response to the data logged during the learning session, and wherein the processor is programmed to calculate a transformation of the robot coordinates in response to an input regarding physical properties, such as length, of a tool to be mounted on the tool center point during control of the robot in response to the data logged during the learning session, and further in response to known properties of the probe sensor.
In general, the data to be logged, i.e. stored for later use, may be the direct user interface, or a processed signal from the robot controller. The signal(s) from the probe sensor may be logged directly or not.
Even though the user is close to the robot, safety features can be easily incorporated to stop motion of the robot when the user does not touch both control elements simultaneously, and using joysticks, the joysticks are preferably supplied with springs to enter a neutral position to when not being actively operated by the user. Alternatively, e.g. for control of smaller robots, the first and second control elements can advantageously be mounted on the robot so as to provide an intuitive control during the learning process. For control of large robots, a separate control console with the two control elements, e.g. joysticks, mounted thereon may be preferred.
It is to be understood that other available and known user interface control elements may be used instead of joysticks as first and second control elements, e.g. push knobs, dial elements, or a combination of these etc.
The processor of the system may be implemented in various ways, as known by the skilled person. E.g. it may form part of the controller of the robot, or it may form part of a computer system separate from the controller of the robot, such as a laptop computer with a suitable software. In the same way, the logged data may be stored or recorded in the memory contained within the controller of the robot, and/or on a separate computer or data storage system being in a wireless of wired connection with the user interface and/or the controller of the robot so as to log and store the sampled data for later use.
The system preferably comprises a safety system in order to protect the user from injuries during the learning session, especially in cases where the user is within the reach of the robot arm during the learning session. Such safety system may be arranged to cause the controller of the robot to stop movement of the robot during a learning session, unless the safety system senses that the user touches both of the first and second control elements. Especially, the safety system may comprise a sensor positioned on each of the first and second control elements for sensing contact with the user's hand, such as a sensor comprising a push button, such as a sensor arranged to sense electrical contact to the user's hand.
It is be understood that the robot may be a robot based on any known robot actuator technology arranged for position in order to follow a trajectory or path in space. The robot arm may have two, three, four, five or more joint arm elements between the base and the TCP. Some existing robots allow for logging of control data from its controller during an operation sequence of the robot, which can be used in a system according to the first aspect, e.g. for logging on a general purpose computer. The ‘controller of the robot’ is to be understood as the processing means which generates the (electrical) control signals to be applied to the various actuators of the robot for controlling the motion of the robot.
The skilled person will know how to program the controller of the robot to provide a desired translation of the signals from the user interface, i.e. control element(s) (e.g. joystick), so as to have a suitable translation between movements of the control element (e.g. joystick) and robot movement. Especially, it may be preferred that the position and orientation of the TCP is controlled such that a tip of the sensor probe mounted on the TCP is in contact with an object to follow, can be controlled independently with respect to position and orientation by the first and second control elements, respectively. This may facilitate the trajectory following task further for the user, during the learning session.
The skilled person will known how to provide a suitable spatial resolution from the probe sensor, and how to sample the signal(s) from the probe sensor to provide suitable input data to the robot controller.
It is to be understood that the system according to the first aspect can be provided as a stand alone system to be mounted on existing robot and robot controllers. However, the system may as well be integrated robot and robot controller systems.
In a second aspect, the invention provides a robot system comprising
Especially, the robot may be arranged to be controlled in response to the data logged during the learning session, and to perform a welding process accordingly, such as the robot being a dedicated welding robot.
Apart from a robot for welding, the robot may be arranged for e.g. cutting, spraying, painting, milling, drawing, grinding, glueing, grouting, sealing and chamfering. Further, the invention may also be suitable for robot learning of pick and place robots, where the robot learning can be used to demonstrate the continuous trajectory motion sequence involved in a pick and placed routine.
In a third aspect, the invention provides a method for controlling a robot during trajectory learning of the robot by demonstration from a user to make the robot follow a desired trajectory during a real-time learning session, the method comprising
In a fourth aspect, the invention provides a computer program product having instructions which, when executed, cause a computing device or system comprising a processor to perform the method according to the third aspect. Especially, the computer program product may be one of: a part of a robot controller software product, and a stand-alone software product for a general computer. It is to be understood that the computer program product instructions in the form of program code which may be implemented on any processing platform, e.g. a robot controller, a general processor in a computer device, e.g. in the form of a downloadable application for a programmable device.
In a fifth aspect, the invention provides a computer readable medium having stored thereon a computer program product according to the fourth aspect.
It is appreciated that the same advantages and embodiments described for the first aspect apply as well for the second, third, fourth, and fifth aspects. Further, it is appreciated that the described embodiments can be intermixed in any way between all the mentioned aspects.
The invention will now be described in more detail with regard to the accompanying figures of which
The figures illustrate specific ways of implementing the present invention and are not to be construed as being limiting to other possible embodiments falling within the scope of the attached claim set.
The robot RB has in the example an arm formed by 5 joint arm elements between a base and a tool center point TCP. A controller of the robot CTL serves to control actuators for moving the arm elements. A probe sensor PS with a probe tip PT is mounted on the TCP of the robot RB, so as to continuously sense distance and orientation between the probe tip PT and the TCP during the real-time trajectory learning session. This allows contact with the probe tip during the trajectory following on an object, e.g. a welding trace. A signal PTL_IN is generated by the probe sensor in response to the sensed distance and orientation.
The user interface comprises two three-axis joysticks J1, J2 both mounted on the robot arm, namely on the TPC, and on an elbow between two arm elements, a suitable distance (e.g. 30-80 cm) away from each other to allow the user to comfortably operate both joysticks J1, J2 simultaneously. A specific example of a low cost joystick is Apem 3140SAL600, which is based on Hall effect sensing technology.
In
In the shown embodiment in
In
In the system, key elements are two three axes joysticks J1, J2 mounted in this example on a six axes industrial robot body, and a preferably three axes probe sensor PS. In the shown example the position controlling joystick J1 is mounted near to the 6'th joint of the robot, i.e. near the TCP. The orientation controlling joystick J2 is mounted near to the robot elbow, or at distance of human shoulder width, e.g. around 45-80 centimeters, in case of a large industrial robot. The three axes probe sensor PS is mounted on the TCP.
In the three sketches of
In
All the axes preferably have spring designs, so the axis will center the position when no forces are applied on the probe sensor PS. The probe tip PT is preferably designed for the actual workpiece and therefore must be easily replaceable. E.g. the probe tip PT may be formed as a small ball to be able to easily slide on a surface.
Equation (1) describes the transformation defined in h∈{1, . . . , n} where n is the number of records. The transformation in equation is calculated by using the rotation and position part in a homogeneous coordinate description. Equation (2) describes the transformation from the robot base to the end of the last joint for the i'th transformation. This equation is input to the robot controller for moving the robot around the scene. Equation (3) describes the transformation from the robot base to the end of the probe sensor for the i'th transformation. During the learning process, the two transformations on the right side are recorded for calculating the desired path during execution described in equation (4):
i
T
6Joint
Probe (1)
i
T
Base
6Joint (2)
i
T
Base
Probe=iTBase6JointiT6JointProbe (3)
i
T′
Base
6Probe=iTBase6JointiT6JointProbeT6JointTool
The learning part, to the left, comprises a main loop while the learning is active: #1: Loop start. #2: Read from analog to digital converter data from position joystick J1, rotation joystick J2, and probe sensor PS. #3: Calculate a transformation from the probe sensor PS equation (1) with a Feedback-Assisted algorithm to avoid outer mechanic limits for the probe sensor PS that adjusts the three probe sensor PS axes so they will be close to a center position during the learning process. Calculate transformations based thereon according to equations (1) and (2), and save them. #4: Determine probe sensor correction. #5: Calculates the new transformation from equation (2) with adjustment calculated in state #4, and send it to the robot controller CTL and return to state #1.
The execution part, to the right, comprises two states in a main loop until (user) stop STP. #6: Calculate equation (4) and send it to the robot controller CTL and return to state #6, as long as there is recorded transformation left.
A real-time learning session may begin with the robot being controlled to start with a tip of the probe sensor mounted on its TCP so as to start on one end of the desired trajectory to be followed. In a continuous real-time manner, the robot is then controlled to follow the trajectory by simultaneously controlling movement of position in space and orientation of the TCP by the user operating e.g. joysticks, and at the same time, the probe sensor provides input to the robot controller regarding distance and orientation deviation between a neutral position of the probe sensor tip and the actual position of the probe sensor tip. This is performed until the end E_T2 of the desired trajectory has been reached. During the controlling, a logging of data from the robot controller is performed with a suitable temporal and spatial precision in response to the user's operation of the user interface during the learning session. Finally, a calculation of a transformation is performed, so as to adapt the robot movement to a process tool with a given length, and this transformation calculation is performed in response to data logged from the probe sensor.
This method can be programmed as a control algorithm to form part of a robot controller software package, or it can be fully or partly implemented as a dedicated computer device.
Each potentiometer (X, Y, Z) in the probe sensor has a center position defined, meaning that the two angle potentiometers (X, Y) will be in center position, and the linear potentiometer (Z) will be partly pressed in, e.g. half way pressed in between its two outer positions. During the real-time learning process, the trajectory generator will try to keep this center position for the probe sensor, and therefore the trajectory generator will add this to the rest of the control system.
The Trajectory Generator measures the current probe sensor position and calculates a new relative robot position as output. The system will force the robot to move in a direction, so the next sensor position will be closer to the neutral or center position of the probe sensor. This behaving will avoid any exceed of the probe sensor position limits. Furthermore, this behaving will be the primary control loop and help the user controlling the robot during the real-time continuous trajectory learning session, where the system logs or records the positions and/or control signals at a predetermined sample rate throughout the learning session.
To sum up: the invention provides a robot learning system for trajectory learning of a robot RB having a robot arm between a base and a tool center point TCP. A user interface allows the user to control the robot arm in order to follow a desired trajectory during a real-time. A probe sensor PS is mounted on the TCP during the learning session. The probe sensor PS measures a distance parameter Z indicative of distance from the TCP and a surface forming the trajectory to be followed, and an orientation parameter X, Y indicative of orientation of the TCP and the surface forming the trajectory to be followed. These distance and orientation data are provided as a feedback to the controller of the robot CTL during the real-time learning session, thereby allowing the robot controller software to assist the user in following a desired trajectory in a continuous manner. Especially, the probe sensor PS may have a displaceable tip TP to follow a surface and having a neutral or center position, and where the robot controller software controls the robot movements to seek the neutral or center position irrespective of the user's control inputs. Data DT is logged during the learning session, so as to allow later control of the robot RB in response to the data DT logged during the learning session.
Although the present invention has been described in connection with the specified embodiments, it should not be construed as being in any way limited to the presented examples. The scope of the present invention is to be interpreted in the light of the accompanying claim set. In the context of the claims, the terms “including” or “includes” do not exclude other possible elements or steps. Also, the mentioning of references such as “a” or “an” etc. should not be construed as excluding a plurality. The use of reference signs in the claims with respect to elements indicated in the figures shall also not be construed as limiting the scope of the invention. Furthermore, individual features mentioned in different claims, may possibly be advantageously combined, and the mentioning of these features in different claims does not exclude that a combination of features is not possible and advantageous.
Number | Date | Country | Kind |
---|---|---|---|
15196074.7 | Nov 2015 | EP | regional |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/DK2016/050392 | 11/23/2016 | WO | 00 |