Automatic calibration method for robot systems using a vision sensor

Abstract

An automatic calibration method for a robot system is disclosed. The automatic calibration method for a robot system includes the steps of calibrating a sensor and a sensor coordinate system of the sensor with respect to a world coordinate system, controlling a robot under the guidance of the sensor to move a point of a tool mounted on the robot to reach a same target point with a plurality of different poses, the point of the tool in a tool coordinate system, and calculating a transformation matrix tcpTt of the tool coordinate system with respect to a tool center point coordinate system based on pose data of the robot at the same target point.

Description

FIELD OF THE INVENTION

The present invention relates to a calibration method for a robot system, and more particularly, to an automatic calibration method for a robot system using a vision sensor.

BACKGROUND

Know calibration methods for robot systems generally involve artificial teaching. For example, an operator manually controls a robot of the robot system to move a tool mounted on the robot to reach the same target point with a plurality of different poses (for a 6-axis robot, generally with four or more different poses).

The operator must visually determine whether the tool is moved to the same target point, and consequently, calibration errors arise leading to inaccurate tool usage. Furthermore, it is extremely time-consuming to repeatedly manually control the robot to reach the same target point and visually verify the movement, greatly decreasing work efficiency. Moreover, the robot system must be re-calibrated every time a tool is replaced, adding to the time burden.

SUMMARY

An object of the invention, among others, is to provide an automatic calibration method for a robot system in which calibration is achieved with high precision and high efficiency. The disclosed automatic calibration method for a robot system includes the steps of calibrating a sensor and a sensor coordinate system of the sensor with respect to a world coordinate system, controlling a robot under the guidance of the sensor to move a point of a tool mounted on the robot to reach a same target point with a plurality of different poses, the point of the tool in a tool coordinate system, and calculating a transformation matrix ^tcpT_t of the tool coordinate system with respect to a tool center point coordinate system based on pose data of the robot at the same target point.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will now be described by way of example with reference to the accompanying figures, of which:

FIG. 1 is a schematic view of a robot system according to the invention;

FIG. 2 is a block diagram of a closed-loop feedback control on a position of a tool of the robot system shown in FIG. 1; and

FIG. 3 is a block diagram of a closed-loop feedback control on a pose of the tool of FIG. 2.

DETAILED DESCRIPTION OF THE EMBODIMENT(S)

The invention is explained in greater detail below with reference to embodiments of an automatic calibration method for a robot system. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and still fully convey the scope of the invention to those skilled in the art.

A robot system according to the invention is shown in FIG. 1. The robot system has a sensor 10, a robot 20, a tool 30, a controller (not shown), and a processor (not shown). The major components of the invention will now be described in greater detail.

The sensor 10 may be a vision sensor, for example, a camera, or any other type of vision sensor known to those with ordinary skill in the art. The sensor 10 has intrinsic parameters including a focal length, a lens distortion, a pixel ratio, and a geometric relationship between a chip pose and a lens pose of the sensor 10. The sensor 10 is configured to capture an image of the tool 30, an object 40, and a target region 50 on the object 40, and identify pose (position and posture) data of the tool 30, the object 40, and the target region 50 based on the captured image.

The sensor 10 calibrates the robot system and guides the robot 20. Although only one sensor 10 is shown in the embodiment of FIG. 1, the robot system may comprise two or more sensors 10. In this case, one sensor 10 may be only used to calibrate the robot system, and the other sensor 10 may be used only to guide the robot 20. The sensor 10 may alternatively be another type of suitable sensor 10, for example a laser tracker.

The robot 20 in the shown embodiment is a 6-axis robot. One with ordinary skill in the art would understand that the robot may be any multi-freedom robot, for example, a four-axis robot or a five-axis robot. Although only one robot 20 is shown in the embodiment of FIG. 1, the robot system may comprise two or more robots 20.

The tool 30 may be any type of tool that can be mounted on the robot 20. The tool 30 is controlled by the robot 20 and is used to machine the object 40.

The controller (not shown) controls the robot system based on a pre-stored program.

The processor (not shown) processes the image data obtained by the sensor 10.

A plurality of coordinate systems are shown in FIG. 1, including a world coordinate system O, a camera coordinate system O_c, a robot coordinate system O_R, a tool center point coordinate system O_tcp, a tool coordinate system O_t, an object coordinate system O_o, and a target region coordinate system O_P. The tool center point (“TCP”) mentioned herein refers to a center point of an end of the robot 20 on which the tool 30 is mounted, and the tool center point coordinate system O_tcp correspondingly refers to a coordinate system of that end point of the robot 20.

Transformation matrices among these coordinate systems are also shown in FIG. 1, including a transformation matrix T_c of the camera coordinate system O_c with respect to the world coordinate system O, a transformation matrix T_R of the robot coordinate system O_R with respect to the world coordinate system O, a transformation matrix ^RT_Tcp of the tool center point coordinate system O_Tcp with respect to the robot coordinate system O_R, a transformation matrix ^TCPT_t of the tool coordinate system O_t with respect to the tool center point coordinate system O_Tcp, a transformation matrix ^RT_t of the tool coordinate system O_t with respect to the robot coordinate system O_R, a transformation matrix T_O of the object coordinate system O_o with respect to the world coordinate system O, and a transformation matrix ^OT_P of the target region coordinate system O_P with respect to the object coordinate system O_o. The transformation matrix ^RT_Tcp of the tool center point coordinate system O_Tcp with respect to the robot coordinate system O_R is a known aspect of the robot 20, and may be provided by the controller of the robot 20 or read from a teaching device of the robot 20.

The calibration process of the robot system will now be described generally with reference to FIG. 1. First, intrinsic parameters of the sensor 10 and the sensor coordinate system O_c of the sensor 10 are calibrated with respect to a world coordinate system O. Calibration of the sensor 10 and sensor coordinate system Oc may be performed by any known means of calibration, including calibration by a calibration plate. Second, after the sensor 10 is calibrated, all transformation matrices ^TcpT_t, T_R, ^RT_t, T_O, and ^OT_P are identified based on the calibrated sensor 10.

Identifying the transformation matrix ^TcpT_t will now be described in greater detail.

First, a target point is defined in the world coordinate system O and recognized using the sensor 10.

Second, the robot 20 is automatically controlled by the controller under the guidance of the calibrated sensor 10 to move a point of the tool 30 in the tool coordinate system O_t to reach the same target point with a plurality of different poses. In order to accurately move the point of the tool 30 to the same target point, closed-loop feedback control shown in FIG. 2 is performed on the robot 20 based on data from the calibrated sensor 10. As shown in FIG. 2, the robot 20 initially moves automatically under the guidance of a control algorithm executed by the controller to move the point of the tool 30. The sensor 10 then captures an image of the tool 30 in the world coordinate system O, processes the captured image to extract features of the captured image, and obtains the actual position of the tool 30 in the world coordinate system O based on the extracted features. The position error between the actual position P and the target position P* can then be obtained. Thereafter, the control algorithm automatically generates a control value, corresponding to the position error, to control the robot 20 until the position error between the actual position P and the target position P* becomes zero.

Third, the transformation matrix ^tcpT_t of the robot 20 is calculated based on pose data of the robot 20 at the same target point. Due to the closed loop feedback shown in FIG. 2, the point of the tool 30 in the tool coordinate system O_t is accurately moved to the same target point, improving the recognition accuracy of the calculated transformation matrix ^TcpT_t.

The robot system may need to frequently replace the tool 30. In this case, after the tool 30 is replaced by a new tool 30, the robot system automatically and immediately re-recognizes a transformation matrix ^TcpT_t of the robot 20.

In an embodiment, the above process identifying the transformation matrix ^TcpT_t may be repeated with a plurality of additional target points to improve the accuracy of the ^TcpT_t matrix calculation. For instance, a second target point is defined in the world coordinate system O and recognized using the sensor 10. The robot 20 is controlled by the controller under the guidance of the calibrated sensor 10 by closed-loop feedback control to move the point of the tool 30 in the tool coordinate system Ot to reach the same second target point with a plurality of different poses. A second transformation matrix ^tcpT_t of the robot 20 is then calculated based on pose data of the robot 20 at the same second target point.

A difference between the first transformation matrix ^tcpT_t and the second transformation matrix ^tcpT_t is then compared to a predetermined allowable range. If the difference is outside the allowable range, the calculation of the transformation matrix ^tcpT_t is restarted. If the difference is within the allowable range, an average of the first transformation matrix ^tcpT_t and the second transformation matrix ^tcpT_t is used as the overall transformation matrix ^tcpT_t.

This process can be repeated for 1 to N target points, obtaining N transformation matrices ^tcpT_t. N may be an integer greater than or equal to 2. The overall transformation matrix ^tcpT_t may be calculated by applying the Least Squares Method to the N transformation matrices ^tcpT_t. As the integer N increases the precision of the overall transformation matrix ^tcpT_t improves.

Identifying the transformation matrix T_R will now be described in greater detail.

First, the transformation matrix ^RT_t is calculated based on the obtained transformation matrix ^TcpT_t according to the following expression (1):^RT_t=^RT_Tcp*^TcpT_t  (1)

Second, a target pose of the tool 30 is defined in the world coordinate system O and recognized by the sensor 10 to obtain a transformation matrix T of the recognized target pose with respect to the world coordinate system O.

Third, the robot 20 is controlled by the controller under the guidance of the sensor 10 to move the tool 30 to reach the recognized target pose. The transformation matrix T_R is calculated based on the transformation matrix T of the recognized target pose with respect to the world coordinate system O and the calculated transformation matrix ^RT_t, according to the following expression (2):T_R=T*^RT_t⁻¹  (2),wherein ^RT_t⁻¹ is an inverse matrix of ^RT_t.

In order to accurately move the tool 30 to the recognized target pose in the third step, closed-loop feedback control shown in FIG. 3 is performed on the robot 20 based on data from the calibrated sensor 10. As shown in FIG. 3, the robot 20 initially automatically moves under the guidance of the control algorithm executed by the controller to move the pose of the tool 30. The sensor 10 then captures an image of the tool 30 in the world coordinate system O, processes the captured image to extract features of the captured image, and obtains the actual pose of the tool 30 in the world coordinate system O based on the extracted features. The pose error between the actual pose S and the target pose S* can then be obtained. Thereafter, the control algorithm automatically generates a control value, corresponding to the pose error, to control the robot 20 until the pose error between the actual pose S and the target pose S* becomes zero.

The actual pose S may be pose information calculated based on the sensor 10 or pose data read from the sensor 10 space. There is a mapping relationship between the sensor space and the pose space, for example, s=L*p, wherein s refers to the pose data read from the sensor space, p refers to the actual pose data in the pose space, and L refers to a mapping matrix between the sensor space and the pose space. Thereby, the robot 20 may be controlled in the pose space, that is, a feedback signal S is a pose signal calculated based on the pose data read from the sensor 10. Alternatively, the robot 20 may be controlled in the sensor space, that is, a feedback signal S is a pose signal directly read from the sensor 10. The particular control algorithm for controlling the robot would differ between the sensor space and the pose space, but would have the same function in both spaces.

Identifying the transformation matrix T_O will now be described in greater detail.

First, an image of the object 40 is captured by the calibrated sensor 10 and processed by the processor to obtain the object coordinate system O_o. Then, the transformation matrix T_O of the object coordinate system O_o with respect to the world coordinate system O can be obtained. Generally, the object 40 to be machined is fixed, and therefore, the pose of the object 40 with respect to the world coordinate system O is constant; the object coordinate system O_o and the transformation matrix T_O only need to be identified once. However, in some conditions, the object 40 to be machined is continuously or intermittently moving and the pose of the object 40 is continuously or intermittently variable. In these conditions, the transformation matrix T_O and object coordinate system O_o must be recalculated.

Identifying the transformation matrix ^OT_P will now be described in greater detail.

First, an image of the target region 50 on the object 40 is captured by the calibrated sensor 10 and processed by the processor to obtain the target region coordinate system O_P. Then, the transformation matrix ^OT_P of the target region coordinate system O_P with respect to the object coordinate system O_O can be obtained. Since the pose of the target region 50 in the object coordinate system O_o is constant, the transformation matrix ^OT_P is constant.

Advantageously, in the automatic calibration method for a robot system according to the invention, the robot 20 can accurately move to the same target point with the plurality of different poses, improving the calibration accuracy of the robot system. Furthermore, the calibration of the robot system is automatically performed by the control algorithm, increasing the calibration efficiency, and simplifying the calibration operation.

Claims

1. An automatic calibration method for a robot system, comprising: calibrating a sensor and a sensor coordinate system of the sensor with respect to a world coordinate system;controlling a robot under the guidance of the sensor to move a point of a tool mounted on the robot to reach a same target point with a plurality of different poses, the point of the tool being in a tool coordinate system; andcalculating a transformation matrix tcpTt of the tool coordinate system with respect to a tool center point coordinate system based on pose data of the robot at the same target point.

2. The automatic calibration method for a robot system according to claim 1, wherein the calibrating step is performed by means of a calibration plate.

3. The automatic calibration method for a robot system according to claim 1, wherein the controlling step includes performing a closed-loop feedback control on the robot until a position error between an actual position of the point of the tool sensed by the sensor and a target position of the target point becomes zero.

4. The automatic calibration method for a robot system according to claim 3, further comprising calculating a transformation matrix RTt of the tool coordinate system with respect to a robot coordinate system based on the transformation matrix tcpTt.

5. The automatic calibration method for a robot system according to claim 4, further comprising controlling the robot under the guidance of the sensor to move the tool to reach a known target pose by closed-loop feedback control, until a pose error between an actual pose of the tool sensed by the sensor and the target pose becomes zero.

6. The automatic calibration method for a robot system according to claim 5, further comprising calculating a transformation matrix TR of the robot coordinate system with respect to the world coordinate system based on a transformation matrix T of the known target pose with respect to the world coordinate system and the transformation matrix RTt.

7. The automatic calibration method for a robot system according to claim 6, further comprising determining a transformation matrix TO of an object to be machined by the tool with respect to the world coordinate system using the sensor.

8. The automatic calibration method for a robot system according to claim 7, further comprising determining a transformation matrix OTP of a target region on the object with respect to an object coordinate system using the sensor.

9. The automatic calibration method for a robot system according to claim 8, wherein the transformation matrix OTP is constant.

10. The automatic calibration method for a robot system according to claim 9, wherein a pose of the object with respect to the world coordinate system is constant.

11. The automatic calibration method for a robot system according to claim 9, wherein a pose of the object with respect to the world coordinate system is continuously variable.

12. The automatic calibration method for a robot system according to claim 1, wherein the sensor is a vision sensor or a laser tracker.

13. The automatic calibration method for a robot system according to claim 12, wherein the sensor is a camera having calibrated parameters including at least one of a focal length, a lens distortion, a pixel ratio, and a geometric relationship between a chip pose and a lens pose of the camera.

14. The automatic calibration method for a robot system according to claim 13, wherein the robot system has a plurality of cameras.

15. The automatic calibration method for a robot system according to claim 14, wherein the robot system has a plurality of robots.

16. The automatic calibration method for a robot system according to claim 13, wherein the camera visually guides the robot to machine an object.

17. The automatic calibration method for a robot system according to claim 1, further comprising re-identifying the transformation matrix tcpTt automatically and immediately after the tool is replaced by a new tool.

18. An automatic calibration method for a robot system, comprising: calibrating a sensor and a sensor coordinate system of the sensor with respect to a world coordinate system;controlling a robot under the guidance of the sensor to move a point of a tool mounted on the robot to reach a same first target point with a first plurality of different poses, the point of the tool in a tool coordinate system;calculating a first transformation matrix tcpTt of the tool coordinate system with respect to a tool center point coordinate system based on pose data of the robot at the first target point;controlling the robot under the guidance of the sensor to move the point of the tool to reach a same second target point with a second plurality of different poses;calculating a second transformation matrix tcpTt of the tool coordinate system with respect to a tool center point coordinate system based on pose data of the robot at the second target point; anddetermining whether an error between the first transformation matrix tcpTt and the second transformation matrix tcpTt is within an allowable range, returning to the calibrating step if the error is not within the allowable range, and using an average of the first transformation matrix tcpTt and the second transformation matrix tcpTt as an overall transformation matrix tcpTt if the error is within the allowable range.

19. An automatic calibration method for a robot system, comprising: calibrating a sensor and a sensor coordinate system of the sensor with respect to a world coordinate system;controlling a robot under the guidance of the sensor to move a point of a tool mounted on the robot to reach a plurality of target points with a plurality of different poses, the point of the tool being in a tool coordinate system;calculating a plurality of transformation matrices tcpTt of the tool coordinate system with respect to a tool center point coordinate system based on pose data of the robot at the plurality of target points, each transformation matrix tcpTt corresponding to one target point; anddetermining an overall transformation matrix tcpTt by using the least squares method on the plurality of transformation matrices tcpTt.

20. The automatic calibration method for a robot system of claim 19, wherein the plurality of target points is at least three target points.