The present disclosure relates to the field of robotics, and particularly to a robot teleoperation system and method.
With the development of technology, robots are widely used in various fields and have different types and structures. For example, an articulated robot has multiple of rotatable joints so as to achieve movement within a certain spatial range.
The control of the robot can be achieved by teleoperation. In an implementation, a master robot is required and manually operated, and a slave robot is configured to follow the motion of the master robot. In one scenario, the robot is taught to perform motions by robot teleoperation, and is programmed based on the data generated during the teaching. The teaching produces operation instructions that can be executed automatically by the robot. In another scenario where the environment is quite complex and the robot is difficult to be operated automatically or to be programed, the robot can be controlled remotely to preform operations in the complex environment.
Existing remote control of robots may lead to unintended movements of the slave robot or overdriving by the operator due to, for example, communication issues or environmental reasons, and consequently produce excessive contact forces between the slave robot and a workpiece or an obstacle, which leads to damage to the workpiece or robot.
One aspect of the present disclosure provides A robot teleoperation system, which includes a master robot, a slave robot and a control system configured to cause the slave robot to follow a motion of the master robot. The control system is further configured to determine a coefficient K, and determine a control force F output by the slave robot at a selected point on the slave robot based on the coefficient K and a displacement error between a reference point on the master robot and the selected point on the slave robot. The selected point corresponds to the reference point.
Another aspect of the present disclosure provides a robot teleoperation method, which includes acquiring a displacement of a reference point on a master robot and a displacement of a selected point on a slave robot corresponding to the reference point, and determine a coefficient K and determining a control force F output by the slave robot at the selected point based on the coefficient K and a displacement error between the displacement of the reference point and the displacement of the selected point.
Yet another aspect of the present disclosure provides a computer device, which includes a memory and a processor, the memory having a computer program stored therein. The computer program, when executed by the processor, causes the processor to perform steps of the above robot teleoperation method.
Yet another aspect of the present disclosure provides a readable storage medium having a computer program stored therein. The computer program, when executed by a processor, causes the processor to perform steps of the above robot teleoperation method.
Details of one or more embodiments of the present disclosure are set forth in the accompanying drawings and description below. Other features, objects, and advantages of the disclosure will become apparent from the description, the drawings, and the claims.
The foregoing and other objects, features and advantages of the invention will become apparent from the following description of particular embodiments of the invention, as illustrated in the accompanying drawings. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention.
In order to make the above objects, features and advantages of the present application more apparent and better understood, embodiments of the disclosure will be fully described hereinafter with reference to the drawings. Numerous specific details are set forth in the following description in order to provide a thorough understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and a person skilled in the art can make similar modifications without departing from the application, and therefore, the present application is not limited by the specific embodiments disclosed below.
The present application provides a robot teleoperation system which includes a master robot, a slave robot, and a control system. The control system is configured to cause the slave robot to follow a motion of the master robot, and configured to determine a coefficient K and determine a control force F output by the slave robot at a selected point on the slave robot based on the coefficient K and a displacement error between a displacement of a reference point on the master robot and a displacement of the selected point on the slave robot. The selected point corresponds to the reference point.
According to the robot teleoperation system of the present application, a reference point and a selected point are set at corresponding positions on the master robot and the slave robot, respectively, and the control force F output by the slave robot at the selected point is determined based on the coefficient K and the displacement error between the displacements of the selected point and the reference point. In some scenarios, it is able to prevent damages to components of the slave robot or external objects such as workpieces or obstacles due to excessive control force.
The master robot 20 and the slave robot 30 may be of the same structure. Taking the slave robot 30 as an example, the slave robot 30 includes a slave end effector 31 and a plurality of arms 32 connected in series, with the end effector 31 located at the end of the connected arms 32. A plurality of joints 33 are formed between adjacent arms 32 and between the arm 32 and slave end effector 31, and each joint 33 is equipped with an actuator 34, such as an electric motor, for driving the arm 32 or the slave end effector 31. The plurality of joints 33 ensure multiple degrees of freedom of movements of the slave end effector 31.
In an embodiment, the control system 40 is configured to determine the coefficient K such that a portion Fext of the control force F at the selected point is less than or equal to a predetermined threshold Flim. The portion Fext includes one portion of the control force F which serves to balance a contact force between the slave robot 30 and an external object and another portion of the control force F which serves to drive the slave robot 30 to follow a motion of the master robot 20.
In an embodiment, the control system 40 is further configured to maintain the coefficient K at a predetermined value K0 when the portion Fext is less than the predetermined threshold Flim, and adjust the coefficient K when the portion Fext reaches the predetermined threshold Flim such that the portion Fext is less than or equal to the predetermined threshold Flim.
In an embodiment, the control system 40 is further configured to, when the portion Fext reaches the predetermined threshold Flim, adjust the coefficient K based on the displacement error such that the portion Fext is less than or equal to the predetermined threshold Flim.
In an embodiment, the control system 40 provides a virtual mechanical-impedance system 50 between the master robot 20 and the slave robot 30 for controlling the slave robot 30 with the master robot 20, which enables teleoperation of the slave robot 30. It is noted that the virtual mechanical-impedance system 50 is not a physical hardware, but is a computer-based control model that is just logically similar to a physical mechanical-impedance system, and is used in this disclosure to calculate and determine the forces and torques applied to the slave robot 30. When the master robot 20 moves by manual control of an operator, the control system 40 is configured to cause the slave robot 30 to follow the motion of the master robot 20. Specifically, the control system 40 generates instructions for controlling the motion of the slave robot 30 based on relevant parameters generated according to the motion of the master robot 20 and also with the help of the virtual mechanical-impedance system 50, and controls the motion of the slave robot 30 based on the instructions, i.e., maintaining same poses and positions as the master robot 20.
It will be understood that the virtual mechanical-impedance system 50 may be provided in the control system 40 in advance, or the virtual mechanical-impedance system 50 may be created and adjusted in real time during teleoperation of the slave robot 30 based on relevant parameters.
In this embodiment, the reference point and the selected point are corresponding points on a master end effector 21 of the master robot 20 and the slave end effector 31 of the slave robot 30, respectively. It will be understood that in other embodiments, the reference point and the selected point may be set on other components of the robots as desired. The application of the virtual mechanical-impedance system 50 of the present disclosure will be described in detail below with reference to specific embodiments.
A dynamics equation of the slave end effector 31 in Cartesian space is:
where x is the displacement of the slave end effector 31 (e.g. using the tool center point TCP as the reference), {umlaut over (x)} is the second order derivative of x, A(x) is the inertia matrix (e.g. under the Cartesian space) of the slave robot 30 at the selected point, i.e., the inertia matrix of the slave robot 30 with the selected point as the origin of a body frame, {dot over (x)} is the first order derivative of x, μ(x, {dot over (x)}) is the centrifugal and Coriolis force matrix of the slave robot 30 at the selected point, p(x) is gravity matrix of the slave robot 30 at the selected point, and F is the control force for controlling the motion of the slave end effector 31.
It should be understood that, in the case that the slave robot is not following the motion of the master robot, the portion Fext only includes an equivalent external force on the slave end effector 31 when the slave robot 30 is in contact with an external object. When the slave end effector 31 moves in space, whether it touches a workpiece or an obstacle, an external force is generated on the slave end effector 31, i.e. the contact force between the slave end effector 31 and the workpiece or obstacle. If the external force is too large, damage to the slave end effector 31, the workpiece or the obstacle may occur. Therefore, the control force F on the slave end effector 31 needs to be well controlled such that the external force on the slave end effector 31 can be maintained within a safe range.
It will be understood that the description “control force or external force at the selected point” or “control force or external force on the selected point” in this disclosure represents an equivalent force of the control force output by the slave robot or the external force at the selected point.
In some embodiments, to implement the virtual mechanical-impedance control system 50, the control system 40 is further configured to dynamically determine a coefficient K and determine a control force F based on the coefficient K and the displacement error between the master end effector 21 and the slave end effector 31. In an embodiment, the control system 40 actively configures the control force based on the following condition:
where {circumflex over (Λ)}(x) is an inertia matrix of the slave robot 30 at the selected point based on a dynamics model of the slave robot, {circumflex over (μ)}(x, {dot over (x)}) is the centrifugal and Coriolis force matrix of the slave robot 30 at the selected point based on the slave robot dynamics model, {circumflex over (p)}(x) is a gravity matrix of the slave robot 30 at the selected point based on the dynamics model, xd is the displacement of the reference point, x is the displacement of the selected point, x′d is the first order derivative of xd, {dot over (x)} is the first order derivative of x, {umlaut over (x)} is the second order derivative of x, and D is a virtual damping coefficient.
It will be understood that in some embodiments, the coefficient K and the virtual damping coefficient D may be a matrix of coefficients under Cartesian space, respectively.
Since the selected point is theoretically expected to follow the motion of the reference point exactly, the displacement xd of the reference point is also an expected displacement of the selected point. The displacement x of the selected point and the displacement xd of the reference point can be measured by various displacement sensors, or obtained from other control parameters. For example, the rotation amount of the joints of the robots can be calculated based on the data of the motor encoders, and the displacements of the robot end effectors can be thus obtained based on the kinematics of the robots. The control system 40 is configured to obtain the displacement x of the selected point and the displacement xd of the reference point from, for example, the displacement sensors mentioned above. In an embodiment, the displacement x of the selected point and the displacement xd of the reference point are vector parameters each involving both moving distance and orientation, which represent the position and pose of the master end effector 21 or the slave end effector 31, respectively.
The component K(xd−x) in equation (2) corresponds to a virtual spring 51 in the virtual mechanical-impedance system 50, where K can be thus called virtual spring coefficient. As the amount of deformation of this virtual spring increases, the elastic force generated by the virtual spring increases accordingly. The component D(x′d−{dot over (x)}) corresponds to a virtual damper 52 in the virtual mechanical-impedance system 50. Both the coefficient K and the coefficient D can be assigned default values in advance, and the default values can be obtained by debugging depending on needs.
According to equations (1) and (2), equation (3) can be obtained:
where xe is the error between xd and x, {dot over (x)}e is the error between x′d and {dot over (x)}, and {umlaut over (x)}e is the error between the second order derivative of xd and the second order derivative of x.
By introducing the virtual mechanical-impedance system 50, the portion Fext calculated according to equation (3) contains an equivalent force generated by two types of the force at the selected point of the slave robot 30, i.e. one portion of the control force F which serves to balance a contact force between the slave robot 30 and an external object and another portion of the control force F which serves to drive the slave robot 30 to follow a motion of the master robot 20.
In an embodiment, the control system 40 is further configured to adjust the coefficient K based on a change of the displacement error xe, such that the portion Fext is maintained less than or equal to a predetermined threshold Flim.
Referring to
The value of D{dot over (x)}e becomes zero when the master end effector 21 and the slave end effector 31 further enter a steady state, i.e., both entering a stationary state or moving in synchronous way at a constant speed. In this case, equation (5) can be obtained:
It can be seen that, in these cases, the portion Fext can be maintained less than or equal to the predetermined threshold Flim by adjusting the coefficient K, thereby ensuring that no damage is caused to the slave end effector 31 and external objects such as workpieces or obstacles.
In an embodiment, the coefficient K in the virtual mechanical-impedance system 50 is maintained at the default value K0 before the portion Fext reaches the predetermined threshold Flim. When the slave end effector 31 and the external object begin to contact with each other, the slave end effector 31 stops moving as it is blocked, and thus the displacement error xe increases gradually as the displacement xd of the reference point increases gradually. According to equation (3), the control force F also increase continuously. During this period, the portion Fext can be continuously calculated or monitored by, for example, detection by a contact force and/or torque sensor located at the slave end effector 31 or estimation through measured joint torques, or by calculation according to the above equation (3).
When the portion Fext reaches the predetermined threshold Flim, the coefficient K is adjusted based on the displacement error to maintain the portion Fext less than or equal to the predetermined threshold. For example, a close-loop control method can be adopted during this period, and the coefficient K is adjusted downward continuously as the displacement error xe continues to rise, which is equivalent to making the virtual spring gradually “softer”. The closed-loop control of K and the portion Fext can be achieved in common ways and is not described in this disclosure.
It will be understood that the description “less than or equal to a threshold” in this disclosure means that the portion Fext can be maintained substantially less than or equal to the predetermined threshold under normal circumstances, but it does not exclude the case that the portion Fext appears above the predetermined threshold briefly or accidentally. For example, there may be a case where the portion Fext oscillates at the predetermined threshold due to, for example, communication or calculation delays.
There may also be a situation where the slave end effector 31 is in dynamic contact with an external object, that is, the slave end effector 31 is in a varying state during the contact. In the case of dynamic contact, according to equation (3), the values of D{dot over (x)}e and A{umlaut over (x)}e are not equal to zero any more. However, it is still able to control the portion Fext by adjusting the coefficient K. For example, the control force F may rise sharply in a short period of time, causing a large acceleration of the slave end effector 31 and therefore generating a large dynamic external force on the end effector 31. At this point, the portion Fext can be controlled to be, for example, less than or equal to the predetermined threshold by adjusting the coefficient K.
In an embodiment, the coefficient K is kept unchanged, and in this case, equation (6) can be obtained according to equation (3):
where Fext(K0) represents a calculated value of the portion Fext with K0 as the coefficient.
It will be understood that, before the portion Fext reaches the predetermined threshold Flim, Fext(K0) represents a true portion of the control force which serves to balance a contact force between the slave robot 30 and an external object and to drive the slave robot 30 to follow a motion of the master robot 20, and when Fext(K0) is greater than the predetermined threshold Flim, Fext(K0) is only a virtual value to be used in the calculation of the required coefficient K in the actual control process.
In an embodiment, the portion Fext is maintained at the predetermined threshold Flim when the portion Fext reaches the predetermined threshold Flim. In this case, the confident K satisfies the following equation:
Equation (8) can be further obtained by transforming the equation (7):
Also, equation (9) can be obtained according to equation (6) and equation (7):
It can be seen that, when the slave end effector 31 is in the dynamic state, the coefficient K can be determined by equation (8) or equation (9), so that the portion Fext is maintained at the predetermined threshold Flim.
When the slave end-effector 31 is in the stationary state, equation (9) can be simplified to equation (10):
It shows that, when the slave end-effector 31 in the stationary state, it is also possible to determine the coefficient K by equation (10) so that the portion Fext is maintained at the predetermined threshold Flim.
According to the embodiment described above, an initial coefficient K0 is used to adjust the control force F before the portion Fext reaches the predetermined threshold Flim, and when the portion Fext reaches the predetermined threshold Flim, the coefficient K is adjusted based on equations (8) and (9) so that the portion Fext can be stabilized at the predetermined threshold Flim. It will be understood that in another embodiment, the coefficient K can be determined directly based on equations (8) and (9) throughout the process, which aims to maintain the portion Fext at the predetermined threshold Flim from the beginning, without depending on the changes of the displacement error.
In an embodiment, the control system 40 is configured to control at least one actuator of the master robot 20 to provide tactile feedback to the operator based on the portion Fext. When a component of the slave robot 30 such as the slave end effector 31 contacts an external object, an external force is generated on the slave end effector 31, which may be detected by a force sensor located at the end of the slave end effector 31, or calculated with joint torques. The control system 40 is configured to feed this external force back to the operator such that the operator can intuitively feel the presence of the external force and its changes when operating, for example, the master end effector 21 of the master robot 20, and accordingly adjust the operation of the master robot 20. For example, if the operator has felt that the external force is too large, he or she may reduce the speed of the master robot 20 or stop the operation of the master robot 20, thereby preventing accidents such as damage from occurring. In addition, when the displacement error between the slave robot 30 and the master robot 20 is large, the operator can also intuitively feel the inertial force that the slave robot 30 has to overcome to follow the motion of the master robot 20, according to the calculation of the portion Fext mentioned previously, and thus adjust the operation accordingly (e.g., slowing down the motion of the master robot 20).
In some embodiments, the control force F applied to the slave end effector 31 includes a combined driving force on the slave end effector 31 generated by at least one actuator of the joint. In another embodiment, the control force F includes a driving force on the slave end effector 31 generated by an actuator that directly drives the slave end effector 30.
In some embodiments, the control system 40 is configured to continuously calculate and adjust the control force F at a predetermined frequency. The higher the predetermined frequency, the more responsive the slave robot 30 is to the motion of the master robot 20, and meanwhile, the control force F is more accurate.
According to an embodiment described above, the control system 40 configures the control force F according to equation (2) such that the control force F includes a virtual spring component and a virtual damper component. It will be understood that in other embodiments, the control force F may include only the virtual spring component, or a combination of the virtual spring component and other components, which can also achieve control of the portion Fext applied to the slave robot 31 by the adjustment of the coefficient K.
The present disclosure also provides a robot teleoperation method, which will be described below, with reference to the teleoperation system according to various embodiments described above.
As shown in
In an embodiment, the control system 40 is configured to determine the coefficient K such that a portion Fext of the control force F at the selected point is less than or equal to a predetermined threshold Flim. The portion Fext includes one portion of the control force F which serves to balance a contact force between the slave robot and an external object and another portion of the control force F which serves to drive the slave robot 30 to follow a motion of the master robot.
In an embodiment, the control system 40 is further configured to maintain the coefficient K at a predetermined value K0 when the portion Fext is less than the predetermined threshold Flim, and adjust the coefficient K when the portion Fext reaches the predetermined threshold Flim such that the portion Fext is less than or equal to the predetermined threshold Flim.
In an embodiment, the control system 40 is further configured to, when the portion Fext reaches the predetermined threshold Flim, adjust the coefficient K based on the displacement error such that the portion Fext is less than or equal to the predetermined threshold Flim.
In an embodiment, determining the control force F output by the slave robot 30 at the selected point includes establishing a virtual impedance control relation between the master robot 20 and the slave robot 30, and determining the control force F based on the above-described equation (2) and determining the portion Fext based on the above-described equation (3).
In an embodiment, determining the coefficient K such that the portion Fext of the control force F at the selected point is less than or equal to a predetermined threshold Flim includes: when the portion Fext is less than the predetermined threshold Flim, maintaining the coefficient K at a predetermined value K0; and when the portion Fext reaches the predetermined threshold Flim, adjusting the coefficient K based on the above-described equations (8) or the above-described equations (9).
In an embodiment, when the slave robot 30 is in a stationary state, the coefficient K is determined based on the above-described equation (10).
In an embodiment, the method further includes controlling an actuator of the master robot 20 to provide haptic feedback to an operator based on the portion Fext.
In an embodiment, the selected point is located on the slave end effector 31 of the slave robot 30 and the reference point is located on the master end effector 21 of the master robot 20.
In an embodiment, the control force F includes a combined driving force on the slave end effector 31 generated by at least one actuator of the joint of the slave robot 30.
In an embodiment, the method further includes continuously calculating and adjusting the control force F at a predetermined frequency.
The present disclosure also provides a computer device, which may be the control system 40 described above, and the architecture of which is illustrated in
In other embodiments, other steps that may be implemented when the computer program is executed by the processor and further features related to the steps may be found in the above description of the robot teleoperation method of various embodiments, which are thus omitted herein.
It will be understood by those skilled in the art that the structure illustrated in
The present disclosure also provides a readable storage medium having a computer program stored thereon, and the computer program, when executed by a processor, causes the processor to the steps of the robot teleoperation method as described above, which includes:
In other embodiments, other steps that may be implemented when the computer program is executed by the processor and further features related to the steps may be found in the above description of the robot teleoperation method of various embodiments, which are thus omitted herein.
It will be understood by those skilled in the art that achieving all or part of the processes in the methods of the above embodiments is possible by means of a computer program to instruct the relevant hardware to do so. The computer program may be stored in a non-volatile readable storage medium, and when the computer program is executed, processes such as those of the embodiments of the methods described above may be performed. Any reference to a memory, storage, database, or other medium used in the embodiments provided in the present disclosure may include at least one of non-volatile and volatile memory. The non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical memory, among others. The volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not a limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The technical features of the above embodiments can be combined in any number of ways. For the sake of brevity, not all possible combinations of the technical features of the above embodiments have been described, however, as long as the combinations of these technical features are not contradictory, they should be considered to be within the scope of the present specification.
The foregoing respective technical features involved in the respective embodiments can be combined arbitrarily, for brevity, not all possible combinations of the respective technical features in the foregoing embodiments are described, however, to the extent they have no collision with each other, the combination of the respective technical features shall be considered to be within the scope of the description.
The above described embodiments express only several embodiments of the present disclosure, and their descriptions are more specific and detailed, but they should not be construed as a limitation of the scope of the present disclosure. It should be noted that for those skilled in the art, a number of variations and improvements can be made without departing from the conception of the present disclosure, and these belong to the scope of present disclosure. Therefore, the scope of the present disclosure shall be subject to the attached claims.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/CN2022/117629 | 9/7/2022 | WO |