The invention relates to a teleoperation system based on a master-slave structure.
Background of the invention is the development of a teleoperation system for a medical application. The teleoperation system is intended to provide haptic feedback for the representation of interaction forces, preferably between an end effector and the surrounding tissue.
For use in surgery, telemanipulation systems, in the following also called teleoperation systems, exist, which can be referred to as a remote-controlled system. In particular, the limited integration of haptics and the implementation as a pure telemanipulation system are limitations for a wider use in surgical environment. Thanks to the use of lightweight robotics with comprehensive integrated force/torque sensors, completely novel approaches for surgical interventions are possible. The integration of haptic processes in the context of therapeutic and diagnostic concepts in medicine represents the next stage for an intuitive human-machine interface. Also, the extension of pure telemanipulation to a teleoperation with the integration of autonomous partial procedures also relieves the doctor of concentration-reducing routines.
The term haptics originates from the Greek. It means “tangible” or “suitable for touching”. In principle, therefore, all media offer the possibility of haptic perception. They feel in a certain way. A table surface can be smooth or rough. It is therefore a perception, which occurs primarily by the fingers of the hand.
With the pseudo-haptic feedback, the user is given a haptic impression via additional visual information. For example, the information on a screen gives the user the impression that a haptic feedback is present, which is actually not the case or only minimal.
In a teleoperation system based on a master-slave structure, the master, at whom the doctor sits, comprises a control unit. The control unit preferably includes two control means for the left and right hand (left, right). The doctor interacts with the control. The operational area is presented to the user by a visual user interface, such as a screen. The doctor should see on the screen only the operation area, or the end effector. For an intuitive operation it can be an advantage if one does not see his own hands during the teleoperation. In this case, it is of advantage, especially in the case of pseudo-tactics, that one cannot see his own fingers, since the irritation due to the missing or deviation of the finger from the expectation does not occur.
The slave, also referred to as a single-port robot, consists of a drive unit. Two parallel kinematic manipulators (left/right) are controlled by push rods with the movements generated in the drive units. At the top of each manipulator is the Tool Center Point (TCP), which is used to hold surgical tools (end effectors), and can be positioned in the situs, for example. The slave has one or more drives, which are arranged distally as far as possible from the end effector in the extension of the push rods of the parallel kinematic manipulator in order not to have any negative effect during sterility. The slave further comprises a camera, illuminants, and preferably a working channel. The two systems are electrically connected in the control computer.
The object of the invention is now to provide a realistic pseudo-tactical feedback without integrating a (further) actuator in the user interface for the active generation of the haptic feedback. Likewise, a sophisticated force sensor system in the end effector can be dispensed with by this method.
The object of the invention is to generate a pseudohaptic feedback in the control unit of a teleoperation system. In this case, compared to the current state of the art, an actuator in the user interface is dispensed with and the measuring technology expenditure in the end effector is reduced. The pseudohaptic feedback is generated by utilizing the visual feedback during the application and the processing of different sensory impressions to a consistent perception by the user.
Specifically, it is a teleoperation system comprising:
In a further embodiment, the Feffector is determined by one or more of the following approaches:
These may be e.g. Struts or guide rods or joints.
In a possible embodiment, the control unit is, in particular the control means, as rigid as possible and has only the flexibility necessary for the gripping force detection. The user interface should be rigid in order to obtain the following advantages (in the pseudo-freedom degree, do not allow any deflection).
However, the control means can also be designed with a constant resilience and thus for a defined deflection. This results in better (more realistic) results for the degree of freedom of pseudohaptic feedback, but loses the advantages described above for the overall system.
In a further embodiment, the gripping force FG is determined by deriving the interaction force between the control means and the user by one or more of the following methods:
This preserves the independence between gripping force (pseudohaptic feedback) and possible active haptic feedback of other degrees of freedom. The differential force measurement is achieved by measuring the gripping force of the thumb and index finger separately from each other. (In practice, the smaller and possibly the larger of the two measured values of the gripping force is presumed). If the differential force of the thumb and index finger is measured separately, the parasitic forces can be calculated by external feedback and as a result on has only the really effective force between thumb and pointing fingers. Differential force measurement thus makes it possible to measure the force independently of disturbances. Disturbances are, in this connection, additional forces which, for example, integrate spatial feedback.
F
G=Kinematic coordinate*Feffector
F
G=Kinematic coordinate+Feffector
or
F
G=Kinematic coordinate*(Feffector+Fmin)+FG_offset
Where Fmin is the force to initially move the effector, and FG_offset is the force to allow the sensor to respond in the control unit. Other dependencies, in particular linear, are also conceivable. It should be noted that the formulas should only represent the basic/general dependency. Alternate parameters can be considered, which are not yet included here. These include scaling factors of the individual forces as well as scaling factors for adapting the units in the equation and adapting them to arbitrary kinematic coordinates. The kinematic coordinate can be represented, inter alia, by the closing angle of an end effector.
For the purposes of the invention, all mathematical relationships between the gripping force FG and the kinematic coordinate of the end effector fulfill their purpose, in which an increase in the acting end effector interaction force leads to a greater necessary gripping force of the user in order to bring about a further increase in the kinematic coordinate.
The selected relationship as well as the scaling factors should be selected depending on the environment manipulated by the end effector.
Pseudohaptic feedback works up to a frequency of approx. 10 Hz. This barrier results from the ability of humans to self-consciously generate forces and movements up to this frequency. (DIN EN ISO 9241 910). This serves mainly to the haptically kinesthetic senses.
Tactile haptic feedback can be output for frequencies that go beyond this. For this purpose, an actuator system can be used in the user interface, which couples a direct or non-direct haptic feedback to the user. (Frequency range approx. 50 Hz-1000 Hz according to DIN EN ISO 9241 910). Thus, information regarding material selectivity, surface structures, etc. can be presented.
In a further embodiment, a unit for generating a tactile haptic feedback is used on the control unit or control means, whereby a signal, which is sent to the unit for generating tactile haptic feedback, is detected by a sensor in the slave, the spectral components of this signal are preferably in the range from 50 to 1000 Hz.
The output of the previously described tactile haptic feedback can be effected by:
1. Force output by inertial mass motors
2. Eccentric motors
3. Piezo actuators—Direct between the control unit and the control unit
4. Piezo actuators—Intermediate between the base of the control unit and the fingers
5. Piezo actuators for generating surface waves at any location of the control means
Thus, on the one hand, controlled force variables, or accelerations, can be represented.
In a further embodiment, the above-mentioned elements are formed in such a way that the acting force direction of the unit for generating tactile haptic feedback exert no or only minimal forces in the direction of the gripping force FG in order to reduce control instability in the system.
In one embodiment, when such a feedback is introduced, an attempt is made to remove the forces and deflections exerted from the actually acting force direction in order to open the control circuit and thus reduce control-technical instabilities in the system. In addition, the position pre-selection signals can be “notch-filtered” (narrow-band filters or notch filters) depending on the output “high-frequency” tactile output values in order to obtain the control-related stability in the haptic system. By using a notch filter a narrow band elimination of a certain frequency is possible. This can be adapted adaptively to the frequency of the tactile feedback. In one embodiment, the position presetting signals may also be passed through a low-pass filter with a cutoff frequency below the typical tactile feedback frequencies, e.g. 40 Hz, so as to separate the frequency ranges of the channels from each other.
In one embodiment, the sensor in the slave is an acceleration sensor. Alternatively, encoder signals of the actuators can be used. High-frequency signals can also be derived from force sensors which have already been described. One could also imagine using “surface acoustic wave” (SAW) sensors to detect surface vibrations in the kinematic components or at the end effector.
Another part of the invention is the construction of the slave for a teleoperation system, e.g. as described above. The slave can of course also be used for other systems and is not limited to a teleoperation system and vice versa. Components can also be used in other combinations.
The slave includes:
At least three pushing rods arranged as tripod, each having two active degrees of freedom in the form of translation and rotation, and each being driven by means of a drive into the degrees of freedom. More pushing rods can also be possible, their arrangement can also be different;
An end effector which is connected to the push rods via kinematic chains, the kinematic chains being designed in such a way that the end effector can be aligned and opened and closed in three dimensions, by translation or rotation of the push rods.
In one possible embodiment, there is a kinematic chain, which is designed as a main chain, the rotation of which leads to a rotation of the end effector and its displacement leads to a displacement of the end effector.
In addition, there are two chains which are formed as side chains whose displacement results in a displacement of the end effector and whose rotation leads to an opening or closing or angling of the end effector.
In one possible embodiment, the rotations of the secondary chains are converted into a linear movement via a spindle and a carriage, which opens or closes the angled end effector.
The kinematic main chain preferably has four degrees of freedom and/or the kinematic secondary chain preferably has six degrees of freedom.
The secondary chain is preferably connected to the main chain or its pushing rod via swivel joints, wherein the swivel joints are designed as U-shaped tensioning elements.
In sum, more favorable, more robust and more easily sterilizable systems can be developed with the invention. The use of pseudo-haptic feedback in teleoperation systems has advantages over conventional haptic teleoperation systems in terms of regulatory stability.
The invention is described below with reference to a teleoperation system for minimally invasive surgery, which is not to be understood as limiting. This transmits control information from the user to an intracorporal manipulator and represents the interaction forces between the end effector of the intracorporal manipulator and tissue as haptic and pseudo-haptic feedback on the control unit.
The slave is shown in
The master consists of two control units according to
The gripping force of the user is used as the control variable for the closing angle phi of an intracorporal end effector (see, for example,
In order to generate the haptic feedback, no actuator is necessary in this case since the user generates the force necessary for a haptic sense impression by virtue of its gripping force. A necessary prerequisite is a direct view of the end effector by the user. The fundamental function of this “pseudohaptic feedback” is known from the realm of virtual reality.
The force FG or also Fgrip is determined on the control means as shown in
In order to ensure haptic feedback of a material in the end effector/gripper, the characteristic curve (
This takes place as a function of the force required for closing or actuating the end effector. This corresponds to the interaction force Feffector due to the adjusting force balance.
The variation of the characteristic curve phi (FG) is thereby possible by adding the measured output end effector force
phi′=phi (FG+Feffector) as well as by multiplying the measured end effector force phi′=phi (FGFeffector). The two cases describe a differently strong weighting of the respectively effective end effector force (Feffector). In both cases, the necessary gripping force, which is necessary to achieve a certain closing angle phi, changes. In connection with the visual feedback on the opening of the gripper, an impression is thus obtained for the user of the material at the end effector, since the interaction force Feffector is inter alia material-dependent.
Preliminary tests show that a coupling of the gripping force and the kinematic component via a multiplication provides the better results and thus makes it easier for the user to differentiate between different material properties. Moreover, it is found that scaling factors and calculation methods can be selected depending on the nature of the environment of the end effector in order to get the optimal dynamic of the haptic perception for distinguishing special material parameters.
A necessary prerequisite for this method is the derivation of the interaction force Feffector between the gripping arms of the end effector (
The haptic feedback of the gripping force thus shown is quasi-static and can therefore be unsatisfactory when used for the representation of certain properties, such as the surface texture and the differentiation of materials and the like.
Therefore, in a further embodiment, this disadvantage is compensated in a simple manner by the integration of a highly dynamic actuator in the control means (piezo, voice coil, eccentric motor, etc.) with very small deflections required. Due to the properties of the human haptic perception, the introduction direction cannot be clearly distinguished in the case of highly dynamic signals, so that haptic feedback, which is felt in several degrees of freedom, can be represented with a one-dimensional movement of the actuator.
The measurement of the high-frequency signals could be carried out by measuring accelerations with miniaturized acceleration sensors arranged sterilisably in the end effector.
By comparing teleoperation systems with haptic feedback known from the literature, the invention not only expands the haptically perceptible range, but also reduces the design complexity of the entire control means. By using serially arranged actuators, a frequency distribution is possible for the haptic feedback. Instead of an actuator with a large bandwidth and, at the same time, great deflections in the base of the control means, the high-frequency portion of the haptic feedback is generated by a dynamic actuator with small deflections. In the end effector, the complexity of the sensor system is reduced so that multi-dimensional, highly dynamic force sensors can be replaced by a one-dimensional force sensor system and a multi-dimensional acceleration measurement. The latter is easier to integrate into the end effector since it does not have to be integrated into the main force flow direction. In addition, peripheral requirements for the sensors in terms of dynamics, overload resistance and sterilizable packaging are decreasing.
The parallel kinematic mechanism consists of kinematic chains composed of rigid or flexible struts and joints. In general, a large number of solutions are conceivable for the implementation of the joints. Thus, in addition to rigid joints, solid body joints or flexible elements, e.g. Springs, film joints, folding bellows and NiTi wires could be used.
In order to move the intracorporal manipulator, six motors are preferably installed in the extra-corporal drive unit per manipulator. A different number of motors and gearboxes are conceivable. The movements generated are transmitted via three pushing rods 4 into the intracorporal region. Two active degrees of freedom are transmitted via a push rod in the form of translation q10-q30 and rotation q40-q60. The intracorporal movements are shaped by the parallel kinematic mechanism, which consists of a kinematic main chain 18 and up to four kinematic secondary chains 8, 9, 14, 15, such that a displacement of the push rods leads to a change in the position of the TCP, a rotation of the main chain Q40 rotates the end effector arbitrarily about its longitudinal axis, and a rotation q50 and q60 opens or closes the gripper. This is achieved e.g. by means of corresponding spindles which can also be seen in
In detail, the parallel kinematic mechanism consists of a tripod-like substructure composed of the kinematic main chain 18 with four degrees of freedom and two kinematic secondary chains 8, 9 each with six degrees of freedom. These kinematic chains are connected to the main shaft 5 via pivot joints. In order to prevent jamming, these joints are realized as U-shaped clamping elements 6, 7. The rotation of the main chain is routed directly to the end effector via a universal joint located at the base so that the latter can be rotated freely about its longitudinal axis.
The rotations of the two remaining pushing rods are also transferred via cross joints along the first and second secondary chain, and are finally converted into a respective displacement via a spindle and a slide 10, 11. Via the third and fourth secondary chains 14, 15, each of which has four degrees of freedom, these movements are transmitted to sleds 21, 22 guided on the main shaft. In order to limit the forces occurring within the mechanism, compliances are integrated in the third and fourth secondary chain, in order to prevent jamming of the sled elements, these are also designed as a U-shaped bracket. The friction moment occurring within the rotary joints 12, 13 is dissipated via the secondary chains. For this purpose, the clamping elements 6 and 7 are connected to the elements 21 and 22 by means of a respective pendulum support.
Each of the displacements generated on the main shaft moves a push rod within the main shaft, this movement being applied to one of the two jaws of the end effector, e.g. by means of a cam disk or a toggle lever 16, 17. The pushing rods are guided by an elongated hole with respect to the main shaft and are locked against twisting by means of a pin. In order to obtain the rotation of the main shaft, the carriage movements 21, 22 produced on the main shaft are transmitted via pivot joints 12, 13 to the pushrod located in the shaft. As a result, the grippers can be opened or closed via a uniform rotation q50 and q60. If the push rods rotate counter-clockwise, the gripper is angled. The described state is invertible.
The parallel kinematic mechanism described has the following transfer characteristics:
1. The position of the TCP is independent of the rotations q40-q60 and is influenced by the shifts q10-q30 alone.
2. The push rods are arranged in a colinear manner so that the working space in the z direction is limited only by the maximum travel distance of the pushing rods. In the z-direction, a constant translation ratio of 1 results.
3. The longitudinal rotation of the end effector depends solely on the rotation q40.
4. The opening angle and the angle of inclination are mainly determined by the rotations q50 and q60.
In order to reference the kinematics with respect to the base plate (19), stops are attached (20) at the ends of the push rods.
The invention is not limited to the above-described embodiments but is intended to be defined by the claims.
Number | Date | Country | Kind |
---|---|---|---|
10 2015 100 694.4 | Jan 2015 | DE | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/EP2016/050901 | 1/18/2016 | WO | 00 |