Patent Application
20240133916
The present invention relates to a method for detecting operating anomalies of an unconstrained master device of a master-slave robotic system for medical or surgical teleoperation, and a corresponding master-slave robotic system for medical or surgical teleoperation equipped so as to perform the aforesaid method.
In the context of robotic teleoperated surgery, in the presence of master-slave robotic systems for medical or surgical teleoperation, it is very important to evaluate in real time whether the master device which imparts the movement to the slave is functioning well and operates in the expected conditions, adapted to ensure effective action and patient safety, and it is also important to verify in real time that the master device is not operating in abnormal conditions or situations.
This need is felt both in the context of master devices with an unconstrained, magnetically or optically detected interface, and in the context of master devices with a mechanically constrained interface.
In the context of mechanically unconstrained or “ungrounded” master devices (recently emerged as an effective and advantageous solution, as for example shown in documents WO-2019-220407, WO-2019-220408 and WO-2019-220409 of the same Applicant) the aforesaid requirement poses complex technical challenges.
In particular, in a master-slave robotic system, in which the master is not mechanically constrained or motorized, the transmission of unintentional commands to the surgical (or micro-surgical) device, deriving from an uncontrolled operating situation of the master device, must be prevented to avoid risks for the patient. Examples of such solutions having an unconstrained master are shown by documents US-2011-118748, in which the master is worn by the surgeon, and WO-2020-0092170, in which the master body has a substantially oval shape.
The known robotic master-slave systems for medical or surgical teleoperation, with a mechanically unconstrained (or “ungrounded” or “groundless”) master device, do not provide fully satisfactory solutions to the aforesaid needs, especially taking into account the very stringent safety requirements which derive from the fact that any anomaly in the operation or condition of the master device can identify consequent anomalies in the operation of the slave device and the surgical instrument associated therewith, intended to act on the patient, with possible even serious consequences on the patient.
Therefore, in this context, the need is strongly felt to apply procedures for verifying any abnormal operating conditions of the master device in real time, conducted automatically by the robot control system for medical or surgical teleoperation, such as to be are efficient and reliable, in order to meet the stringent safety requirements which are required by such applications.
It is the object of the present invention to provide a method for detecting operating anomalies of a master device of a robotic master-slave system for medical or surgical teleoperation, which allows at least partially overcoming the drawbacks mentioned above with reference to the prior art, and responding to the needs mentioned above particularly felt in the technical field considered. Such an object is achieved by a method according to claim 1.
Further embodiments of such a method are defined by claims 2-21.
It is another object of the present invention to provide a method for managing anomalies detected in a master device comprising carrying out the aforesaid method for detecting anomalies of the master device. This method is defined by claim 22.
It is also an object of the present invention to provide a robotic system for medical or surgical teleoperation equipped to perform the aforesaid anomaly detection method.
Such an object is achieved by a system according to claim 23.
Further embodiments of such a method are defined by claims 24-44.
Further features and advantages of the system and method according to the invention will become apparent from the following description of preferred embodiments, given by way of indicative, non-limiting examples, with reference to the accompanying drawings, in which:
bis diagrammatically show a teleoperated surgery system, according to some embodiments;
ter diagrammatically shows a portion of a teleoperated surgery system, according to an embodiment;
With reference to
Such a method comprises the steps of detecting and/or calculating, by one or more sensors, the acceleration vector of at least one point belonging to or integral with the master device, or of a virtual point uniquely and rigidly associated with the master device; and then identifying and recognizing and/or discriminating at least one detectable anomaly/fault condition based on at least one component or modulus of said detected and/or calculated acceleration vector.
The aforesaid detectable anomalies/faults comprise at least one of the following: involuntary drop of the master device and/or excessive acceleration of the master device and/or sudden and/or involuntary opening of the master device.
Furthermore, each of the detectable anomalies/faults is associated with at least one system state change to be performed if the anomaly/fault is detected.
According to an implementation option of the method, the aforesaid system state change, to be performed if an anomaly is detected, comprises exiting the teleoperation.
According to an implementation option, the identifying step comprises identifying the at least one detectable anomaly condition based on at least one component of the aforesaid detected and/or calculated acceleration vector.
According to an implementation option, the identifying step comprises identifying the at least one detectable anomaly condition based on the modulus of the aforesaid detected and/or calculated acceleration vector.
In accordance with an embodiment, the method comprises the further step of measuring and/or detecting, by one or more sensors, the position vector of said at least one point belonging to or integral with the master device and the evolution over time of such a position vector.
In such a case, the step of detecting and/or calculating said acceleration vector comprises detecting and/or calculating the acceleration vector based on the evolution over time of the respective measured and/or detected position vector.
According to an implementation option of such an embodiment, the aforesaid step of detecting and calculating the acceleration vector comprises calculating the acceleration vector by movable windows of N samples of the vector representing the position vector evolution over time, and by interpolation with second order polynomials, for the degree of freedom related to the grip, and with third order polynomials, for the degrees of freedom related to the master device translation and orientation.
According to another implementation option of such an embodiment, the aforesaid step of detecting and calculating the acceleration vector comprises calculating the acceleration vector by a Kalman-type predictive filter which uses a movement model with random acceleration dynamics adapted to estimate the master device position state and correct the estimate based on information provided by the measurement system. Such a model is preferably parameterized with regard to the process noise thereof, and with regard to the noise of the measurements.
According to a further implementation option of such an embodiment, which combines a predictive approach with that of delayed smoothing, the method considers a window of N samples, predicts forward to the end of the window, and then corrects the prediction in the past to the point in the center of the window.
Kalman's filter-based solution, with respect to the polynomial fitting option, has the potential to adapt to different observation noise values, but may be subject to an adaptation time of the filter to the input values.
In accordance with an implementation option, the acceleration is calculated based on the detection of the velocity vector and the evolution over time of the velocity vector.
According to an implementation option, the acceleration is directly detected by one or more sensors, in which such one or more sensors are accelerometers.
According to various possible embodiments of the method, as a basis for detecting anomalies, for each of the aforesaid at least one points belonging to or integral with the master device, or virtual point uniquely and rigidly associated with the master device, the linear acceleration and/or the angular acceleration and/or the linear velocity and/or the angular velocity and/or the position in Cartesian coordinates and/or the position in polar or angular coordinates are calculated or detected.
In accordance with a method embodiment, the aforesaid step of detecting and/or calculating the acceleration vector comprises detecting and/or calculating by at least two sensors the acceleration vector of each of at least two points belonging to or integral with the master device; and then calculating the acceleration vector of a virtual point uniquely and rigidly associated with the master device, corresponding to the midpoint between the points where the sensors are located.
For example, in a “gripper” master device, such a midpoint can be located on the opening circumference arc described by the one or more sensors of the “gripper” master device.
If there is only one control point, 6 degrees of freedom, i.e., 3 degrees of position and 3 degrees of orientation, can be detected.
If two control points are provided, it is also possible to detect a seventh degree of freedom, associated with the grip, representative of the opening/closing angle of the master device body.
In accordance with an embodiment, the method is performed in the context of a robotic system for medical or surgical teleoperation comprising the aforesaid master device and slave device, and further comprising a control unit.
The master device is mechanically ungrounded and adapted to be held in hand by a surgeon during surgery, and is configured to detect a manual command of the surgeon and generate a respective first electrical command signal.
The at least one slave robotic assembly comprises at least one slave surgical instrument configured to operate on the anatomy of a patient, in a manner controlled by the master device.
The control unit provided with a computer is configured to receive the aforesaid first electrical command signal from the master device, generate a second electrical command signal, based on the first electrical command signal, and provide the second electrical command signal to the slave robotic assembly, to actuate the at least one slave surgical instrument.
Furthermore, the control unit is operatively connected to the aforesaid one or more sensors to receive at least a third electrical signal representative of the detected and/or measured acceleration vector and/or the related evolution over time, and is configured to perform the aforesaid steps of calculating and detecting at least one detectable anomaly.
According to an implementation option, said third electrical signal corresponds to said first electrical signal.
According to a method embodiment, the aforesaid step of measuring and/or detecting is performed with respect to a reference coordinate frame associated with the robotic system for teleoperated surgery, and having predetermined axes and origin in a predetermined point.
For example, a fixed set of three Cartesian coordinates x, y, z is defined, having an origin O in a preset point.
In particular, according to a method embodiment, in which the robotic system for medical or surgical teleoperation comprises an operating console, the aforesaid reference coordinate frame is integral with the robotic system console.
In a method embodiment, such an operating console comprising at least one surgical chair comprising at least one seating surface for the surgeon to sit on during surgery, the aforesaid reference coordinate frame is integral with the aforesaid at least one surgical chair, for example with the seat of the surgical chair. The reference coordinate frame can be integral with a support for the seat of the surgical chair, the seat being rotatable with respect to the support for the seat.
According to an embodiment, the robotic system for medical or surgical teleoperation comprises a tracking system, which is suitable for detecting the input position and orientation of the master device within a predetermined tracking volume, so that the actuation of the slave surgical instrument depends on the manual command given by the surgeon by means of the master device and/or on the position and orientation of the master device within the tracking volume.
In a method embodiment, said operating console comprises at least one display which displays the slave workspace.
The operating console can be a console which is remote with respect to the slave device, or it can be a console located in the operating room together with the slave device.
In accordance with a method embodiment, the said step of measuring and/or detecting is performed by one or more magnetic sensors.
Each of the magnetic sensors is arranged at a respective one of the at least one point belonging to or integral with the master device, and is configured to detect a local value of a magnetic field generated by a magnetic field generator constrained to a part of the robotic system for teleoperated surgery.
In such a case, the reference coordinate frame originates at the aforesaid magnetic emitter, and has three orthogonal axes x, y, z.
According to an implementation option, said magnetic field generator belongs to sais tracking system.
In accordance with another method embodiment, the aforesaid step of measuring and/or detecting is performed by at least one optical sensor or camera, associated with and/or constrained to the robotic system for medical or surgical teleoperation. Multiple cameras can be included in stereoscopic configuration.
In such a case, the aforesaid reference coordinate frame is an internal reference coordinate system of the optical sensor or camera.
According to different possible implementation options of the embodiment disclosed above, the aforesaid at least one optical sensor or camera is constrained to and/or integral with the surgical chair, and/or is mounted on a support which is wearable by the surgeon, so as to be integral with the surgeon.
According to a method embodiment, in which the master device is a hand-held, unconstrained master device, comprising two rigid parts constrained to relatively rotate or translate with respect to a common axis, the aforesaid step of detecting and/or calculating comprises detecting and/or calculating, by respective sensors, the acceleration vector and/or the acceleration vector evolution over time, of at least two detectable points, a first point belonging to or integral with one of the aforesaid rigid parts of the master device and a second point belonging to or integral with the other one of the aforesaid rigid parts of the device.
In fact, the method can be applied, for example, to a master device having a “gripper” structure (shown for example in
The method can also be applied, for example, to a master device having a “pen” structure, having two rigid parts constrained, elastically, to translate along a longitudinal axis coinciding with the longitudinal extension of at least one (or both) of the aforesaid parts of the master device.
According to various possible embodiments of the method, said calculating step comprises calculating the acceleration vector and/or the velocity vector of said at least two detectable points, or calculating the acceleration vector and/or the velocity vector of one of said at least two detected points.
According to further implementation options, said calculating step further comprises calculating the acceleration vector and/or the velocity vector and/or the position vector of at least one of the following further points: midpoint between said two detected points and/or center of gravity of said master device, and/or of a master device rotational joint, and/or of a master device prismatic joint.
In accordance with a method embodiment, in which the master device body comprises two tips or free ends, a first tip or free end belonging to or integral with one of the rigid parts of the master device and a second tip or free end belonging to or integral with the other of the rigid parts of the device, the aforesaid two detectable points correspond to and/or are associated with a respective one of the aforesaid two tips or free ends of the master device.
In accordance with a method embodiment, in which the detectable anomaly/fault is an involuntary drop of the master device, the method comprises the following steps:
According to an implementation option the vertical acceleration threshold value ay_thr is equal to the gravity acceleration g, or is a value around g (e.g., 80% of g).
According to an implementation option, the acceleration vector of each of the aforesaid at least two detection points of the master device is calculated to provide redundancy and/or a further verification.
In fact, the consistency of the calculated acceleration measurement of the aforesaid at least two points makes it possible to improve the estimation of the anomaly determination, further reducing the time window necessary for the estimation process.
An inconsistency of the calculated measurement of acceleration of the two points can be associated with a drop with rotation of the master device, or the breakage of the rigid constraints between the two sensors.
In the specific case of mounting two sensors in the same mechanical part of the master device, the behavior of the measurements is pure redundancy.
With reference to what has been described above, according to an implementation option, in order to detect a drop of the master device from the surgeon's hand, it is possible to rely on the information of the acceleration and/or the velocity component of the master device facing downwards.
With reference to what has been described above, according to an implementation option, the master device having two rigid parts constrained to each other to rotate about a common axis, said rigid parts being elastically biased to relatively move away, in order to detect a drop of the master device from the surgeon's hand it is possible to rely on detecting an involuntary opening of the master device i.e., detecting a relative distancing movement between the two rigid parts determined by the elastic biasing action. The detection of the involuntary opening can be obtained by information on angular velocity, and/or angular acceleration, and/or linear velocity, and/or linear acceleration.
According to an implementation option, the drop of the master device is detected using information on the relation between the accelerations of two sensors, said two sensors being respectively associated with the two rigid parts of the master device. For example, a relation function between the accelerations of the two sensors can compare the relative direction between the accelerations. According to an implementation option, the relation function processes the scalar product between the two acceleration vectors of the respective sensors to give an indication of the drop of the master device; in such an implementation option, it is possible to resort to a polynomial estimator of the position or the evolution of the position over time of the scalar product between the two sensors, said polynomial estimator having peaks at the drop event of the master device.
According to an implementation option as diagrammatically shown in
In such an implementation option, the detected angle between the two acceleration vectors is very low, for example minimum, at the drop; the evaluation of the residual oscillations given by the difference between the acceleration vectors of the respective sensors are very high (for example maximum). In other words, the angle between the acceleration vectors of the two sensors when very low (e.g., minimum) indicates a high degree of consistency between the accelerations in the specific case when both facing downwards; a high degree of fluctuation in the comparison (difference) between the two acceleration vectors indicates a drop event.
According to an implementation option, the master device having two rigid parts constrained to rotate about a common axis, said rigid parts being elastically biased to relatively move away, during the drop of the master device two phenomena occur (i) involuntary opening of the master device, and (ii) inertial rotation of the master device; therefore the velocity vectors of the respective (detected and/or calculated) sensors have at least one transient of limited duration (but also for the entire duration of the drop in some cases) a first component facing downwards and a second component that can be attributed to the inertial rotation of the master device.
In such an implementation option, the detected angle between the two velocity vectors is very low, for example minimum, at the drop; the evaluation of the residual oscillations given by the difference between the velocity vectors of the respective sensors are very high (for example maximum). In other words, the angle between the velocity vectors of the two sensors when very low (e.g., minimum) indicates a high degree of consistency between the velocities in the specific case when both facing downwards; a high degree of fluctuation in the comparison (difference) between the two velocity vectors indicates a drop event.
In accordance with a method embodiment, in which the detectable anomaly/fault is an excessive acceleration of the master device (e.g., imparted in the handling by the user), the method comprises the following steps:
According to an implementation option, said vertical acceleration threshold ay_thr is lower than said total acceleration threshold atot_thr.
For example, the relation “atot=3·ay” can be used
In accordance with an implementation option, the total acceleration threshold value atot_thr (in modulus) belongs to the range between 2 g and 5 g.
According to an embodiment, the accelerations of both detection points of the master device are calculated.
According to different possible implementation options of such an embodiment, the alarm trigger condition is raised if at least one of the aforesaid detected points exceeds the threshold acceleration, or if the virtual midpoint exceeds the threshold acceleration, or if the relative acceleration between the aforesaid two points is above threshold.
According to an implementation option, the aforesaid total acceleration threshold atot_thr is defined so as to increase with the decrease of the scale factor of the motion between the master device and the slave device, and/or with the decrease of a scale factor selected by the user and applied to the teleoperated Master-Slave movement.
According to an application example, in the field of robotic micro-surgery, the scale factor can be defined in a range between 5× and 20×. Obviously, the greater such a scale factor (for example, the slave movement is scaled 20 times), the greater the trigger threshold so as to allow accelerated movements.
For example, atot_thr1 is the threshold chosen for a scale factor S1, and T is the time taken by the system to recognize the acceleration and therefore to interrupt the teleoperation. In such a case, a maximum uncontrolled movement of the slave equal to D=1/2 S1 atot_thr1 T2 is accepted. Then for the scale factor S2, setting the same distance traveled D and time T, atot_thr2=atot_thr1 S1/S2.
It should be noted that in a typical implementation option, the scale factor can be set by the user depending on the specific circumstances, and can be varied, reset by the user even during the movement of the slave.
In accordance with a method embodiment, in which the master device consists of two rigid parts mutually connected in an elastic joint which tends to open such parts at least angularly when not pressed or held firmly in the user's hand, the detectable anomaly/fault is an involuntary opening of the master device. Such a situation can occur, in particular, if is the surgeon loses control, for example because the master device has escaped his hands, and the master device, dropping, opens by snapping due to the spring of the joint.
In such a case, the method comprises the following steps:
In accordance with an implementation option, the threshold angular acceleration value ω_thr depends on the elastic rigidity of the spring associated with the elastic joint and decreases with the increasing degree of opening between the rigid parts.
According to other implementation options, the aforesaid steps of calculating, comparing and identifying are performed not on the angular acceleration, but on the angular velocity, or on the linear velocity.
For example, according to another implementation option, again referring to the case in which the master device consists of two rigid parts mutually connected in an elastic joint which tends to open such parts at least angularly when not pressed or held firmly in the user's hand, and the detectable anomaly/fault is an involuntary opening of the master device, the method comprises the following steps:
In accordance with a method embodiment, in which the master device has two rigid parts elastically constrained to translate along a longitudinal axis coinciding with the longitudinal extension of at least one of the aforesaid parts of the master device, the method comprises the following steps:
In accordance with a method embodiment, the aforesaid anomalies of involuntary drop of the master device, excessive acceleration of the master device and sudden and/or involuntary opening of the master device are all detected, and at the same time.
Such an embodiment allows a wide spectrum of controls to be obtained, aiming at the maximum possible safety.
According to a preferred implementation option, the detection of the aforesaid anomalies of involuntary drop of the master device, excessive acceleration of the master device and sudden and/or involuntary opening of the master device are subject to the further constraint that the master device is within a predeterminable working volume assigned to the master device. Such a working volume can be defined so that it is positioned around the position taken by the surgeon during the teleoperation.
According to an embodiment, the method comprises the further step of detecting the position of at least one point belonging to or integral with the master device. In such a case, the further anomaly/fault associated with the detection of the master device positioning outside permitted limits can be detected through a comparison of the detected position of at least one point belonging to or integral with the master device with respect to a limit surface, in the absolute reference frame X, Y, Z.
Such a limit surface is, in several possible implementation options, a ball or a box (parallelepiped), or in general a polytope or the convex intersection of half-spaces.
A method for managing anomalies/faults identified in a master device of a master-slave robotic system for surgical or medical teleoperation is also comprised in the present invention.
Such a method includes performing a method for identifying and recognizing and/or discriminating at least one anomaly/fault condition according to any of the embodiments described above.
Such a method further includes, if at least any of the aforesaid anomalies/faults are determined, the step of immediately interrupting the teleoperation and the movements of the surgical instrument (or “end-effector”) of the slave device, to safeguard the patient's safety.
A robotic system for medical or surgical teleoperation comprising at least one master device, at least one slave device and a control unit is further comprised in the present invention.
The at least one master device is mechanically ungrounded and adapted to be held in hand by a surgeon during surgery, and is configured to detect a manual command of the surgeon and generate a respective first electrical command signal.
The at least one slave device, or slave robotic assembly, comprises at least one slave surgical instrument configured to operate on the anatomy of a patient, in a controlled manner by the respective at least one master device.
The control unit provided with a computer is configured to receive the aforesaid first electrical command signal from the master device, generate a second electrical command signal, based on the first electrical command signal, and provide the second electrical command signal to the slave robotic assembly, to actuate the at least one slave surgical instrument.
The control unit is further configured to perform a method for identifying and recognizing and/or discriminating at least one anomaly/fault condition according to any of the previously disclosed embodiments.
According to an implementation option, the control unit is further configured to perform a method for managing identified anomalies/faults, according to any one of the previously described embodiments of the method for managing anomalies/faults.
In an implementation option of the system, the master device body comprises seats for receiving the one or more sensors in respective predeterminable positions.
According to a system embodiment, the master device body is disposable and thus typically made of plastic.
According to another system embodiment, the master device body is made of metal (e.g., titanium) and is sterilizable.
With reference to
The anomaly checks of the master device are introduced into the robotic system for teleoperation in order to intervene with the minimum latency with respect to the actual movement.
In an embodiment, the sequence of operating actions carried out includes an acquisition of information on all the degrees of freedom of movement of the master device, for example in terms of acceleration; then, filtering the signals obtained; evaluating one or more anomaly checks on the master; detecting any faults or anomalies of the master device, based on the checks that are carried out; communicating with the control unit of the machine state of the robotic system, with the user interface UI and with the end points of the slave device.
Further details on the anomaly checks carried out in some embodiments of the method (already mentioned above) will be provided below, by way of non-limiting example.
The objective of this check is to identify an involuntary drop of the master device from the surgeon's hands. Such a check is based on the detection of acceleration (or position) of the master device (without the need for further sensors to detect other quantities, such as pressure-sensitive surfaces).
The principle consists in detecting the acceleration, or in deriving the acceleration from position information (even affected by noise) and calculating the instantaneous value of the acceleration along the (downwards) direction of the gravity vector.
When such an acceleration reaches a threshold which is comparable to the gravity acceleration, the anomaly warning is issued with respect to this check.
It is assumed in the following that, in the Global Reference System, the gravitational field is oriented along the −Y axis.
For example, the acceleration estimation is based on the use of a polynomial fitting of the Y axis, and then the double derivation of the polynomial by manipulating the coefficients thereof.
Among the different fitting techniques that can be used, it is possible to mention, for example, the solution based on the Solezky-Golay filter, which is characterized in that it expresses the polynomial derived as in the FIR (Finite Impulse Response) filter, which operatively consists of taking a window of 2W+1 samples and multiplying it by a constant matrix. Such a matrix depends on two parameters: the size of the window (with half-width W) and the order of the polynomial.
The size of the window depends on the sampling time, the desired latency in computation, and the signal noise.
The order of the polynomial depends on the nature of the positional signal.
The filter is a low-pass filter, with cutoff frequency which can be expressed according to relations known in the literature, for example:
Cutoff (Hz)=Dt(Order+1)/(3.2 Window−4.6).
According to an implementation option, the master device has two detection positions (i.e., two sensors). In such a case, when any one thereof exceeds the threshold, an anomaly warning is issued.
It should be noted that the wider the window used for the estimate, the better the estimate itself, with the aforesaid algorithm. On the other hand, the narrower the window, the faster the reaction time.
A criterion for choosing an appropriate compromise between the aforesaid needs is the amount of space traveled by the controlled slave device during the unintuitive and undesired movement of the master device (for example, the dropping movement of the master). The maximum distance allowed to the path by the controlled slave device during the non-intuitive movement defined as D, and the maximum velocity of the master in this situation defined as M, then the maximum window width W is expressed by the relation:
W=2D/M/T+1,
Another type of anomaly check is related to an unintuitive movement is an excess acceleration of the master device along any direction. This event can be identified based on a detection or estimation of acceleration component by component, using the same techniques described above for the “Master Drop” case.
In this case, the vector modulus of the three components is compared with a threshold to issue any anomaly warning.
In the case of a master device having a degree of freedom related to grip, a further check can be carried out on a possible sudden opening (i.e., excessively fast with respect to what can be expected in normal conditions of voluntary control by the operator) of the grip of the master device, which is considered indicative of an involuntary opening, i.e., for example, of the abnormal situation in which the operator loses control of the master device, or of the grip on the master device, where the degree of freedom of grip of the master device is elastically preloaded at opening or closing.
The estimation of the opening velocity is performed, for example, using the same polynomial fitting described above for the “Master Drop” case, but with different parameters, associated with this particular condition.
The estimated velocity of the opening angle (or “grip angle”) obtained from the fitting is used for the evaluation of this anomaly.
In one case, the opening acceleration is estimated: the estimation of the opening acceleration is performed, for example, using the same polynomial fitting described above for the “Master Drop” case, but with different parameters, associated with this particular condition. The opening acceleration depends on the preload elasticity (for example, a spring can be included between two rigid parts constrained to rotate). The estimated acceleration of the opening angle (or “grip angle”) obtained from the fitting is used for the evaluation of this anomaly.
Another anomaly check is related to the spatial limits prescribed for operator movement. These limits are defined based on usability considerations of the specific surgical target and limitations of the sensor system used to calculate the position of the master device.
Two main scenarios can be identified in relation to such limits: a sphere centered in the center of the workspace; or a parallelepiped-shaped surface; or a geometry in which it is computationally efficient to calculate whether a point is therein or not.
When the limits of such volumes are reached by the master device, an anomaly notification is provided to the user.
In the context of a master device with constrained mechanical interface, these limits depend on the limits of the mechanical interface.
In the context of a master device with an unconstrained (ungrounded) mechanical interface, if an optical detection is considered, the workspace is the intersection of the cone trunks of each camera, built taking into account the minimum resolution necessary to identify the features being tracked. When considering magnetic tracking systems, the workspace has limits which depend on the attenuation of the magnetic field.
Thus, in summary, according to a method embodiment, by measuring or calculating the vector acceleration, i.e., modulus and direction, linear acceleration or angular acceleration of the master device, at least the following information is obtained:
The vector acceleration of the master device is either directly detected by one or more accelerometers, or derived from monitoring the evolution of the position vector, in turn detected.
As disclosed above, the present method relates to a broad class of master device interfaces for robotic systems of surgical teleoperation, characterized by position and orientation measurements.
In particular, master devices with two parts, or tips, which can be closed with a hinge or hinge joint are for example considered. Each part is associated with a position measurement, which is directly measured or deducted.
In such a case, the measurements performed on the two parts of the master device can provide up to 12 degrees of freedom detected: 3 position coordinates and 3 orientation values for the first master device portion; 3 position coordinates and 3 orientation values for the second master device portion.
Such detections always allow (and also with redundancy) to detect the 7 degrees of freedom of the mechanical structure of the master device.
The examples shown in
In
The position and rotation measurements of each of the two sensors can be represented by a three-dimensional vector of the position (thus obtaining two position vectors which we indicate as p1 and p2) and by a rotation matrix for each arm (thus obtaining two rotation matrices which we indicate as R1 and R2). Each sensor is then associated with respective position and rotation information, (p1, R1) and (p2, R2).
It should be noted that the rotation can be associated with the three-dimensional orthogonal subgroup SO(3) and thus the number of degrees of freedom is always 3 (regardless of the type of representation, whether it is based on a rotation matrix with 9 numbers, as exemplified herein, or based on 3 Euler angles (3), or based on quaternions.
The arrangement (i.e., pose, i.e., position and rotation) of the reference points (or tips) of the arms allows calculating an arrangement (i.e., pose, i.e., position and rotation) of the entire master device, for example with a position calculated as the average pM of the two positions p1 and p2, and rotation as an average of the rotations (i.e., a matrix RM having as elements the averages of the respective elements of R1 and R2). The opening angle α of the gripper can be calculated using the distances between the tips and the known lengths of the master device arms, i.e., the known distances between the joint OJ and each of the reference points provided with sensors S1, S2 (assuming that the sensors are placed at equidistant points from the joint OJ, the aforesaid two distances are equal, and are referred to herein as D).
According to an implementation option, shown for example in
In such a case, as already observed, the method comprises: detecting and/or calculating the acceleration vector and/or the respective evolution over time of each of said two detectable points; calculating the relative linear acceleration of translation of the two rigid parts of the master device constrained to translate elastically, based on said detected and/or calculated acceleration vectors; comparing the calculated relative linear acceleration of translation with a threshold linear acceleration which depends on the elastic rigidity of the constraint; and finally identifying the anomaly condition if the calculated relative linear acceleration of translation is greater than the threshold linear acceleration.
The threshold linear acceleration can be an approaching acceleration threshold between the two rigid portions and/or a distancing acceleration threshold. For example, when in operating conditions, a radial pressure action on the master device body causes the two rigid parts to move away, imparting a gripping action (opening/closing) to the surgical slave instrument.
Preferably, this implementation option which provides calculating the relative linear acceleration of translation of the two rigid parts of the master device constrained to each other in translation allows identifying an indicative anomaly condition of an uncontrolled behavior of the master device, like the implementation described above with reference to the involuntary opening in a master device having a rotational joint between the two rigid parts.
According to an embodiment shown in
According to an embodiment shown in
With respect to the coordinate frames shown in the figures and used in the method, it should be noted that the reference frame denoted as MFO is a general reference frame for the master device (e.g., associated with the workspace of the master device); the reference frames denoted as MF #1 and M2 (
According to an embodiment shown in
In particular,
A robotic system 400 for medical or surgical teleoperation comprising a master device, at least one slave device and a control unit is described below.
The master device (110; 310; 410, 420; 510) is mechanically ungrounded and adapted to be held in hand by a surgeon 450 during surgery, and configured to detect a manual command of the surgeon and generate a respective first electrical command signal. The at least one slave device 440, or slave robotic assembly, comprises at least one slave surgical instrument 460, 470 configured to operate on the anatomy of a patient, in a manner controlled by the master device.
The control unit provided with a computer is configured to receive said first electrical command signal from the master device, generate a second electrical command signal, based on the first electrical command signal, and provide the second electrical command signal to the slave robotic assembly, to actuate the at least one slave surgical instrument.
The control unit is configured to identify and recognize and/or discriminate at least one anomaly/fault condition, among a detectable set of anomalies/faults comprising at least one of the following: involuntary drop of the master device and/or excessive acceleration of the master device and/or sudden and/or involuntary opening of the master device.
Each of such detectable anomalies/faults is associated with at least one system state change to be performed if the anomaly is detected.
To identify and recognize and/or discriminate at least one anomaly/fault condition, the control unit is configured to perform the following actions:
According to an embodiment of the robotic system, the aforesaid master device consists of two rigid parts 580, 590 mutually connected in an elastic joint which tends to open such parts at least angularly when not pressed or held firmly in hand by the user.
The anomaly to be detected is an involuntary opening of the master device, and the control unit is configured to: detect and/or calculate the acceleration vector and/or the respective evolution over time of each of two detectable points; calculate the opening angular acceleration ω of the two rigid parts of the master device, based on the aforesaid detected and/or calculated acceleration vectors; compare the calculated opening angular acceleration ω with a respective threshold angular acceleration which depends on the elastic rigidity of the elastic joint; identify the anomaly condition associated with an involuntary opening of the master device if the aforesaid calculated opening angular acceleration ω is greater than said threshold angular acceleration, according to the relation ω>threshold angular acceleration.
According to an embodiment of the robotic system, the aforesaid master device comprises two rigid parts 380, 390 elastically constrained to translate along a longitudinal axis coinciding with the longitudinal extension of at least one of the aforesaid parts of the master device.
The control unit is configured to detect and/or calculate the acceleration vector and/or the respective evolution over time of each of said two detectable points; calculate the distancing/approaching linear acceleration of the two rigid parts of the master device, based on the aforesaid detected and/or calculated acceleration vectors; compare the calculated distancing/approaching linear acceleration with a threshold linear velocity which depends on the elastic rigidity of the constraint 375; identify the anomaly/fault condition associated with an involuntary opening of the master device if the aforesaid calculated distancing/approaching linear acceleration is greater than the aforesaid threshold linear acceleration.
According to an embodiment of the robotic system, the aforesaid master device comprises two rigid parts 580, 590 mutually connected in an elastic joint which tends to open said parts at least angularly when not pressed or held firmly in the user's hand.
The anomaly to be detected is a drop of the master device, and the control unit is configured to detect the drop of the master device based on the relation between the accelerations of two sensors respectively associated with the aforesaid two rigid parts of the master device.
According to different embodiments of the robotic system, the control unit is configured to identify and recognize and/or discriminate at least one anomaly condition by performing a method according to any of the embodiments of such a method disclosed in this description.
According to an embodiment of the robotic system, the control unit is configured to manage anomalies found by performing a method according to any of the embodiments of such a method disclosed in this description.
As can be seen, the objects of the present invention as previously indicated are fully achieved by the method described above by virtue of the features disclosed above in detail.
In fact, the method and system described allow an effective and real-time verification of detecting several possible operating anomalies/faults of the master device, or possible abnormal situation of the master device, and recognizing the type of anomaly.
Thus, the need to apply procedures for verifying any abnormal operating conditions of the master device in real time, conducted automatically by the robot control system for medical or surgical teleoperation, such as to be efficient and reliable, in order to meet the stringent safety requirements which are required by such applications, are met.
This is achieved by measuring or calculating the linear and/or angular acceleration of at least one point which can be associated with the master, preferably when the unconstrained or “flying” master device is within a certain working volume, and comparing the one or more detected quantities with one or more respective predeterminable threshold values.
Since the master device is intended to be mechanically hand-held by the surgeon during teleoperation, the detection of accelerations of the master device outside permitted limits when the master device is within the workspace assigned to the master is an indication of anomaly.
Once a structural or functional anomaly of the master device has been identified, the teleoperation can be immediately and promptly interrupted, thus avoiding that such an anomaly is reflected in a consequent anomaly in the operation of the slave device and the surgical instrument associated therewith, intended to act on the patient, with possible even serious consequences on the patient himself.
Thereby, the objective of improving patient safety is achieved, meeting the very strict safety requirements which must be respected in the operating environment considered.
In order to meet contingent needs, those skilled in the art may make changes and adaptations to the embodiments of the method described above or can replace elements with others which are functionally equivalent, without departing from the scope of the following claims. All the features described above as belonging to a possible embodiment may be implemented irrespective of the other embodiments described.
| Number | Date | Country | Kind |
|---|---|---|---|
| 102021000003416 | Feb 2021 | IT | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/IB2022/051293 | 2/14/2022 | WO |
