Patent Grant
11712308
The invention relates to a surgical system and, more particularly, to method and apparatus for controlling a haptic device.
Minimally invasive surgery (MIS) is the performance of surgery through incisions that are considerably smaller than incisions used in traditional surgical approaches. For example, in an orthopedic application such as total knee replacement surgery, an MIS incision length may be in a range of about 4 to 6 inches whereas an incision length in traditional total knee surgery is typically in a range of about 6 to 12 inches. As a result of the smaller incision length, MIS procedures are generally less invasive than traditional surgical approaches, which minimizes trauma to soft tissue, reduces post-operative pain, promotes earlier mobilization, shortens hospital stays, and speeds rehabilitation.
MIS presents several challenges for a surgeon. For example, in minimally invasive orthopedic joint replacement, the small incision size reduces the surgeon's ability to view and access the anatomy, which increases the complexity of sculpting bone and assessing proper implant position. As a result, accurate placement of implants may be difficult. Conventional techniques for counteracting these problems include, for example, surgical navigation, positioning the leg for optimal joint exposure, and employing specially designed, downsized instrumentation and complex surgical techniques. Such techniques, however, typically require a large amount of specialized instrumentation, a lengthy training process, and a high degree of skill. Moreover, operative results for a single surgeon and among various surgeons are not sufficiently predictable, repeatable, and/or accurate. As a result, implant performance and longevity varies among patients.
Conventional efforts to facilitate the performance and improve the outcome of minimally invasive and traditional orthopedic joint procedures may include the use of a robotic surgical system. For example, some conventional techniques include autonomous robotic systems, such as the ROBODOC system (formerly available from Integrated Surgical Systems, Inc., Sacramento, Calif.). Such systems, however, typically serve primarily to enhance bone machining by performing autonomous cutting with a high speed burr. Although such systems enable precise bone resections for improved implant fit and placement, they act autonomously (rather than cooperatively with the surgeon) and thus require the surgeon to cede a degree of control to the robot. Additional drawbacks of autonomous systems include the large size of the robot, poor ergonomics, increased incision length for adequate robot access, and limited acceptance by surgeons and regulatory agencies due to the autonomous nature of the system. Such systems also typically require rigid clamping of the bone during registration and cutting and thus lack real-time adaptability to the dynamic intraoperative scene.
Other conventional robotic systems include non-autonomous robots that cooperatively interact with the surgeon, such as the ACROBOT system (The Acrobot Company Limited, London, Great Britain). One drawback of conventional interactive robotic systems, however, is that such systems lack the ability to adapt surgical navigation in real-time to a dynamic intraoperative environment. For example, U.S. Pat. No. 7,035,716, which is hereby incorporated by reference herein in its entirety, discloses an interactive robotic system programmed with a three-dimensional virtual region of constraint that is registered to a patient. The robotic system includes a three degree of freedom (3 DOF) arm having a handle that incorporates force sensors. The surgeon utilizes the handle to manipulate the arm and move the cutting tool. Moving the arm via the handle is required so that the force sensors can measure the force being applied to the handle by the surgeon. The measured force is then used to control motors to assist or resist movement of the cutting tool. For example, during a knee replacement operation, the femur and tibia of the patient are fixed in position relative to the robotic system. As the surgeon applies force to the handle to move the cutting tool, the interactive robotic system applies an increasing degree of resistance to resist movement of the cutting tool as the tool approaches a boundary of the virtual region of constraint. In this manner, the robotic system guides the surgeon in preparing the bone by maintaining the tool within the virtual region of constraint. As with the above-described autonomous systems, however, the interactive robotic system functions primarily to enhance bone machining. Additionally, the 3 DOF configuration of the arm and the requirement that the surgeon manipulate the arm using the force handle results in limited flexibility and dexterity, making the robotic system unsuitable for certain MIS applications. The interactive robotic system also requires the anatomy to be rigidly restrained and the robotic system to be fixed in a gross position and thus lacks real-time adaptability to the intraoperative scene.
Although some interactive robotic systems may not require fixation of the anatomy, such as the VECTORBOT system (BrainLAB, Inc., Westchester, Ill.), such systems do not enable bone sculpting but instead merely function as intelligent tool guides. For example, such systems may control a robotic arm to constrain movement of a drill along a pre-planned drilling trajectory to enable a surgeon to drill a hole in a vertebra for placement of a pedicle screw. Similarly, other robotic systems, such as the BRIGIT system (Zimmer, Inc., Warsaw, Ind.), simply position a mechanical tool guide. For example, the robotic system disclosed in International Pub. No. WO 2005/0122916, and hereby incorporated by reference herein in its entirety, discloses a robotic arm that positions a mechanical tool guide. Using the robot-positioned tool guide, the surgeon manually manipulates a conventional surgical tool, such as a saw or drill, to make cuts to the patient's anatomy while the robot constrains movement of the tool guide. Although such systems may increase the accuracy and repeatability of the bone cuts, they are limited to performing the functions of a conventional tool guide and thus lack the ability to enable the surgeon to sculpt complex shapes in bone, as may be required for minimally invasive modular implant designs.
Some non-robotic conventional surgical tools useful for bone sculpting do not require fixation of the relevant anatomy, such as the Precision Freehand Sculptor (Blue Belt Technologies, Inc., Pittsburgh, Pa.). One drawback of such tools, however, is that they do not function in a manner that is transparent to the user. For example, U.S. Pat. No. 6,757,582, which is hereby incorporated by reference herein in its entirety, discloses a handheld surgical tool that can be used for sculpting a target shape into a bone. The handheld tool is a freehand cutting tool that is manipulated by the surgeon to grind away portions of the bone to form a desired target shape in the bone. The target shape is defined, for example, by a voxel-based model that is registered to the physical bone. During cutting, both the bone and the cutting tool are tracked to enable a controller to determine whether the cutting tool is impinging on the boundaries of the target shape and therefore cutting away bone that should be left intact. If so, the controller may shut off or retract the cutting tool to protect the bone. Although the bone is protected, the operation of the surgical tool is interrupted during the surgical procedure and the length of time to perform the procedure may increase. Further, interruption of cutting may also result in a rough surface cut. Additionally, such systems merely disable the cutting tool based on a position of the tool relative to the target shape but do not actually constrain the surgeon's manipulation of the cutting tool, for example, to prevent contact between the cutting tool and sensitive anatomy, or address other adverse situations, such as when rapid motion of the anatomy is detected. Thus, such systems may not include adequate safeguards to protect the patient. Moreover, a handheld tool that incorporates a shutoff mechanism may be bulky and heavier than a normal freehand tool or a gravity compensated interactive arm. Thus, it may be difficult for a surgeon to maneuver such a handheld tool to produce fine cutting motions, which makes such tools unsuited for applications that require complex shapes to be sculpted in bone, especially in a minimally invasive surgical environment such as when cutting in the gap between the femur and the tibia in a knee replacement operation without dislocating or distracting the joint.
In view of the foregoing, a need exists for a surgical system that is able to cooperatively interact with a surgeon to enable the surgeon to sculpt complex shapes in bone in a minimally invasive manner and that has the ability to dynamically compensate for motion of objects in the intraoperative environment in a manner that safeguards the patient and is substantially transparent to the surgeon.
According to an aspect of the present invention, a method for calibrating a surgical device includes acquiring first data including a position and/or an orientation of a first object disposed on the surgical device at a first location; acquiring second data including a position and/or an orientation of a second object disposed on the surgical device at a second location; determining third data including a position and/or an orientation of the first object relative to the second location; and determining a position and/or an orientation of the second object relative to the second location based at least in part on the first data, the second data, and the third data.
According to another aspect, a system for calibrating a surgical device includes a first object configured to be disposed on the surgical device, a second object configured to be disposed on the surgical device, and a computing system. The computing system is programmed to determine first data including a position and/or an orientation of the first object when the first object is disposed on the surgical device at a first location; determine second data including of a position and/or an orientation of the second object when the second object is disposed on the surgical device at a second location; determine third data including a position and/or an orientation of the first object relative to the second location; and determine a position and/or an orientation of the second object relative to the second location based at least in part on the first data, the second data, and the third data.
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain principles of the invention.
Presently preferred embodiments are illustrated in the drawings. An effort has been made to use the same or like reference numbers to refer to the same or like parts.
The computing system 20 includes hardware and software for operation and control of the surgical system 10 and may comprise a computer 21, a computer 31, a display device 23, an input device 25, and a cart 29. The computing system 20 is adapted to enable the surgical system 10 to perform various functions related to surgical planning, navigation, image guidance, and/or haptic guidance. The computer 21 is preferably customized for surgical planning and navigation and includes algorithms, programming, and software utilities related to general operation, data storage and retrieval, computer aided surgery (CAS), and/or any other suitable functionality. In contrast, the computer 31 is preferably customized for controlling performance, stability, and/or safety of the haptic device 30 and includes haptic control utilities and programs that enable the haptic device 30 to utilize data from the tracking system 40.
The haptic device 30 is a surgical device configured to be manipulated by a user (such as a surgeon) to move a surgical tool 50 to perform a procedure on a patient, such as sculpting a surface of a bone to receive an implant. During the procedure, the haptic device 30 provides haptic guidance to the surgeon, for example, to maintain the tool 50 within a predefined virtual boundary. As disclosed in the above-referenced Pub. No. US 2006/0142657, the virtual boundary may be defined by a virtual haptic object that is generated by the computing system 20 and registered to (associated with) the anatomy of the patient. The haptic object establishes a desired relationship between the anatomy and the tool 50, such as a desired position, orientation, velocity, and/or acceleration of the tool 50 relative to the anatomy. In operation, when the surgeon moves the tool 50 in a manner that violates the desired relationship (such as when the tool 50 contacts a virtual boundary), the haptic device 30 provides haptic guidance in the form of tactile feedback (e.g., vibration) and/or force feedback (e.g., force and/or torque) to the surgeon. The haptic guidance may be experienced by the surgeon, for example, as resistance to further tool movement in the direction of the virtual boundary. As a result, the surgeon may feel as if the tool 50 has encountered a physical object, such as a wall. In this manner, the virtual boundary functions as a virtual cutting guide. Thus, the surgical system 10 limits the surgeon's ability to physically manipulate the haptic device 30 (e.g., by providing haptic guidance and/or a limit on user manipulation of the haptic device 30) by implementing control parameters based on a relationship between the anatomy and a position, an orientation, a velocity, and/or an acceleration of a portion of the haptic device 30, such as the tool 50. In addition to haptic objects, the relationship may be based on predefined parameters, such as a predefined depth that limits total travel of the tool 50.
Guidance from the haptic device 30 coupled with computer aided surgery (CAS) enables a surgeon to actively and accurately control surgical actions, such as bone cutting, and delivery of localized therapies (e.g., in the brain). In orthopedic applications, the haptic device 30 can be applied to the problems of inaccuracy, unpredictability, and non-repeatability in bone preparation by guiding the surgeon in proper sculpting of bone to thereby enable precise, repeatable bone resections while maintaining intimate involvement of the surgeon in the bone preparation process. Moreover, because the haptic device 30 guides the surgeon during cutting, the skill level of the surgeon is less critical. Thus, surgeons of varying skill degree and experience are able perform accurate, repeatable bone resections.
The haptic device 30 may be robotic, non-robotic, or a combination of robotic and non-robotic systems. In one embodiment, the haptic device 30 is a robotic system as disclosed in the above-referenced Pub. No. US 2006/0142657. In a preferred embodiment, the haptic device is the HAPTIC GUIDANCE SYSTEM™ available from MAKO SURGICAL CORP.® in Ft. Lauderdale, Fla. As shown in
The base 32 provides a foundation for the haptic device 30. The base 32 supports the arm 33 and may also house other components, such as, for example, controllers, amplifiers, actuators, motors, transmission components, clutches, brakes, power supplies, sensors, computer hardware, and/or any other well-known robotic component.
The arm 33 is disposed on the base 32 and is adapted to enable the haptic device 30 to be manipulated by the user. The arm 33 may be an articulated linkage such as serial device, a parallel device, or a hybrid device (i.e., a device having both serial and parallel elements). In a preferred embodiment, the arm 33 is a serial device having four or more degrees of freedom (axes of movement), such as, for example, a robotic arm known as the “Whole-Arm Manipulator” or WAM™ currently manufactured by Barrett Technology, Inc. The arm 33 includes a proximal end disposed on the base 32 and a distal end that includes the end effector 35 to which the surgical tool 50 is coupled. To manipulate the haptic device 30, a user 160 simply grasps and moves the arm 33 (as shown in
To enable the haptic device 30 to provide haptic guidance to the user, the arm 33 incorporates a drive system, such as the drive system disclosed in the above-referenced Pub. No. US 2006/0142657. The drive system includes actuators (e.g., motors) and a mechanical transmission. In an exemplary embodiment, the drive system includes a high-speed cable transmission and zero backlash, low friction, cabled differentials. The cable transmission may be, for example, a cable transmission used in the WAM™ robotic arm currently manufactured by Barrett Technology, Inc. and/or a cable transmission as described in U.S. Pat. No. 4,903,536, which is hereby incorporated by reference herein in its entirety.
The arm 33 also includes position sensors (not shown) for determining a position and an orientation (i.e., pose) of the arm 33, such as encoders and/or resolvers mounted on the joints 33d and 33e and/or encoders and/or resolvers mounted on a shaft of each motor.
The end effector 35 comprises a working end of the haptic device 30. As shown in
In one embodiment, the tool holder 51 includes a holding device 151 configured to couple the surgical tool 50 to the haptic device 30. As shown in
In one embodiment, the holding device 151 is configured so that an axis of the holding device 151 corresponds to a desired axis of the tool 50 when the holding device 151 is disposed on the haptic device 30. For example, in one embodiment (shown in
The user interface 37 enables physical interaction between the user and the haptic device 30. The interface 37 is configured so that the user can grasp the interface 37 and manipulate the tool 50 while simultaneously receiving haptic guidance from the haptic device 30. The interface 37 may be a separate component affixed to the haptic device 30 (such as a handle or hand grip) or may simply be part of the existing structure of the haptic device 30 (such as the arm 33). Because the interface 37 is affixed to or is an integral part of the haptic device 30, any haptic feedback output by the haptic device 30 is transmitted directly to the user when the user is in contact with the interface 37. Thus, the interface 37 advantageously enables the haptic device 30 to hold the tool 50 cooperatively with the surgeon (as shown in
The tracking system 40 of the surgical system 10 is configured to track one or more objects during a surgical procedure to detect movement of the objects. As described in the above-referenced Pub. No. US 2006/0142657. The tracking system 40 includes a detection device that obtains a pose (i.e., position and orientation) of an object with respect to a coordinate frame of reference of the detection device 41. As the object moves in the coordinate frame of reference, the detection device tracks the object. A change in the pose of the object indicates that the object has moved. In response, the computing system 20 can make appropriate adjustments to the control parameters for the haptic device 30. For example, when the anatomy moves, the computing system 20 can make a corresponding adjustment to a virtual haptic object (e.g., a virtual cutting boundary) that is registered to the anatomy. Thus, the virtual cutting boundary moves along with the anatomy.
Pose data from the tracking system 40 is also used to register (i.e., map or associate) coordinates in one space to those in another space to achieve spatial alignment, for example, using a coordinate transformation process. Registration may include any known registration technique, such as, for example, image-to-image registration; image-to-physical space registration; and/or combined image-to-image and image-to-physical-space registration. In one embodiment, the anatomy and the tool 50 (in physical space) are registered to a representation of the anatomy (such as an image 614 in image space) as disclosed in the above-referenced Pub. No. US 2006/0142657 and shown in
The tracking system 40 may be any tracking system that enables the surgical system 10 to continually determine (or track) a pose of the relevant anatomy of the patient and a pose of the tool 50 (and/or the haptic device 30). For example, the tracking system 40 may comprise a non-mechanical tracking system, a mechanical tracking system, or any combination of non-mechanical and mechanical tracking systems suitable for use in a surgical environment.
In one embodiment, the tracking system 40 includes a non-mechanical tracking system as shown in
The anatomy tracker 43 is disposed on the patient's anatomy and enables the anatomy to be tracked by the detection device 41. The anatomy tracker 43 includes a fixation device for attachment to the anatomy, such as a bone pin, surgical staple, screw, clamp, intramedullary rod, or the like. In one embodiment, the anatomy tracker 43 is configured for use during knee replacement surgery to track a femur F and a tibia T of a patient. In this embodiment, as shown in
The haptic device tracker 45 is disposed on the haptic device 30 and enables the surgical system 10 to monitor a global or gross position of the haptic device 30 in physical space so that the surgical system 10 can determine whether the haptic device 30 has moved relative to other objects in the surgical environment, such as the patient, or whether the detection device 41 has moved relative to the haptic device 30. Such information is important because the tool 50 is attached to the haptic device 30. For example, if the user repositions or inadvertently bumps the haptic device 30 while cutting the femur F with the tool 50, the tracking system 40 will detect movement of the haptic device tracker 45. In response, the surgical system 10 can make appropriate adjustments to programs running on the computing system 20 to compensate for movement of the haptic device 30 (and the attached tool 50) relative to the femur F. As a result, integrity of the bone preparation process is maintained.
The haptic device tracker 45 includes a unique array S3 of markers (e.g., reflective spheres) and is adapted to be mounted on the base 32 of the haptic device 30 in a manner that enables the tracker 45 to be secured in a fixed position relative to the base 32. The fixed position is calibrated to the haptic device 30 during a haptic device registration calibration (discussed below) so that the surgical system 10 knows where the tracker 45 is located relative to the base 32. Once calibrated, the fixed position is maintained during the surgical procedure. In one embodiment, as shown in
Adjustability may be imparted to the arm 34 in any known manner (e.g., an articulating linkage, a flexible neck, etc.). For example, in the embodiment of
The end effector tracker 47 enables the surgical system 10 to determine a pose of a distal end of the haptic device 30. The tracker 47 is preferably configured to be disposed on the haptic device 30 at a distal end of the arm 33 (e.g., on the segment 33c, the end effector 35, the tool 50, and/or the tool holder 51). In one embodiment (shown in
To install the end effector tracker 47 on the haptic device 30, the first and second portions 1505 and 1510 of the clamp 1500 are disposed around a cylindrical portion of the tool 50 or the tool holder 51 and tightened together using the thumbscrew 1515. The effector 47 may include a feature configured to aid in positioning the tracker 47 relative to the end effector 35. For example, the tracker 47 may include one or more surfaces 1503 (shown in
Alternatively, instead of a separate end effector tracker 47, the haptic device 30 may incorporate fiducials on the end effector 35. The fiducials may be similar to the unique array S4 of markers and may include, for example, reflective spheres. In contrast to the end effector tracker 47, the fiducials are not removed from the end effector 35 prior to surgery. One disadvantage of not removing the fiducials is that blood and debris may contaminate the fiducials during surgery, which occludes the fiducials and degrades their ability to reflect light to the detection device 41. Thus, the fiducials preferably include a smooth plastic coating so that any surface contamination can be easily removed. The fiducials should be mounted in a location on the end effector 35 that is visible to the detection device 41 during the haptic device registration calibration (described below) but that will not impede the surgeon during the surgical procedure. For example, the fiducials may be mounted on an underside of the end effector 35. Alternatively, the fiducials may be mounted on an adjustable linkage that can be positioned in a registration calibration position where there is a clear line of site between the fiducials and the detection device 41 and a stowed position where the fiducials will not hamper the surgeon during the surgical procedure.
In one embodiment, the end effector tracker 47 is used only during the haptic device registration calibration (discussed below) and is removed prior to performance of the surgical procedure. In this embodiment, the end effector tracker 47 is disposed on the end effector 35 and the haptic device tracker 45 is mounted to the base 32 (e.g., via the adjustable arm 34) so that a position of the haptic device tracker 45 with respect to the haptic device 30 is adjustable. Because the position of the haptic device tracker 45 is adjustable, the surgical system 10 does not know the location of the haptic device tracker 45 relative to the haptic device 30. To determine the geometric relationship between the haptic device 30 and the haptic device tracker 45, the registration calibration process utilizes the end effector tracker 47 (as described below). Although the end effector tracker 47 may remain on the haptic device 30 for the surgical procedure and can be continuously monitored, it is advantageous to remove the end effector tracker 47 when the registration calibration is complete to prevent the tracker 47 from impeding the surgeon during the surgical procedure. Another advantage of removing the tracker 47 is that movement of the tracker 47 during the surgical procedure may result in degraded performance of the surgical system 10 due to delays or limited bandwidth as the tracking system 40 detects and processes movement of the tracker 47.
In an alternative embodiment, the end effector tracker 47 may be eliminated. In this embodiment, the haptic device tracker 45 is fixed in a permanent position on the haptic device 30. Because the haptic device tracker 45 is permanently fixed on the haptic device 30, the relationship between the haptic device tracker 45 and the coordinate frame of the haptic device 30 is known. Accordingly, the surgical system 10 does not need the end effector tracker 47 for the registration calibration to establish a relationship between the haptic device tracker 45 and the coordinate frame of the haptic device 30. In this embodiment, the haptic device tracker 45 may be rigidly mounted on the haptic device 30 in any position that permits the tracking system 40 to see the array S3 of the haptic device tracker 45, that is close enough to the surgical site so as not to degrade accuracy, and that will not hinder the user or interfere with other personnel or objects in the surgical environment.
In another alternative embodiment, the haptic device 30 is firmly locked in position. For example, the haptic device 30 may be bolted to a floor of the operating room or otherwise fixed in place. As a result, the global or gross position of the haptic device 30 does not change substantially so the surgical system 10 does not need to track the global or gross position of the haptic device 30. Thus, the haptic device tracker 45 may be eliminated. In this embodiment, the end effector tracker 47 may be used to determine an initial position of the haptic device 30 after the haptic device 30 is locked in place. One advantage of eliminating the haptic device tracker 45 is that the surgical system 10 does not need to include monitoring data for the haptic device tracker 45 in the control loop. As a result, noise and errors in the control loop are reduced. Alternatively, the haptic device tracker 45 may be retained but is monitored only for detecting excessive motion of the base 32 or the tracking system 40 rather than being included in the control loop.
In another alternative embodiment, the tracking system 40 is attached to the haptic device 30 in a permanently fixed position. For example, the tracking system 40 (including the detection device 41) may be mounted directly on the haptic device 30 or connected to the haptic device 30 via a rigid mounting arm or bracket so that the tracking system 40 is fixed in position with respect to the haptic device 30. In this embodiment, the haptic device tracker 45 and the end effector tracker 47 may be eliminated because a position of the tracking system 40 relative to the haptic device 30 is fixed and can be established during a calibration procedure performed, for example, during manufacture or set up of the haptic device 30.
In another alternative embodiment, the tracking system 40 is attached to the haptic device 30 in an adjustable manner. For example, the tracking system 40 (including the detection device 41) may be connected to the haptic device 30 with an arm, such as the adjustable arm 34 (described above in connection with the haptic device tracker 45) so that the tracking system 40 is moveable from a first position to a second position relative to the haptic device 30. After the arm and the tracking system 40 are locked in place, a calibration can be performed to determine a position of the tracking system 40 relative to the haptic device 30. A calibration to determine the position of the tracking system 40 relative to the haptic device 30 may be performed, for example, by viewing the end effector tracker 47 with the tracking system 40.
The instrument tracker 49 is adapted to be coupled to an instrument 150 that is held manually in the hand of the user. The instrument 150 may be, for example, a probe, such as a registration probe. As shown in
The tracking system 40 may additionally or alternatively include a mechanical tracking system. In contrast to the non-mechanical tracking system (which includes a detection device 41 that is remote from the trackers 43, 45, 47, and 49), a mechanical tracking system may be configured to include a detection device (e.g., an articulating linkage having joint encoders) that is physically connected to the tracked object. The tracking system 40 may include any known mechanical tracking system, such as a mechanical tracking system as described in U.S. Pat. Nos. 6,033,415, 6,322,567, and/or Pub. No. US 2006/0142657, each of which is hereby incorporated by reference herein in its entirety, or a fiber optic tracking system.
For example, as described in U.S. Pub. No. 2006/0142657, the tracking system 40 includes a mechanical tracking system having a jointed mechanical arm 241 (e.g., an articulated arm having six or more degrees of freedom) adapted to track a bone of the patient. As shown in
In operation, the computing system 20, the haptic device 30, and the tracking system 40 cooperate to enable the surgical system 10 to provide haptic guidance to the user during a surgical procedure. The haptic guidance manifests as a result of the user's interaction with a virtual environment generated by a haptic rendering process. The haptic rendering process may include any suitable haptic rendering process, such as, for example, a haptic rendering process as described in U.S. Pat. No. 6,111,577, which is hereby incorporated by reference herein in its entirety. In a preferred embodiment, the haptic rendering process includes a haptic rendering algorithm as disclosed in the above-referenced Pub. No. US 2006/0142657 and/or U.S. patent application Ser. No. 11/646,204, filed Dec. 27, 2006, and incorporated by reference herein in its entirety. In the preferred embodiment, the surgical system 10 employs point-based haptic interaction where only a virtual point, or haptic interaction point (HIP), interacts with virtual objects in the virtual environment. The HIP corresponds to a physical point on the haptic device 30, such as, for example, a tip of the tool 50. The HIP is coupled to the physical point on the haptic device 30 by a virtual spring/damper model. The virtual object with which the HIP interacts may be, for example, a haptic object 705 (shown in
One embodiment of a haptic rendering process is represented generally in
In one embodiment, the surgical system 10 includes a haptic rendering process as shown in
As shown in
A first coordinate transformation T1 (shown in
A second coordinate transformation T2 (shown in
A third coordinate transformation T3 (shown in
A fourth coordinate transformation T4 (shown in
A fifth coordinate transformation T5 (shown in
A sixth coordinate transformation T6 (shown in
One advantage of the present invention is that the surgical system 10 is able to compensate for motion of objects during the surgical procedure in a dynamic manner that is transparent to the user. Specifically, the surgical system 10 operates synchronously by continually monitoring motion of the anatomy, the haptic device tracker 45, and the arm 33 and continually updating the transformations T2, T4, and T5 without interrupting operation of the haptic device 30. In contrast, conventional surgical systems typically operate asynchronously, for example, by requiring the user to stop and reset the system or reregister tracked objects when movement of a tracked object is detected. As a result, with conventional systems, the operation of the system may be interrupted or impeded when motion of a tracked object is detected. Although the present invention can operate synchronously without interrupting the operation of the haptic device 30, it is advantageous to occasionally restrict operation of the haptic device 30, for example, when the surgical system 10 detects abnormal motion, such as when a tracked object moves too fast and/or too far.
In one embodiment, a method of compensating for motion of objects during a surgical procedure includes (a) determining a pose of the anatomy; (b) determining a pose of the tool 50; (c) determining at least one of a position, an orientation, a velocity, and an acceleration of the tool 50; (d) associating the pose of the anatomy, the pose of the tool 50, and a relationship between the pose of the anatomy and the at least one of the position, the orientation, the velocity, and the acceleration of the tool 50; and (e) updating the association in response to motion of the anatomy and/or motion of the tool 50 without interrupting operation of the surgical device during the surgical procedure. The method may also include the step of providing haptic guidance, based on the relationship, to the user to constrain the user's manipulation of the surgical tool. The relationship may be based, for example, on a desired interaction between the anatomy and a position, an orientation, a velocity, and/or an acceleration of the tool 50. In one embodiment, the relationship is defined by a virtual object or parameter positioned relative to the anatomy and representing a desired location of an implant and/or cut surfaces for installing the implant. The step of associating the pose of the anatomy, the pose of the tool 50, and the relationship may be accomplished, for example, using registration processes, coordinate transformation processes (e.g., block 2506 of
In this embodiment, the pose of the tool 50 is determined by determining a pose of a first portion of the haptic device 30 to which the tool 50 is coupled, determining a pose of a second portion of the haptic device 30, and calculating the pose of the tool 50 based at least in part on the poses of the first and second portions of the haptic device 30 and a known geometric relationship between the tool 50 and the first portion of the haptic device 30. In one embodiment, the first portion of the haptic device 30 comprises the distal end of the arm 33, and the second portion of the haptic device 30 comprises the base 32 of the haptic device 30. In another embodiment, the second portion of the haptic device 30 comprises an intermediate portion of the arm 33 (e.g., the segments 33a, 33b, or 33c). In one embodiment, rather than mounting the end effector tracker 35 to a distal end of the arm, the end effector tracker 35 could be mounted to an intermediate portion of the arm, such as the elbow. The step of determining the pose of the second portion of the haptic device 30 includes determining a pose of the haptic device tracker 45 (which is mounted on the second portion of the haptic device 30, e.g., to the base 32 or an intermediate portion of the arm 33). Because the pose of the tool 50 is determined based on the poses of the first and second portions of the haptic device 30 and because the surgical system 10 continually updates the poses of the first and second portions (e.g., based on joint encoder data and a position of the haptic device tracker 45), the pose of the tool 50 is updated to account for motion of the first and second portions. As a result, motion to the tool 50 is determined based on motion of the first and second portions. In this manner, the surgical system 10 is able to compensate for motion of objects during a surgical procedure.
In one embodiment, the tracking system 40 is a non-mechanical tracking system (e.g., as described above in connection with the tracking system 40) that operates at a different update rate than the haptic device 30. For example, the haptic device 30 may update at 2000 Hz while the tracking system 40 updates at 15-30 Hz. The lower update rate of the tracking system 40 limits the dynamic performance of the motion compensation because the 15-30 Hz updates are separated by 1/15 to 1/30 seconds during which time no tracking information is available. Additionally, higher frequency motions of a tracked object will not be present in the output data of the tracking system 40. One disadvantage of poor dynamic performance is that the surgical system 10 may not have sufficient data to move the haptic object precisely in sync with the physical anatomy. As a result, any cuts the surgeon makes may have reduced accuracy. For example, when the surgical system 10 is providing haptic guidance to guide the surgeon in cutting a planar bone surface with a spherical burr, momentary motion of one of the tracked objects combined with poor dynamic performance may result in divots or peaks in the final cut surface. The worse the dynamic performance, the larger the divots and peaks will be. If the bone cuts are for a cemented implants, small divots are acceptable because cement will simply fill the divots. For press fit implants, however, divots cause gaps between the implant and the bone that may potentially inhibit full in-growth of the bone into the implant. Peaks are less critical than divots because they can be easily removed with the burr but will increase the amount of time required to complete bone preparation.
For motion compensation applications, several techniques are beneficial in maximizing performance from tracking systems with dynamic performance issues. First, because of the different update rates, if the coordinate transformation process 2506 uses data directly from the tracking system 40, the desired cut surfaces defined by the virtual object move in abrupt steps in response to detected motion of the anatomy. As a result, a user manipulating the haptic device 30 may experience a rough or “jumpy” feeling when interacting with a haptic object. To address this problem, the surgical system 10 may include an interpolation or other appropriate filter (represented by the blocks 4512 in
In their traditional implementation, all of the above filters are designed for scalar input signals. However, the tracking system 40 will generally output multiple position and orientation signals. In a preferred embodiment, the filter 4512 receives the tracking system output, expressed as a homogenous transformation four by four size matrix that contains the position and orientation information. It is not desirable to filter the elements of this matrix directly because the result will not be a valid homogenous matrix and the orientation will not be filtered properly. Instead, the homogenous transformation is first converted to a three element position vector and a quaternion, which is a four element vector that represents the orientation information. For the small motions between samples, these seven values can then be independently filtered. The quaternion may be normalized before taking the filtered values and converting them back to a homogenous transformation, which is then output from the filter 4512.
In most cases, the position output of the tracking system 40 represents the position of the relevant tracked object at some point in the past. The latency is the time interval between the time when the tracking system 40 samples the tracked object's position and the time when the surgical system 10 receives this position output. This time interval may include processing time of the tracking system 40, communication delays, and a fraction or multiple of the sampling time of the tracking system 40. The filter 4512 adds additional latency based on the phase delay of the particular filter selected. These latency sources all combine to degrade the dynamic tracking performance and cause the haptic surfaces to lag behind the motion of their associated tracked objects. However, these latency values are usually known or can be measured or estimated fairly accurately. Thus, the latency effect can be partially compensated for. For example, if the combined latency of the tracking system 40 and the filter 4512 is t1, then the filtered position output p may be corrected by Δp=νt1. The velocity value ν can be computed by a (possibly filtered) difference of successive position values or with a washout filter, as described below. In another embodiment, as is well known to those skilled in the art of control theory, a state estimator, state observer, or Kalman filter, which include a simple simulated model of the anatomy and/or the base 32 of the haptic device 30 and internally compute both the position and velocity of the tracked object, could be used to eliminate the latency of the filter 4512. Alternatively, the filter 4512 could be eliminated by utilizing a higher frequency tracking system, such as an encoder-based mechanical tracking system or high speed optical tracking system.
Some tracking systems, notably optical tracking systems, may not produce accurate outputs when tracked objects are moving relative to the camera (i.e., the detection device 41). Errors may result, for example, from motion blur caused by the exposure time or scanning rate of the camera. If the velocity of the tracked object is computed using one of the methods described above and an error model of the tracking system as a function of velocity and/or position is known or determined, these errors may be corrected by adding this error value to the filtered position output.
Dynamic performance of the tracking system 40 is only relevant if the haptic device 30 is capable of rendering a moving haptic object effectively. The haptic rendering capabilities of the haptic device 30 are impacted by the type of haptic control scheme used. The haptic device 30 may utilize any suitable haptic control scheme, such as, for example, admittance control, impedance control, or hybrid control. In an admittance control mode, the haptic device 30 accepts force input and yields position (or motion) output. For example, the haptic device 30 measures or senses a wrench at a particular location on the haptic device 30 (e.g., the user interface 37) and acts to modify a position of the haptic device 30. In an impedance control mode, the haptic device 30 accepts position (or motion) input and yields wrench output. For example, the haptic device 30 measures, senses, and/or calculates a position (i.e., position, orientation, velocity, and/or acceleration) of the tool 50 and applies an appropriate corresponding wrench. In a hybrid control mode, the haptic device 30 utilizes both admittance and impedance control. For example, a workspace of the haptic device 30 may be divided into a first subspace in which admittance control is used and a second subspace in which impedance control is used and/or both position and force inputs may be used to compute a force or position output. For example, in a substantially impedance controlled device, force inputs may be used to cancel out some of the natural friction of the system. In a preferred embodiment, the haptic device 30 is designed for impedance control, where the haptic device 30 reads the position and/or orientation of the user-manipulated surgical tool 50 and outputs an appropriate force and/or torque. Impedance control devices have the advantage of simplicity (no force sensor is required), better stability properties when the tool contacts physical objects (such as when cutting bone), and better performance when moving in free space. Admittance control devices, however, have an advantage in that they can render haptic objects with very stiff walls more easily than impedance control devices. With regard to motion tracking, impedance control devices are advantageous in that their performance is related to the open-loop force bandwidth and physical system dynamics. In contrast, the performance of an admittance control device depends on the closed-loop position control performance, which tends to be slower than the open-loop force and physical system dynamics.
Returning to the haptic rendering algorithm of
As shown in
where s is the Laplace transform variable, and where p determines the location of poles and in general should be located about two to three times faster than the fastest system pole. In one embodiment, the pole is placed at about 80 Hz. The filtered velocity is then multiplied by a Jacobian matrix, J, to obtain the Cartesian endpoint velocity, x, of the haptic device 30.
The washout filter limits the high-frequency gain thereby limiting the amplification of noise inherent in a derivative or differencing operation and removing sampling-rate artifacts. The washout filter has a single parameter, p, which simplifies design and tuning of the filter, compared with separate velocity differencing and smoothing operations. The Laplace domain representation given above can be transformed into a discrete-time representation that is suitable for implementation on a digital computer using the well-known bilinear transform or z-transform. In an alternative embodiment to the washout filter, a simple differenced position signal can be filtered with a Butterworth or other filter described above to provide a velocity measure. Alternatively, a filtered position signal can be differenced and possibly filtered again using any of the filters described above.
As shown in
The haptic device 30 is preferably configured to operate in various operating modes. For example, the haptic device 30 may be programmed to operate in an input mode, a hold mode, a safety mode, a free mode, an approach mode, a haptic (or burring) mode, and/or any other suitable mode. The operating mode may be selected manually by the user (e.g., using a selection button represented graphically on the display device 23 or a mode switch located on the haptic device 30 and/or the computing system 20) and/or automatically by a controller or software process. In the input mode, the haptic device 30 is enabled for use as an input device to input information to the surgical system 10. When the haptic device 30 is in the input mode, the user may operate the haptic device 30 as a joystick or other input device, for example, as described above in connection with the end effector 35 and/or in U.S. patent application Ser. No. 10/384,078 (Pub. No. US 2004/0034282), which is hereby incorporated by reference herein in its entirety.
In the hold mode, the arm 33 of the haptic device 30 may be locked in a particular pose. For example, the arm 33 may be locked using brakes, control servoing techniques, and/or any other appropriate hardware and/or software for stabilizing the arm 33. The user may desire to place the haptic device 30 in the hold mode, for example, during an activity such as bone cutting to rest, confer with a colleague, allow cleaning and irrigation of the surgical site, and the like. In the safety mode, the tool 50 coupled to the haptic device 30 may be disabled, for example, by shutting off power to the tool 50. In one embodiment, the safety mode and the hold mode may be executed simultaneously so that the tool 50 is disabled when the arm 33 of the haptic device 30 is locked in position.
In the free mode, the end effector 35 of the haptic device 30 is freely moveable within the workspace of the haptic device 30. Power to the tool 50 is preferably deactivated, and the haptic device 30 may be adapted to feel weightless to the user. A weightless feeling may be achieved, for example, by computing gravitational loads acting on the segments 33a, 33b, and 33c of the arm 33 and controlling motors of the haptic device 30 to counteract the gravitational loads (e.g., as described below in connection with block 4518 of
In the approach mode, the haptic device 30 is configured to guide the tool 50 to a target object, such as, for example, a surgical site, feature of interest on the patient's anatomy, and/or haptic object registered to the patient, while avoiding critical structures and anatomy. For example, in one embodiment, the approach mode enables interactive haptic positioning of the tool 50 as described in U.S. patent application Ser. No. 10/384,194 (Pub. No. US 2004/0034283), which is hereby incorporated by reference herein in its entirety. In another embodiment, the haptic rendering application may include a haptic object defining an approach volume (or boundary) that constrains the tool 50 to move toward the target object while avoiding sensitive features such as blood vessels, tendons, nerves, soft tissues, bone, existing implants, and the like. For example, as shown in
Another embodiment of the approach mode is shown in
In the haptic (or burring) mode, the haptic device 30 is configured to provide haptic guidance to the user during a surgical activity such as bone preparation. In one embodiment, as shown in
In operation, the surgical system 10 may be used for surgical planning and navigation as disclosed in the above-referenced Pub. No. US 2006/0142657. The surgical system 10 may be used, for example, to perform a knee replacement procedure or other joint replacement procedure involving installation of an implant. The implant may include any implant or prosthetic device, such as, for example, a total knee implant; a unicondylar knee implant; a modular knee implant; implants for other joints including hip, shoulder, elbow, wrist, ankle, and spine; and/or any other orthopedic and/or musculoskeletal implant, including implants of conventional materials and more exotic implants, such as orthobiologics, drug delivery implants, and cell delivery implants. Prior to performance of the procedure, the haptic device 30 is initialized, which includes a homing process, a kinematic calibration, and a haptic device registration calibration.
The homing process initializes the position sensors (e.g., encoders) in the arm 33 of the haptic device 30 to determine an initial pose of the arm 33. Homing may be accomplished in any known manner such as by manipulating the arm 33 so that each joint encoder is rotated until an index marker on the encoder is read. The index marker is an absolute reference on the encoder that correlates to a known absolute position of the joint. Thus, once the index marker is read, the control system of the haptic device 30 knows that the joint is in an absolute position. As the arm 33 continues to move, subsequent positions of the joint are calculated based on the absolute position and subsequent displacement of the encoder.
The kinematic calibration identifies errors in the kinematic parameters of the forward kinematics process 2504 (shown in
The haptic device registration calibration establishes a geometric relationship or transformation between a coordinate system of the haptic device tracker 45 (e.g., the coordinate system X3 shown in
The registration calibration involves securing the haptic device tracker 45 in a fixed position on the haptic device 30 and temporarily affixing the end effector tracker 47 to the end effector 35, for example, with the clamp 1500 shown in
In one embodiment, the user manually moves the end effector 35 to the various positions while the end effector 35 in the free mode. In another embodiment, the surgical system 10 controls the haptic device 30 to automatically move the end effector 35 to the various positions. In yet another embodiment, the haptic device 30 provides haptic guidance to guide the user in moving the end effector 35 to predefined points in the workspace of the haptic device 30. To improve the accuracy of the registration calibration, the predefined points are preferably located in a vicinity of the surgical site (e.g., close to the actual bone preparation site). The predefined points may include, for example, vertices of a shape, such as a two or three dimensional polytope (e.g., a polygon or polyhedron). In one embodiment, the shape is a cube centered at the relevant anatomy, such as the knee joint. The vertices are preferably displayed on the display device 23 along with an arrow indicating an allowable direction of motion for the end effector 35. One advantage of utilizing haptic guidance to guide the user in moving the end effector 35 to predefined points is that the user is able to move the end effector 35 to a plurality of positions in a repeatable fashion, which improves the accuracy of the registration calibration.
In addition to capturing data relating the pose of the trackers 45 and 47 to the tracking system 40, the surgical system 10 determines a pose of the end effector 35 relative to the haptic device 30 (e.g., relative to a reference point fixed in relation to the base 32 of the haptic device 30) based on data from the position sensors (e.g., joint encoders) in the arm 33. The surgical system 10 uses the data obtained during the registration calibration to calculate the geometric relationship between the haptic device tracker 45 and the coordinate frame of reference of the haptic device 30 (e.g., the coordinate system X4 shown in
In one embodiment, a transformation, TRB, of the haptic device tracker 45 relative to the base 32 of the haptic device 30 (e.g., relative to a reference point fixed in relation to the base 32 of the haptic device 30) is calculated as follows. As the end effector 35 is moved to the various positions, the surgical system 10 records (a) a position of the end effector tracker 47 (e.g., a known position on the end effector 35) relative to the tracking system 40, PCE, which is obtained from the tracking system 40; (b) a position and an orientation of the haptic device tracker 45 relative to the tracking system 40, TCB, which is obtained from the tracking system 40; and (c) a position of the end effector tracker 47 relative to the base of the haptic device 30, r, which is obtained from the joint encoders of the haptic device 30. If noise is present in the tracking system output, multiple samples can be taken for each end effector position. In the event the haptic device 30 moves during data sampling, the surgical system 10 can alert the user. Additionally, any affected data points should be thrown out because there will be latency between the data from the tracking system 40 and the data from the joint encoders of the haptic device 30. A position of the end effector tracker 47 relative to the haptic device tracker 45 is computed as bi=TB,iCPC,iE for each test location, i. A position of the end effector tracker 47 relative to the base 32 of the haptic device 30 at each test location, i, is denoted by ri.
After data collection is complete, the transformation of the haptic device tracker 45 relative to the base 32 of the haptic device 30 (i.e., relative to a reference point on the base 32 of the haptic device 30), TRB, is separated into orientation and position terms.
The orientation component RRB is found by solving the equation RRB
A least-squares estimator using singular value decomposition is used to solve for RRB. The position vector PRB can then be found from the equation PRB=rm−RRBbm. The transformation of the haptic device tracker 45 relative to the base 32 of the haptic device 30, TRB, can then be reconstructed according to
After the registration calibration is complete, the end effector tracker 47 is removed from the haptic device 30. During surgery, the surgical system 10 can determine a pose of the tool 50 based on (a) a known geometric relationship between the tool 50 and the end effector 35, (b) a pose of the end effector 35 relative to the haptic device 30 (e.g., from the position sensors in the arm 33), (c) the geometric relationship between the haptic device 30 and the haptic device tracker 45 determined during the registration calibration, and (d) the global or gross position of the haptic device 30 (e.g., from the pose of the haptic device tracker 45 relative to the tracking system 40). The registration calibration need not be performed if the haptic device tracker 45 has not moved with respect to the haptic device 30 since the previous registration calibration and the previously acquired registration calibration data is still reliable.
In one embodiment, a method for performing the registration calibration includes (a) acquiring first data including at least one of a position and an orientation of a first object disposed on the haptic device 30 at a first location; (b) acquiring second data including at least one of a position and an orientation of a second object disposed on the haptic device 30 at a second location; (c) determining third data including at least one of a position and an orientation of the first object relative to the second location; and (d) determining at least one of a position and an orientation of the second object relative to the second location based at least in part on each of the first data, the second data, and the third data. The method may also include (e) moving the first object (e.g., the end effector tracker 47 disposed on the arm 33 of the haptic device 30) to a plurality of positions; (f) providing haptic guidance (e.g., force feedback) to guide the user in moving the first object to at least one of the plurality of positions; (g) acquiring the first data or the second data when the first object is in each of the plurality of positions; and (h) alerting the user if the first object, the second object, the first location, and/or the second location moves during acquisition of the first data, the second data, and/or the third data.
In one embodiment, the first object is the end effector tracker 47, and the second object is the haptic device tracker 45. In this embodiment, the steps of acquiring the first data and the second data include detecting the trackers 45 and 47 with the detection device 41. Alternatively, the second object may comprise one or more components of the tracking system 40, such as the detection device 41. As described above in connection with the end effector tracker 47, the end effector tracker 47 may be disposed at a location (e.g., the first location) on the haptic device 30 that includes a locating feature, such as a cylindrical feature of the tool 50 or the tool holder 51. In this case, the step of acquiring the first data may include determining a position and/or an orientation of a point and/or an axis of the cylindrical feature (e.g., the axis H-H shown in
After the haptic device 30 is initialized, the surgeon can register the patient and the surgical tool 50 to a representation of the anatomy (such as a CT image) and perform a surgical procedure, such as preparing a bone to receive an implant based on a surgical plan. Registration, implant planning, and surgical navigation may be accomplished, for example, as described in the above-referenced Pub. No. US 2006/0142657. Throughout the surgical procedure, the surgical system 10 monitors a position of the bone to detect movement of the bone and makes corresponding adjustments to programs running on the computer 21 and/or the computer 31. For example, the surgical system 10 can adjust a representation (or image) of the bone in response to detected movement of the bone. Similarly, the surgical system 10 can adjust a representation (or image) of the surgical tool 50 in response to detected movement of the surgical tool 50. Thus, images of the bone and the surgical tool on the display device 23 move dynamically in real-time as the bone and the surgical tool 50 move in physical space. The surgical system 10 can also adjust a virtual object associated with the bone in response to detected movement of the bone. For example, the virtual object may define a virtual cutting boundary corresponding to a shape of a surface of the implant. As the bone moves, the surgical system 10 adjusts the virtual object so that the virtual cutting boundary moves in correspondence with the physical bone. In this manner, the surgeon can make accurate bone cuts even when the bone is moving. Additionally, adjustment of the images and the haptic object are transparent to the surgeon so that the surgeon's operation of the haptic device 30 is not interrupted during the surgical procedure.
To improve the safety of the surgical system 10, the surgical system 10 may include a safety feature adapted to constrain the user's operation of the tool 50 when an unsafe condition exists. For example, if an unsafe condition is detected, the surgical system 10 may issue a fault signal. A fault condition may exist if there is a system problem (e.g., a problem with the hardware or software), if an occlusion detection algorithm (e.g., as described below) detects an occluded condition, if a tracked object is moving too fast for the tracking system to process (e.g., when the patient's leg or the haptic device tracker 45 suddenly drops), when the tracking data is questionable, when the user is pushing too hard on the interface 37, and/or if the tool 50 is in an undesirable location. In one embodiment, the surgical system 10 is programmed to issue a fault if a relationship between the anatomy and a position, an orientation, a velocity, and/or an acceleration of the tool 50 does not correspond to a desired relationship and/or if the detection device 41 is unable to detect the position of the anatomy or the position of the surgical tool 51. In response to the fault signal, the surgical system 10 may impose a constraint on the haptic device 30. The constraint may include, for example, providing haptic guidance to the user (e.g., to prevent the user from moving the tool 50 in an unsafe manner) or changing the mode of the haptic device 30 (e.g., from a haptic mode to a free mode). In the preferred embodiment, the constraint is applied to the interface 37, which is both manipulated by the user and is proximal to the surgical site. For a teleoperated haptic device, which includes a “master” device that is operated by the surgeon and is typically remote from the surgical site and a “slave” device that holds the surgical tool proximal to the surgical site and is controlled by the master device, the constraint may be applied to the master device, the slave device, or both.
In one embodiment, a fault signal may be issued if the haptic rendering algorithm determines that a penetration depth of the tool 50 into a haptic boundary exceeds a predetermined threshold. The predetermined threshold may be, for example, a penetration depth in a range of about 1 mm to about 1.25 mm. In one embodiment, the haptic rendering algorithm determines whether the predetermined threshold is exceeded based on the haptic wrench (i.e., force and/or torque) being applied by the haptic device 30 to the user. For example, the haptic rendering algorithm may include a linear force versus position curve where the haptic force is set to about 20,000 N/m (or 20 N/mm). Thus, if the user moves the tip of the tool 50 to a penetration depth of 1 mm, the haptic device 30 outputs a haptic force of about 20 N. Similarly, if the user moves the tip of the tool 50 to a penetration depth of 1.25 mm, the haptic device 30 outputs a haptic force of about 25 N. In this embodiment, the fault signal is triggered when the haptic force reaches about 22.5 N, which corresponds to a penetration depth of about 1.125 mm. Additionally, a threshold haptic force value can be used to protect against the haptic device 30 generating excessively high forces. For example, haptic objects can be designed as independent primitives (e.g., simple geometric shapes) and combined during haptic rendering. If the cumulative effect of the primitives is undesirable (e.g., the total haptic force is too high), a fault signal can be issued.
In another embodiment, a fault signal may issue if rapid motion of the anatomy is detected as indicated, for example, by a velocity of the anatomy trackers 43a and 43b. Rapid motion may be caused, for example, when the anatomy shifts or a tracking element or the detection device 41 is bumped. In one embodiment, the fault signal issues if a velocity of the anatomy tracker 43a is greater than about 40 mm/s or a velocity of the anatomy tracker 43b is greater than about 26 mm/s. The indication of rapid motion may also be based on position (as opposed to velocity) such as when a position of the anatomy tracker 43a or 43b abruptly changes significantly. An abrupt change may be indicated, for example, if a change from the last known good position reported by the tracking system 40 to the current position reported by the tracking system 40 is greater than a predetermined threshold. In addition to rapid motion of the anatomy, the fault signal may issue if rapid motion of the haptic device tracker 45 is detected, such as when the haptic device tracker 45 has a high velocity or an abrupt change in position, which may indicate that the tracker 45 has been bumped or is not securely secured to the arm 34.
The surgical system 10 may have different levels or stages of faults. For example, in one embodiment, there are three stages of faults. The first fault stage applies when the tip of the tool 50 penetrates too deeply into or beyond a haptic boundary. The second fault stage applies when rapid motion of the anatomy is detected. The third fault stage applies when a system fault is present. The surgical system 10 responds to the fault stages by imposing a constraint on the haptic device 30. For example, the surgical system 10 may respond to the first fault stage by disabling the tool 50. The surgical system 10 may respond to the second fault stage by disabling both the tool 50 and the haptic guidance. Disabling the haptic guidance when rapid motion of the anatomy is detected (e.g., when the patient's leg slips off the operating table) advantageously prevents the virtual haptic surfaces that define the haptic cutting volume from moving with the falling bone and dragging the tool 50 along. In contrast, if the haptic surfaces are not disabled when the bone moves rapidly, the haptic surfaces will follow the bone and the haptic device 30 will exert a large force on the arm 33 to maintain the tool 50 within the falling haptic volume. As a result, the arm 33 will be dragged downward as the bone falls. Disabling the haptic guidance avoids this dangerous situation. The surgical system 10 may respond to the third fault stage by disabling the tool 50, shutting off power to the arm 33, and locking the brakes of the arm 33. In one embodiment, the surgical system 10 responds to a fault signal by disabling the tool 50 and placing the haptic device 30 in the free mode (rather than applying the brakes) so that the arm 33 does not pull or apply stress to the anatomy. In this manner, the surgical system 10 avoids damaging the anatomy by preventing the user from operating the tool 50 and/or the arm 33 when an unsafe condition exists.
In one embodiment, a safety feature of the surgical system 10 includes a tool disabling feature. For example, if the tool 50 is an electric tool, the surgical system 10 may include a relay disposed along an electrical connection between the tool 50 and a user input device for controlling the tool 50. For example, the relay may be located between a foot pedal and a tool control console (e.g., the ANSPACH® foot pedal and console described above in connection with the tool 50). Alternatively, the relay could be disposed along a control cable for a handheld instrument. In the case of a pneumatic tool, a pneumatic shutoff valve may be disposed in an air connection between the user input device and the tool motor. In lieu of a relay, the surgical system 10 could supply a digital or analog signal to a “disable input” port on the tool control console. In one embodiment, the surgical system 10 includes a relay that is closed under normal operating conditions so that the tool 50 is activated when the user depresses the foot pedal. If a fault condition is detected, the surgical system 10 issues a fault signal and commands the relay to open so that the tool 50 cannot be activated even when the user depresses the foot pedal. In another embodiment, the relay is a “normally open” relay so that the tool 50 will be remain shut off or disabled unless the tool 50 is specifically enabled by the surgical system 10. One advantage of a “normally open” relay is that if the haptic device 30 completely shuts down, the tool 50 will be disabled. Alternatively or in addition to disabling the tool 50 by commanding a relay or shut off valve, a fault condition may trigger the surgical system 10 to disable the tool 50 by commanding a power shutoff to the console or to the power supplies or amplifiers that drive the tool 50.
In one embodiment, a method of controlling the haptic device 30 based on the tool disabling features includes (a) enabling operation of the haptic device 30; (b) manipulating the haptic device 30 to perform a procedure on a patient; (c) determining whether a relationship between the anatomy of the patient and a position, an orientation, a velocity, and/or an acceleration of the tool 50 of the haptic device 30 corresponds to a desired relationship; and (d) issuing a fault signal and/or imposing a constraint on the haptic device 30 if the relationship does not correspond to the desired relationship or if the detection device 41 is unable to detect the anatomy or the tool 50. The relationship may be based, for example, on a desired interaction between the anatomy and the tool 50. In one embodiment, the relationship is defined by a virtual object positioned relative to the anatomy and representing a desired location of an implant and/or cut surfaces for installing the implant. The method may further include implementing control parameters for controlling the haptic device 30 to provide at least one of haptic guidance to the user and a limit on user manipulation of the surgical device based on the relationship. In one embodiment, in response to the fault signal, the surgical system 10 disables operation of the haptic device 30, locks a portion of the haptic device 30 in position, and/or places the haptic device 10 in a safety mode. In the safety mode, operation of and/or manipulation of the haptic device 30 may be impeded or constrained. To determine whether the relationship corresponds to the desired relationship, the surgical system 10 may, for example, determine whether a penetration depth of the tool 50 into a virtual boundary associated with the anatomy exceeds a desired penetration depth, determine whether the haptic device 30 has violated an operational constraint (e.g., a parameter generated by the haptic rendering algorithm), and/or determine whether the detection device 41 is able to detect a position of the anatomy and/or a position of the tool 50.
In another embodiment, a safety feature of the surgical system 10 includes an occlusion detection algorithm adapted to mitigate risk during a cutting operation in the event tracking elements (e.g., the trackers 43a, 43b, 45) associated with the haptic device 30 and/or the anatomy become occluded. An occluded state may exist, for example, when the detection device 41 is unable to detect a tracking element (e.g., when a person or object is interposed between the tracking element and the detection device 41), when a lens of the detection device 41 is occluded (e.g., by dust), and/or when reflectivity of markers on a tracking element is degraded (e.g., by blood, tissue, dust, bone debris, etc.). If an occluded state is detected, the occlusion detection algorithm alerts the user, for example, by causing a warning message to be displayed on the display device 23, an audible alarm to sound, and/or the generation of tactile feedback (e.g., vibration). The occlusion detection algorithm may also issue a control signal, such as a command to the surgical system 10 to shut off power to or otherwise disable the tool 50 or to impose a constraint on the haptic device 30 (e.g., providing haptic guidance, changing a mode of the haptic device 30, etc.). In this manner, the occlusion detection algorithm prevents the tool 50 from damaging the anatomy when the tracking system 40 is not able to accurately determine relative positions of the tool 50 and the anatomy.
In one embodiment, the occlusion detection algorithm considers a position of the tool 50 relative to a haptic boundary. In this embodiment, if the occlusion detection algorithm detects an occluded state, the surgical system 10 determines whether the tool 50 is touching a haptic boundary of a haptic object. If the tool 50 is not in contact with a haptic boundary at the time of an occlusion event, the occlusion detection algorithm disables the tool 50 and places the haptic device 30 in the free mode so that the tool 50 will move with the patient and, if necessary, can be withdrawn from the patient. When the occluded state ends (e.g., when all occluded trackers become visible), the surgical system 10 places the haptic device 30 in the approach mode so that the user may resume the procedure. In this manner, the occlusion detection algorithm permits the haptic boundary to be deactivated if the user isn't pushing against the haptic wall at the time of the occlusion event. In contrast, if the surgical system 10 determines that the tool 50 is touching the haptic boundary and/or exceeding the haptic boundary at the time of the occlusion event, the occlusion detection algorithm waits for a predetermined period of time (e.g., 1 second) to see if the occluded tracker(s) become visible. During this time, the tool 50 is disabled, and the user is alerted that the tracker(s) are occluded (e.g., via a visual, audible, or tactile signal). If the haptic device tracker 45 and the anatomy trackers 43a and 43b all become visible within the predetermined period of time, the haptic (or burring) mode is resumed. Otherwise, the haptic device 30 is placed in the free mode so that the tool 50 will move with the patient and, if necessary, can be withdrawn from the patient. As before, when the occluded state ends (e.g., when all occluded trackers again become visible), the surgical system 10 places the haptic device 30 in the approach mode so that the user may resume the procedure. One advantage of utilizing the predetermined period of time (or time interval) is that the occlusion detection algorithm allows the haptic wall to remain active during momentary occlusion events. Additionally, sudden removal of the haptic walls, which might result in sudden motion from the surgeon during cutting, is avoided. Additionally, if the occluded condition ceases to exist within the predetermined period of time, the low pass filter utilized for dynamic tracking (motion compensation) is reset to prevent the tracking system 40 from perceiving small motions as discontinuous motion.
In contrast, if the detection device 41 in step S3502 is unable to detect the haptic device tracker 45 and/or the relevant anatomy tracker, the algorithm concludes that at least one of the trackers is occluded and proceeds to step S3506. The surgical tool 50 may be disabled, for example, by shutting off power to the tool 50 so that the tool 50 cannot be activated by the user even if the user attempts to activate the tool 50, such as by depressing a foot pedal. After the tool 50 is disabled, the algorithm the proceeds to step S3508 and provides an indication to the user that an occluded state exists. The indication may be any suitable signal, such as a visual signal on the display device 23, an audible signal (e.g., a beep, alarm, or other warning sound), a tactile signal (e.g., vibration), and/or a control signal (e.g., a control signal that commands the haptic device 30 to lock the arm 33 in position). In step S3510, the algorithm determines whether a haptic force is detected. A haptic force is detected, for example, when the haptic device 30 is providing force feedback to the user (e.g., haptic guidance and/or a limit on user manipulation of the arm 33). If a haptic force is not detected in step S3510, the algorithm proceeds to step S3518, deactivates the haptic mode, and enables the free mode. When the haptic device 30 is in the free mode, the tool 50 will move with the patient and, if necessary, can be withdrawn from the patient. When the occluded state ends, the surgical system 10 places the haptic device 30 in the approach mode so that the surgeon may resume the procedure.
In contrast, if a haptic force is detected, the algorithm proceeds to step S3512 and maintains the haptic device 30 in the haptic mode. In step S3514, the algorithm determines whether the haptic device tracker 45 and/or the relevant anatomy tracker is still occluded. If the trackers are not occluded, the algorithm returns to step S3500 where the haptic device 30 is maintained in the haptic mode 30 so that the surgeon may continue the procedure. In contrast, if at least one of the trackers is still occluded, the algorithm proceeds to step S3516 and determines whether a time t has elapsed since the occluded state was detected. The time t may be chosen based on the application. In one embodiment, the time t is about 1 second. If the time t has not elapsed, the algorithm returns to step S3514. If the time t has elapsed, the algorithm proceeds to step S3518, deactivates the haptic mode, and enables the free mode. When the haptic device 30 is in the free mode, the tool 50 will move with the patient and, if necessary, can be withdrawn from the patient. When the occluded state ends, the surgical system 10 places the haptic device 30 in the approach mode so that the surgeon may resume the procedure. In this manner, the occlusion detection algorithm advantageously limits the user's ability to activate the tool 50 when the surgical system 10 is not able to determine the relative positions of the haptic device 30 and the anatomy. As a result, the risk of damaging the anatomy is mitigated.
Another embodiment of the occlusion detection algorithm includes a method for controlling the haptic device 30 comprising the following steps: (a) detecting with the detection device 41 a first object comprising at least one of the anatomy and a tracking element associated with the anatomy; (b) detecting with the detection device 41 a second object comprising at least one of the haptic device 30 and a tracking element associated with the haptic device 30; and (c) providing an indication to the user if the detection device 41 is unable to detect the first object and/or the second object. The indication may be, for example, a signal, such as a visual, an audible, a tactile, and/or a control signal, or may be provided by disabling at least a portion of the haptic device 30, such as the tool 50. In one embodiment, the method includes imposing a constraint on the haptic device 30, such as limiting movement of at least a portion of the haptic device 30 (e.g., the arm 33, the tool 50) or limiting operation of the haptic device 30 (e.g., shutting off power to or otherwise disabling the tool 50, changing a mode of the haptic device, etc.). The constraint is preferably removed after a predetermined time interval (e.g., 1 second as discussed above in connection with step S3516 of
In one embodiment, the occlusion detection algorithm determines whether the haptic device 30 is providing haptic guidance to the user and/or a limit on user manipulation of the haptic device 30. The haptic guidance and/or the limit on user manipulation may be based, for example, on a virtual boundary associated with the anatomy. If haptic guidance and/or a limit on user manipulation is being provided, the haptic guidance and/or the limit on user manipulation is preferably maintained to avoid damage to the anatomy (e.g., damage caused by sudden removal of the virtual boundary or haptic wall when the user is pushing against the virtual boundary with the tool 50). Accordingly, the virtual boundary is preferably maintained if a portion of the haptic device 30 (e.g., the tip of the tool 50) is proximate to, in contact with, or exceeding the virtual boundary. The method may also include deactivating the virtual boundary if the portion of the haptic device 30 is not interacting with the virtual boundary (e.g., if the tool 50 is not in contact with the virtual boundary or haptic wall). In this situation, because the user is not pushing against the virtual boundary with the tool 50, the tool 50 is not likely to damage the anatomy if the virtual boundary is suddenly removed. As a result, the risk of damaging the anatomy is reduced.
Thus, embodiments of the present invention provide a surgical system that is able to cooperatively interact with a surgeon to enable the surgeon to sculpt complex shapes in bone in a minimally invasive manner and that has the ability to dynamically compensate for motion of objects in the intraoperative environment in a manner that safeguards the patient and is substantially transparent to the surgeon.
A system and method for verifying calibration of a surgical device is disclosed in U.S. patent application Ser. No. 11/750,807, entitled System and Method for Verifying Calibration of a Surgical Device, by Louis Arata, Sherif Aly, Robert Van Vorhis, Sandi Glauser, Timothy Blackwell, Rony Abovitz, and Maurice R. Ferre, filed on May 18, 2007, the disclosure of which is hereby incorporated herein by reference in its entirety.
This application is a continuation of U.S. application Ser. No. 16/509,651, filed Jul. 12, 2019, which is a continuation of U.S. application Ser. No. 15/288,769, filed Oct. 7, 2016, which is a continuation of U.S. application Ser. No. 11/750,815, filed May 18, 2007, which claims the benefit of and priority to U.S. Provisional Application No. 60/801,378, filed May 19, 2006, all of which are hereby incorporated by reference herein in their entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4903536 | Salisbury et al. | Feb 1990 | A |
| 4979949 | Matsen et al. | Dec 1990 | A |
| 5046375 | Salisbury et al. | Sep 1991 | A |
| 5086401 | Glassman et al. | Feb 1992 | A |
| 5142930 | Allen et al. | Sep 1992 | A |
| 5154717 | Matsen et al. | Oct 1992 | A |
| 5207114 | Salisbury et al. | May 1993 | A |
| 5230338 | Allen et al. | Jul 1993 | A |
| 5236432 | Matsen et al. | Aug 1993 | A |
| 5299288 | Glassman et al. | Mar 1994 | A |
| 5343385 | Joskowicz et al. | Aug 1994 | A |
| 5383901 | Mcgregor et al. | Jan 1995 | A |
| 5388480 | Townsend | Feb 1995 | A |
| 5399951 | Vallee et al. | Mar 1995 | A |
| 5408409 | Glassman et al. | Apr 1995 | A |
| 5445144 | Wodicka et al. | Aug 1995 | A |
| 5445166 | Taylor | Aug 1995 | A |
| 5452941 | Halse et al. | Sep 1995 | A |
| 5551429 | Fitzpatrick et al. | Sep 1996 | A |
| 5572999 | Funda et al. | Nov 1996 | A |
| 5576727 | Rosenberg et al. | Nov 1996 | A |
| 5587937 | Massie et al. | Dec 1996 | A |
| 5611353 | Dance et al. | Mar 1997 | A |
| 5625576 | Massie et al. | Apr 1997 | A |
| 5630431 | Taylor | May 1997 | A |
| 5638819 | Manwaring et al. | Jun 1997 | A |
| 5676673 | Ferre et al. | Oct 1997 | A |
| 5682886 | Delp et al. | Nov 1997 | A |
| 5688280 | Booth et al. | Nov 1997 | A |
| 5694013 | Stewart et al. | Dec 1997 | A |
| 5695500 | Taylor et al. | Dec 1997 | A |
| 5701140 | Rosenberg et al. | Dec 1997 | A |
| 5704791 | Gillio | Jan 1998 | A |
| 5727554 | Kalend et al. | Mar 1998 | A |
| 5792147 | Evans et al. | Aug 1998 | A |
| 5799055 | Peshkin et al. | Aug 1998 | A |
| 5806518 | Mittelstadt | Sep 1998 | A |
| 5831408 | Jacobus et al. | Nov 1998 | A |
| 5855553 | Tajima et al. | Jan 1999 | A |
| 5871018 | Delp et al. | Feb 1999 | A |
| 5887121 | Funda et al. | Mar 1999 | A |
| 5888220 | Felt et al. | Mar 1999 | A |
| 5898599 | Massie et al. | Apr 1999 | A |
| 5928137 | Green | Jul 1999 | A |
| 5950629 | Taylor et al. | Sep 1999 | A |
| 5971997 | Guthrie et al. | Oct 1999 | A |
| 5976156 | Taylor et al. | Nov 1999 | A |
| 5978696 | Vomlehn et al. | Nov 1999 | A |
| 5980535 | Barnett et al. | Nov 1999 | A |
| 5984930 | Maciunas et al. | Nov 1999 | A |
| 5987960 | Messner et al. | Nov 1999 | A |
| 6002859 | Digioia, III et al. | Dec 1999 | A |
| 6006126 | Cosman | Dec 1999 | A |
| 6006127 | Van Der Brug et al. | Dec 1999 | A |
| 6017305 | Bonutti | Jan 2000 | A |
| 6033415 | Mittelstadt et al. | Mar 2000 | A |
| 6084587 | Tarr et al. | Jul 2000 | A |
| 6104158 | Jacobus et al. | Aug 2000 | A |
| 6109270 | Mah et al. | Aug 2000 | A |
| 6111577 | Zilles et al. | Aug 2000 | A |
| 6147674 | Rosenberg et al. | Nov 2000 | A |
| 6161032 | Acker | Dec 2000 | A |
| 6188728 | Hurst | Feb 2001 | B1 |
| 6191796 | Tarr | Feb 2001 | B1 |
| 6205411 | Digioia, III et al. | Mar 2001 | B1 |
| 6219032 | Rosenberg et al. | Apr 2001 | B1 |
| 6223100 | Green | Apr 2001 | B1 |
| 6226566 | Funda et al. | May 2001 | B1 |
| 6228089 | Wahrburg | May 2001 | B1 |
| 6231526 | Taylor et al. | May 2001 | B1 |
| 6233504 | Das et al. | May 2001 | B1 |
| 6259806 | Green | Jul 2001 | B1 |
| 6285902 | Kienzle et al. | Sep 2001 | B1 |
| 6288705 | Rosenberg et al. | Sep 2001 | B1 |
| 6292174 | Mallett et al. | Sep 2001 | B1 |
| 6300936 | Braun et al. | Oct 2001 | B1 |
| 6322467 | Hook et al. | Nov 2001 | B1 |
| 6322567 | Mittelstadt et al. | Nov 2001 | B1 |
| 6325808 | Bernard et al. | Dec 2001 | B1 |
| 6337994 | Stoianovici et al. | Jan 2002 | B1 |
| 6366273 | Rosenberg et al. | Apr 2002 | B1 |
| 6369834 | Zilles et al. | Apr 2002 | B1 |
| 6377011 | Ben-Ur | Apr 2002 | B1 |
| 6377839 | Kalfas et al. | Apr 2002 | B1 |
| 6385475 | Cinquin et al. | May 2002 | B1 |
| 6385509 | Das et al. | May 2002 | B2 |
| 6393340 | Funda et al. | May 2002 | B2 |
| 6405072 | Cosman | Jun 2002 | B1 |
| 6405158 | Massie et al. | Jun 2002 | B1 |
| 6417638 | Guy et al. | Jul 2002 | B1 |
| 6421048 | Shih et al. | Jul 2002 | B1 |
| 6424885 | Niemeyer et al. | Jul 2002 | B1 |
| 6430434 | Mittelstadt | Aug 2002 | B1 |
| 6434416 | Mizoguchi et al. | Aug 2002 | B1 |
| 6443894 | Sumanaweera et al. | Sep 2002 | B1 |
| 6466815 | Saito et al. | Oct 2002 | B1 |
| 6468265 | Evans et al. | Oct 2002 | B1 |
| 6484118 | Govari | Nov 2002 | B1 |
| 6493608 | Niemeyer | Dec 2002 | B1 |
| 6494039 | Pratt et al. | Dec 2002 | B2 |
| 6522906 | Salisbury et al. | Feb 2003 | B1 |
| 6533737 | Brosseau et al. | Mar 2003 | B1 |
| 6546277 | Franck et al. | Apr 2003 | B1 |
| 6547782 | Taylor | Apr 2003 | B1 |
| 6551325 | Neubauer et al. | Apr 2003 | B2 |
| 6552722 | Shih et al. | Apr 2003 | B1 |
| 6583161 | Medina | Jun 2003 | B1 |
| 6642686 | Ruch | Nov 2003 | B1 |
| 6671651 | Goodwin et al. | Dec 2003 | B2 |
| 6674916 | Deman et al. | Jan 2004 | B1 |
| 6690964 | Bieger et al. | Feb 2004 | B2 |
| 6692485 | Brock et al. | Feb 2004 | B1 |
| 6701174 | Krause et al. | Mar 2004 | B1 |
| 6704694 | Basdogan et al. | Mar 2004 | B1 |
| 6711431 | Sarin et al. | Mar 2004 | B2 |
| 6711432 | Krause et al. | Mar 2004 | B1 |
| 6748819 | Maeguchi et al. | Jun 2004 | B2 |
| 6750877 | Rosenberg et al. | Jun 2004 | B2 |
| 6757582 | Brisson et al. | Jun 2004 | B2 |
| 6778850 | Adler et al. | Aug 2004 | B1 |
| 6785572 | Yanof et al. | Aug 2004 | B2 |
| 6786896 | Madhani et al. | Sep 2004 | B1 |
| 6801801 | Sati | Oct 2004 | B1 |
| 6810281 | Brock et al. | Oct 2004 | B2 |
| 6816148 | Mallett et al. | Nov 2004 | B2 |
| 6831640 | Shih et al. | Dec 2004 | B2 |
| 6845691 | Hsien | Jan 2005 | B2 |
| 6850794 | Shahidi | Feb 2005 | B2 |
| 6853965 | Massie et al. | Feb 2005 | B2 |
| 6859661 | Tuke | Feb 2005 | B2 |
| 6877239 | Leitner et al. | Apr 2005 | B2 |
| 6894678 | Rosenberg et al. | May 2005 | B2 |
| 6920347 | Simon et al. | Jul 2005 | B2 |
| 6985133 | Rodomista et al. | Jan 2006 | B1 |
| 6987504 | Rosenberg et al. | Jan 2006 | B2 |
| 7001346 | White | Feb 2006 | B2 |
| 7035716 | Harris et al. | Apr 2006 | B2 |
| 7039866 | Rosenberg et al. | May 2006 | B1 |
| 7131073 | Rosenberg et al. | Oct 2006 | B2 |
| 7168042 | Braun et al. | Jan 2007 | B2 |
| 7199790 | Rosenberg et al. | Apr 2007 | B2 |
| 7206626 | Quaid, III | Apr 2007 | B2 |
| 7206627 | Abovitz et al. | Apr 2007 | B2 |
| 7491198 | Kockro | Feb 2009 | B2 |
| 7660623 | Hunter et al. | Feb 2010 | B2 |
| 7717932 | Mcfarlin et al. | May 2010 | B2 |
| 7742804 | Faul | Jun 2010 | B2 |
| 7747311 | Quaid, III | Jun 2010 | B2 |
| 7774044 | Sauer et al. | Aug 2010 | B2 |
| 7831292 | Quaid et al. | Nov 2010 | B2 |
| 8010180 | Quaid et al. | Aug 2011 | B2 |
| 8095200 | Quaid, III | Jan 2012 | B2 |
| 8391954 | Quaid, III | Mar 2013 | B2 |
| 8571628 | Kang et al. | Oct 2013 | B2 |
| 8911499 | Quaid et al. | Dec 2014 | B2 |
| 9002426 | Quaid et al. | Apr 2015 | B2 |
| 9492237 | Kang et al. | Nov 2016 | B2 |
| 9724165 | Arata et al. | Aug 2017 | B2 |
| 10028789 | Quaid et al. | Jul 2018 | B2 |
| 10231790 | Quaid et al. | Mar 2019 | B2 |
| 10350012 | Kang et al. | Jul 2019 | B2 |
| 11033343 | Dozeman | Jun 2021 | B2 |
| 11123143 | Kang et al. | Sep 2021 | B2 |
| 20010034530 | Malackowski et al. | Oct 2001 | A1 |
| 20010037064 | Shahidi | Nov 2001 | A1 |
| 20010039422 | Carol et al. | Nov 2001 | A1 |
| 20010041838 | Holupka et al. | Nov 2001 | A1 |
| 20020062177 | Hannaford et al. | May 2002 | A1 |
| 20020082498 | Wendt et al. | Jun 2002 | A1 |
| 20020107521 | Petersen et al. | Aug 2002 | A1 |
| 20020108054 | Moore et al. | Aug 2002 | A1 |
| 20020120188 | Brock et al. | Aug 2002 | A1 |
| 20030093103 | Malackowski et al. | May 2003 | A1 |
| 20030112281 | Sriram et al. | Jun 2003 | A1 |
| 20030187351 | Franck et al. | Oct 2003 | A1 |
| 20030209096 | Pandey | Nov 2003 | A1 |
| 20040009459 | Anderson et al. | Jan 2004 | A1 |
| 20040012806 | Murata | Jan 2004 | A1 |
| 20040024311 | Quaid, III | Feb 2004 | A1 |
| 20040034282 | Quaid et al. | Feb 2004 | A1 |
| 20040034283 | Quaid | Feb 2004 | A1 |
| 20040034302 | Abovitz et al. | Feb 2004 | A1 |
| 20040102866 | Harris et al. | May 2004 | A1 |
| 20040106916 | Quaid et al. | Jun 2004 | A1 |
| 20040115606 | Davies | Jun 2004 | A1 |
| 20040127788 | Arata | Jul 2004 | A1 |
| 20040128026 | Harris et al. | Jul 2004 | A1 |
| 20040143243 | Wahrburg | Jul 2004 | A1 |
| 20040157188 | Luth et al. | Aug 2004 | A1 |
| 20040167654 | Grimm et al. | Aug 2004 | A1 |
| 20040171924 | Mire et al. | Sep 2004 | A1 |
| 20040236424 | Berez et al. | Nov 2004 | A1 |
| 20040242993 | Tajima | Dec 2004 | A1 |
| 20050001831 | Shih et al. | Jan 2005 | A1 |
| 20050053200 | Sukovic et al. | Mar 2005 | A1 |
| 20050062738 | Handley et al. | Mar 2005 | A1 |
| 20050093821 | Massie et al. | May 2005 | A1 |
| 20050107801 | Davies et al. | May 2005 | A1 |
| 20050113677 | Davies et al. | May 2005 | A1 |
| 20050154471 | Aram et al. | Jul 2005 | A1 |
| 20050165489 | Michelson | Jul 2005 | A1 |
| 20050197800 | Goodwin et al. | Sep 2005 | A1 |
| 20050203384 | Sati et al. | Sep 2005 | A1 |
| 20050215879 | Chuanggui | Sep 2005 | A1 |
| 20050215888 | Grimm et al. | Sep 2005 | A1 |
| 20050217394 | Langley et al. | Oct 2005 | A1 |
| 20050222830 | Massie et al. | Oct 2005 | A1 |
| 20060033707 | Rodomista et al. | Feb 2006 | A1 |
| 20060058616 | Marquart et al. | Mar 2006 | A1 |
| 20060084867 | Tremblay et al. | Apr 2006 | A1 |
| 20060109266 | Itkowitz et al. | May 2006 | A1 |
| 20060133827 | Becouarn et al. | Jun 2006 | A1 |
| 20060142657 | Quaid et al. | Jun 2006 | A1 |
| 20060207419 | Okazaki et al. | Sep 2006 | A1 |
| 20060265179 | Jansen et al. | Nov 2006 | A1 |
| 20060293598 | Fraser | Dec 2006 | A1 |
| 20070142751 | Kang et al. | Jun 2007 | A1 |
| 20070260140 | Solar et al. | Nov 2007 | A1 |
| 20070270685 | Kang et al. | Nov 2007 | A1 |
| 20080004633 | Arata et al. | Jan 2008 | A1 |
| 20080010705 | Quaid et al. | Jan 2008 | A1 |
| 20080010706 | Moses et al. | Jan 2008 | A1 |
| 20080058945 | Hajaj et al. | Mar 2008 | A1 |
| 20080125637 | Geist et al. | May 2008 | A1 |
| 20090000626 | Quaid et al. | Jan 2009 | A1 |
| 20090000627 | Quaid et al. | Jan 2009 | A1 |
| 20090012531 | Quaid et al. | Jan 2009 | A1 |
| 20090012532 | Quaid et al. | Jan 2009 | A1 |
| 20100137882 | Quaid, III | Jun 2010 | A1 |
| 20100170362 | Bennett et al. | Jul 2010 | A1 |
| 20100198219 | Mcfarlin et al. | Aug 2010 | A1 |
| 20100225209 | Goldberg et al. | Sep 2010 | A1 |
| 20110130761 | Plaskos et al. | Jun 2011 | A1 |
| 20110270120 | Mcfarlin et al. | Nov 2011 | A1 |
| 20120176306 | Lightcap et al. | Jul 2012 | A1 |
| 20130096573 | Kang et al. | Apr 2013 | A1 |
| 20130096574 | Kang et al. | Apr 2013 | A1 |
| 20140350571 | Maillet et al. | Nov 2014 | A1 |
| 20160097676 | Kurasawa et al. | Apr 2016 | A1 |
| 20160124022 | Tadano | May 2016 | A1 |
| 20160153777 | Nl et al. | Jun 2016 | A1 |
| 20160155097 | Venkatesha | Jun 2016 | A1 |
| 20190290367 | Andersson et al. | Sep 2019 | A1 |
| 20210128253 | Kang | May 2021 | A1 |
| 20210307843 | Kang | Oct 2021 | A1 |
| 20220031404 | Kang | Feb 2022 | A1 |
| Number | Date | Country |
|---|---|---|
| 1684729 | Oct 2005 | CN |
| 1 059 067 | Dec 2000 | EP |
| 1 184 684 | Mar 2002 | EP |
| 1 380 266 | Jan 2004 | EP |
| 1 574 186 | Jun 2008 | EP |
| 1 871 267 | Sep 2018 | EP |
| 08-215211 | Aug 1996 | JP |
| 09-330016 | Dec 1997 | JP |
| 2000-279425 | Oct 2000 | JP |
| 2002-102251 | Apr 2002 | JP |
| 2003-053684 | Feb 2003 | JP |
| 2004-513684 | May 2004 | JP |
| WO-9501757 | Jan 1995 | WO |
| WO-9617552 | Jun 1996 | WO |
| WO-0035336 | Jun 2000 | WO |
| WO-0224051 | Mar 2002 | WO |
| WO-02060653 | Aug 2002 | WO |
| WO-02061371 | Aug 2002 | WO |
| WO-03007101 | Jan 2003 | WO |
| WO-03077101 | Sep 2003 | WO |
| WO-2004069036 | Aug 2004 | WO |
| WO-2004069040 | Aug 2004 | WO |
| WO-2004069041 | Aug 2004 | WO |
| WO-2004070573 | Aug 2004 | WO |
| WO-2004070577 | Aug 2004 | WO |
| WO-2004070580 | Aug 2004 | WO |
| WO-2004070581 | Aug 2004 | WO |
| WO-2004075987 | Sep 2004 | WO |
| WO-2005009215 | Feb 2005 | WO |
| WO-2005013841 | Feb 2005 | WO |
| WO-2005072629 | Aug 2005 | WO |
| WO-2005120380 | Dec 2005 | WO |
| WO-2005122916 | Dec 2005 | WO |
| WO-2006004894 | Jan 2006 | WO |
| WO-2006091494 | Aug 2006 | WO |
| WO-2007117297 | Oct 2007 | WO |
| Entry |
|---|
| U.S. Appl. No. 60/801,378, filed May 19, 2006. (Year: 2006). |
| Abovitz et al., “The Future Use of Networked Haptic Learning Information Systems in Computer-Assisted Surgery,” CAOS 2001, Jul. 6-8, 2001, pp. 337-338. |
| Abovitz, “Digital surgery: the future of medicine and human-robot symbiotic interaction,” Industrial Robot: an International Journal, Oct. 2001, vol. 28, Issue 5, pp. 401-406 (abstract only). |
| Abovitz, “Human-Interactive Medical Robotics,” CAOS 2000, Jun. 15-17, 2000, pp. 71-72. |
| Abovitz, “Human-Interactive Medical Robotics,” CAOS 2001, Jul. 6-8, 2001, pp. 81-82. |
| Acosta, et al., “Development of a Haptic Virtual Environment”, Computer-Based Medical Systems, Proceedings 12th IEEE Symposium, pp. 35-39, 1999. |
| Bennett et al., “Autonomous Calibration of Single-Loop Kinematic Chains Formed by Manipulators With Passive End-Point Constraints,” IEEE Transactions on Robotics and Automation, vol. 7, pp. 597-606, 1991. |
| Bettini et al., “Vision assisted control for manipulation using virtual fixtures: Experiments at macro and micro scales,” in Proc. 2002 IEEE Intl. Conf. on Robotics and Automation, (Washington, DC), May 2002, 8 pages. |
| Chapter II Demand and Response to Written Opinion for PCT/US2006/005700, submitted Dec. 15, 2006, 16 pages. |
| Chapter II Demand and Response to Written Opinion for PCT/US2006/049216, submitted Jul. 15, 2008, 19 pages. |
| Chen et al., “Force Feedback for Surgical Simulation,” Proceedings of the IEEE, New York, US, vol. 86, No. 3, Mar. 1, 1998. pp. 524-530. |
| Cobb et al., “A robotic system for TKR surgery,” in Third Annual North American Program on Computer Assisted Orthopaedic Surgery, (Pittsburgh, PA), pp. 70-74, Jun. 1999. |
| Colgate, J. Edward, et al., “Cobots: Robots for Collaboration with Human Operators,” proceedings of International Mechanical Engineering Congress & Exhibition, DSC-vol. 58, 1996, pp. 433-439. |
| Davies et al., “Acrobot-using Robots and Surgeons Synergistically in Knee Surgery”, 1997 British Crown Copyright, pp. 173-178. |
| Davies et al., “The use of force control in robot assisted knee surgery,” in Proceedings of the First Annual Symposium on Medical Robotics and Computer Assisted Surgery, vol. 2, (Pittsburgh, PA), pp. 258-262, Sep. 1994. |
| Decision to Refuse a European Patent Application for EP Application No. 07756266.8 dated Aug. 3, 2016, 32 pages. |
| Definition of devices, web thesaurus, printers Mar. 4, 2021. (Year: 2021). |
| Definition of Object, web dictionary, printed Mar. 4, 2021 (Year: 2021). |
| Examination report for EP 04757075.9, dated Jan. 12, 2011, 5 pages. |
| Fritz, et al., “Design of a Haptic Data Visualization System for People with Visual Impairments”, IEEE Trans. on Rehabiliation Engineering, vol. 7, No. 3, Sep. 1999, 13 pages. |
| Germano et al., Clinical Use of the Optical Digitizer for Intracranial Neuronavigation, Neurosurgery, vol. 45(2), Aug. 1999, 15 pages. |
| Goswami, et al., “Identifying Robot Parameters Using Partial Pose Information,” IEEE Control Systems Magazine, vol. 13, No. 5, Oct. 1993, 11 pages. |
| Ho, S.C. et al., “Robot Assisted Knee Surgery Establishing a Force Control Strategy Incorporating Active Motion Constraint,” IEEE Engineering in Medicine and Biology Magazine, vol. 14, No. 3, May 1, 1995, col. 2-3, p. 293. |
| Hollerbach, J.M. & D. E. Johnson. Virtual Environment Rendering. To appear in Human and Machine Haptics, M. Cutkosky, R. Howe, K. Salisbury, and M. Srinivasan (eds.), MIT Press, 2000 (available at http://www.cs.ubc.ca/labs/spin/publications/related/hollerbach00.pdf), 25 pages. |
| International Preliminary Examination Report for PCT/US2003/007063, dated Sep. 2, 2004 (2 pages). |
| International Preliminary Report on Patentability for PCT/US2004/022978 including International Search Report and Written Opinion, dated Feb. 13, 2007 (6 pages). |
| International Preliminary Report on Patentability for PCT/US2006/005700, dated May 8, 2007 (7 pages). |
| International Preliminary Report on Patentability for PCT/US2006/049216, dated Sep. 10, 2008, 9 pages. |
| International Search Report and Written Opinion for corresponding PCT Application No. PCT/US2006/049216, dated May 8, 2008 (15 pgs.). |
| International Search Report and Written Opinion for PCT/US2006/005700, dated Jun. 27, 2006, 10 pages. |
| International Search Report for PCT/US2003/007063, dated Apr. 16, 2004 (7 pages). |
| Kanazides, Peter et al., “An Integrated System for Cementless Hip Replacement”, Integrated Surgical Systems Department of Orthopedic Surgery, Sutter General Hospital, May/Jun. 1995, pp. 307-313. |
| Leeser et al., “Computerassisted teach and play: Novel user-friendly robot teach mode using gravity compensation and backdrivability,” in Proceedings of the Robotics International/SME Fifth World Conference on Robotics Research, (Cambridge, MA), Sep. 1994, 7 pages. |
| Leeser, Karl, et al., “Control and Exploitation of Kinematic Redundancy in Torque-Controllable Manipulators via Multiple-Jacobian Superposition,” to the International Conf. on Field & Service Robotics, Dec. 8-10, 1997, 7 pages. |
| London Press Services, “‘Acrobot’ capable of delicate knee surgery,” Can. Med. Assoc. J., Jun. 15, 1997, 156(12), p. 1690. |
| Matsuoka, Yoky, et al., “Design of Life-Size Haptic Environments,” Experimental Robotics VII, 2001, pp. 461-470. |
| Meggiolaro, et al., “Manipulator calibration using a single endpoint contact constraint,” in 26th ASME Bienniel Mechanisms Conference, (Baltimore, MD), 2000, 9 pages. |
| Moore, Carl A., et al., “Cobot Implementation of 3D Virtual Surfaces,” proceedings of the 2002 Institute of Electrical and Electronics Engineers International Conference on Robotics & Automation, May 2002, pp. 3242-3247. |
| Niki, et al., “Simple Haptic Display and Object Data Design”, Proceedings of the 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 967-972, 2000. |
| Otmane, S., et al., “Active Virtual Guides as an Apparatus for Augmented Reality Based Telemanipulation System on the Internet,” presented at Institute of Electrical and Electronics Engineers Computer Society 33rd Annual Simulation Symposium ANSS 2000, held Apr. 16-20, 2000, pp. 185-191. |
| Park et al., “Virtual fixtures for robotic cardiac surgery,” in Proc. Medical Image Computing and Computer-Assisted Intervention, (Utrecht, Netherlands), Oct. 2001, 2 pages. |
| PCT/US2006/049216, Partial Intl. Search Report, dated Jan. 18, 2008 (2 pgs.). |
| Press Release, “The Acrobot Company Wins Best Surgical Innovation Award,” Acrobot Precision Surgical Systems, May 24, 2002, 1 page. |
| Provision of the Minutes in Accordance with Rule 124(4) EPC for EP Application No. 07756266.8 dated Aug. 2, 2016, 5 pages. |
| Quaid et al., “Haptic Information Displays for Computer-Assisted Surgery,” Proceedings of the 2002 IEEE International Conference on Robotics & Automation, May 2002, pp. 2092-2097. |
| Quaid, Arthur E., et al., “FGS WAM: First Cadaver Trial,” Z-Kat, Inc. Confidential Material, Sep. 28, 2001, pp. 1-7. |
| Quaid, Arthur E., et al., “FGS WAM: Integration of Fluorotactic Guidance with the Whole-Arm Manipulator,” Z-Kat, Inc. Confidential Material, Dec. 28, 2000, pp. 1-6. |
| Quaid, et al., “The Use of Haptic Information Displays for Assisting in the Execution of Image-Guided Surgery Plans,” Syllabus of the Computer Assisted Orthopaedic Surgery Meeting, Jul. 2001, pp. 338-340. |
| Roche, “Changing the way surgeons plan and execute minimally invasive unicompartmental knee surgery,” Orthopaedic Product News, pp. 16-18, Jul./Aug. 2006. |
| Rosenberg, “Virtual Fixtures: Perceptual Tools for Telerobotic Manipulation”, 1993 IEEE, 76-82. |
| Rosenberg, Virtual Fixtures: Perceptual overlays enhance operator performance in telepresence tasks. PhD thesis, Stanford University, Aug. 1994, 7 pages. |
| Sayers, Craig P., et al., “An Operator Interface for Teleprogramming Employing Synthetic Fixtures,” to appear in Presence, Special Issue on Networked Virtual Environments and Teleoperation, Jun. 1994, pp. 1-27. |
| Schneider, O., et al., “Synergistic Robotic Assistance to Cardiac Procedures,” presented to Computer Assisted Radiology and Surgery on Jun. 23-26, 1999, 5 pages. |
| Sensable Technologies, Inc., “Freeform Feel the Difference”, 2001, 4 pages. |
| Sensable Technologies, Inc., “FreeForm Modeling—Technical Features,” 2003, 2 pages. |
| Staecker et al., “Use of the LandmarX (tm) Surgical Navigation System in Lateral Skull Base and Temporal Bone Surgery”, SkullBase, vol. 11, No. 4, 2001, pp. 245-255; Thieme Medical Publishers, Inc. 11 pages. |
| Taylor, Russell et al., “An Image-Directed Robotic System for Precise Orthopaedic Surgery”, IEEE Transactions on Robotics and Automation, vol. 10, No. 3, Jun. 1994, pp. 261-275. |
| Taylor, Russell et al., “Redundant Consistency Checking in a Precise Surgical Robot”, Annual International Conference of the IEEE Engineering in Medicine and Biology Society, vol. 12, No. 5, 1990, pp. 1933-1935. |
| Taylor, Russell et al., “Robotic Joint Replacement Surgery”, NSF Engineering Research Center for Computer-Integrated Surgical Systems and Technology, 2000, 2001, 2004, 71 pages. |
| Tognetti, Lawrence Joseph, “Actuator Design for a Passive Haptic Display,” Georgia Institute of Technology, Jun. 1999, 33 pages. |
| Townsend et al., “Teleoperator slave—WAM design methodology,” Industrial Robot, vol. 26, No. 3, pp. 167-177, 1999. |
| World Wide Web, http://haptics.me.jhu.edu/r.sub.--hapt.html, “Haptic Interfaces and Virtual Environments,” printed on Jun. 12, 2003, 2 pages. |
| World Wide Web, http://haptics.me.jhu.edu/r.sub.--kine.html, “Robot Design and Kinematics,” printed on Jun. 12, 2003, 2 pages. |
| World Wide Web, http://www.acrobot.co.uk/background.html, “The Acrobot Company Limited—Background,” printed on Jul. 10, 2002, 1 page. |
| World Wide Web, http://www.acrobot.co.uk/home.html, “The Acrobot Company Limited—Precision Surgical Systems,” printed on Jul. 10, 2002, 1 page. |
| World Wide Web, http://www.acrobot.co.uk/meetings.html, “The Acrobot Company Limited—Meetings and Publications,” printed on Jul. 10, 2002, pp. 1-3. |
| World Wide Web, http://www.acrobot.co.uk/products.html, “The Acrobot Company Limited—Products,” printed on Jul. 10, 2002, pp. 1-6. |
| World Wide Web, http://www.fcs-cs.com/robotics/content/assistance.htm, “Surgical Assistance,” printed on Jun. 12, 2003, 1 page. |
| World Wide Web, http://www.fcs-cs.com/robotics/content/design.htm, “Virtual Design, Assembly & Maintenance,” printed on Jun. 12, 2003, 1 page. |
| World Wide Web, http://www.fcs-cs.com/robotics/content/endeffectors.htm, “End effectors,” printed on Jun. 12, 2003, 1 page. |
| World Wide Web, http://www.fcs-cs.com/robotics/content/hapticmaster.htm, “HapticMASTER”, printed on Jun. 12, 2003, 1 page. |
| World Wide Web, http://www.fcs-cs.com/robotics/content/reality.htm, “Virtual Reality,” printed on Jun. 12, 2003, 1 page. |
| World Wide Web, http://www.fcs-cs.com/robotics/content/rehabilitation.htm, “Rehabilitation,” printed on Jun. 12, 2003, 1 page. |
| World Wide Web, http://www.fcs-cs.com/robotics/content/research.htm, “Research,” printed on Jun. 12, 2003, 1 page. |
| World Wide Web, http://www.fcs-cs.com/robotics/content/simulation.htm, “Simulation & Training,” printed on Jun. 12, 2003, 1 page. |
| World Wide Web, http://www.fcs-cs.com/robotics/content/software.htm, “Software,” printed on Jun. 12, 2003, 1 page. |
| World Wide Web, http://www.merl.com/projects/surgSim99/, “Knee Arthroscopy Simulation,” printed on Jun. 12, 2003, 2 pages. |
| Written Opinion for PCT/US2006/049216, dated May 8, 2008, 12 pages. |
| Zilles, et al., “A Constraint-Based God-object Method for Haptic Display”, IEEE Proceedings, pp. 146-151, 1995. |
| Number | Date | Country | |
|---|---|---|---|
| 20210361362 A1 | Nov 2021 | US |
| Number | Date | Country | |
|---|---|---|---|
| 60801378 | May 2006 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16509651 | Jul 2019 | US |
| Child | 17395668 | US | |
| Parent | 15288769 | Oct 2016 | US |
| Child | 16509651 | US | |
| Parent | 11750815 | May 2007 | US |
| Child | 15288769 | US |
