Robot assisted volume removal during surgery

Abstract

Described herein is a device and method used to effectively remove volume inside a patient in various types of surgeries, such as spinal surgeries (e.g. laminotomy), neurosurgeries (various types of craniotomy), ENT surgeries (e.g. tumor removal), and orthopedic surgeries (bone removal). Robotic assistance linked with a navigation system and medical imaging it can shorten surgery time, make the surgery safer and free surgeon from doing repetitive and laborious tasks. In certain embodiments, the disclosed technology includes a surgical instrument holder for use with a robotic surgical system. In certain embodiments, the surgical instrument holder is attached to or is part of an end effector of a robotic arm, and provides a rigid structure that allows for precise removal of a target volume in a patient.

Description

BACKGROUND OF THE INVENTION

Spinal stenosis (narrowing of the spinal canal) is the single most common diagnosis that leads to spinal surgery. A laminotomy may be performed to alleviate pain associated with spinal stenosis by partially removing/trimming the vertebra lamina shown in FIG. 1, thereby decompressing the corresponding spinal cord and/or spinal nerve root. During this type of surgery, after obtaining access to the vertebrae, a surgeon manually removes portions of the lamina. The surgical instruments used for this task include surgical pliers and a high-speed burr as shown in FIGS. 2A and 2B, respectively.

Removal/trimming of the lamina is a difficult process. The surgeon must be particularly careful not to damage spinal dura mater (tissue protecting spinal cord) which would lead to serious complications. This is a lengthy process which may take around 20 min per vertebra (e.g., 60 minutes for three vertebra). Furthermore, removing bony tissue requires repetitive movements which are tiring and may lead to repetitive motion disorders for surgeons performing these surgeries daily.

SUMMARY OF THE INVENTION

Described herein is a device and method used to effectively remove volume inside a patient in various types of surgeries, such as spinal surgeries (e.g. laminotomy), neurosurgeries (various types of craniotomy), ENT surgeries (e.g. tumor removal), and orthopedic surgeries (bone removal). The disclosed technology provides robotic assistance linked with a navigation system and medical imaging to shorten surgery time, make the surgery safer and free surgeons from doing repetitive and laborious tasks. The disclosed technology is also compatible with robotic surgical system described in U.S. patent application Ser. No. 14/266,769, filed Apr. 30, 2014, and entitled “Apparatus, Systems, and Methods for Precise Guidance of Surgical Tool” which is hereby incorporated by reference in its entirety, thereby, for example, allowing the same robotic surgical system to assist with different aspects of spinal surgery.

In certain embodiments, the disclosed technology includes a surgical instrument holder for use with a robotic surgical system. In certain embodiments, the holder is attached to or is part of an end effector of a robotic arm, and provides a rigid structure that can be used with a surgical tool to affect precise removal of a target volume in a patient.

In certain embodiments, the holder has a tubular shape with a longitudinal notch along a portion of its length. The holder is sized to allow a surgical instrument to slide through the holder in a fixed orientation while the holder is held by the robotic arm. The surgical instrument, in certain embodiments, is fitted with a tool support having a peg sized to fit the notch. A navigational marker (e.g., a multipoint, planar marker) may be attached thereto via the peg. Thus, because of the notch, movement of the surgical instrument is constrained in a fixed orientation as it slides along the axis defined by the holder. When a navigation marker is coupled to the peg, the navigation marker can be tracked (e.g., and hence the location of a surgical instrument connected to the navigation marker) when the surgical instrument is fully inserted and secured in the holder (e.g., in part by a locking mechanism). This facilitates and simplifies tracking of the marker, for example, via a remote tracking system that displays real-time tracking of the surgical instrument during the surgical procedure.

The disclosed technology, in certain embodiments, includes an instrument holder with a rigid hollow tubular structure having a proximal open end and a distal open end, said structure defining an axis along which movement of a surgical instrument sliding through the structure is restricted, wherein the tubular structure has an interior surface shaped and sized to accommodate the surgical instrument sliding through the surgical instrument holder such that movement of the surgical instrument is constrained to move along the axis defined by the surgical instrument holder (e.g., thereby allowing rapid exchange of surgical instruments held by the surgical instrument holder); and a connector element associated with (e.g., attached to) the holder that, when engaged (e.g., mated with) a corresponding connector element associated with a surgical instrument, provides power to the surgical instrument (e.g., and receives measurement information from a force sensor in the surgical instrument).

In certain embodiments, the tubular structure has an exterior surface including at least one flange that is sized and shaped to secure coupling of the surgical instrument holder to an end effector of the robotic surgical system.

In certain embodiments, the tubular structure includes a longitudinal notch along its length, wherein the longitudinal notch (e.g., slot) is sized in relation to a peg on (e.g., directly or indirectly) the surgical instrument to permit the surgical instrument to slide along the axis defined by the surgical instrument holder. In certain embodiments, the longitudinal notch is sized in relation to a peg to (i) permit a navigation marker attached to the surgical instrument at least in part via the peg to be viewable by a navigation camera along an entire range of movement of the surgical instrument through the surgical instrument holder, and (ii) constrain movement of the marker in a fixed orientation along the axis defined by the surgical instrument holder. In certain embodiments, navigation marker is used by navigation camera to track the surgical instrument.

In certain embodiments, a lock that, when engaged, restricts (e.g., prevents) movement of a surgical instrument within the rigid hollow tubular structure (e.g., such that the surgical instrument is constrained within the tubular structure in all directions). In certain embodiments, the lock, when engaged, prevents removal of the surgical instrument from the surgical instrument holder.

In certain embodiments, a force sensor that measures one or more forces and/or torques (e.g., 1 to 3 forces and 1 to 3 torques) applied to at least a portion of the surgical instrument. In certain embodiments, the surgical instrument includes a force sensor that measures one or more forces and/or torques (e.g., 1 to 3 forces and 1 to 3 torques) applied to at least a portion of the surgical instrument, and the measurement information provided to the surgical instrument holder via the connector comprises the one or more forces and/or torques.

In certain embodiments, the surgical instrument is a drill (e.g., with a drill bit). In some implementations, the portion of the surgical instrument to which the one or more forces and/or torques are applied is the drill bit. In certain embodiments, the surgical instrument is a drill (e.g., for preparing a hole for receiving a screw). In certain embodiments, the surgical instrument is a milling device, shaver, laser, or ultrasonic scalpel. In certain embodiments, the surgical instrument holder is for use in spinal surgery. In certain embodiments, the surgical instrument is a screw driver (e.g., for placing a screw in a hole).

In certain embodiments, a user interface (e.g., touch screen, one or more buttons, and/or a display). In certain embodiments, the tubular structure has an interior surface sized and shaped to accommodate a tool support (e.g., sliding surface) of the surgical instrument. In certain embodiments, a second connector associated with the surgical instrument holder that communicates with a sensor (e.g., the force sensor(s)) measuring the position of the surgical instrument. In certain embodiments, the second connector is one or more brushes (e.g., that physically contact the surgical instrument). In certain embodiments, the rigid hollow tubular structure is a cylindrical structure.

The disclosed technology, in certain embodiments, includes a robotic surgical system for performing surgery. In certain embodiments, a robotic arm with an end effector comprising a surgical instrument holder sized and shaped to hold and/or restrict movement of a surgical instrument therethrough, the surgical instrument holder comprising: a rigid hollow tubular structure having a proximal open end and a distal open end, said structure defining an axis along which movement of a surgical instrument (e.g., fitted with a tool support) sliding through the structure is restricted, wherein the tubular structure has an interior surface shaped and sized to accommodate the surgical instrument sliding through the surgical instrument holder such that movement of the tool support is constrained to move along the axis defined by the surgical instrument holder, wherein the tubular structure has an exterior surface comprising at least one flange that is sized and shaped to securely couple of the surgical instrument holder to an end effector of the robotic surgical system, and wherein the tubular structure comprises a longitudinal notch along its length, wherein the longitudinal notch is sized in relation to a peg on the tool support to permit the tool support to slide along the axis defined by the surgical instrument holder; a lock that, when engaged, restricts (e.g., prevents) movement of a surgical instrument within the rigid hollow tubular structure (e.g., such that the surgical instrument is constrained within the tubular structure in all directions); and a connector element associated with (e.g., attached to) the surgical instrument holder that, when engaged with (e.g., mated with) a corresponding connector associated with the surgical instrument connector, provides power to the surgical instrument (e.g., and information transfer, e.g., information concerning measured forces).

The disclosed technology, in certain embodiments, includes a manipulator that allows robotically-assisted or unassisted positioning and/or movement of the surgical instrument holder by a user with at least four degrees of freedom to align an axis defined by the instrument holder at a desired trajectory in relation to a patient situation.

The disclosed technology, in certain embodiments, includes a surgical instrument holder for use with a robotic surgical system. In certain embodiments, the surgical instrument holder includes a rigid hollow tubular structure having a proximal open end and a distal open end, said structure defining an axis along which movement of a surgical instrument (e.g., fitted with a tool support) sliding through the structure is restricted, wherein the tubular structure has an interior surface shaped and sized to accommodate the surgical instrument sliding through the surgical instrument holder such that movement of the tool support is constrained to move along the axis defined by the surgical instrument holder, wherein the tubular structure has an exterior surface comprising at least one flange that is sized and shaped to securely couple the surgical instrument holder to an end effector of the robotic surgical system, and wherein the tubular structure comprises a longitudinal notch along its length, wherein the longitudinal notch is sized in relation to a peg on the tool support to permit the tool support to slide along the axis defined by the surgical instrument holder; and a lock that, when engaged, restricts (e.g., prevents) movement of a surgical instrument within the rigid hollow tubular structure (e.g., such that the surgical instrument is constrained within the tubular structure in all directions); a connector element associated with (e.g., attached to) the surgical instrument holder that, when engaged with (e.g., mated with) a corresponding connected associated with the surgical instrument, provides power to the surgical instrument (e.g., and information transfer, e.g., information concerning measured forces).

In certain embodiments, a force sensor for measuring one or more forces and/or torques (e.g., 1 to 3 forces and 1 to 3 torques) applied to at least a portion of the surgical instrument. In certain embodiments, the surgical instrument comprises a force sensor for measuring one or more forces and/or torques (e.g., 1 to 3 forces and 1 to 3 torques) applied to at least a portion of the surgical instrument.

The disclosed technology, in certain embodiments, includes a surgical instrument for preparing a hole in bone tissue of a patient.

In certain embodiments, the surgical instrument includes an elongate structure having a proximal end with at least one of a drilling, milling, or shaving surface and a distal end with a shank sized and shaped to be grasped by a drill; and a force sensor integrated directly in the elongate structure for measuring one or more forces and/or torques (e.g., 1 to 3 forces and 1 to 3 torques) applied to at least a portion of the surgical instrument.

In certain embodiments, the surgical instrument includes an elongate structure having a proximal end with a milling surface and a distal end with a shank sized and shaped to be grasped by a drill, wherein the proximal end of the surgical instrument is flat and substantially perpendicular to the axis of the elongate structure, thereby reducing skidding (e.g., unintentional lateral movement of the surgical instrument) of the surgical instrument upon contact of the milling surface with the bone tissue; and a force sensor integrated directly in the elongate structure for measuring one or more forces and/or torques (e.g., 1 to 3 forces and 1 to 3 torques) applied to at least a portion of the surgical instrument.

In certain embodiments, a portion of the structure of the surgical instrument closest to the milling surface is for milling (e.g., rather than drilling) and the remaining portion of the surgical instrument is for drilling. In certain embodiments, the surgical instrument is an anti-skip surgical instrument. In certain embodiments, the surgical instrument is for use in spinal surgery. In certain embodiments, the surgical instrument is insertable into a surgical instrument holder such that the surgical instrument is constrained by the surgical instrument holder. In certain embodiments, the surgical instrument holder comprises a rigid hollow tubular structure having a proximal open end and a distal open end, said structure defining an axis of the tubular structure along which movement of a surgical instrument sliding through the structure is restricted, wherein the tubular structure has an interior surface shaped and sized to accommodate the surgical instrument sliding through the surgical instrument holder such that movement of the surgical instrument (e.g., fitted with a tool support) is constrained to move along the axis defined by the surgical instrument holder.

In certain embodiments, the surgical instrument is fitted with a tool support shaped and sized to slide through the surgical instrument holder along the axis defined by the surgical instrument holder. In certain embodiments, the surgical instrument is a drill bit and the surgical instrument holder is a drill bit guide holder. In certain embodiments, the surgical instrument is held by a robotic surgical system comprising a robotic arm.

In certain embodiments, the robotic arm has an end effector comprising a surgical instrument holder attached thereto, the surgical instrument holder sized and shaped to hold and/or restrict movement of a surgical instrument therethrough (e.g., via a lock). In certain embodiments, a navigation marker is used by a navigation camera to track the surgical instrument.

The disclosed technology, in certain embodiments, includes a method of performing surgery with a robotic surgical system. The method, in certain embodiments, includes moving a mobile cart transporting a robotic surgical system comprising a robotic arm in proximity to an operating table, wherein the robotic arm has an end effector comprising a surgical instrument guide attached thereto, the surgical instrument guide sized and shaped to hold and/or restrict movement of one of a plurality of surgical instruments therethrough, wherein the plurality of surgical instruments comprises a first surgical instrument (e.g., for removing tissue from the body) and a second surgical instrument (e.g., for preparing a screw placement in a vertebra); stabilizing the mobile cart; maneuvering the first surgical instrument in a manner that is constrained by a surgical instrument guide comprising a rigid hollow tubular structure having a proximal open end and a distal open end, said structure defining the axis along which movement of a surgical instrument (e.g., fitted with a tool support) sliding through the structure is restricted, wherein: the tubular structure of the surgical instrument guide has an interior surface shaped and sized to accommodate the surgical instrument sliding through the guide such that movement of the surgical instrument is constrained to move along the axis defined by the guide, and the tubular structure comprises a longitudinal notch along its length, wherein the longitudinal notch is sized in relation to a peg to (i) permit a marker attached to the surgical instrument at least in part via the peg to be viewable by a navigation camera along an entire range of movement of the surgical instrument through the guide, (ii) constrain movement of the navigation marker in a fixed orientation along the axis defined by the guide, and/or (iii) permit the surgical instrument to slide along the axis defined by the guide while the guide is held in a fixed position by the robotic surgical system; removing the first surgical instrument from the surgical instrument guide, wherein the surgical instrument guide facilitates rapid exchange of surgical instruments (e.g., without requiring reregistration); and maneuvering the second surgical instrument in a manner that is constrained by a surgical instrument guide.

In certain embodiments, the first surgical instrument is a drill (e.g., with a drill bit). In certain embodiments, the first surgical instrument is a milling device, shaver, laser, and ultrasonic scalpel.

In certain embodiments, the second surgical instrument is a screw driver (e.g., for placing a screw in a hole). In certain embodiments, the second surgical instrument is a drill (e.g., for preparing a hole for receiving a screw). In certain embodiments, the tubular structure has an exterior surface comprising at least one flange that is sized and shaped to securely couple the surgical instrument holder to an end effector of the robotic surgical system. In certain embodiments, the robotic surgical system is for use in spinal surgery. In certain embodiments, the rigid hollow tubular structure is a cylindrical structure. In certain embodiments, the longitudinal notch is a slot. In certain embodiments, the navigation marker is used by a navigation camera to track the surgical instrument. In certain embodiments, the second surgical instrument is used to guide a screw implant and a tissue protector. In certain embodiments, the robotic arm comprises a manipulator attached to the robotic arm. In certain embodiments, the robotic arm comprises a manipulator molded into the robotic arm.

In certain embodiments, stabilizing the mobile cart comprises extracting one or more rigid legs on the mobile cart such that the mobile cart rests on the one or more rigid legs of the mobile cart. In certain embodiments, stabilizing the mobile cart comprises retracting one or more wheels on the mobile cart such that the mobile cart rests on one or more rigid legs of the mobile cart.

The disclosed technology, in certain embodiments, includes a method of performing surgery with a robotic surgical system. In certain embodiments, the method includes identifying a volume to be removed, wherein medical images (e.g., obtained intra-operatively or pre-operatively) of the patient situation displayed on a display (e.g., on the robotic surgical system) are automatically updated to show feedback about the planning (e.g., displaying the volume identified for removal by shading); removing the planned volume using robotic assistance, the removing comprising: storing, by a processor of the robotic surgical system, a location of the volume to be removed as “stay-in zone”, and storing, by the processor, a location of a second volume to protect from removal, wherein the location of the second volume defines a “no-go zone”; maintaining, by the processor, the surgical instrument in the “stay-in zone” and/or out of the “no-go zone”, thereby removing the volume; after removing at least a portion of the volume, moving at least a portion of the robotic surgical system away from the patient; and manually completing the surgery.

The disclosed technology, in certain embodiments, includes a method of performing surgery with a robotic surgical system. The method, in certain embodiments, includes identifying a volume to be removed, wherein medical images (e.g., obtained intra-operatively or pre-operatively) of the patient situation displayed on a display (e.g., on the robotic surgical system) are automatically updated to show feedback about the planning (e.g., displaying the volume identified for removal by shading); removing the planned volume using robotic assistance, the removing comprising: preventing, by the robotic surgical system, a surgical instrument from leaving the volume until the volume is completely removed or the surgeon voluntarily wants to leave the volume; after removing at least a portion of the volume, moving at least a portion of the robotic surgical system away from the patient; and manually completing the surgery.

In certain embodiments, the volume is identified by identifying (e.g., using a navigation system) a plurality of points on the patient anatomy. In certain embodiments, the plurality of points are identified by a surgeon point to a plurality of points on the patient anatomy (e.g., using a pointer tracked by a navigation system). In certain embodiments, the plurality of points are identified by following a path identified by a surgeon (e.g., via a pointer tracked by a navigation system) such that the plurality of points are automatically collected. The volume can be identified using a navigation system. Additionally, the volume can be identified using automatic segmentation, semi-automatic segmentation (e.g., using surgeon-defined points or corrections), or fully manual when surgeon/assistant/neurologist defines volume by manually selecting etc. the “pixels”/“voxels” to be removed.

In certain embodiments, the method includes manually completing the surgery includes removing, by the surgeon, a portion of the volume to be removed. In certain embodiments, manually completing the surgery includes removing, by the surgeon, a portion of a second volume adjacent the volume removed with assistance of the robotic surgical system.

In certain embodiments, repulsive/wall-like forces prevent the surgeon from moving a position of the surgical instrument into the second volume.

In certain embodiments, the method includes triggering a dead-man switch (e.g., via voice recognition, a gesture, presence or absence of physical contact with a portion of the robotic surgical system), thereby causing the robotic surgical system to stop.

In certain embodiments, the method includes, upon receiving a trigger signal (e.g., from a volume removal force sensor, a bio-sensing device such as PediGuard® by SpineGuard S.A. of Vincennes, France, and/or a neuro-monitoring device), preventing movement of the surgical instrument further in a forbidden direction.

In certain embodiments, the method includes, after identifying the volume to be removed, bringing the robot to the volume (e.g., automatically or using hands-on control). In certain embodiments, the identification of the volume is performed using a navigation system pointer. In certain embodiments, the identification of the volume is performed using the robotic surgical system in a force control mode.

In certain embodiments, the method includes maneuvering a surgical instrument to make an incision, thereby exposing a vertebra; and attaching a frame of a navigation system to the patient.

In certain embodiments, the method includes moving a mobile cart transporting a robotic surgical system comprising a robotic arm in proximity to an operating table, wherein the robotic arm has an end effector comprising a surgical instrument holder attached thereto, the surgical instrument holder sized and shaped to hold and/or restrict movement of a surgical instrument therethrough; and stabilizing the mobile cart.

In certain embodiments, the method includes removing a least a portion of the spinous process, discs, facets, facet joints, pedicles, and/or vertebral bodies. In certain embodiments, the method includes a first force sensor and a second force sensor, each of the first and second force sensors for measuring one or more forces and/or torques (e.g., 1 to 3 forces and 1 to 3 torques) applied to at least a portion of the surgical instrument.

In certain embodiments, the surgical instrument holder, surgical instrument, and/or robotic surgical system is for use in at least one of spinal surgery (e.g. laminotomy), neurosurgery (various types of craniotomy), ENT surgery (e.g. tumor removal), and orthopedic surgery (bone removal).

In another aspect, the disclosed technology includes a tool attachment device for attaching a tool to a robotic surgical arm, the tool attachment device including: a lever for transmitting a fixation force to a quick lock latch via a plurality of links; and the quick lock latch for releasably securing a tool to the robot when an activation force is applied to the lever (e.g., the lever pulling the tool tight against a robot flange).

In certain embodiments, the lever is activated by pushing the lever toward a robot flange such that quick lock latch pulls the tool tight against the robot flange.

In certain embodiments, the quick lock latch comprises a quick lock hook that engages a pin head on the tool when the tool is inserted fully into the robot flange.

In certain embodiments, a positioning module for precisely positioning the surgical instrument holder relative to the robotic surgical arm (e.g., restricting orientation of the tool when mounting the tool to the robot).

In certain embodiments, the positioning module comprises one or more pins on the robot flange, wherein, upon mechanically coupling the tool to the robotic surgical arm, the each pin of the one or more pins engage an opening in the tool thereby precisely positioning the tool relative to the robotic surgical arm.

In certain embodiments, the positioning module comprises one or more openings on the robot flange, wherein, upon mechanically coupling the tool to the robotic surgical arm, the each opening of the one or more opening engages a pin on the tool thereby precisely positioning the tool relative to the robotic surgical arm.

In another aspect, the disclosed technology includes a tool attachment device for attaching a tool to a robotic surgical arm, the tool attachment device including: a robot flange on the robotic surgical arm, the robot flange comprising (i) an open portion to receive a protrusion of a tool and (ii) a notch that permits a width of the opening to be at least partially decreased; and a lever connected to the robot flange that causes the width of the open portion of the robot flange to decrease when the lever is engaged, thereby securing the tool with the protrusion positioned in the opening to the robot flange.

In another aspect, the disclosed technology includes a tool attachment device for attaching a tool to a robotic surgical arm, the tool attachment device including: a robot flange on the robotic surgical arm, the robot flange comprising a shape lock (e.g., bayonet mount) having at least two or more openings each arranged to receive and secure a protrusion on the tool or at least two or more protrusions each arranged to engage and secure an opening on the tool.

In another aspect, the disclosed technology includes a drill for use with a surgical robotic arm, the drill including: a chuck for securely holding a drill bit; and a body comprising a positioning module for precisely positioning the drill relative to the robotic surgical arm.

In certain embodiments, the body of the drill comprises a protrusion and the protrusion comprises a pin head.

In certain embodiments, the positioning module comprises one or more pins on the body, wherein, upon mechanically coupling the drill to the robotic surgical arm, each pin of the one or more pins engages an opening in the surgical robotic arm thereby precisely positioning the drill relative to the robotic surgical arm.

In certain embodiments, the positioning module comprises one or more openings on the body, wherein, upon mechanically coupling the drill to the robotic surgical arm, the each opening of the one or more opening engages a pin on the robot thereby precisely positioning the drill relative to the robotic surgical arm.

In certain embodiments, the positioning module comprising a hole that passes from a first side of the body to a second side of the body (e.g., such that the body can be mounted on the robot (e.g., by a bolt extending from the robotic surgical arm that slides through the hole and a nut that securely holds the body on the robotic surgical arm)).

In certain embodiments, the positioning module is a friction based module and comprises a protrusion that fits into a portion of the robot flange and is secured therein by a force applied by a lever attached to the robot flange.

BRIEF DESCRIPTION OF THE FIGURES

The foregoing and other objects, aspects, features, and advantages of the present disclosure will become more apparent and better understood by referring to the following description taken in conjunction with the accompanying drawings, in which:

FIG. 1 is a superior view and left posterolateral view of articulated vertebrae;

FIGS. 2A and 2B are illustrations of surgical pliers and a high-speed burr, respectively;

FIG. 3 is an illustration of an example robotic surgical system in an operating room;

FIG. 4 is an illustration of an example robotic surgical system;

FIGS. 5A through 5C are illustrations of an example force sensor implementations;

FIG. 6 is an illustration of an example surgical instrument;

FIGS. 7A through 7D are illustrations of an example implementations of a force sensor integrated in a surgical drill;

FIG. 8 is an illustration of an example surgical instrument holder;

FIG. 9 is an illustration of volumetric planning;

FIG. 10 is an illustration of volume removal;

FIGS. 11A through 11F illustrate an example system for securing an instrument to a robot in accordance with an embodiment of the invention;

FIG. 12 illustrates an example system for securing an instrument to a robot in accordance with an embodiment of the invention;

FIG. 13 illustrates an example system for securing an instrument to a robot in accordance with an embodiment of the invention;

FIG. 14 illustrates an example system for securing an instrument to a robot in accordance with an embodiment of the invention;

FIG. 15 is an illustration of an example method for volume definition;

FIG. 16 shows a block diagram of an exemplary cloud computing environment; and

FIG. 17 is a block diagram of a computing device and a mobile computing device.

The features and advantages of the present disclosure will become more apparent from the detailed description set forth below when taken in conjunction with the drawings, in which like reference characters identify corresponding elements throughout. In the drawings, like reference numbers generally indicate identical, functionally similar, and/or structurally similar elements.

DETAILED DESCRIPTION OF THE INVENTION

The disclosed technology includes the robotic surgical system used to remove a target volume from a patient. Initially, an incision is made and the vertebra is exposed. In some implementations, the frame of a navigation system is attached to the patient in the place selected by the surgeon. Intra-operative medical images of the target anatomy may be obtained. Alternatively, images are acquired pre-operatively. Once the images are obtained, the images must be matched to the actual patient position by a process called registration. For intra-operative images, an automatic algorithm may be used to register the actual patient position with the intra-operative images. Alternatively, point-to-point registration or surface matching may be used. The disclosed technology provides an effective and quick way for the surgeon to define volume to be removed and thereafter remove the volume. In another example, the robotic surgical system may be used to place a screw in a vertebra by assisting in drilling a hole and inserting the screw in the hole as described in U.S. patent application Ser. No. 14/266,769, filed Apr. 30, 2014, and entitled “Apparatus, Systems, and Methods for Precise Guidance of Surgical Tool” which is hereby incorporated by reference in its entirety.

FIG. 3 illustrates an example robotic surgical system in an operating room 300. In some implementations, one or more surgeons, surgical assistants, surgical technologists and/or other technicians, (306a-c) perform an operation on a patient 304 using a robotic-assisted surgical system. In the operating room the surgeon may be assisted by the robotic system to accurately execute an operation.

In some implementations, the surgical robotic system includes a surgical robot 302 on a mobile cart. The surgical robot 302 may be positioned in proximity to an operating table 312 without being attached to the operating table itself, thereby providing maximum operating area and mobility to surgeons around the operating table and reducing clutter on the operating table. In alternative embodiments, the surgical robot (or cart) is securable to the operating table. In certain embodiments, both the operating table and the cart are secured to a common base to prevent any movement of the cart or table in relation to each other, even in the event of an earth tremor.

The mobile cart may permit a user (operator) 306a, such as a technician, nurse, surgeon, or any other medical personnel in the operating room, to move the surgical robot 302 to different locations before, during, and/or after a surgical procedure. The mobile cart enables the surgical robot 302 to be easily transported into and out of the operating room 300. For example, a user 306a may move the surgical robot into the operating room from a storage location. In some implementations, the mobile cart may include wheels, a track system, such as a continuous track propulsion system, or other similar mobility systems for translocation of the cart. The mobile cart may include an attached or embedded handle for locomotion of the mobile cart by an operator.

For safety reasons, the mobile cart may be provided with a stabilization system that may be used during a surgical procedure performed with a surgical robot. The stabilization mechanism increases the global stiffness of the mobile cart relative to the floor in order to ensure the accuracy of the surgical procedure. In some implementations, the wheels include a locking mechanism that prevents the cart from moving. The stabilizing, braking, and/or locking mechanism may be activated when the machine is turned on. In some implementations, the mobile cart includes multiple stabilizing, braking, and/or locking mechanisms. In some implementations, the stabilizing mechanism is electro-mechanical with electronic activation. The stabilizing, braking, and/or locking mechanism(s) may be entirely mechanical. The stabilizing, braking, and/or locking mechanism(s) may be electronically activated and deactivated.

In some implementations, the surgical robot 302 includes a robotic arm mounted on a mobile cart. An actuator may move the robotic arm. The robotic arm may include a force control end-effector configured to hold a surgical tool. The robot may be configured to control and/or allow positioning and/or movement of the end-effector with at least four degrees of freedom (e.g., six degrees of freedom, three translations and three rotations).

In some implementations, the surgical system includes a tracking detector 308 that captures the position of the patient and different components of the surgical robot 302, and a display screen 310 that displays, for example, real time patient data and/or real time surgical robot trajectories.

For example, a tracking detector 308 monitors the location of patient 304 and the surgical robot 302. The tracking detector may be a camera, a video camera, an infrared detector, field generator and sensors for electro-magnetic tracking or any other motion detecting apparatus. In some implementation, based on the patient and robot position, the display screen displays the target volume to be removed and/or the volume already removed. These different volumes may be differentiated on the display using various colors or shading. By continuously monitoring the patient and robotic arm positions using tracking detector 308, the surgical system can visually display the target and/or removed volume on display screen 310 to inform and guide surgeons and/or technicians in the operating room using the surgical robot. For instance, the location of the end-effector can be automatically adjusted in real time to account for movement of the vertebrae or other part of the patient during the surgical procedure.

FIG. 4 illustrates an example configuration 400 of a robotic arm for performing a surgical operation involving the removal of a volume from a patient. A surgical tool holder 408, in some implementations, is connected (e.g., removably; e.g., directly or indirectly) to the robot arm 404. For example, the surgical tool holder 408 is removably connected to the robotic arm 404 via connector 406 and a force sensor 430 as shown in FIG. 4. In some implementations, the robot arm 414 is extended so that a tool holder 408 may be placed on the robot arm connector 406. In some embodiments, a robot arm connector 406 is located at the end of the robot manipulator 404. Although not shown in FIG. 4, a surgical drape may be placed over the robot arm 414 and cart 402 when the tool holder 408 is inserted into, or otherwise attached to, the connector 406. The connector 406 may be configured to at least partially protrude from the surgical drape, such that a sterile cap, collar, or other covering of the connector may be installed prior to attachment of the tool holder 408 into the connector 406 to maintain sterile seal.

The robot arm connector 406 is configured to pass electrical signals from the rest of robot arm 414 to the tool holder 408. The electrical signals may be passed by a separate cable that passes through the sterile drape. The cable may be integrated with the sterile drape to simplify handling for the user. An operator attaches the tool holder 408 to the robot arm connector 406. In the embodiment shown in FIG. 4, the tool holder 408 is a drill holder for securely holding a drill.

The tool holder 408, in some implementations, is a surgical tool holder that may be used to hold various surgical tools and/or allow insertion of one or more surgical tools/implements therethrough. The surgical tool 410, in some implementations, provides an accurate tool guide for surgical bone drilling, e.g., by providing a precisely aligned shaft through which a drill bit or drill may be inserted. Surgical tool 410 is used in spinal surgeries to allow for accurate surgical instrument placement. In some implementations, the surgical tool 410 is a drill that is directly connected to the tool holder 408, thereby securely holding a drill for removing a volume from a patient. The surgical tool may be any instrument which can be used for removing tissues. For example, the surgical tool 410 may be a drill, mill, shaver, laser, or ultrasonic scalpels. In some implementations, the tool holder 408 may be used both to securely hold a tool for removing a volume from a patient and placing a screw in a vertebra.

A force sensor 430, in some implementations, is placed between the robot arm connector 406 and the tool holder 408. The force sensor 430 measures forces applied to a surgical instrument 410 held by the tool holder 408. The force sensor 430 may be placed in a variety of locations. In some implementations, multiple force sensors 430 are used. For example, multiple force sensors 430 may be used to provide redundant measurements for safety reasons. Additionally, multiple force sensors 430 may be used to extract additional force information (e.g., determining where the forces are applied). In some implementations, the robot arm manipulator 404 includes an emergency stop switch 412. The emergency stop switch may be placed on the robot arm, robot cart, or on the operator's panel. A user interface 436, in some implementations, is integrated into the surgical instrument 410 to allow a user quick and easy interaction with the system. Several placements of user interfaces can be used. For direct and immediate interaction a sterile user interface is placed on the surgical instrument or tool holder. In other implementations, the interface can be placed on the robot arm as described in U.S. patent application Ser. No. 14/858,325, filed Sep. 18, 2015, and entitled “Robot-Mounted User Interface For Interacting with Operation Room Equipment” which is hereby incorporated by reference in its entirety. In some implementations, user interfaces are provided in multiple locations, such as on the robot arm, the surgical instrument, tool holder, and/or manipulator (e.g., handle) as described in U.S. patent application Ser. No. 14/619,732, filed Feb. 11, 2015, and entitled “Sterile Handle for Controlling a Robotic Surgical System from a Sterile Field” a which is hereby incorporated by reference in its entirety. The sterile interfaces can be reusable or disposable.

FIGS. 5A-C illustrate example locations for mounting a force sensor (e.g., force/torque sensor 430). In some implementations, as shown in FIG. 5A, the force sensor 502a is located between the tool holder 506a and robot 504a. Using this configuration, the sterile cover 508a may be wrapped around the robot arm and between the force sensor and the tool holder to ensure sterilization. The force sensor 502a may provide for direct measurement of forces (e.g., forces and/or torques) on the tool. The force sensor 502a may be designed to resist flexing. The force sensor 502a may be designed to flex under the stress of certain external forces. The displacement caused when an external force is applied may be calculated based on the force and/or torque applied to the tool, radial force stiffness, axial torque stiffness, and the diameter of the holder to which the tool is attached.

As shown in FIGS. 5B and 5C, respectively, the force sensor (e.g., 502b in FIG. 5B or 502c in FIG. 5C) may be located on the robot or the tool holder, respectively. These configurations may exclusively measure the forces and/or torques applied by the user. The force sensor 508 may be connected to the robot with an intermediary analog box which measures forces and torques and transmits them via a network (e.g., Ethernet, CAN, wireless, internet, private LAN, public LAN, etc.). Combinations of the above mentioned force sensor positions are possible to achieve pre-defined behavior (e.g. the first sensor in the base FIG. 5A and the second one in the handle FIG. 5B may be positioned to allow the feedback control system to decouple forces applied to the surgical tool from forces and/or torque applied by a user).

Additionally, in some implementations the force sensor is integrated directly in the surgical instrument. For example, the force sensor may be integrated directly in the surgical drill bit as illustrated in FIG. 6. While the implementation of the force sensor 604 is described in relation to a drill bit 602 as shown in FIG. 6, the force sensor 604 may be similarly integrated in other surgical instruments. Integrating the force sensor 604 in a surgical instrument, such as a drill bit 602, may be more robust as it minimizes the impact of external disturbances for measuring forces applied to the drill bit.

In the example configuration shown in FIG. 6, the force sensor 604 is integrated in the shaft of the drill bit 602. The force sensor 604, in some implementations, is located on the drill bit 602 outside of the body 610 of the drill as shown in FIG. 6. In other implementations, the force sensor 604 is located inside the body 610 of the drill, thereby better protecting the force sensor 604 from external influences. Force sensor can have multiple degrees of freedom and measure, for example, 1 to 3 forces and/or 1 to 3 torques. Forces are transmitted from the rotating shaft through a connector 606. The connector, in some implementations, is one or more brushes that provide an electrical connection to the force sensor 604. If the force sensor is an optical sensor, the connector may be an optical transmitter (e.g. LED) and/or optical receiver (e.g., photodiode). In this example, the brushes contact the drill bit thereby forming an electrical connection with the force sensor 604. In some implementations, the brushes touch one or more contacts on the drill bit to form the electrical connection.

An electric or pneumatic motor 608 rotates the drill bit 602 shaft. In some implementations, a sensor 612 (e.g., an encoder) measures position of the shaft. The sensor 612 measures the position of the shaft in order to correlate forces measured by the force sensor to the relative position of the shaft. For example, if the force sensor is located in a drill bit, the measurement of the direction of the force will vary as the drill bit rotates. Specifically, the force sensor measures force and the direction of the force periodically (e.g., every millisecond, every microsecond, or somewhere therebetween). The drill bit rotates as the surgeon pushes it into bone. When the drill contacts the bone, the force sensor will indicate some force (F1) in a direction (D1). One period later (e.g., one millisecond), the drill bit will rotate slightly so the force sensor will indicate force of the same value (F1) (assuming a constant force is applied) in a different direction (D2). The direction of the force will continue to change relative to a single perspective as the drill bit rotates even if surgeon pushes into the bone with a constant force. A constantly changing force direction is not acceptable. In order to correlate the directions (e.g., D1, D2) with the global direction of the force (D) coming from the bone (seen by the surgeon, robotic system etc.) the position of the drill in the global space must be calculated as the drill bit rotates. The sensor 612 is used to measure the position of the shaft and thus determine the global direction of the force (D). The sensor 612 may be located on the back of the motor 608 as shown in FIG. 6. The sensor 612 may be located in other locations relative to the motor 608 as well.

The force sensor 604 may be provided in various configurations as shown in FIGS. 7A-D. In each configuration, the goal is to measure forces on the tip of the tool (e.g., drill bit ultrasound bit, etc.). In the example shown in FIG. 7A the force sensor 604 is integrated in the shaft of the drill bit 602 as described in relation to FIG. 6. The force sensor 604 may communicate with a connector 606 (shown in FIG. 6) via a sensor cable 702. The sensor cable 702, in some implementations, is routed inside the drill bit 602. In some implementations, the connector 606 (shown in FIG. 6) is electrically connected to the sensor cable 702 via one or more connection pads.

The force sensor 604 in this example may be a miniaturized industrial sensor (e.g., the multi-axis force/torque sensor from ATI Industrial Automation, Inc. of Apex, NC) that measures, for example, all six components of force and torque using a transducer. Alternatively, the force sensor 604 may be an optical sensor. Alternatively, the force sensor 604 may comprise a strain gauge 706 integrated directly into the shaft of the drill bit 602 as shown in FIG. 7B.

As shown in FIG. 7C, the force sensor 604, in some implementations, measures forces on the motor instead of measuring forces on the drill bit 602 itself. As shown in FIG. 7D, the shaft of the drill bit 602, in some implementations, includes a flexible element 708 that allows the drill bit 602 to bend (e.g., only slightly) such that after deflection of the shaft of the drill bit 602, forces can be measured by the force sensor 604. In some implementations, for the configuration shown in FIGS. 7C and 7D, the measurement of shaft positions (e.g., by sensor 612 as shown in FIG. 6) may be omitted as the forces are measured directly in the instrument coordinate frame.

The tool holder 408 as shown in FIG. 4 may have different implementations. In certain embodiments the tool holder 408 is a flange that a surgical instrument tool may be secured to via, for example, bolts. In some implementations, the tool holder includes a rapid connection mechanism allowing for quick interchange of surgical instruments. In particular it may allow for attaching different surgical instruments/guides necessary for pedicle screw placement as described in U.S. patent application Ser. No. 14/266,769, filed Apr. 30, 2014, and entitled “Apparatus, Systems, and Methods for Precise Guidance of Surgical Tool” and U.S. patent application Ser. No. 14/597,883, filed Jan. 15, 2015, and entitled “Notched Apparatus for Guidance of an Insertable Instrument Along an Axis During Spinal Surgery,” both of which are hereby incorporated by reference in their entirety.

An example implementation of the tool holder with a rapid connection mechanism is shown in FIG. 8. The tool holder 802 has similar form to the one described in U.S. patent application Ser. No. 14/597,883, filed Mar. 14, 2014, and entitled “Notched Apparatus for Guidance of an Insertable Instrument Along an Axis During Spinal Surgery” a copy of which is included as Appendix B, and is hereby incorporated by reference.

The tubular structure of the tool holder 802, in some implementations, has one or more flanges that are configured for secure coupling of the holder 802 to an end effector of the robotic surgical system. The tubular structure, in some implementations, defines an axis along which movement of a surgical instrument (fitted with a tool support) sliding through the structure is restricted. The tubular structure is configured (e.g., an interior surface of the structure is shaped and sized) to permit a tool support to slide through the holder 802 such that movement of the tool support is constrained to move along the axis (e.g., constrained in all directions except along the axis) defined by the holder 802. Once the surgical instrument with the tool support is fully inserted into the tool holder 802, the surgical instrument 816 can be locked in place using a lock 804. This allows the surgical instrument 816 to be rigidly fixed to the robotic surgical system.

In some implementations, the tool holder 802 has a longitudinal notch 806 which is interfaced with peg 808. In some implementations, the peg 808 is a single pin as shown in FIG. 8. In other implementations, the peg 808 supports a navigation tracker such the navigation marker is viewable by a navigation camera along an entire range of movement of the tool support through the holder 802. The navigation marker may be used by navigation camera to track the surgical instrument. The navigation marker may be, for example, navigation tracker such as the Dedicated NavLock™ tracker from Medtronic, Inc. of Minneapolis, MN The intra-operative imaging system (not shown) may be, for example, the O-Arm from Medtronic, Inc. of Minneapolis, MN, the Airo from Mobius Imaging of Ayer, MA, or the BodyTom® from Neurologica Corp. of Danvers, MA.

The longitudinal notch 806, in some implementations, is sized in relation to a peg 808. In some implementations, the surgical drill 816 has a sliding surface on the tool support 814 which interface with internal diameter of the holder 802. Sliding surface slides along the interior surface of the holder 802 and permits the tool 816 to slide into the holder such that movement of the tool 816 is constrained in all directions except along the axis defined by the holder 802. The sliding surface 814 is designed in order to slide into the holder 802 allowing the surgeon to achieve a linear motion of the instrument along the holder's 802 axis such that the tool 816 is fixedly attached to the robotic surgical system when fully inserted into the tool holder 802 and the lock 804 is engaged. In some implementations, the sliding surface 814 comprises more than one interface band.

Connectors 810 and 812 can be used to transmit power to the drill 816 and for transmitting information, such as forces measured by a force sensor as described above, between the instrument and the robotic surgical system. The connectors 810 and 812, in some implementations, are positioned such that when the drill 816 is slide completely into the holder 802 the connectors 810 and 812 are electrically engaged.

Several system can be used for attachment of a drill to the robot. In each instance, for example, the following requirements should be met: (i) fixation has high rigidity, (ii) fixation of the instrument shall be done without the need of additional tools, (iii) the external part of the robot-side fixation (flange) shall be easy to clean, and (iv) the instrument-side of the fixation shall be sterilizable in an autoclave.

In certain embodiments, the instrument (e.g., drill) can be secured to the robot using a bolt and pin based fixation system, such as the system described in U.S. patent application Ser. No. 14/695,154, filed Apr. 24, 2015, entitled “Surgical Instrument Holder for Use with a Robotic Surgical System,” the contents of which are hereby incorporated by reference in its entirety. In order to use this type of system for rigidly fixing a surgical drill to the robot flange, the part referred to as the instrument holder base 1012 in U.S. patent application Ser. No. 14/695,154 is integrated into the surgical drill. Thus, the drill can slide onto a protruding bolt and be secured by a nut, similar to the instrument holder base described in U.S. patent application Ser. No. 14/695,154.

FIGS. 11A through 11F illustrate an example lever system for securing an instrument to a robot.

Robot flange 1152 is fixed to the robot. It contains the lever system 1102 used to generate fixation force through the lever 1102a. Instrument 1150 is shown as a surgical instrument holder, such as that described in U.S. patent application Ser. No. 14/695,154. However, this can be adapted to other instruments, such as a surgical drill.

FIG. 11C shows the lever system in more detail. It includes a lever system 1102 (including components 1102a, 1102b, 1102c, and 1102d), quick lock mechanism 1104 (including components 1104a and 1104b) and positioning pins 1106 (including 1106a, 1106b, and 1106c). The lever system 1102 is used to generate high forces. User pushes the lever 1102a in the direction of the arrow. Thanks to the level ratio the forces are increased and transmitted through the intermediate elements (1102b, 1102c, and 1102d) to the rod (1102e). On the rod, the quick lock mechanism 1104 is mounted. It has a quick lock hook 1104a which automatically captures a pin head 1104b which is part of the attached instrument 1150 (e.g., including pins 1106 and pin head 1104b). The guide plate 1108 allows the pin head 1104b to pass therethrough and also allows the positioning pins 1106 (such as those described in U.S. patent application Ser. No. 14/695,154) to interface with the guide plate 1108 to ensure proper alignment of the instrument 1150.

As the lever system 1102 is activated by being pushed in the direction of the arrow, the quick lock mechanism pulls the surgical instrument tight against the flange 1152. The locked position of the mechanism is shown in FIGS. 11B and 11D. When the lever is in the closed position, joint 1120e is on the left side of the line L going through the joints 1120a and 1120c. Thanks to this configuration, a bi-stable behavior is achieved and the lever remains closed until being slightly raised. In this configuration very high forces are achieved because of alignment of intermediate elements 1102c and 1102d (illustrated by the bold line L). This configuration is called a toggle mechanism. In this example, the lever system 1102 is activated by being pushed in the direction of the arrow, element 1102b rotates around pin 1120b, lever 1102a rotates about pin 1120a, element 1102c rotates about pin 1120c, and pin 1120d connects elements 1102b, 1102c, and 1102c. In this example, when the lever 1102a is released, the quick lock hook 1104a can be released (e.g., by pressing tab 1124 directly or indirectly be a separate lever or button) such that the instrument 1150 can be removed. In some embodiments, the linkage is arranged such that the bold line L is straight.

FIG. 12 illustrates an example system for securing an instrument to a robot. FIG. 12 illustrates a friction based system. The instrument 1250 is blocked in robot flange 1252 due to the friction between the two parts. The force is applied by the lever 1202 and thanks to the material compliance achieved by adding a notch (grey) the instrument part is tightly blocked inside. It is similar mechanism to the one known in bicycles seats.

FIG. 13 illustrates an example system for securing an instrument to a robot. In this example, a shape lock system is used to secure the instrument to the robot. Shape locks can be achieved between two elements having specific forms. An example is a Bayonet mount shown in this example. The robot flange 1352 includes a pair of shape lock cutouts 1354a and 1354b (two cutouts are shown here, however, more cutouts may be used, such as 4, 5, or 6 cutouts) and the instrument 1350 includes a pair of protrusions 1356 (only one is shown in this illustration). The protrusions 1356 engage the cutouts 1354a and 1354b when the instrument 1350 slides into the flange 1352. The instrument 1350 can be twisted such that is it locked to the flange 1352 by the protrusions 1356 and the cutouts 1354a and 1354b

FIG. 14 illustrates an example system for securing an instrument to a robot. Linear guides, such as guide 1454, can be used to lock the instrument in the robot flange. A spring or other device can be used to remove any mechanical backlash such that the instrument cannot be removed after being inserted into the guide until the spring or other mechanism is released. A corresponding member 1456 is attached to the instrument and engages the guide 1454 which is attached to the robot flange.

The disclosed technology, in some implementations, includes a method of performing surgery. Initially, an incision is made and the vertebra is exposed. In some implementations, the frame of the navigation system is attached to the patient in the place selected by the surgeon. Intra-operative medical images of the target anatomy may be obtained. Alternatively, images are acquired pre-operatively. Once the images are obtained, the images must be matched to the actual patient position by a process called registration. For intra-operative images, an automatic algorithm may be used to register the actual patient position with the intra-operative images. Alternatively, point-to-point registration or surface matching may be used. During the surgery, if the surgeon decides to perform fixation of the spine using pedicle screws, he can place them with assistance of the robotic system as described in U.S. patent application Ser. No. 14/266,769, filed Apr. 30, 2014, and entitled “Apparatus, Systems, and Methods for Precise Guidance of Surgical Tool” which is hereby incorporated by reference in its entirety.

Some surgeries require the surgeon to remove vertebrae volume. The disclosed technology provides an effective and quick way for the surgeon to define volume to be removed. FIG. 9 is an illustration of volumetric planning. In some implementations, volumetric planning is linked with registration precision improvement algorithms. For example, the surgeon may use the navigation system pointer to measure points or follow a path traced by the surgeon using the pointer, thereby generating a set of points that identify (e.g., outline) the volume to be removed. Alternatively, in some implementations, the surgeon uses robot end effector in force control mode (e.g., as described in U.S. patent application Ser. No. 14/266,769, filed Apr. 30, 2014, and entitled “Apparatus, Systems, and Methods for Precise Guidance of Surgical Tool,” which is hereby incorporated by reference in its entirety, to identify the points (e.g., the points identifying the volume to be removed). The position of the robot end effector may be tracked by the navigation system using a navigation marker fixed in relation to the surgical instrument Additionally, the volume can be identified using automatic segmentation, semi-automatic segmentation (e.g., using surgeon-defined points or corrections), or fully manual when surgeon/assistant/neurologist defines volume by manually selecting etc. the “pixels”/“voxels” to be removed.

In some implementations, a surgeon first removes the spinous process 902 using, for example, pliers and cutting along the red zigzag line as shown in FIG. 9. Next, the surgeon identifies the extremities of the volume to be removed (shown in blue and labeled as 904. The surgeon can do this by using a navigation pointer and pointing to separate points (e.g. points (1), (2), (3), (4), (5), (6), and (7)) on the patient anatomy or, alternatively, the navigation system may follow a path identified by the surgeon and automatically collect of points. In some implementations, the system generates the volume to be removed by taking a set of points (e.g., identified by the surgeon) and/or patient volumetric information (e.g., medical images) and identifying parts of the vertebrae to be removed. The system, in some implementations, combines the points identified by the surgeon and the patient volumetric information to identify the volume to be removed. In some implementations, the system utilizes medical image segmentation (e.g., statistical shape modeling) to identify the volume. The navigation system may track the points identified by the surgeon and medical images on the navigation screen, in some implementations, are automatically updated to show feedback about the planning. This can be achieved, for example, using different colors on several views of medical images (e.g., the blue section shown in FIG. 9).

The planning algorithms recognize which part of the spinous process was removed. In some implementations, this is achieved if the surgeon shows a point on the patient anatomy which normally should be inside bone. If surgeon is able to point there, it demonstrates that this part of the bone was removed and it should be tagged and shown on a display (e.g., using different color such as red as shown in FIG. 9).

In this type of surgery the precision is crucial. Usually after screw placement the relative positions of the vertebrae may change due to forces applied by the surgeon. In order to improve precision points (e.g. points (1), (2), (3), (4), (5), (6), and (7)) on the patient anatomy or a path may be followed to collect a plurality of points that identify the volume to be removed. By matching these points and medical images (e.g. using surface matching algorithms and initial registration as a starting point) an updated, more precise position of the vertebra can be found and used in the next steps of the surgery. In particular, in some implementations, the combination of the points identified by the surgeon and the patient volumetric information (e.g., medical images) allows for a more precise mapping of the volume to be removed. In certain embodiments, these points are collected using the robot in manual or lose control as described in relation to FIG. 9. The force sensor can be used to automatically detect bone versus tissue, something that is difficult to do manually. In certain embodiments, automatic and semi-automatic segmentation algorithms for use van be used to plan a volume to be removed as well as other surgeries. FIG. 15 is an illustration of an example method for volume definition.

Object characterization as shown in FIG. 15 defines the object to be removed. The characterization can be identified spatially, such as set of points delimiting the volume to be removed, or by identifying the type of surgery or volume to remove, e.g. spinous process, lamina, etc. As such, the object characterization can be the set of points identified by the surgeon (e.g., using a pointer) to delineate the area to remove (or operate on in another example), the operation being performed, the specific area to be removed (e.g., identification of a specific portion of the body (e.g., lamina of c7 vertebrae), or a set of the above.

Patient images as shown in FIG. 15 represents the set of images for which the actual volume definition should apply. These images can be CT, MRI, X-ray and/or fluoroscopy images.

The segmentation algorithm takes the object characterization and patient images as an input and gives on the output an exact volume definition which in objective, measurable terms specifying the volume to be removed (e.g. series of spatial coordinates, volume mesh, voxel data etc.). In some implementations, the object characterization is not an input into the algorithm. For example, the patient images and the specific area to be removed (e.g., identification of a specific portion of the body (e.g., lamina of c7 vertebrae) can be the input to the algorithm and the algorithm can determine the area to be removed. In some implementations, the area identified for removal by the algorithm is displayed back to the user (e.g., via a display on the robot) for confirmation by the surgeon.

Good results of the automatic or semi-automatic segmentation in medical domain were obtained using atlas based segmentation, e.g. Expectation-Maximization algorithm or Statistical Shape Modelling algorithm. In Statistical Shape Modelling (SSM) a statistical atlas of the segmented anatomy is created. For example, in the case of the vertebrae lamina, it would require analyzing e.g. 30 representative laminas on medical images and defining a probabilistic representative of the shape called atlas. Once the atlas is generated, its relative position to the patient images and object characterization can be found (this step is called registration). Most often it is defined as a registration matrix. Next, a segmentation algorithm kernel, e.g. Gaussian kernel or multi-scale Gaussian kernel, transforms the atlas to fulfill the constraints of the object definition and patient images. As a result a volume definition is obtained and can be reused in further algorithm steps.

Next, the robot can be automatically brought to the volume or a surgeon may move the robot to the arm using the hands-on control mode as described in U.S. patent application Ser. No. 14/266,769, filed Apr. 30, 2014, and entitled “Apparatus, Systems, and Methods for Precise Guidance of Surgical Tool” which is hereby incorporated by reference in its entirety. The planned volume may be removed using robotic assistance in several ways. In some implementations, the volume may be removed based on stay-in/no-go zones as described in U.S. patent application Ser. No. 14/009,050, filed Jan. 10, 2014, and entitled “Robotic System and Method for Spinal and Other Surgeries” which is hereby incorporated by reference in its entirety. The removing tool stays blocked inside virtual volume (constrained movement) as long as it is not completely removed or the surgeon voluntarily wants to quit the volume.

An alternative approach for removing the appropriate volume is shown in FIG. 10. In this implementation, the robot automatically moves the drilling/milling/erasing instrument 1002 in a type of “painting” trajectory while being supervised by the surgeon. It is important to indicate to user the volume that has be removed 1004 and remaining volume 1006. This may be shown on a display of the robot or on a display separate from the robot.

A surgeon can supervise the robot by looking at the actual position on the navigation screen. The surgeon can stop the robot movement at any time, for example, if the surgeon is concerned about the position of the removal instrument. In some implementations, a dead-man switch is used. The switch may be activated if the surgeon places his/her hands on the tool holder interface or if the surgeon removes his/her hands from the tool holder interface. Alternatively, or in addition, presence detection switches may be used and/or voice recognition may be used to recognize gestures and voice commands, respectively.

During volume removal a force sensor signal can be used to detect if the surgical instrument goes across the edge of the vertebra and, in response, the robotic surgical system can prevent movement of the surgical instrument and destruction of surrounding tissue. This is achieved, for example, using edge detection algorithms which use relative changes in force. Additionally, in some implementations, neuro-monitoring may be used for detection of nervous system infraction.

In some implementations, it is not necessary for a robotic system to completely remove the volume. In the clinical scenario the robot may be used to partially remove the target volume, thereby leaving the most difficult and less laborious parts of the tissue to be removed manually by the surgeon. The amount of the target volume removed by the robot, in some implementations, is controlled during the volume planning phase.

After the robot finishes removing the target volume (e.g., the entire volume or a portion of the volume), the robot is moved away from the patient. In some implementations, this is done automatically by the robot itself. In other implementations, the robotic end effect is manually moved using the hands-on control mode. In some implementations, after moving the robot away the surgeon may inspect the removal of the volume by the robot and/or complete the surgery manually.

As shown in FIG. 16, an implementation of a network environment 1600 for use with the robotic surgical system is shown and described. In brief overview, referring now to FIG. 16, a block diagram of an exemplary cloud computing environment 1600 is shown and described. The cloud computing environment 1600 may include one or more resource providers 1602a, 1602b, 1602c (collectively, 1602). Each resource provider 1602 may include computing resources. In some implementations, computing resources may include any hardware and/or software used to process data. For example, computing resources may include hardware and/or software capable of executing algorithms, computer programs, and/or computer applications. In some implementations, exemplary computing resources may include application servers and/or databases with storage and retrieval capabilities. Each resource provider 1602 may be connected to any other resource provider 1602 in the cloud computing environment 1600. In some implementations, the resource providers 1602 may be connected over a computer network 1608. Each resource provider 1602 may be connected to one or more computing device 1604a, 1604b, 1604c (collectively, 1604), over the computer network 1608.

The cloud computing environment 1600 may include a resource manager 1606. The resource manager 1606 may be connected to the resource providers 1602 and the computing devices 1604 over the computer network 1608. In some implementations, the resource manager 1606 may facilitate the provision of computing resources by one or more resource providers 1602 to one or more computing devices 1604. The resource manager 1606 may receive a request for a computing resource from a particular computing device 1604. The resource manager 1606 may identify one or more resource providers 1602 capable of providing the computing resource requested by the computing device 1604. The resource manager 1606 may select a resource provider 1602 to provide the computing resource. The resource manager 1606 may facilitate a connection between the resource provider 1602 and a particular computing device 1604. In some implementations, the resource manager 1606 may establish a connection between a particular resource provider 1602 and a particular computing device 1604. In some implementations, the resource manager 1606 may redirect a particular computing device 1604 to a particular resource provider 1602 with the requested computing resource.

FIG. 17 shows an example of a computing device 1700 and a mobile computing device 1750 that can be used to implement the techniques described in this disclosure. The computing device 1700 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The mobile computing device 1750 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smart-phones, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to be limiting.

The computing device 1700 includes a processor 1702, a memory 1704, a storage device 1706, a high-speed interface 1708 connecting to the memory 1704 and multiple high-speed expansion ports 1710, and a low-speed interface 1712 connecting to a low-speed expansion port 1714 and the storage device 1706. Each of the processor 1702, the memory 1704, the storage device 1706, the high-speed interface 1708, the high-speed expansion ports 1710, and the low-speed interface 1712, are interconnected using various busses, and may be mounted on a common motherboard or in other manners as appropriate. The processor 1702 can process instructions for execution within the computing device 1700, including instructions stored in the memory 1704 or on the storage device 1706 to display graphical information for a GUI on an external input/output device, such as a display 1716 coupled to the high-speed interface 1708. In other implementations, multiple processors and/or multiple buses may be used, as appropriate, along with multiple memories and types of memory. Also, multiple computing devices may be connected, with each device providing portions of the necessary operations (e.g., as a server bank, a group of blade servers, or a multi-processor system).

The memory 1704 stores information within the computing device 1700. In some implementations, the memory 1704 is a volatile memory unit or units. In some implementations, the memory 1704 is a non-volatile memory unit or units. The memory 1704 may also be another form of computer-readable medium, such as a magnetic or optical disk.

The storage device 1706 is capable of providing mass storage for the computing device 1700. In some implementations, the storage device 1706 may be or contain a computer-readable medium, such as a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid state memory device, or an array of devices, including devices in a storage area network or other configurations. Instructions can be stored in an information carrier. The instructions, when executed by one or more processing devices (for example, processor 1702), perform one or more methods, such as those described above. The instructions can also be stored by one or more storage devices such as computer- or machine-readable mediums (for example, the memory 1704, the storage device 1706, or memory on the processor 1702).

The high-speed interface 1708 manages bandwidth-intensive operations for the computing device 1700, while the low-speed interface 1712 manages lower bandwidth-intensive operations. Such allocation of functions is an example only. In some implementations, the high-speed interface 1708 is coupled to the memory 1704, the display 1716 (e.g., through a graphics processor or accelerator), and to the high-speed expansion ports 1710, which may accept various expansion cards (not shown). In the implementation, the low-speed interface 1712 is coupled to the storage device 1706 and the low-speed expansion port 1714. The low-speed expansion port 1714, which may include various communication ports (e.g., USB, Bluetooth®, Ethernet, wireless Ethernet) may be coupled to one or more input/output devices, such as a keyboard, a pointing device, a scanner, or a networking device such as a switch or router, e.g., through a network adapter.

The computing device 1700 may be implemented in a number of different forms, as shown in the figure. For example, it may be implemented as a standard server 1720, or multiple times in a group of such servers. In addition, it may be implemented in a personal computer such as a laptop computer 1722. It may also be implemented as part of a rack server system 1724. Alternatively, components from the computing device 1700 may be combined with other components in a mobile device (not shown), such as a mobile computing device 1750. Each of such devices may contain one or more of the computing device 1700 and the mobile computing device 1750, and an entire system may be made up of multiple computing devices communicating with each other.

The mobile computing device 1750 includes a processor 1752, a memory 1764, an input/output device such as a display 1754, a communication interface 1766, and a transceiver 1768, among other components. The mobile computing device 1750 may also be provided with a storage device, such as a micro-drive or other device, to provide additional storage. Each of the processor 1752, the memory 1764, the display 1754, the communication interface 1766, and the transceiver 1768, are interconnected using various buses, and several of the components may be mounted on a common motherboard or in other manners as appropriate.

The processor 1752 can execute instructions within the mobile computing device 1750, including instructions stored in the memory 1764. The processor 1752 may be implemented as a chipset of chips that include separate and multiple analog and digital processors. The processor 1752 may provide, for example, for coordination of the other components of the mobile computing device 1750, such as control of user interfaces, applications run by the mobile computing device 1750, and wireless communication by the mobile computing device 1750.

The processor 1752 may communicate with a user through a control interface 1758 and a display interface 1756 coupled to the display 1754. The display 1754 may be, for example, a TFT (Thin-Film-Transistor Liquid Crystal Display) display or an OLED (Organic Light Emitting Diode) display, or other appropriate display technology. The display interface 1756 may comprise appropriate circuitry for driving the display 1754 to present graphical and other information to a user. The control interface 1758 may receive commands from a user and convert them for submission to the processor 1752. In addition, an external interface 1762 may provide communication with the processor 1752, so as to enable near area communication of the mobile computing device 1750 with other devices. The external interface 1762 may provide, for example, for wired communication in some implementations, or for wireless communication in other implementations, and multiple interfaces may also be used.

The memory 1764 stores information within the mobile computing device 1750. The memory 1764 can be implemented as one or more of a computer-readable medium or media, a volatile memory unit or units, or a non-volatile memory unit or units. An expansion memory 1774 may also be provided and connected to the mobile computing device 1750 through an expansion interface 1772, which may include, for example, a SIMM (Single In Line Memory Module) card interface. The expansion memory 1774 may provide extra storage space for the mobile computing device 1750, or may also store applications or other information for the mobile computing device 1750. Specifically, the expansion memory 1774 may include instructions to carry out or supplement the processes described above, and may include secure information also. Thus, for example, the expansion memory 1774 may be provide as a security module for the mobile computing device 1750, and may be programmed with instructions that permit secure use of the mobile computing device 1750. In addition, secure applications may be provided via the SIMM cards, along with additional information, such as placing identifying information on the SIMM card in a non-hackable manner.

The memory may include, for example, flash memory and/or NVRAM memory (non-volatile random access memory), as discussed below. In some implementations, instructions are stored in an information carrier. that the instructions, when executed by one or more processing devices (for example, processor 1752), perform one or more methods, such as those described above. The instructions can also be stored by one or more storage devices, such as one or more computer- or machine-readable mediums (for example, the memory 1764, the expansion memory 1774, or memory on the processor 1752). In some implementations, the instructions can be received in a propagated signal, for example, over the transceiver 1768 or the external interface 1762.

The mobile computing device 1750 may communicate wirelessly through the communication interface 1766, which may include digital signal processing circuitry where necessary. The communication interface 1766 may provide for communications under various modes or protocols, such as GSM voice calls (Global System for Mobile communications), SMS (Short Message Service), EMS (Enhanced Messaging Service), or MMS messaging (Multimedia Messaging Service), CDMA (code division multiple access), TDMA (time division multiple access), PDC (Personal Digital Cellular), WCDMA (Wideband Code Division Multiple Access), CDMA2000, or GPRS (General Packet Radio Service), among others. Such communication may occur, for example, through the transceiver 1768 using a radio-frequency. In addition, short-range communication may occur, such as using a Bluetooth®, Wi-Fi™, or other such transceiver (not shown). In addition, a GPS (Global Positioning System) receiver module 1770 may provide additional navigation- and location-related wireless data to the mobile computing device 1750, which may be used as appropriate by applications running on the mobile computing device 1750.

The mobile computing device 1750 may also communicate audibly using an audio codec 1760, which may receive spoken information from a user and convert it to usable digital information. The audio codec 1760 may likewise generate audible sound for a user, such as through a speaker, e.g., in a handset of the mobile computing device 1750. Such sound may include sound from voice telephone calls, may include recorded sound (e.g., voice messages, music files, etc.) and may also include sound generated by applications operating on the mobile computing device 1750.

The mobile computing device 1750 may be implemented in a number of different forms, as shown in the figure. For example, it may be implemented as a cellular telephone 1780. It may also be implemented as part of a smart-phone 1782, personal digital assistant, or other similar mobile device.

Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor, and can be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms machine-readable medium and computer-readable medium refer to any computer program product, apparatus and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term machine-readable signal refers to any signal used to provide machine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

In view of the structure, functions and apparatus of the systems and methods described here, in some implementations, a system and method for performing surgery with a robotic surgical system are provided. Having described certain implementations of methods and apparatus for supporting a robotic surgical system, it will now become apparent to one of skill in the art that other implementations incorporating the concepts of the disclosure may be used. Therefore, the disclosure should not be limited to certain implementations, but rather should be limited only by the spirit and scope of the following claims.

Throughout the description, where apparatus and systems are described as having, including, or comprising specific components, or where processes and methods are described as having, including, or comprising specific steps, it is contemplated that, additionally, there are apparatus, and systems of the disclosed technology that consist essentially of, or consist of, the recited components, and that there are processes and methods according to the disclosed technology that consist essentially of, or consist of, the recited processing steps.

It should be understood that the order of steps or order for performing certain action is immaterial so long as the disclosed technology remains operable. Moreover, two or more steps or actions may be conducted simultaneously.

Claims

1. A method of performing surgery with a robotic surgical system, the method comprising: identifying a volume to be removed, wherein medical images of the patient situation displayed on a display are automatically updated to show feedback about the planning;removing the planned volume using robotic assistance, the removing comprising: storing, by a processor of the robotic surgical system, a location of the volume to be removed as stay-in zone, andstoring, by the processor, a location of a second volume to protect from removal, wherein the location of the second volume defines a no-go zone;maintaining, by the processor, the surgical instrument in the stay-in zone and/or out of the no-go zone, thereby removing the volume;robotically maneuvering a surgical instrument to make an incision, thereby exposing a vertebra;attaching a frame of a navigation system to the patient;after removing at least a portion of the volume, moving at least a portion of the robotic surgical system away from the patient; and manually completing the surgery.

2. A method of performing surgery with a robotic surgical system, the method comprising: identifying a volume to be removed, wherein medical images of the patient situation displayed on a display are automatically updated to show feedback about the planning;removing the planned volume using robotic assistance, the removing comprising: preventing, by the robotic surgical system, a surgical instrument from leaving the volume until the volume is completely removed or the surgeon voluntarily wants to leave the volume;robotically maneuvering a surgical instrument to make an incision, thereby exposing a vertebra;attaching a frame of a navigation system to the patient;after removing at least a portion of the volume, moving at least a portion of the robotic surgical system away from the patient; andmanually completing the surgery.

3. The method of claim 2, wherein the volume is identified by identifying, using a navigation system, a plurality of points on the patient anatomy.

4. The method of claim 3, wherein the plurality of points are identified by a surgeon point to a plurality of points on the patient anatomy using a pointer tracked by a navigation system.

5. The method of claim 4, wherein the plurality of points are identified by following a path identified by a surgeon such that the plurality of points are automatically collected.

6. The method of claim 2, wherein manually completing the surgery comprises: removing, by the surgeon, a portion of the volume to be removed.

7. The method of claim 2, wherein manually completing the surgery comprises: removing, by the surgeon, a portion of a second volume adjacent the volume removed with assistance of the robotic surgical system.

8. The method of claim 2, wherein repulsive/wall-like forces prevent the surgeon from moving a position of the surgical instrument into a second volume.

9. The method of claim 2, comprising triggering a dead-man switch via voice recognition, a gesture, presence or absence of physical contact with a portion of the robotic surgical system, thereby causing the robotic surgical system to stop.

10. The method of claim 2, comprising: upon receiving a trigger signal from a volume removal force sensor or other devices, such as neuro-monitoring devices preventing movement of the surgical instrument further in a forbidden direction.