Robotic surgical systems have been used in minimally invasive medical procedures. During a medical procedure, the robotic surgical system is controlled by a surgeon interfacing with a user interface. The user interface allows the surgeon to manipulate an end effector of a surgical instrument that acts on a patient. The user interface includes an input controller or handle that is moveable by the surgeon to control the robotic surgical system.
Different robotic surgical systems exist in the market each with different controls and displays. As a surgeon moves from one robotic surgical system to another, the surgeon must familiarize themselves with the controls and the displays of the particular robotic surgical system. In addition, during a surgical procedure, each system may have different alerts or alarms to indicate a condition of the robotic surgical system or the patient which may delay recognition of the alarm to a surgeon not familiar with the particular robotic surgical system.
This disclosure relates generally to standardized controls and displays for robotic surgical systems.
In an aspect of the present disclosure, a graphical user interface on a display of a user interface of a robotic surgical system includes a work area and a ribbon. The work area is configured to display a representation of a first end effector at a surgical site. The ribbon has a first icon that is configured to display end effector data of the first end effector.
In aspects, the end effector data includes an identifier of the first end effector, a type of the first end effector, a selected function of the first end effector, and a state of the first end effector. The state of the first end effector may be indicative of a supply level of an exhaustible resource of the first end effector.
In some aspects, the ribbon includes an endoscope icon that is configured to display endoscope data of an endoscope providing a view of the surgical site. The endoscope data may include at least one of a rotation indicator or an inclination indicator. The first icon may be displayed on a left side of the endoscope icon when the first end effector is controlled by a left handle of the user interface and on a right side of the endoscope icon when the first end effector is controlled by a right handle of the user interface. The ribbon may include a second icon that is configured to display end effector data of a second end effector having a representation displayed in the work area. The second icon may be displayed on a left side of the endoscope icon when the second end effector is controlled by a left handle of the user interface and on a right side of the endoscope icon when the second end effector is controlled by a right handle of the user interface.
In certain aspects, the ribbon includes an endoscope control icon to indicate when an endoscope providing a view of the surgical site is being controlled by a user. The endoscope control icon may be displayed on a side of the ribbon indicative of a control handle of a surgical robot in control of the endoscope.
In some aspects, the ribbon includes a clutch icon to indicate when a control handle of the user interface is clutched. The clutch icon may be displayed on a side of the ribbon indicative of the control handle of the user interface that is clutched.
In particular aspects, the graphical user interface includes a firing indicator that is displayed along two edges forming a corner opposite the ribbon. The firing indicator may be displayed when the first end effector is being fired. The graphical user interface may include an off-screen indicator that is displayed along an edge of the display when the first end effector is outside of the working area shown on the display. The off-screen indicator may be positioned along an edge of the display indicative of a direction of the first end effector outside of the working area.
In another aspect of the present disclosure, a graphical user interface of a display of a user interface of a robotic surgical system includes a representation of a respective handle of the user interface, and a plurality of selection boxes. The representation has a plurality of control interfaces. Each selection box of the plurality of selection boxes having an indicator associating the selection box with one of the control interfaces. Each of the selection boxes of the plurality of selection boxes displaying text of a function associated with the respective control interface. The at least one of the selection boxes is selectable to modify a function associated with the respective control interface.
In another aspect of the present disclosure, a method of interfacing with a user interface to simulate or control a surgical robot includes visualizing a first icon in a ribbon of a graphical user interface on a display of the user interface and moving a first control handle of the user interface to move a first end effector within a work area of the display. The first icon provides end effector data of the first end effector associated with the first control handle. The first icon has a stat indicator that provides indicia that the first end effector is active when the first control handle is associated with the first control handle. The first icon has a mode indicator that provides indicia of a current mode of the first end effector.
In aspects, the method includes firing the first end effector. The display may indicate a firing indicator along two edges forming a corner opposite the ribbon while the first end effector is being fired to provide visual indicia of the firing of the first end effector.
In some aspects, moving the first control handle includes moving the first control handle such that the first end effector is moved outside of the work area of the display. The display may include an off-screen indicator along an edge of the display indicative of a direction of the first end effector.
In certain aspects, the method includes associating an endoscope with the first control handle to move an endoscope providing a view on the display. An endoscope control icon may appear in the ribbon on a side indicative of the first control handle while movement of the first control handle is associated with the endoscope.
Further, to the extent consistent, any of the aspects described herein may be used in conjunction with any or all of the other aspects described herein.
Various aspects of the present disclosure are described hereinbelow with reference to the drawings, which are incorporated in and constitute a part of this specification, wherein:
Embodiments of the present disclosure are now described in detail with reference to the drawings in which like reference numerals designate identical or corresponding elements in each of the several views. As used herein, the term “clinician” refers to a doctor, a nurse, or any other care provider and may include support personnel. Throughout this description, the term “proximal” refers to the portion of the device or component thereof that is closer to the clinician and the term “distal” refers to the portion of the device or component thereof that is farther from the clinician.
Referring to
The user console 40 includes a display device 44 which is configured to display three-dimensional images. The display device 44 displays three-dimensional images of the surgical site “S” which may include data captured by imaging devices 16 positioned on the ends 14 of the linkages 12 and/or include data captured by imaging devices that are positioned about the surgical theater (e.g., an imaging device positioned within the surgical site “S”, an imaging device positioned adjacent the patient “P”, imaging device 56 positioned at a distal end of an imaging arm 52). The imaging devices (e.g., imaging devices 16, 56) may capture visual images, infra-red images, ultrasound images, X-ray images, thermal images, and/or any other known real-time images of the surgical site “S”. The imaging devices transmit captured imaging data to the processing unit 30 which creates three-dimensional images of the surgical site “S” in real-time from the imaging data and transmits the three-dimensional images to the display device 44 for display.
The user console 40 also includes input handles 42 which are supported on control arms 43 which allow a clinician to manipulate the surgical robot 10 (e.g., move the linkages 12, the ends 14 of the linkages 12, and/or the tools 20). Each of the input handles 42 is in communication with the processing unit 30 to transmit control signals thereto and to receive feedback signals therefrom. Additionally or alternatively, each of the input handles 42 may include input devices (not explicitly shown) which allow the surgeon to manipulate (e.g., clamp, grasp, fire, open, close, rotate, thrust, slice, etc.) the tools 20 supported at the ends 14 of the linkages 12.
Each of the input handles 42 is moveable through a predefined workspace to move the ends 14 of the linkages 12, e.g., tools 20, within a surgical site “S”. The three-dimensional images on the display device 44 are orientated such that the movement of the input handles 42 moves the ends 14 of the linkages 12 as viewed on the display device 44. The three-dimensional images remain stationary while movement of the input handles 42 is scaled to movement of the ends 14 of the linkages 12 within the three-dimensional images. To maintain an orientation of the three-dimensional images, kinematic mapping of the input handles 42 is based on a camera orientation relative to an orientation of the ends 14 of the linkages 12. The orientation of the three-dimensional images on the display device 44 may be mirrored or rotated relative to the view captured by the imaging devices 16, 56. In addition, the size of the three-dimensional images on the display device 44 may be scaled to be larger or smaller than the actual structures of the surgical site permitting a clinician to have a better view of structures within the surgical site “S”. As the input handles 42 are moved, the tools 20 are moved within the surgical site “S” as detailed below. Movement of the tools 20 may also include movement of the ends 14 of the linkages 12 which support the tools 20.
For a detailed discussion of the construction and operation of a robotic surgical system 1, reference may be made to U.S. Pat. No. 8,828,023, the entire contents of which are incorporated herein by reference.
The user console 40 further includes one or more foot pedals 60 that can be used to control various aspects of the robotic surgical system 1. For example, the foot pedal 60 may be selectively associated with an input handle, e.g., input handle 42, to actuate a tool 20 associated with the respective input handle. Additionally or alternatively, the foot pedal 60 may be associated with a camera, e.g., camera 56, to move the camera about the surgical site “S”. For a detailed discussion of suitable foot pedals, reference may be made to U.S. Provisional Patent Application Ser. No. 62/510,502, filed May 24, 2017, entitled “PEDAL CONTROL FOR ROBOTIC SURGICAL SYSTEMS,” and U.S. Provisional Patent Application Ser. No. 62/566,100, filed Sep. 8, 2017, entitled “HIGH PRECISION INSTRUMENT CONTROL MODE FOR ROBOTIC SURGICAL SYSTEMS,” the entire contents of each of the above applications are hereby incorporated by reference.
Referring now to
The information ribbon 120 is presented in an area of the display 44 that minimizes obscuring of the work area 110 while providing a clinician information concerning the surgical instruments, end effectors, and/or endoscope viewable in the work area 110, e.g., end effectors A, B, and C and endoscope E. In addition, the information ribbon 120 may provide information to a clinician about surgical instruments, end effectors, and/or endoscopes of a surgical robot, e.g., surgical robot 10, that are outside of the work area 110. As shown the information ribbon 120 is located across the top of the display 44; however, it is contemplated that the information ribbon 120 may be located across the bottom of the display 44 and/or along a side of the display 44.
The information ribbon 120 includes icons displaying information relevant to the end effectors A, B, and C and the endoscope E. As shown, the information ribbon 120 includes an icon 122 which provides information with respect to the end effector A, an icon 124 which provides information with respect to the end effector B, an icon 126 which provides information with respect to the end effector C, and an icon 128 which provides information with respect to the endoscope E. The information ribbon 120 may have a defined border, e.g., a line, or may be borderless. The icons 122-128 are arranged across the information ribbon 120 with icons 122, 126 associated with end effectors manipulated by a left hand of the clinician, e.g., end effectors A and C, displayed on the left side of the information ribbon 120; icon 124 associated with end effectors manipulated by a right hand of the clinician, e.g., end effector B, displayed on the right side of the information ribbon 120; and the icon 128 which provides information with respect to the endoscope E is positioned between instruments controlled by the left hand and the right hand of the clinician and may be substantially centered in the information ribbon 120. The icons 122 and 126 are both associated with end effectors manipulated by the left hand of a clinician are arranged with the icon 126 to the left of the icon 122 to correspond to the position of the end effector C relative to the end effector A in the work area 110. The positions of the icons 122, 126 may switch or swap relative to one another as the positions of the end effectors A, C change during a surgical procedure. The positions may swap in real-time or there may be a slight delay such that the positions only swap if the relative positions of the end effectors A, C are incorrect, e.g., end effector C is positioned to the right of end effector A in the work area 110, for more than a predetermined amount of time, e.g., 5 seconds. This may reduce the number of swaps of icons 122, 126 which may distract a clinician during a surgical procedure.
The icons 122, 124, 126 provide information with respect to the end effector associated with the respective icon. The information is displayed in a standard format for each end effector but may differ based on the type of instrument of the end effector. For example, an icon for an electrosurgical instrument may have different information from an icon for a stapler or grasper.
With particular reference to
The instrument type indicator 165 displays a type and/or name of an end effector or tool secured to the respective arm of the surgical robot 10 to allow a clinician to quickly identify a type of a tool secured to the respective arm. As shown, the instrument type 165 of the end effector C is an “Electrosurgical Forceps”.
The mode indicator 166 of the icon 126 provides information relevant to a selected mode of the associated end effector. As shown, the mode indicator 166 may be a first color, e.g., yellow, when the associated end effector is in a first mode, e.g., monopolar or MonoCUT, and may be a second color, e.g., green, when the associated end effector is in a second mode, e.g., bipolar or BiSEAL. When different types of end effectors are associated with the icon 126, the mode indicator 166 may display different modes. For example, when a stapling end effector is associated with the icon 126, the first mode may be indicative of a grasping mode and the second mode may be indicative of a stapling mode.
The status indicator 168 provides information relevant to a status of the associated end effector. For example, the status indicator 168 may display a first color, e.g., blue or cyan, when the associated end effector is active, e.g., under the control of a handle 42 of the user interface 40 (
With reference to
With reference to
It will be appreciated that each of the icons 122, 124, 126 may have a style similar to the icons 126, 226, and 326 as detailed above. The icons 122, 124, 126 may have the same style or may each have a different style depending on the type of end effector associated with the respective icon. The information of each of the icons 122, 124, 126 may increase a clinician's situational awareness. Increasing a clinician's situational awareness may reduce duration of a procedure and/or improve a patient's outcome.
Referring back to
With reference to
Referring now to
With reference to
Referring now to
As detailed above and shown in
While several embodiments of the disclosure have been shown in the drawings, it is not intended that the disclosure be limited thereto, as it is intended that the disclosure be as broad in scope as the art will allow and that the specification be read likewise. Any combination of the above embodiments is also envisioned and is within the scope of the appended claims. Therefore, the above description should not be construed as limiting, but merely as exemplifications of particular embodiments. Those skilled in the art will envision other modifications within the scope of the claims appended hereto.
This application claims benefit of, and priority to, U.S. Provisional Patent Application Ser. No. 62/471,716, filed Mar. 15, 2017, the entire contents of which are hereby incorporated by reference.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/US2018/022288 | 3/14/2018 | WO | 00 |
Number | Date | Country | |
---|---|---|---|
62471716 | Mar 2017 | US |