Patent Application
20230225810
The present disclosure is generally related to a robotic surgical system in a Virtual Reality (VR) environment.
The subject matter discussed in the background section should not be assumed to be prior art merely as a result of its mention in the background section. Similarly, a problem mentioned in the background section or associated with the subject matter of the background section should not be assumed to have been previously recognized in the prior art. The subject matter in the background section merely represents different approaches, which in and of themselves may also correspond to implementations of the claimed technology.
Robotic surgical devices are now routinely used for numerous surgical procedures such as general surgery, pediatric surgery, and those related to the medical fields of gynecology, urology, cardiology, and otorhinolaryngology. Robotic devices continue to evolve and are being more frequently utilized in surgical procedures.
The robotic devices are used most in surgical procedures that require a high degree of accuracy and/or precision. Such robotic devices include autonomous, tele-operated, and interactive type robotic systems. Interactive robotic systems are most frequently used for providing the surgeon with direct hands-on control of the surgical procedure, thus achieving a high degree of accuracy and/or precision. For example, in a knee surgery, a surgeon can use an interactive robotic arm to sculpt a bone and/or to receive a knee implant. In a laparoscopic surgical procedure, using a robotic system, a surgeon may directly control and manipulate tissue, albeit at some distance from the patient through a fulcrum point in the abdominal wall.
In other surgical procedures performed using robotic devices, the surgeon may sit at a console in the operating room, but outside the sterile field, directing and controlling the movements of one or more robotic arms. However, robotic devices can be intrusive during a surgical procedure, blocking the surgeon's point of view and occupying substantial space around an operating table, increasing the likelihood of an operator error.
Instruments for robotic devices, like movement detection equipment and navigation markers, may be implemented in surgical procedures as safeguards. Such instruments help guide the robotic devices and assist the surgeons in avoiding errors. The movement detection equipment and the navigation markers help determine the position of the instrument in space and prevent the instrument from deviating beyond the path set by the surgeon. For example, in neurosurgery, neuromonitors with sensors are used to detect the threshold level, and a signal is sent to an appropriate system to stop insertion of the surgical instrument or to move the instrument away for preventing any damage when an error is detected as being imminent. Further, actuators are also used for controlled movement and positioning of end effectors in the robotic arm.
The accompanying drawings illustrate various embodiments of systems, methods, and embodiments of various other aspects of the disclosure. Any person with ordinary skills in the art will appreciate that the illustrated element boundaries (e.g. boxes, groups of boxes, or other shapes) in the figures represent one example of the boundaries. It may be that in some examples one element may be designed as multiple elements or that multiple elements may be designed as one element. In some examples, an element shown as an internal component of one element may be implemented as an external component in another, and vice versa. Furthermore, elements may not be drawn to scale. Non-limiting and non-exhaustive descriptions are described with reference to the following drawings. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating principles.
Some embodiments of this disclosure, illustrating all its features, will now be discussed in detail. The words “comprising,” “having,” “containing,” and “including,” and other forms thereof, are intended to be equivalent in meaning and be open ended in that an item or items following any one of these words is not meant to be an exhaustive listing of such item or items, or meant to be limited to only the listed item or items.
It must also be noted that as used herein and in the appended claims, the singular forms “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise. Although any systems and methods similar or equivalent to those described herein can be used in the practice or testing of embodiments of the present disclosure, the preferred, systems and methods are now described.
Embodiments of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings in which like numerals represent like elements throughout the several figures, and in which example embodiments are shown. Embodiments of the claims may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. The examples set forth herein are non-limiting examples and are merely examples among other possible examples.
The image database 106 may store images of a subject patient, as well as images of previous patients who have undergone similar surgeries. The images may be captured using an X-ray, ultrasound, and Magnetic Resonance Imaging (MRI). Further, the images may be present in raw form, as Three-Dimensional (3D) models, Augmented Reality (AR) images, Virtual Reality (VR) images, and Point of View (PoV) images. The position database 108 may store real-time position information of a PoV surgical drill 122 and that of a virtual drill that may he shown to a surgeon during surgery.
The communication network 104 may be either of a wired and/or a wireless network. The communication network 104, if wireless, may be implemented using communication techniques such as Visible Light Communication (VLC), Worldwide Interoperability for Microwave Access (WIMAX), Long Term Evolution (LTE™), Wireless Local Area Network (WLAN), Infrared (IR) communication, Public Switched Telephone Network (PSTN), Radio waves, and other communication techniques known in the art.
The system 102 may further include a processor 110, interface(s) 112, and a memory 114. The processor 110 may execute an algorithm stored in the memory 114 for processing the PoV images and for guiding the robotic surgical system when performing a surgical procedure. The processor 110 may also be configured to decode and execute any instructions received from one or more other electronic devices or server(s).
In at least one embodiment, the processor 110 may include one or more general purpose processors (e.g., INTEL® or Advanced Micro Devices® (AMD) microprocessors) and/or one or more special purpose processors (e.g., digital signal processors or Xilinx System On Chip (SOC) Field Programmable Gate Array (FPGA) processor). The processor 110 may be configured to execute one or more computer-readable program instructions, such as program instructions to carry out any of the functions described in this description.
The interface(s) 112 may facilitate interaction between a surgeon and the system 102. The interface(s) 112 may accept an input from the surgeon or other user who is associated with an on-going surgery and/or provide an output to the surgeon or other user. The interface(s) 112 may either be a Command Line Interface (CLI), Graphical User Interface (GUI), or a voice interface.
The memory 114 may include, but is not limited to, fixed (hard) drives, magnetic tape, floppy diskettes, optical disks, Compact Disc Read-Only Memories (CD-ROMs), and magneto-optical disks, semiconductor memories, such as ROMs, Random Access Memories (RAMS), Programmable Read-Only Memories (PROMs), Erasable PROMs (EPROMs), Electrically Erasable PROMs (EEPROMs), flash memory, magnetic or optical cards, or other type of media/machine-readable medium suitable for storing electronic instructions.
The user device 116 is shown as a tablet in
In at least one embodiment, a drill bit may be placed in an opening 208 of a drill holder 210 of the PoV surgical drill 122. Once the drill bit is placed in the drill holder 210, a module 212 connected to the drill holder 210 may identify parameters of PoV surgical drill 122, including a type of the surgical drill, a type of the drill bit, a size of the drill bit, and an absolute position of a tip of the drill bit in an XYZ coordinate system referencing the operating table 202.
In at least one embodiment, the module 212 may further comprise a surgical drill reader configured to read a serial number present on the drill bit. The serial number may be related to the PoV surgical drill 122 and/or the drill bit thereof. Serial numbers respectively corresponding to different drill bits and different categories of surgical drills may be stored in a memory corresponding to the module 212. The received serial number may be matched with the serial numbers stored in the memory to identify details related to the PoV surgical drill 122 and the drill bit. In at least one example, the surgical drill reader may be implemented as a Near Field Communication (NFC) reader, and NFC encoded chip may be attached to the drill bit. The NFC reader may therefore communicate with the NFC encoded chip to receive the serial number of the drill bit.
In at least one embodiment, the module 212 may identify and/or determine the drill bit being cradled, reference the position of the drill bit with the virtual grid 200, identify the surgical drill and the drill bit, convert the surgical drill identification to an associated virtual surgical icon, and convert the drill bit identification to an associated virtual surgical drill bit icon. The module 212 may further transmit the virtual surgical icon and the virtual drill bit icon, referenced to the XYZ coordinate system of the operating table 202, to an AR imaging system and to the reference holder system 126.
In at least one other embodiment, the reference holder system 126, shown and described with regard to
In at least one embodiment, images captured by any one or more of the cameras 308, 310, and 312 may be integrated to produce one composite PoV image using known image processing tools and techniques. In at least one example, the composite PoV image may be cropped in a circle and centered with regard to the drill bit 302 based on defaults settings stored by the surgeon. The cropped image may then be sent to the reference holder system 126.
In at least one embodiment, while performing a surgical procedure on the subject patientm the surgeon may maneuver the guidance drill 120 to control the PoV surgical drill 122 based on the PoV images seen on the AR/VR display 118. As set forth above, the PoV images may be collected using one or more of the cameras 308, 310, and 312 positioned on the head of the PoV surgical drill 122. Thus, content of the PoV images may change based on an orientation and direction faced by the PoV surgical drill 122.
In at least one embodiment, the processor 110 may synchronize the position of the PoV surgical drill 122 with a reference linked with augmented images shown on the AR display device 118. Based on such synchronization, a Virtual Reality (VR) drill may be shown to the surgeon on the augmented images displayed using the AR display device 118. The VR drill may move based on changes in position of the PoV surgical drill 122, controlled by the surgeon controlling the guidance drill 120. Thus, such synchronization of the VR drill and the PoV surgical drill 122 provides a realistic experience to the surgeon. Further, operating room cameras may also be used for capturing images of the surgical procedure from a fixed angle, as set by the surgeon or based on a positioning of the operating room cameras. Such images may be stored in the image database 106, and may be displayed to the surgeon using an image display, e.g., AR display device 118.
In an illustrative embodiment, any of the operations, processes, etc. described herein can be implemented as computer-readable instructions stored on a computer-readable medium. The computer-readable instructions can be executed by a processor of a mobile unit, a network element, and/or any other computing device.
There is little distinction left between hardware and software implementations of aspects of systems; the use of hardware or software is generally (but not always, in that in certain contexts the choice between hardware and software can become significant) a design choice representing cost vs. efficiency tradeoffs. There are various vehicles by which processes and/or systems and/or other technologies described herein can be effected (e.g., hardware, software, and/or firmware), and that the preferred vehicle will vary with the context in which the processes and/or systems and/or other technologies are deployed. For example, if an implementer determines that speed and accuracy are paramount, the implementer may opt for a mainly hardware and/or firmware vehicle; if flexibility is paramount, the implementer may opt for a mainly software implementation; or, yet again alternatively, the implementer may opt for some combination of hardware, software, and/or firmware.
The foregoing detailed description has set forth various embodiments of the devices and/or processes via the use of block diagrams, flowcharts, and/or examples. Insofar as such block diagrams, flowcharts, and/or examples contain one or more functions and/or operations, it will be understood by those within the art that each function and/or operation within such block diagrams, flowcharts, or examples can be implemented, individually and/or collectively, by a wide range of hardware, software, firmware, or virtually any combination thereof. In one embodiment, several portions of the subject matter described herein may be implemented via Application Specific. Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), digital signal processors (DSPs), or other integrated formats. However, those skilled in the art will recognize that some aspects of the embodiments disclosed herein, in whole or in part, can be equivalently implemented in integrated circuits, as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more processors (e.g., as one or more programs running on one or more microprocessors), as firmware, or as virtually any combination thereof, and that designing the circuitry and/or writing the code for the software and or firmware would be well within the skill of one of skill in the art in light of this disclosure. In addition, those skilled in the art will appreciate that the mechanisms of the subject matter described herein are capable of being distributed as a program product in a variety of forms, and that an illustrative embodiment of the subject matter described herein applies regardless of the particular type of signal bearing medium used to actually carry out the distribution. Examples of a signal bearing medium include, but are not limited to, the following: a recordable type medium such as a floppy disk, a hard disk drive, a CD, a DVD, a digital tape, a computer memory, etc.; and a transmission type medium such as a digital and/or an analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communications link, a wireless communication link, etc.).
Those skilled in the art will recognize that it is common within the art to describe devices and/or processes in the fashion set forth herein, and thereafter use engineering practices to integrate such described devices and/or processes into data processing systems. That is, at least a portion of the devices and/or processes described herein can be integrated into a data processing system via a reasonable amount of experimentation. Those having skill in the art will recognize that a typical data processing system generally includes one or more of a system unit housing, a video display device, a memory such as volatile and non-volatile memory, processors such as microprocessors and digital signal processors, computational entities such as operating systems, drivers, graphical user interfaces, and applications programs, one or more interaction devices, such as a touch pad or screen, and/or control systems including feedback loops and control motors (e.g., feedback for sensing position and/or velocity; control motors for moving and/or adjusting components and/or quantities). A typical data processing system may be implemented utilizing any suitable commercially available components, such as those typically found in data computing/communication and/or network computing/communication systems.
The herein described subject matter sometimes shows different components contained within, or connected with, different other components. It is to be understood that such depicted architectures are merely examples, and that in fact many other architectures can be implemented which achieve the same functionality, In a conceptual sense, any arrangement of components to achieve the same functionality is effectively “associated” such that the desired functionality is achieved. Hence, any two components herein combined to achieve a particular functionality can be seen as “associated with” each other such that the desired functionality is achieved, irrespective of architectures or intermedial components. Likewise, any two components so associated can also be viewed as being “operably connected”, or “operably coupled”, to each other to achieve the desired functionality, and any two components capable of being so associated can also be viewed as being “operably couplable”, to each other to achieve the desired functionality. Specific examples of operably couplable include but are not limited to physically rateable and/or physically interacting components and/or wirelessly interactable and/or wirelessly interacting components and/or logically interacting and/or logically interactable components.
From the foregoing, it will be appreciated that various embodiments of the present disclosure have been described herein for purposes of illustration, and that various modifications may be made without departing from the scope and spirit of the present disclosure. Accordingly, the various embodiments disclosed herein are not intended to be limiting.
| Number | Date | Country | |
|---|---|---|---|
| 62526439 | Jun 2017 | US | |
| 62528102 | Jul 2017 | US | |
| 62529019 | Jul 2017 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16023014 | Jun 2018 | US |
| Child | 18174287 | US |
