Patent Application
20220192752
The present disclosure relates to a system and a method for registering operating images.
With the increase of aging population and the influence of modern life characteristics such as obesity, sedentary, etc., diseases of spine lesions are increasing year by year. If conservative treatment fails, it is often necessary to rely on implants to reduce pain and maintain basic functions. The spine is responsible for protecting the central nervous system, but the portion where the implants that can be applied are quite narrow. For example, a pedicle screw may damage the central nervous system. Although an orthopedic minimally invasive surgery is available in the market, how to accurately track the position of the spine during the surgery is still an issue since the position of the spine changes with the patient's posture.
Embodiments of the present disclosure provide a system for registering operating images. The system includes an X-ray imaging machine, a visible-light camera disposed on the X-ray imaging machine, a rectifier, and a computer system. The X-ray imaging machine is configured to capture a first X-ray image of the rectifier, and the visible-light camera is configured to capture a first visible-light image of the rectifier at the same time. The computer system is configured to recognize the rectifier in the first X-ray image and in the first visible-light image to compote a conversion model between a first coordinate system of the X-ray imaging machine and a second coordinate system of the visible-light camera. The X-ray imaging machine is further configured to capture a second X-ray image of a patient on which a positioning mark is set, and the visible-light camera is configured to capture a second visible-light image of the positioning mark. The computer system is configured to provide a navigation interface by combining the second X-ray image and the second visible-light image based on the conversion model.
In some embodiments, coordinates of the rectifier in the first coordinate system are represented as a vector X. Coordinates of the rectifier in the second coordinate system are represented as a vector Y. The computer system is configured to compute the conversion model according to the following equation (1), in which the conversion model is a matrix TXM.
X=T
XM
Y (1)
In some embodiments, the rectifier includes multiple metal balls.
In some embodiments, the X-ray imaging machine is a C-arm X-ray machine which includes an emitting terminal and a receiving terminal. The visible-light camera is disposed on the receiving terminal.
In some embodiments, the system further includes a surgical instrument and an instrument camera which is fixed on the surgical instrument.
From another aspect, embodiments of the present disclosure provide a method for registering operating images for a computer system. The method including: capturing, by an X-ray imaging machine, a first X-ray image of a rectifier, and capturing, by a visible-light camera, a first visible-light image of the rectifier at same time, in which the visible-light camera is disposed on the X-ray imaging machine; recognizing the rectifier in the first X-ray image and in the first visible-light image to compote a conversion model between a first coordinate system of the X-ray imaging machine and a second coordinate system of the visible-light camera; capturing, by the X-ray imaging machine, a second X-ray image of a patient on which a positioning mark is set; capturing, by the visible-light camera, a second visible-light image of the positioning mark; and providing a navigation interface by combining the second X-ray image and the second visible-light image based on the conversion model.
In some embodiments, the method further includes computing the conversion model according to the equation (1).
In the system and the method, the system is simplified by disposing the visible-light camera on the X-ray imaging machine. A conversion model between the X-ray imaging machine and the visible-light camera is computed by using a rectifier. Therefore, the X-ray image and the visible-light image can be combined to provide a navigation interface.
The invention can be more fully understood by reading the following detailed description of the embodiment, with reference made to the accompanying drawings as follows.
Specific embodiments of the present invention are further described in detail below with reference to the accompanying drawings, however, the embodiments described are not intended to limit the present invention and it is not intended for the description of operation to limit the order of implementation. Moreover, any device with equivalent functions that is produced from a structure formed by a recombination of elements shall fall within the scope of the present invention. Additionally, the drawings are only illustrative and are not drawn to actual size.
The using of “first”, “second”, “third”, etc. in the specification should be understood for identifying units or data described by the same terminology, but are not referred to particular order or sequence.
In the embodiment, the X-ray imaging machine 110 is a C-arm X-ray machine which includes an emitting terminal 112 and a receiving terminal 111. The visible-light camera 120 is disposed on the receiving terminal 111.
The visible-light camera 120 may include a Charge-Coupled Device (CCD) sensor, a Complementary Metal-Oxide Semiconductor (CMOS) sensor or other suitable optical sensors. In some embodiment, the visible-light camera 120 may also include an infrared transmitter, an infrared sensor, dual cameras, a structured light sensing device or any device that can sense the depth of the scene.
The computer system 140 is communicatively connected to the X-ray imaging machine 110 and visible-light camera 120 by any wire or wireless communication means. The rectifier 130 can be imaged in an X-ray image. For example, the rectifier 130 includes multiple metal balls 131. X-ray images and visible-light images are combined herein to provide a navigation interface for surgery. Therefore, a coordinate system of the X-ray imaging machine 110 has to be matched to a coordinate system of the visible-light camera 120.
First, the X-ray imaging machine 110 captures a first X-ray image of the rectifier 130, and the visible-light camera 120 captures a first visible-light image of the rectifier 130 at the same time. The computer system 140 can recognize the rectifier 130 in the first X-ray image and in the first visible-light image by any image processing algorithm and computer vision approach. For example, the shape of the rectifier 130 is known, and therefore, the computer system 140 can search the rectifier 130 in the first X-ray image and the first visible-light image based on a preset pattern. After recognizing the rectifier 130, a conversion model between a coordinate system (also referred to as a first coordinate system) of the X-ray imaging machine 110 and a coordinate system (also referred to as a second coordinate system) of the visible-light camera 120 is generated. To be specific, referring to
X=T
XM
Y (2)
Referring to
The visible-light camera 120 captures a second visible-light image of the positioning mark 330. Note that X-ray is harmful, and therefore X-ray images cannot be captured frequently. In the embodiment, images of the positioning mark 330 are captured by the visible-light camera 120, and the position of the vertebra 320 is computed by recognizing the positioning mark 330 in the images.
Next, the computer system 140 combines the second X-ray image and the second visible-light image to provide a navigation interface based on the conversion model (i.e. the matrix TXM). In the embodiment, the navigation interface may be shown on the screen of the computer system, but it may be shown on any head mounted device, tablet, or transparent display in other embodiments. Any suitable technology of virtual reality, augmented reality, alternative reality or mixed reality can be used to generate the navigation interface. For example, the spine in the second X-ray image can be divided as vertebras to generate corresponding virtual objects based on an image processing algorithm. The computer system 140 can recognize the positioning mark 330 in the second visible-light image to compute the position of the positioning mark 330, and accordingly renders the corresponding virtual object in the navigation interface. As a result, the doctor can see the position of the vertebra through the navigation interface. In some embodiments, the computer system can transform coordinates of the positioning mark 330 in the second visible-light image into coordinates in the second X-ray image based on the matrix TXM, and accordingly make appropriate operations such as segmentation, deformation, scaling, and displacement to at least a portion of the second X-ray image. The processed second X-ray image can be rendered in the navigation interface to reflect the real position of the vertebras.
In some embodiments, the system also includes a surgical instrument 340 and an instrument camera 350 which is fixed on the surgical instrument 340. For example, the surgical instrument 340 includes drills, files, scrapers, saws, screwdrivers or other tools that are commonly used in surgical operations to repair or remove parts of an anatomical area by drilling, grinding, cutting or scraping. The instrument camera 350 may include a CCD sensor, a CMOS sensor or other suitable optical sensors. In some embodiments, the instrument camera 350 may include an infrared transmitter, an infrared sensor, dual cameras, a structured light sensing device or any device that can sense the depth of the scene. The instrument camera 350 captures an image of the positioning mark 330. A conversion model between a third coordinate system of the instrument camera 350 and the second coordinate system is computed by calibration in advance. Therefore, a virtual object 341 of the surgical instrument 340 can be rendered in the navigation interface. The doctor can see the position of the surgical instrument 340 relative to the vertebra through the navigation interface.
In the aforementioned embodiments, the system is simplified by disposing the visible-light camera on the X-ray imaging machine. The conversion model between the X-ray imaging machine and the visible-light camera is computed by a rectifier such that the X-ray images and the visible-light images are combined to provide the navigation interface.
Although the present invention has been described in considerable detail with reference to certain embodiments thereof, other embodiments are possible. Therefore, the spirit and scope of the appended claims should not be limited to the description of the embodiments contained herein. It will be apparent to those skilled in the art that various modifications and variations can be made to the structure of the present invention without departing from the scope or spirit of the invention. In view of the foregoing, it is intended that the present invention cover modifications and variations of this invention provided they fall within the scope of the following claims.
