The invention relates to a method and a device for displaying a device inserted into a vascular system, in particular a medical instrument, in a 3-D volumetric data set by creating 2-D fluoroscopic images of the device.
In increasing measure, procedures carried out on patients are minimally invasive, where, for example, an x-ray control is used to guide a catheter or other medical instrument through the bloodstream to the diseased point in the body. This can be used to treat inter alia tumors, aneurisms, AVMs (arteriovenous malformations (Arterio Venous Malformatio))and stenoses for example. The navigation of the catheter from the point it enters into the body to the site of the disease presents a big challenge even for experienced medical practitioners. In this respect, navigation in the neural area stands out in particular, as targeted navigation in the filigree branching of the vascular system in the brain is extremely demanding.
The problem here is that although the catheter is clearly visible during the x-ray fluoroscopy, the anatomy of the patient, in particular his vascular structure, is, on the other hand, scarcely recognizable in the fluoroscopic image, or, as the case may be, only recognizable after a contrast medium has been injected.
Since 3-D image data sets have been increasingly employed in recent years, and are generally available, as can be seen, for example, from the U.S. Pat. No. 5,764,719 or the DE 101 46 915 B, the aim is not to just have the catheter visible only in the 2-D fluoroscopic image, but also in a 3-D data set of the patient.
The previous solutions to the problem can be divided into 2-D and 3-D visualization of the medical instrument in the vascular system.
2-D visualization of the vascular system and the catheter: It has been prior art and current practice for many years to administer the contrast medium in the vascular system area, in which, for example, a catheter is just being moved, to store a fluoroscopic image, in which the contrast medium makes the blood vessels visible, as a reference image and to place this image under the subsequent fluoroscopic imaging. A visualization of this kind is known as Roadmap Method, known, for example, from U.S. Pat. No. 4,709,385.
3-D visualization of the catheter in the vascular system: The difficulty here is that a correspondence must be created between at least one point in the 2-D fluoroscopic image (e.g. the tip of the catheter) and the 3-D data set.
The invention is based on the task of developing a method of the type mentioned in the beginning in such a way that, from 2-D fluoroscopic images the positions of a medical instrument inserted into the vascular system of a patient can be superimposed on a 3-D image of a volumetric data set in a simple manner.
According to the invention, the task is solved in that information is integrated recursively from consecutive 2-D fluoroscopic images of the device into previous information or previous knowledge about the 3-D positions, wherein from the 3-D volumetric data set information is determined which 3-D positions are plausible for the device, and in that because of the information from the previous information or knowledge and the determinations, the currently possible position of the device is superimposed on the image of the 3-D volumetric data set. This makes it possible to determine, in a simple fashion, the spatial position of a medical instrument inserted in the vascular system of a patient from a sequence of fluoroscopic images. An exact estimation of the 3-D position only becomes possible when several 2-D images are integrated, as only in their entirety (integration) do the 2-D images contain information, which no single image, or, for the most part, any subset of these images contains.
An essential characteristic of the method is that the information from new images can be integrated without the old images having to be present, as expressed by “recursively”.
All possible catheter positions that describe its movement in its temporal sequence belong to the information or to the knowledge. This also includes the corresponding projection matrix for each position of the catheter, as it is only in this way that a relation can be established between 2-D coordinates in the image and 3-D coordinates in 3-D volumes. The presence of the projection matrices also ensures that external equipment parameters, such as C-arm angulation, zoom, table position, etc., do not have to continue to be considered separately.
2-D fluoroscopic images with same positions of the instrument also provide valuable new information: this thus makes the estimation more exact. From images of the same instrument position, it is true that many kinds of ambiguities are not resolved, but new 2-D fluoroscopic images, which, according to the invention, are produced at different positions of the instrument, are always a gain in information that improves the estimation.
It has proved to be advantageous if the determination and visualization of the 3-D coordinates of the medical instrument inserted in the vascular system is made, taking into consideration ambiguities, i.e. ambiguities with two or more possible interpretations, from a sequence of fluoroscopic images, wherein the information on the position of the medical instrument is merged over time to obtain a current estimation of the most likely 3-D positions at any given point in time.
Advantageously, the method can consist of the following steps:
The “possible 3-D coordinates” according to step e) results from the quality of all possible 3-D positions, by selecting, for example, only the n-best or those that achieve a certain minimum quality.
The jump from step g) back to step c) is only necessary if the 2-D/3-D registration is no longer valid, whereby, in principle, it does no harm always to carry out the 2-D/3-D registration.
According to the invention, as step d) it is possible to make a probalistic estimation of the 3-D position of a medical instrument inserted into the vascular system of the examination object using a density that, for each 3-D position, expresses the probability of the instrument being at this position.
It has proved to be of advantage if as first step d) after the creation of a first fluoroscopic image, the corresponding initial position in the fluoroscopic image is selected manually.
Alternatively as first step d) after the creation of a first fluoroscopic image, the corresponding initial position in the fluoroscopic image is selected using the differential image method or by identifying the corresponding initial position in the fluoroscopic image by object or pattern recognition methods.
The object is achieved for a device according to the invention in that with an x-ray diagnostic device with a detector for capturing fluoroscopic images and viewing monitors for reproducing the fluoroscopic image, the detector is linked to a mixer stage for superimposing a point determined by selection circuitry, and in that the outlet of the mixer stage is linked to one of the viewing monitors for reproducing the fluoroscopic image and is further linked to the input of a rear projection stage, to which a projection matrix of the 2-D/3-D registration is fed from a matrix storage device, and in that a stage for vascular segmentation is connected to a 3-D image storage device for a 3-D volumetric data set, and in that the rear projection stage and the stage for the vascular segmentation are linked to a device for point determination, the outlet of which is connected to a 3-D monitor from the viewing monitors.
Advantageously, the method to display a device, in particular a medical instrument, inserted into a vascular system, in a 3-D volumetric data set by creating 2-D fluoroscopic images of the device, wherein, from consecutive 2-D fluoroscopic images of the device information [lacuna] recursively into the previous information or the previous knowledge about the 3-D positions, can be defined by the following steps:
The invention is described below in more detail using embodiments illustrated in the drawing, in which;
In
Instead of stand 1, floor and/ or ceiling mounts can also be used. The C-arm 2 can also be replaced by a so-called electronic C-arm 2, where x-ray device 3 and x-ray image detector 4 are coupled electronically.
The x-ray image detector 4 can be a rectangular or square, flat semiconductor detector that is made preferably from amorphous silicon (aSi).
A patient 6 is placed on an examination table 5 for examination and x-rayed using the x-ray device 3, so that an attenuated signal corresponding to the radiation transparency of the patient 6 falls on the x-ray image detector 4.
The x-ray diagnostic device delivers the output signal of the x-ray image detector 4 to an image system 7 that manages the control of the x-ray diagnostic device and the further processing of the digital image signals. The digital image signals processed in this way are fed to a device for representing positions of a medical instrument inserted in a vascular system in a 3-D volumetric data set, which will be described below. A storage device 8 for a 3-D volumetric data set and a projection matrix are connected to the image system 7.
The image system 7 generates video signals in known way from the digital picture signals, which video signals can be reproduced on viewing monitors 9. Here, a fluoroscopic image can be represented on the one monitor and an image of a 3-D volumetric data set on the second.
In
The output signal of mixer stage 10 is reproduced on one of the viewing monitors 9 for the fluoroscopic image. The mixer stage 10 remains connected to the input of a rear projection stage 12, to which a projection matrix Pt, is fed from a matrix storage device 13, which contains the 2-D/3-D registration.
The 3-D volumetric data set contained in a 3-D image storage device 14 is linked to a stage 15 for vascular segmentation, the output signal of which is fed to a device 16 for point determination, which device is linked to a 3-D monitor of the viewing monitors 9.
The matrix storage device 13 for the projection matrix Pt and the 3-D image storage device 14 can be contained in the storage device 8 or be separate data storage devices in the image system 7.
The method described below, according to the invention, realized in the image system 7 according to
Determining the 3-D Coordinates
The following data and data structures are presumed:
In the first fluoroscopic image f0 (i.e. at the point in time t=0) illustrated in
A more exact position determination is possible if information on the vascular system is integrated. That means that only points on g0 that are at a sufficiently small distance from a blood vessel, as is shown by the marked points C1 and C2 in
Thus for every voxel of the 3-D volumetric data set, a probability can be stated that this voxel represents the true 3-D position of the medical instrument. There the above formula is a way of calculating this probability: for a specific state q0, look for the voxel on the line of sight g0, which has a minimal distance and calculate using a suitable statistical model d ( . . ) the probability that this state describes the true 3-D position allowing for the distance vector.
If one integrates the findings of the chronologically consecutive fluoroscopic images into these densities, then existing ambiguities can be resolved over time. With the help of
By allowing for the context, i.e. the existing information about the 3-D positions, new ambiguities arise not because when there is an unequivocal 3-D position at the point in time t-1, at the subsequent point in time t the line of sight gt suddenly runs through two blood vessels. This is achieved because information as to how probable it is that this is achieved in the vascular tree taking the 3-D position at the point in time t-1 as starting point, flows into the calculation of the probability of a 3-D position at the point in time t. To explain clearly, this means: is it possible to push the catheter from the 3-D position t-1 to the 3-D position t in the given space of time? With the help of the illustrations in
New ambiguities only arise if a blood vessel divides into at least two branches, and the line of sight intersects both branches. In
In formal terms, this means that
must be calculated, whereby the probability of a state at the point in time t results analogously from equation (1).
The “state transition” in equation (2) is in that case the context component described and is calculated from the length of the path from the 3-D position qt-1 to qt along the blood vessel center lines. If there is no link between qt-1 and qt, then the distance is infinite and thus p (qt|qt-1) practically 0.
The “recursive component” is the knowledge about the 3-D position that was available at the point in time t-1. This was also calculated according to equation (2). In the end, the equation (2) reduces recursively until the point in time t=0 has been reached. The system is initialized with equation (1) in this initial point in time.
Non-probabilistic modelings in terms of an optimization problem can also be realized for equations (1) to (3). Therefore, the method does not depend on the mathematical formulation but on the fact that one integrates new information recursively into the previous knowledge about the 3-D position and at the same time allows information to flow in regarding which 3-D positions are at all plausible, as was explained using the example according to
Visualization of the 3-D Position
In the 3-D volumetric data set, each connected area whose probability exceeds a certain threshold value according to equation (2) is suitably highlighted or one takes, for example, the n-best 3-D points. Thereby, the 3-D volumetric data set can either be represented together with a reconstructed visualization of the medical instrument, or color or graphics are used to highlight the corresponding areas. The representation of the points C1 to C9 using circles is a very simple graphic representation possibility. Other kinds of visualization are also possible.
In summary the method is essentially characterized by the following procedure steps described using FIG.11:
The inventive method produces the advantages stated below:
The essential in the method according to the invention is that the 3D position of the medical instrument is determined
Thereby recursive information means the chronological component. After a sequence of images has been created, recursive means that one integrates the information from the latest image into the previous knowledge of the position of the instrument. The advantage thereby is that with the “recursive approach” all that is required is the previous knowledge about the position and the current image. If no recursive information were used, then all the previous images would have to be stored. Then when a new image is available, said image would have to be used together with all the old images for determining the position. This approach is not practicable due to the memory requirements and the fact that the complexity of the computation increases with the number of images. With the recursive approach is ensured that a Markov process is modeled. That means that only our previous results (knowledge about the 3D position) are required and the new observations (new image with, for example, tip of the catheter located). All previous, old images are no longer required.
Furthermore, “plausible 3D positions” refer to the context information. Since all information on depth is missing from the images that are (never) at an angle to each other, which information also cannot be gained through the chronological components, information about depth must stem principally from another source. That is defmed by the “plausible 3D position”. Thereby it is assumed that the appliance is in a vessel. That means that one knows that for a point of the 2D image which shows the instrument, the only possible positions are those that are on the straight line and intersect a blood vessel. That means that of all the possible 3D positions on the straight line, only those that are within a blood vessel are plausible.
If one looks only at the context information in one single image, a large number of ambiguities will occur, as the straight line may possibly intersect many different blood vessels. If one looks only at the chronological sequence of images (without context information) then the depth is missing. Not until one combines both is a good 3D position estimation provided over time.
Number | Date | Country | Kind |
---|---|---|---|
10 2005 039 657.7 | Aug 2005 | DE | national |
This application claims priority of German application No. 10 2005 039 657.7 filed Aug. 22, 2005, which is incorporated by reference herein in its entirety.