Patent Application
20080199048
This invention relates generally to an image processing system and method for alignment of a sequence of images with a previously obtained sequence of images in respect of the same body volume, and more particularly, to such a system and method for effecting such image alignment with compensation for motion of said body volume between said images.
The imaging of body volumes is practiced notably in the field of medical diagnostics and therapy, that is, in the context of X-ray fluoroscopy. Therefore, the X-ray projection of a biological body volume will be considered hereinafter by way of example, but the present invention is not intended to be restricted thereto and can be used in all fields of application with similar secondary conditions.
In treatment of coronary heart disease, both pre-interventional and interventional X-ray images are used. In pre-interventional coronary angiograms, a radio-opaque contrast agent injected in the coronary artery is used to make the respective artery visible. A number of angiograms are recorded and serve for diagnosis of, for example, stenoses, and as roadmaps for the subsequent X-ray controlled intervention. A special medical application in treatment of coronary heart disease is provided by the fluoroscopic observation of the propagation of a catheter in the vascular system of the patient. Thus, during intervention, a catheter or guidewire is advanced under X-ray surveillance (fluoroscopy) through the vessels to the lesion. The tip of the catheter must be advanced as accurately as possible into a region of interest to be treated or examined, for example, a stenosis, or a guidewire should be positioned behind the region of interest in such a manner that the tip of the catheter is correctly positioned. While this procedure is performed, the vessel structures are made visible for short periods of time by introducing short bursts of contrast agent through the catheter.
In order to assist navigation, it is known to display a manually selected one of the above-mentioned pre-interventional coronary angiograms of the body volume on a second monitor adjacent the current image of the body volume. The selected angiographic image supports the orientation for the attendant physician as a “vascular map” or “road map” of the vascular system. This roadmap, however, is naturally static, and is hence not suitable to achieve a desirable exact alignment of corresponding locations in the angiographic image and the current image with an accuracy in the millimetre or sub-millimetre range, because the body volume of a patient being observed is subject to motion which is caused, notably, by heartbeat and respiration. In other words, because the above-mentioned roadmap is static, it is not consistent with the instantaneous heartbeat and respiration movements in the fluoroscopic images.
In order to improve guidance during catheter placement, methods have been developed to overlay motion compensated roadmap information from the angiograms on the fluoroscopic images, as described in, for example, WO2004/034329. This overlaying is performed under the constraints of a latency budget and is therefore limited regarding the complexity of the chosen algorithms.
The choice of angiograms used in these known methods depends on the respiration status and the contraction status of the patient (and on the level of contrast agent filling). While the contraction status of fluoroscopies and angiographies can be readily compared by analysing the electrocardiographs (ECGs) of both sequences, the important registration of the respiration status is not so straightforward. Currently, both the angiographies and the fluoroscopies are compared to a reference angiography that shows the patient in either complete inhaled or exhaled state.
The similarity values of fluoroscopies and the angiographic reference frame cannot be compared directly to the similarity values of angiographies and the reference angiography, since the fluoroscopies are inherently less similar to the (angiographic) reference frame than other angiographies are. Therefore, the similarity value of any incoming frame is compared to the similarity span of the fluoroscopies and a respiration phase is calculated from this proportion. The angiographic frames are processed in similar ways: for each angiography, the similarity with the reference frame is calculated. From this similarity value and the span of occurring similarity values, the respiration phase is calculated. The alignment is then done by pairing fluoroscopies and angiographies with similar respiration phases.
One problem associated with this method is that it requires that both the angiographic sequence and the fluoroscopic sequence show approximately the same respiratory span. Typically, small changes in respiration depth are compensated by using a sliding max/min window for the estimation of the fluoroscopic respiration span, but with this method it is not possible to detect frames that have no matching angiographic respiration state because they are outside the angiographic breathing span, nor is it possible to cope with systematically deviating respiration depths in angiographies or fluoroscopies.
It is therefore an object of the present invention to address the above issue and provide an improved image processing system and method.
In accordance with the present invention, there is provided an image processing system comprising an input for receiving data representative of a current image of a body volume, said body volume being subject to a motion cycle comprising several phases between first and second extreme phases of motion, means for receiving data representative of the phase of motion of said body volume in said current image, storage means in which is stored a plurality of previously-obtained images of said body volume together with data representative of the respective phase of motion of said body volume in each image, means for selecting at least one of said previously-obtained images of said body volume having substantially the same phase of motion as that of said current image; wherein said data representative of said phase of motion of said body volume is determined by providing first and second static images of said body volume at respective said first and second extreme phases of motion, comparing an image under consideration with said first static image and generating a first value representative of its similarity thereto, comparing said image under consideration with said second static image and generating a second value representative of its similarity thereto, said first and second values together being representative of said phase of motion of said body volume captured in said image under consideration.
Beneficially, the system further comprises means for aligning said selected image with said current image, and preferably comprises means for superposing said selected image on said current image.
Thus, with the proposed system, instead of one angiographic reference frame, two angiographic reference frames are used that show the extreme inhaled and exhaled condition of the patient as recorded by angiographies. During fluoroscopy, each acquired angiography is compared to both reference frames. The result pair gives the fractional respiration position of the fluoroscopic frame in between the two angiographic reference frames. Thereby, the estimation of the matching angiographies for an incoming fluoroscopic frame is done more robustly than with the single-reference-frame method.
As an additional feature, with the two-reference-frame-method it is possible to detect fluoroscopic frames that are outside of the angiographic breathing span:
First the respiratory similarities of an incoming fluoroscopy image are calculated. These values are now compared with the corresponding values of the previous fluoroscopy frame. Under normal conditions, by stepping from the first fluoroscopy to the second fluoroscopy the similarity to reference frame A is increasing while the similarity to reference frame B is decreasing or vice versa. Only when the second fluoroscopy is outside the span of the angiographic respiration states, the similarity to both angiographic reference frames A and B is at the same time either increasing or decreasing. In this case, appropriate measures can be taken: for instance the assignment of angiographies and fluoroscopes can be suspended until the fluoroscopic sequence again enters the angiographic respiration span. Alternatively, the angiographies now can be extrapolated using a model of the influence of respiration on the body, especially the heart.
Thus, means may be provided for determining if the phase motion of said body volume in said current image falls outside one of said extreme phases of motion. In one exemplary embodiment, the system may comprise an input for receiving a temporal sequence of current images of said body volume, wherein said phase of motion of said body volume in one of said current images is determined to fall outside one of said extreme phases of motion if said first and second values relating to said current image indicates that the similarity thereof to both said first and second static images at respective said first and second extreme phases of motion is either increasing or decreasing relative to the first and second values relating to the image immediately preceding said current image in said sequence. If the phase of motion of said body volume in a current image is determined to fall outside one of said extreme phases of motion, said selection of one or more of said previously-obtained images may be interrupted until the phase of motion of said body volume in a subsequent image in said sequence is determined to fall between said first and second extreme phases of motion. Alternatively, if the phase of motion of said body volume in a current image is determined to fall outside an extreme phase of motion, the static image of said body volume at said extreme phase of motion may be extrapolated using a predetermined model defining the influence of one or more parameters on said motion.
The body volume may be a biological body volume and motion of said body volume may be caused by heartbeat and/or respiration. The phase of motion may be detected by means of an electrocardiogram.
The present invention extends to a medical imaging apparatus, comprising means for capturing images of a body volume and an image processing system as defined above; and further X-ray apparatus including an image processing system as defined above.
Also in accordance with the present invention, there is provided a method of identifying in respect of a current image of a body volume one or more previously-obtained images of said body volume to be associated therewith, the method comprising receiving data representative of a current image of a body volume, said body volume being subject to motion cycle comprising several phases between first and second extreme phases of motion, receiving data representative the phase of motion of said body volume in said current image, and selecting from a plurality of previously-obtained images of said body volume at least one of said previously-obtained images of said body volume having substantially the same phase of motion as that of said current image; wherein said data representative of said phase of motion of said body volume is determined by providing first and second static images of said body volume at respective said first and second extreme phases of motion, comparing an image under consideration with said first static image and generating a first value representative of its similarity thereto, comparing said image under consideration with said second static image and generating a second value representative of its similarity thereto, said first and second values together being representative of said phase of motion of said body volume captured in said image under consideration.
Still further in accordance with the present invention, there is provided apparatus for generating data representative of a phase of motion of a body volume captured in an image frame, said body volume being subject to motion of several phases between first and second extreme phases of motion, the method comprising providing first and second static images of said body volume at respective said first and second extreme phases of motion, comparing said image frame with said first static image and generating a first value representative of its similarity thereto, comparing said image frame with said second static image and generating a second value representative of its similarity thereto, said first and second values together being representative of said phase of motion of said body volume captured in said image frame; and a method of generating data representative of a phase of motion of a body volume captured in an image frame, said body volume being subject to motion of several phases between first and second extreme phases of motion, the method comprising providing first and second static images of said body volume at respective said first and second extreme phases of motion, comparing said image frame with said first static image and generating a first value representative of its similarity thereto, comparing said image frame with said second static image and generating a second value representative of its similarity thereto, said first and second values together being representative of said phase of motion of said body volume captured in said image frame.
An embodiment of the present invention will now be described by way of example only and with reference to the accompanying drawings, in which:
The invention will be described in detail hereinafter on the basis of a medical application, although the invention is not intended to be restricted in any way to this field.
Referring to
The image processing system 5 comprises a memory 4 in which previous images of the body volume of the patient are stored. Such images may notably be angiographic images which have been acquired by means of the X-ray apparatus 1, 3 while utilising a radio-opaque contrast medium and which represent the vascular tree in the body volume in highlighted form. However, the previous images may also be buffered images or image sequences concerning the current intervention which have been acquired by means of the X-ray apparatus 1, 3. Images of this kind can reproduce in particular the position of an instrument, such as that of a catheter which has been introduced into the vascular system of the patient and has a catheter tip, or of a guide wire.
The image processing system 5 is also coupled to (at least) two monitors 6, 7 and is arranged to display the current image 8 “live” on both monitors 6, 7 and to display on the monitor, superposed thereon, one of the previous images derived from the memory 4. The parallel (superposed or separate) display of a previous image serves to facilitate the navigation of the instrument in the vascular tree of the patient 2 by the physician. For example, a previous angiographic image offers a sort of vascular map (“road map”), or a previous image of the same medical intervention shows, for example, the position of a stenosis dilated by a bulb catheter or the position of a previously-placed stent. In the latter cases, the previous image assists the physician in repositioning the instrument to a previously adopted location.
For the previous image to be useful, it is important that the position of the organs and vessels represented therein corresponds as accurately as possible to the situation in the current image. For given applications, and in the case of superposed reproduction of the current and previous images, a precision in the range of a millimetre or even sub-millimetre is required.
The above precision is achieved in accordance with this exemplary emboidment of the present invention in that the natural motion of the body volume itself, caused by the heartbeat and/or respiration, is taken into account for associating a previous image from the memory 4 with the current image 8.
In order to enable the heartbeat to be taken into account, the ECG over the duration of at least one heartbeat, during which the previous image was generated, as well as the instant of acquisition relative to the ECG are also stored in the memory 4, together with respective previous images. The respiratory phase domain starts at phase 0 corresponding to the fully exhaled phase, passes phase π corresponding to the fully inhaled phase and ends at phase 2π, which again corresponds to the fully exhaled phase. Thus, mapping the entire ECG cycle on the interval [0, 2π] enables the acquisition instant to be expressed as a value from this interval which reflects the heartbeat position of a previous image and subsequently serves as a first index of the previous image.
In order to enable the respiration to be taken into account, moreover, a second index is provided for the previous images; this second index reflects their relative position in the respiratory cycle. The second index is also typically normalised to the interval [0, 2π]
In accordance with this exemplary embodiment of the present invention, the second index is acquired by way of a similarity comparison of the previous images with two reference images R1 and R2 which which the body volume under consideration is in respective first and second extreme instants of the respiratory cycle, i.e. “deep inhalation” and “deep exhalation”. The second index of any given previous image then indicates its similarity distance relative to the reference images R1 and R2 and thus reflects the relative position in the respiratory cycle.
The reference images R1, R2 themselves may have been selected from the previous images. In order to find an image defining an extreme phase of respiration from among the previous images, for each previous image its similarity measure relative to series of sequential images can be calculated experimentally. If these similarity measures change, for example, periodically with approximately double the respiratory frequency, the experimentally considered previous image will be an image from a central phase of the respiratory cycle; however, if the similarity measures change periodically with approximately the single respiratory frequency, the previous image considered belongs to an extreme phase of the respiration so that it is suitable for use as a reference image. In fact, for the purposes of this exemplary embodiment of the present invention, it is not necessary to know which of the reference images corresponds to extreme inhalation and which corresponds to extreme exhalation—it is sufficient that the two extreme states are identified as such.
An exemplary method of selecting a previous image to be associated with a current image will now be described in detail.
A respective previous image 10a, 10b, . . . is superposed on the live images on the monitor 7 of
First, those images which have approximately the same similarity gap in respect of the respiratory cycle as the current image 8, relative to the predetermined reference images R1, R2, are selected from the memory 4. To this end, the current image 8 is compared with each reference image R1, R2 such that respective similarity measures r1, r2 can be calculated.
Analogously, the similarity measures can be calculated between the reference images R1, R2 and all previous images present in the memory 4. The latter has to be performed once for a given quantity of previous images, because the measures do not change. As previously stated, the pairs of similarity measures may be normalised and added as a (respiration) index to the stored previous images. Therefore, the amount of calculation work required during operation is comparitively small.
Using the similarity measures, or indices, a sub-quantity U of the previous images can be determined, whose members have approximately the same degree of similarity with respect to the reference images R1, R2 as the current image 8. This means that the similarity measures of these images lie, for example, within a window (r1±Δ),(r2±Δ). If the memory 4 does not contain any image that satisfies this condition, the selection method may be interrupted at this point.
However, if the sub-quantity U contains at least one element, a second selection in respect of the respiratory cycle is carried out in a second step. The previous images in the sub-quantity U are then individually compared with the current image 8, that is, the associated similarity pairs r1′,r2′ are calculated and a sub-quantity V≦U is determined whose images exceed a limit value in respect of the similarity to the current image 8. The calculation-intensive individual comparison with the image 8 is minimised by the pre-selection of the quantity U.
Finally, in order to take into account the heartbeat, in a third step the previous image 10a, whose relative instant in the ECG is closest to the relative ECG instant of the current image 8, is selected from the sub-quantity V. For comparison between the electrocardiograms of the current image 8 and a previous image, a transformation is determined, for example, by means of a dynamic programming algorithm; this transformation maps the electrocardiograms on one another in an optimum fashion, thus enabling an exact prediction of the phase differences between the striking features (R, S, T lobes) of the electrocardiograms.
In a post-processing step (not shown) subsequent to the selection process, an estimate of motion and a correction between the selected previous image 10a and the current image 8 can be carried out so as to compensate for changes due to a (whole body) motion of the patient.
For the above exemplary method, the histogram energy of the image differences is a suitable exemplary measure of similarity between two images. The images to be compared are then subtracted from one another one pixel after the other and the histogram of the is difference image is calculated. This process is performed to compare the first reference image R1 with each of the previous images stored in the memory 4 and to compare the second reference image R2 with each of the previous images stored in the memory 4. Each resultant histogram indicates how many pixels n(G) of the difference image have each time a given grey scale value G. The similarity measure (r1,r2) can then be calculated as the respective histogram energy which is, by definition the square sum of all the pixels:
This definition means that histograms with a concentration of greyscale values have a high histogram energy, whereas histograms with as uniform as possible distribution of greyscale values over all pixels have a low histogram energy. The similarity measure according to this definition, therefore, has a small numerical value when the compared images have a high degree of similarity; and vice versa. A person skilled in the art can readily define alternative similarity measures on the basis of, for example, local correlation, cross-correlation or “mutual information” techniques.
The selected previous image 10a can be displayed on a monitor either separately or superposed on the current image 8.
Thus, with the proposed system, instead of one angiographic reference frame, two angiographic reference frames are used that show the extreme inhaled and exhaled condition of the patient as recorded by angiographies. During fluoroscopy, each acquired angiography is compared to both reference frames. The result pair gives the fractional respiration position of the fluoroscopic frame in between the two angiographic reference frames. Thereby, the estimation of the matching angiographies for an incoming fluoroscopic frame is done more robustly than with the single-reference-frame method.
As an additional feature, with the two-reference-frame-method it is possible to detect fluoroscopic frames that are outside of the angiographic breathing span:
First the respiratory similarities of an incoming fluoroscopy image are calculated. These values are now compared with the corresponding values of the previous fluoroscopy frame. Under normal conditions, by stepping from the first fluoroscopy to the second fluoroscopy the similarity to reference frame A is increasing while the similarity to reference frame B is decreasing or vice versa. Only when the second fluoroscopy is outside the span of the angiographic respiration states, the similarity to both angiographic reference frames A and B is at the same time either increasing or decreasing. In this case, appropriate measures can be taken: for instance the assignment of angiographies and fluoroscopes can be suspended until the fluoroscopic sequence again enters the angiographic respiration span. Alternatively, the angiographies now can be extrapolated using a model of the influence of respiration on the body, especially the heart. Possible options in this regard, include:
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be capable of designing many alternative embodiments without departing from the scope of the invention as defined by the appended claims. In the claims, any reference signs placed in parentheses shall not be construed as limiting the claims. The word “comprising” and “comprises”, and the like, does not exclude the presence of elements or steps other than those listed in any claim or the specification as a whole. The singular reference of an element does not exclude the plural reference of such elements and vice-versa. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
| Number | Date | Country | Kind |
|---|---|---|---|
| 05300045.1 | Jan 2005 | EP | regional |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/IB2006/050170 | 1/17/2006 | WO | 00 | 4/30/2008 |
