Patent Application
20100209012
The present invention relates to imaging techniques for imaging moving structures of interest and, more particularly, to techniques for enhancement of discrete pixel images in an image sequence comprising a moving structure, such as those produced in medical imaging systems. Further, the invention may be used by an imaging system for Percutanerous Coronary Intervention (PCI) in catheter laboratories, to image cardiac stenosis, or during x-ray operation, e.g. angiography where potential stenosis may assessed.
In enhancing processes, images are registered with respect to a moving structure of interest as a as devices, e.g. stents biopsy needles, cardiac valves, catheter tips or leads, etc and then temporally integrated. This procedure can be extended to the boosting of anatomy parts, wherein e.g. a stenosis forms said structure of interest.
However, the visualisation requirements for moving structures as stenosis are much more constraining than for others, regarding to a clear view of the surrounding parts of said structure, the preservation of the natural structure deformation, and the selection of an optimally boosted image. A good visualization of such structures is mandatory because their grading, either visual or automatic, may directly impacts a treatment decision. The contrast of the stenosis is not always very high even after contrast-agent injection and the stenosis is submitted to large movements, both cardiac and respiratory. Of course, the actual grading of the stenosis is usually made on a static image, which may be selected. This suppresses the motion difficulty, but this also masks the dynamic local behaviour of the lesion which might influence diagnostic.
A device boosting technique, as presented in “Registration and Integration for Fluoroscopy Device Enhancement” James C. Ross, David Langan, RaviManjeshwar, John Kaufhold, Joseph Manak, and David Wilson. Miccai 2005, which is herewith incorporated by reference, can be used to improve a lesion visibility by temporally averaging the motion-compensated stenosis images. This drastically decreases the noise level, while homogenizing the contrast agent variations.
Applying the traditional device-boosting technique to (for instance) a stenosis is a priori valuable concerning noise reduction or contrast agent homogenizing, but may also creates several severe problems due to the fact that, when it comes to anatomy assessment, the visualisation requirements are much more constraining than for devices e.g. stents for the following reasons.
The stenosis is not isolated but part of a vessel tree, including many bifurcations and side-branches. It is important to keep a decent visualisation of those surrounding vessels because they may play a part in the pathology evaluation. The so-called device boosting technique has precisely the property of blurring the background while improving the visibility of the moving device. Applied to the stenosis, this may leads to a strong blurring of the surrounding vessels, which might constitute a very strong problem for the diagnostic integrity.
Likewise, the local deformation of the lesion is also to be taken into account when assessing a stenotic situation. Again, the traditional device-boosting technique leads to the freezing of that deformation, thus potentially impairing diagnostic.
Finally, because the contrast-agent appearance strongly varies during the acquisition, and because the stenosis registration process is bound to make some errors for some frames, it might be penalizing to rely on all the stenosis-boosting images, since that some images might be of lesser quality, including those built during poor contrast periods, or impacted by transitory registration errors.
According to the aforesaid problems and limitations, it might be an object of the present invention to solve at least partly some of the issues presented above.
The present invention provides a technique for enhancing digital pixel images designed to respond to these needs.
An exemplary embodiment of the invention provides a method for enhancing a moving structure of interest in a sequence of images, wherein images of the sequence are captured at different times and defined by a matrix of discrete pixels. The method comprising the steps of generating data representative of pixels defining a first sequence of images, the sequence comprising a plurality of images I(t) each captured at a different time t with an image I(t0) captured at reference time t0, generating data representative of a displacement V(t→t0) for pixels of the structure of interest between the images I(t) and the image I(t0) of the first sequence of images, warping the data representative of pixels defining the images I(t) by using the data representative of the displacements V(t→t0) to obtain data representative of pixels defining a second sequence of images A(t), applying an enhancement operation to images of the second sequence A(t) to obtain data representative of pixels defining a third sequence of images B(t), selecting data representative of pixels defining at least one image B(s) from the third sequence of images B(t), generating data representative of a reverse displacement V(t0→t) for pixels of the selected structure of interest between the image I(t0) captured at reference time t0 and each image I(t) of the sequence of images captured at time t, and warping of the at least one image B(s) using the data representative of the reverse displacement V(t0→t) to obtain data representative of pixels defining a fourth sequence of images E(t).
Moreover, an exemplary embodiment of the invention provides an imaging system for enhancing a moving structure of interest in a sequence of images, wherein images of the sequence are captured at different times and defined by a matrix of discrete pixels. The imaging system comprises a data acquisition unit configured to generate data representative of pixels defining a first sequence of images, the sequence comprising a plurality of images I(t) each captured at a different time t with an image I(t0) captured at reference time t0 and a signal processing circuit configured to execute the above mentioned steps.
Referring to the claims a method and system for enhancing of a moving target-object as a structure of interest, for instance a stenosis, is proposed that includes a boosting treatment in such way that the target-object is temporally boosted, wherein the surrounding visualisation, as side-branches in the case of the stenosis, or the local deformation of the target-object may kept intact (bending in the case of the stenosis).
In accordance with certain aspects of the method as claimed in claim 8, the global motion is compensated, thus offering stabilisation and zoom possibilities as claimed in claim 12.
Further, according to certain technical aspects of the claims at least one optimized boosted object view B(s) is selectable, manually or automatically, thus may excluding lesser quality images which otherwise may be present in a boosted sequence.
An essential feature of one exemplary consists in creating a result sequence R(t) in which at least one optimally boosted object image B(s) is first computed, and then inlayed in a non-boosted sequence I(t), wherein the natural motion of the object is kept intact, but with an optional global registration that compensates for the overall motion of the object, e.g. a stenosis.
The technique has been design for the optimal view of stenosis, but it can be extended to other moving anatomy parts or devices, in all the situations where temporal boosting may improves the visibility of the target-object, in particular in at least one image, while the requirement to keep both the deformation of the target-object as the structure of interest and the visibility of the surrounding is important. Thus, possible applications for the invention are biopsy needles, cardiac valves, catheter tips or leads.
In a further exemplary embodiment of the invention, the enhancement operation is selected from a group comprising an operation using temporal integration of at least two images of the second sequence of images A(t) and an operation using a spatial enhancement technique. Enhancement operations are disclosed e.g. in “Image Enhancement in Digital X-Ray Angiography, Eric Meijering, 2000, Ponsen & Looijen, Wageningen which is herewith integrated by reference.
In a further exemplary embodiment of the invention, the method further comprising the step of segmenting the structure of interest in every image of the first sequence of images to deriving data representative of a sequence of mask images F(t).
In yet another exemplary embodiment of the invention, the data representative of the displacement V(t→t0) and/or the reverse displacement V(t0→t) are generated using data representative of a sequence of mask images F(t).
In yet another exemplary embodiment of the invention, the data representative of the sequence of mask images F(t) comprising pixel values that are representative for the probability for pixels of said pixel values to belong to the structure of interest.
In yet another exemplary embodiment of the invention, the method further comprising the step of combining of data representative of the first sequence of images and of data representative of the fourth sequence of images E(t) to obtain data representative of pixels defining a fifth sequence of images M(t).
In yet another exemplary embodiment of the invention, the merging is performed by using the data representative of the sequence of mask images F(t).
In yet another exemplary embodiment of the invention, the method further comprising the step of applying a geometrical transformation, precisely, a global geometrical transformation, to data representative of pixels defining the structure of interest in images of one of the sequences to obtain data G(t), wherein the geometrical transformation is applied in order to compensate a global motion of the structure of interest.
In yet another exemplary embodiment of the invention, the geometrical transformation is performed by using data representative of the sequence of mask images F(t).
In yet still another exemplary embodiment of the invention, a geometric barycentre of the structure of interest is generated from data representative of the sequence of mask images F(t). The said geometric barycentre is preferably used to define the global geometrical transformation G(t).
In yet another exemplary embodiment of the invention, the method further comprising the step of applying data G(t) to data representative of pixels defining the fifth sequence of images M(t) in order to obtain data representative of a final sequence of images R(t).
In yet another exemplary embodiment of the invention, the method further comprising the step of applying a zoom function to data representative of a final sequence of images R(t).
In yet another exemplary embodiment of the invention, the method further comprising the step of displaying at least one sequence of images of a group comprising:
I(t), A(t), B(t), E(t), M(t) F(t) and R(t).
In yet another exemplary embodiment of the invention, the first sequence of images is acquired via a digital x-ray imaging system.
Aspects defined above and further aspects of the invention are apparent from the examples of embodiment to be described hereinafter and are explained with reference to the examples of embodiment.
The invention will be described in more detail hereinafter with reference to examples of embodiment but to which the invention is not limited.
According to
The block diagram illustrates method steps for a method, executable by an imaging system 100, a computer readable medium 200, or of a program element 300 for enhancing a moving structure of interest in a sequence of images. The method comprises the steps of
First the target-object must be somehow designated in one image It0 (at reference time t0) (step 10). For any sequence S of images S(t) indexed by time t, it is defined that St refers to the image S(t). The said designation can be carried out through touch-screen pointing or via any other pointing devices, but it can also be automatic. For instance, in the case of the stenosis, automatic designation can be achieved through the detection of the contrast agent arrival at the location of a device, itself in the vicinity of the lesion.
Thanks to the above designation, the segmentation (possibly fuzzy) of the target-object is computed. Any segmentation method is possible. This leads to the creation of a fuzzy mask 12 of the target-object, where each pixel value is representative of the probability of this pixel to belong to the target-object. In case of non-fuzzy segmentation, only the probability values 0 and 1 are possible. This step is applied to every image t, producing a fuzzy-mask Ft. Of course, tracking techniques can be involved to deduce Ft from the previous masks.
c) Target-Object Motion Estimation from t to t0
The motion field linking the target-object at time t to the same object at time t0 is computed in step 14. Any motion estimation method can be used for that task. It can for instance rely on the computed fuzzy masks Ft and Ft0, (12), (dashed arrow 18), but it can also directly be estimated form the images It (16) and It0. This creates a vector field V(t→t0).
d) Warping of It from V(t→t0)
Image It is warped (step 20) towards reference time t0 thanks to the computed field V(t→t0). This produces a series of image At. In case of complex motions, such as a bending stenosis, elastic warping is needed.
The images At are boosted (step 22) into a sequence Bt. This boosting operation usually involves temporal integration (using a plurality of images At such as At1, At2, At3 for Bt1 and At2, At3, At4 for Bt4 and so forth) but it might also depend on spatial enhancement techniques (e.g. high-frequency enhancement). In particular, combining temporal integration and edge enhancement is a good way to reach strong noise reduction without excessive contour blurring (due to imperfect registration prior temporal integration).
Not all the boosted frames in sequence Bt are necessarily of good quality. Some of them might be based on the integration of poorly contrasted images, others might be based on badly registered images (incorrect V(t→t0)). This is why, in step 24, a selection of the best boosted images is achieved. This selection can be performed manually, but it can also rely on automatic measurements (contrast, registration confidence, etc). The selection result is the images Bs. At least one image Bs is selected.
The inverse motion field linking the target-object at time t0 to time t is evaluated in step 26. This can be based on the simple inversion of the direct field V(t→t0) (dashed arrow 28), or this can be achieved as in the case of the direct estimation procedure (relying on the images It (arrow 30), and/or on the fuzzy masks Ft (arrow 32).
h) Warping of Bs with V(t0→t)
In step 34, the selected boosted image is warped back to the location of the target-object at time t thanks to vector field V(t0→t). This creates the sequence Et whose gray-level content is only constituted from Bs values (however warped to match the moving target-object location at time t).
The content of both It and Et are merged/combined thanks to the fuzzy mask Ft in step 36. Basically, where Ft indicates a high probability of presence of the target-object, image Et predominates in the merging, and in the opposite situation, It predominates. For every pixel site x, this can be achieved by:
Mt(x)=Ft(x)*Et(x)+(1−Ft(x))*It(x)
After merging, data of the computed sequence Mt contains both the optimal boosted view(s) of the target-object, together with the non-boosted background (keeping intact the bifurcations). In addition, the natural deformations of the target-object are also preserved.
In order to compensate for the target-object global motion (not its deformation), and in order to comply with a zooming operation, a global geometrical transform is estimated in step 38. For instance, the barycentre of Ft is computed and the translation that compensates the motion of this barycentre between t and t0 is incorporated to the geometrical transform, referred to as Gt.
Gt is applied to Mt in step 40 to produce the final result sequence Rt. In this sequence, a zoom is applied and the global motions of the target-object are compensated for. But Rt retains the natural motion of the target-object visualised in its optimal boosted version, and the background is preserved, including branching vessels.
Obviously for the skilled person, the final global registration and/or zooming are optional.
Additionally, instead of choosing only one optimal boosted view Bs, a sequence part Bj can be selected from Bt. In that case, an association procedure selecting for every image Bt its counterpart image Bj has to be defined (for instance based on the ECG, or on the respective motion content of Bt and Bj). This allows the inlaying of an optimally boosted sequence part in the final result Rt. In fact, the sequence Bj can range from a single image (for every j, j=constant=s) to the full sequence Bt (j=t).
The visualisation result (selected boosted interval warped back to the current frame, with background preservation, and optional global compensation and zoom) can be displayed by a display device, not shown here.
The shown method steps may aim to improve the visibility of stenosis and its grading. The method contributes to make the procedures quicker and safer.
The method described above can be extended to any moving anatomy parts or devices, in all the situations where temporal boosting improves the visibility of the target-object, in particular in at least one image, while the requirement to keep both the deformation of the target-object and the visibility of the surrounding is important. Possible applications: biopsy needles, cardiac valves, catheter tips or leads, etc.
In a further embodiment shown in
After the processing steps as claimed above, another final picture sequence R(t), represented by the two images R1 and R20 in the lower half of
It should be noted that the term “comprising” does not exclude other elements or steps and the “a” or “an” does not exclude a plurality. Also elements described in association with different embodiments may be combined.
It should also be noted that reference signs in the claims shall not be construed as limiting the scope of the claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 07116978.3 | Sep 2007 | EP | regional |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/IB08/53749 | 9/16/2008 | WO | 00 | 3/22/2010 |
