Patent Application
20140031676
The present invention relates to an image processing device for guidance support, a medical imaging system for providing guidance support, a method for guidance support, a method for operating an image processing device for guidance support, as well as a computer program element, and a computer readable medium.
Guidance support can be provided, for example, to a surgeon during an interventional procedure, such as an examination or operation of a patient. One example for an interventional procedure is the placing of a stent in a so-called minimal invasive procedure. In order to provide the surgeon with information about the current situation, which is, needless to say, generally not directly visible for the surgeon himself, image data of a region of interest of an object, for example of a region of a patient, is provided on a display. For example, US 2010/0061603 A1 describes the acquisition of 2D live images, which are combined with pre-operationally 3D image data and displayed as image composition to a user. It has been shown that the information generated prior to the operation may deviate from the current situation. It has been shown further that for providing improved image information about the current situation, the user has to rely on the acquired image data.
It is an object of the present invention to provide enhanced and easily perceptible information about the actual situation.
The object of the present invention is solved by the subject-matter of the independent claims, wherein further embodiments are incorporated in the dependent claims.
It should be noted that the following described aspects of the invention apply also for the image processing device, the medical imaging system, the method for guidance support, the method for operating an image processing device for guidance support, the computer program element, and the computer readable medium.
According to an aspect of the invention, an image processing device for guidance support is provided, comprising a processing unit, an input unit, and an output unit. The input unit is adapted to provide 3D data of a region of interest of an object, and to provide image data of at least a part of the region of interest, wherein a device is arranged at least partly within the region of interest. The processing unit comprises a generation unit to generate a 3D model of the device from the image data. The processing unit comprises an embedding unit to embed the 3D model within the 3D data. The output unit is adapted to provide a model-updated 3D image with the embedded 3D model.
According to the present invention, the term “guidance support” refers to providing information to a user, for example a surgeon or an interventional radiologist which supports, helps or facilitates any intervention where a device or other equipment or part has to be moved or steered inside a volume while it Is not directly visible to the user. The “guidance support” can be any type of information providing a better understanding about the current situation, preferably by visible information.
According to an exemplary embodiment, the image data comprises at least one 2D image and the generation unit is adapted to generate the 3D model from the at least one 2D image. The generation unit is adapted to generate a 3D representation of the region of interest from the 3D data, and the processing unit is adapted to embed the 3D model within the 3D representation.
According to a further aspect of the invention, a medical imaging system for providing guidance support is provided, comprising an image acquisition arrangement, a display unit, and an image processing device according to the above mentioned aspect and exemplary embodiment. The image acquisition arrangement is adapted to acquire the image data and to provide the data to the processing unit. The output unit is adapted to provide the model-updated 3D image to the display unit, and the display unit is adapted to display the model-updated 3D image.
According to an exemplary embodiment, the image acquisition arrangement is an X-ray imaging arrangement with an X-ray source and an X-ray detector. The X-ray imaging arrangement is adapted to provide 2D X-ray images as image data.
According to a further aspect of the invention, a method for guidance support is provided, comprising the following steps:
a) providing 3D data of a region of interest of an object;
b) providing image data of at least a part of the region of interest, wherein a device is located at least partly within the region of interest;
c) generating a 3D model of the device from the image data; and
d) providing data for a model-updated 3D image by embedding the 3D model within the 3D data.
According to an exemplary embodiment of the invention, a spatial relationship between the 3D model and the 3D data is predetermined, and for the embedding, the 3D model is adjusted accordingly.
According to a further exemplary embodiment of the invention, predetermined features of the device and/or the object are detected in the model-updated 3D image.
For example, the predetermined features are highlighted in the model-updated 3D image.
For example, measurement data of the detected features in relation to the object is determined and the measurement data is provided to define and/or adapt a steering or guiding strategy of an intervention.
According to a further aspect of the invention, a method for operating an image processing device for guidance support is provided, comprising the following steps:
providing 3D data of a region of interest of an object from an input unit to a processing unit;
providing image data of at least a part of the region of interest from the input unit to the processing unit, wherein a device is arranged at least partly within the region of interest;
generating a 3D model of the device from the image data by the processing unit; and
embedding the 3D model within the 3D data by the processing unit to provide a model-updated 3D image via an output unit.
It can be seen as the an aspect of the invention to take the image data, for example image data reflecting the current situation, such as live fluoroscopy images, as a basis for modeling the device itself. Thus, a model of the device is generated which is in exact congruence with the current situation, i.e. which represents the current situation. This so-to-speak live model is then shown in the context of 3D data in order to provide the user with easily perceptible and precise information about the current situation.
These and other aspects of the present invention will become apparent from and elucidated with reference to the embodiments described hereinafter.
Exemplary embodiments of the invention of the invention will be described in the following with reference to the following drawings.
For example, the 3D data 18 and the image data 20 can be provided to the input unit 14 from external sources, as indicated with respective dotted arrows 22 and 24. For example, the 3D data 18 can be provided from a storage unit, not further shown; the image data 20 can be provided from an image acquisition device, as will be explained with reference to
The processing unit 12 comprises a generation unit 26 to generate a 3D model 28 of the device from the image data 20. The processing unit 12 further comprises an embedding unit 30 to embed the 3D model 28 within the 3D data 18. Thus, data for a model-updated 3D image 32 with the embedded 3D model is achieved. The output unit 16 is adapted to provide the model-updated 3D image 32, for example to a further external component, as indicated with dotted arrow 34.
According to a further exemplary embodiment, shown in
The display unit 54 is shown with several display areas, which can be arranged as different monitors or also with different sub-areas of a larger monitor. The different sub-areas form a display area 70. The display unit 54 can be suspended from a ceiling via a display support structure 72, for example.
It must be noted that the X-ray imaging arrangement 56 is shown in form of a C-arm device as an example only. Of course, other imaging modalities can be provided, for example other movable arrangements, such as a CT with a gantry, or static imaging devices, for example those where the patient is arranged in a horizontal manner as well as those where the patient is in an upright standing position, such as mammography imaging devices.
According to a further example, although not shown, the image acquisition arrangement is provided as an ultrasonic image acquisition arrangement to provide ultrasonic images instead of X-ray images for the image data 20.
The medical imaging system 50 of
The medical imaging system 50 and the method described in the following can be used, for example, during endovascular surgery procedures, such as endovascular aneurism repair, which will be explained further below with reference to
When defining that the image data, for example live 2D image data, is having a device arranged at least partly within the region of interest, the device can be a stent, a catheter, or a guide-wire, for example, or any other interventional tool or endo-prosthesis. It is not necessary to fully arrange the device in the region of interest, but only a part of it as a minimum. This part has to be sufficient in order to be able to generate a model therefrom in three dimensions.
For example, the model of the device can be static. According to another example, it's a dynamic model. Of course, it is also possible to have a part of the model static and a part of the model dynamic, for example in case a part of the model relates to a (moving) guide-wire as a dynamic part and another part relates to an implant or prosthesis as a static part. However, it must be noted that moving relates primarily to the movement in relation to the object, but of course movement of the body or body parts, for example caused by breathing or heart beat related movements, can also be considered.
The first provision step 110 is also referred to as step a), the second provision step 114 as step b), the generation step 118 as step c), and the third provision step 122 as step d).
According to an exemplary embodiment, shown in
For example, the 3D data 112 is registered with the image data 116.
According to a further example, the 3D data 112 in step a) is also referred to as first image data, and the image data 116 in step b) is referred to as second image data.
For example, the image data 116 comprises at least one 2D image.
According to a further example, shown in
According to a further example, not further shown, the image data 20, or the so-to-speak second image data, comprises a set of live 2D images. Following, step c) comprises building or generating the 3D model from the set of live 2D images.
The model-updated 3D image 124 can be used as a steering guidance image.
Also with reference to
The 3D data or first image data may comprise pre-interventional image data. The image data 116 or second image data may comprise live images or intra-operational, or intra-interventional images.
Thus, it is possible to show current, i.e. actual, information about the situation, in combination with 3D data acquired or generated before the intervention. Thus, the 3D data can show enhanced visibility and improved perceptibility, whereas the image data 116 provides the current, i.e. live information.
As mentioned above, the 3D data may comprise X-ray CT image data, or MRI image data.
The image data 116 may be provided as 2D X-ray image data, since such image acquisition is possible with, for example, a C-arm structure with only minimally disturbing or influencing other interventional procedures.
For example, the image data 116 is provided as at least one fluoroscopic X-ray image. Preferably, at least two fluoroscopy X-ray images are acquired from different directions in order to facilitate the modeling of the device in step c).
According to an example, not further shown, following step d), a step e) is provided in which a 3D view of the reconstructed device within the 3D data is displayed to the user.
For example, as shown in
As will be explained also with reference to
For example, the device is an artificial heart valve in a deployed state.
According to a further exemplary embodiment, shown in
For example, the expected relationship can comprise the location within a vessel structure, for example when placing a stent inside a vessel tree. In such case, it can be assumed that the stent itself must be placed inside a vessel structure. Thus, if the embedding would result in a location of the model of the stent such that it would only be partly placed inside a vessel structure, or even next to or outside a vessel structure, it must be assumed that this is not reflecting the actual position, but is rather based on an incorrect spatial arrangement, for example an incorrect registration step. In such case, the expected relationship can be used to adapt or modify the positioning accordingly.
It must be noted that, in the drawing, the adjustment arrow 150 enters the embedding box 126. However, according to a further example, the adjusting arrow 150 can also be provided as entering the model generation box 118 of step c), which is not further shown. It is further noted that the predetermination 148 can also be provided in combination with the transformation as explained with reference to
According to a further example, shown in
According to a further exemplary embodiment, shown in
For example, the predetermined features 156 are highlighted in the model-updated 3D image 124, which is indicated, with highlighting arrow 160.
According to a further example, which can be provided alternatively or in addition to the highlighting 160 and which is shown also in
For example, the device is a first part of first stent body of a stent graft and a gate of the first part is detected and the position data of the gate is used for placing a second part of a stand graft such that the two parts sufficiently overlap, which will be explained with reference to
The 3D data 214 may be provided from an external data source, such as a storage medium, as indicated with a first provision arrow in a dotted manner, with reference numeral 234. The image data 222 may be provided, for example, from an image acquisition device, as indicated with a second dotted provision arrow 236. The model-updated 3D image 230 may be provided, for example, to display device, as indicated with dotted output arrow 238.
An example for an application of the above-mentioned procedures will be described in the following with reference to
In endovascular surgery procedures, the so-called endovascular aneurism repair (EVAR) is an important interventional procedure.
In order to facilitate the insertion of the guide wire 320 into the gate 318 of the stent main body 313, the deployed prosthesis, as shown in
According to one example, to this end, only the gate needs to be modeled and embedded in the pre-operative CT data. The gate appears in the fluoroscopy images as an ellipse-shape wiry structure. As such it can be automatically detected (for instance relying on a gradient-based Hough-transform for the finding of a parametric shape such as an ellipse), and it can be segmented in two images corresponding to distinct angulations. From these two segmentation results (two elliptical 2D lines), a 3D elliptical line can be computed, the projections of which onto the two originating image planes correspond to the observed gates in those images.
The CT data can be processed such that mainly the vessel boundaries are represented (for instance as a surface or as a mesh). The embedding then consists in representing the 3D elliptical line modeling the device (here the gate) together with the vessel boundaries.
Of course this joint representation should be achieved in a common frame of reference for both the model and the pre-operative data. This might require co-registration of the model with the pre-operative data in case the frames of reference of these two data sources do not natively correspond to each other. In particular this is the case when combining CT and X-Ray-originated data. This is not the case when the 3D data are created with a C-arm CT technique (rotational X-Ray). In this case the 3D data and the model, which is computed from 2D X-ray projections, originate from the same system and can be natively expressed in the same frame of reference, making co-registration superfluous
In addition to the gate, the guide-wire itself (or simply its distal tip) can also be modeled as a 3D line. One can then visualise within a single representation the triplet vessel-wall, prosthesis entry point, intervention device to be threaded through this entry point and potentially taking support onto the vessel walls.
Thus, additional image acquisition steps under X-ray fluoroscopy are not necessary in case the wire tip and its relative depth with respect to the gate's location cannot be properly estimated in the projective view of a fluoroscopy image. Rather, this information, which is of crucial importance to the surgeon, since threading the prosthesis gate is one of the most delicate phases of the intervention, is provided with the respective information by the model-updated 3D image, generated and embedded according to the invention. In other words, the insertion of a guide wire 320, as shown in
According to the invention, it is also possible to facilitate the steering of a respective short extension piece 326, as third part, shown in
It must be noted that, according to an exemplary embodiment of the invention, the model-updated 3D representation is only valid as long as the modeled objects correspond with the live 2D projections. The gate being rather static, this remains true for a long period. Gate-upgraded 3D data can then be computed only once and can be used for the gate passing intervention step. But the guide-wire is naturally steered and does not remain static. This implies that joint gate-plus-wire modeling is only valid when corresponding live images are available. In particular this is the case with a bi-plane system that can constantly produce pairs of projections that can be used for the constant generation of gate-plus-wire modeling and 3D upgrading.
As a second input 414, live images 416 from an X-ray system are provided, for example fluoroscopic images taken from a reduced number of views after the deployment of the first part of the stent graft. These are the usual views used to assess the current situation. From the pre-interventional CT-scan 412, a segmentation 418 is provided, segmenting aorta and iliac arteries in 3D. This can be achieved by automatic or semi-automatic algorithmic solutions extracting tubular structures in a 3D data 112 volume. Further, segmentation of abdominal aortic aneurism can also be applied. The pre-interventional CT scan 412 and the live images 416 are further registered in a 2D/3D registration step 420. Therewith, the position of the pre-interventional CT scan, or of the 3D aorta segmentation, in the X-ray system frame of reference is found. 2D/3D registration algorithms are used to retrieve that particular position from one or several X-ray projections. For example, the vertebrae and the pelvis could be used to register the whole CT scan. Angiograms from the aorta could also be used to register the 3D segmented aorta.
According to the invention, in a modeling step 422, the stent graft main body is modeled in 3D. It is noted that the shape of a stent graft, principally at the gate level, is simple and quite regular, i.e. a tubular structure with a bifurcation. It is possible to use assumptions about its shape, such that it can be modeled from reduced set of fluoroscopic images. The result is a 3D model 424 obtained in the X-ray system frame of reference. The 3D segmentation and the 3D model are then provided to an adjustment step 426 in which the stent model is adjusted within the 3D reconstruction of the aorta. Depending on the 2D/3D registration algorithm, the model stent could not be properly positioned within the 3D segmentation of the aorta. Therefore, a residual transformation, for example, to place the stent within the 3D segmentation of the aorta, is computed. As a result, an adjusted model within the 3D reconstruction is provided, also referred to with reference numeral 428.
Further, the 3D segmentation is used in an embedding step 430, in which the 3D view of the stent graft is embedded within the 3D segmentation of the aorta. The interventionist can then use this particular view to assess the position of the gate within the aorta and adapt its strategy to insert the guide wire.
According to a further example, also with reference to
It is noted that according to a further example, the method as shown in
According to a further exemplary embodiment of the invention, although not shown, the modeling of a device from live data into preoperative CT is also applied in other interventions, such as transcatheter valve implantation.
In another exemplary embodiment of the present invention, a computer program or a computer program element is provided that is characterized by being adapted to execute the method steps of the method according to one of the preceding embodiments, on an appropriate system.
The computer program element might therefore be stored on a computer unit, which might also be part of an embodiment of the present invention. This computing unit may be adapted to perform or induce a performing of the steps of the method described above. Moreover, it may be adapted to operate the components of the above described apparatus. The computing unit can be adapted to operate automatically and/or to execute the orders of a user. A computer program may be loaded into a working memory of a data processor. The data processor may thus be equipped to carry out the method of the invention.
This exemplary embodiment of the invention covers both, a computer program that right from the beginning uses the invention and a computer program that by means of an up-date turns an existing program into a program that uses the invention.
Further on, the computer program element might be able to provide all necessary steps to fulfill the procedure of an exemplary embodiment of the method as described above.
According to a further exemplary embodiment of the present invention, a computer readable medium, such as a CD-ROM, is presented wherein the computer readable medium has a computer program element stored on it which computer program element is described by the preceding section.
A computer program may be stored and/or distributed on a suitable medium, such as an optical storage medium or a solid state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.
However, the computer program may also be presented over a network like the World Wide Web and can be downloaded into the working memory of a data processor from such a network. According to a further exemplary embodiment of the present invention, a medium for making a computer program element available for downloading is provided, which computer program element is arranged to perform a method according to one of the previously described embodiments of the invention.
It has to be noted that embodiments of the invention are described with reference to different subject matters. In particular, some embodiments are described with reference to method type claims whereas other embodiments are described with reference to the device type claims. However, a person skilled in the art will gather from the above and the following description that, unless otherwise notified, in addition to any combination of features belonging to one type of subject matter also any combination between features relating to different subject matters is considered to be disclosed with this application. However, all features can be combined providing synergetic effects that are more than the simple summation of the features.
While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. The invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing a claimed invention, from a study of the drawings, the disclosure, and the dependent claims.
In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. A single processor or other unit may fulfill the functions of several items re-cited in the claims. The mere fact that certain measures are re-cited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.
| Number | Date | Country | Kind |
|---|---|---|---|
| 11305428.2 | Apr 2011 | EP | regional |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/IB2012/051700 | 4/5/2012 | WO | 00 | 10/10/2013 |
