Patent Application
20160242854
This disclosure relates to medical instruments and imaging and more particularly to employing instrument shape sensing to enhance images by removing instrument artifacts from the images.
In angiographic studies during interventional procedures, catheters are used to inject contrast agent into the vascular structure of interest. Examples of such procedures include coronary angiography procedures, e.g., Percutaneous Transluminal Coronary Intervention (PTCI) or valvular procedures such as, Transcatheter Aortic-Valve Implantation (TAVI). Balloons, stents, and other filter device procedures also have catheter-based deployment and very often, there is a need for rotational X-ray (RX) imaging to acquire projections for 3D volumetric reconstruction (e.g., 3D atriography) and visualization. The contrast agent enhanced projections can be used as input for those 3D image reconstructions. While the catheter is a prerequisite to apply the contrast agent, it is not necessarily favorable to see the catheter in a reconstructed field of view. This is particularly unfavorable when the catheter is only partially inside the field of view (not visible in all projections), when the catheter is only filled with contrast agent in a part of the rotational sequence, or when the catheter shows strong motion during the acquisition.
In accordance with the present principles, a medical system for imaging includes a processor and memory coupled to the processor. The memory includes a shape sensing module configured to receive shape sensing data from a shape sensing system coupled to a medical instrument. The shape sensing system is configured to measure a shape and position of the medical instrument. An image generation module is configured to detect and digitally remove image artifacts of the medical instrument from a generated image based on the shape and the position of the medical instrument as measured by the shape sensing system.
Another medical system includes a medical instrument and a shape sensing system coupled to the medical instrument to measure a shape and position of the medical instrument. An imaging system is configured to image a subject wherein image artifacts of the medical instrument are at least partially present in the image of the subject. An image generation module is configured to detect and digitally remove at least the artifacts of the medical instrument from a generated image based on at least the shape and the position of the medical instrument as measured by the shape sensing system.
A method for image processing includes gathering shape sensing data from an instrument; imaging the instrument in an internal image volume; detecting imaging artifacts caused by the instrument in the image volume by employing the shape sensing data from the instrument; and removing the imaging artifacts by employing an image interpolation process and the shape sensing data.
These and other objects, features and advantages of the present disclosure will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings.
This disclosure will present in detail the following description of preferred embodiments with reference to the following figures wherein:
In accordance with the present principles, safe and real time instrument identification and removal from an image are provided. This instrument identification and removal from the image is a valuable feature as it minimizes the impact of associated imaging artifacts. Such a solution is applicable to catheters, implanted electrode leads, guidewires, etc. in an imaging field-of-view for X-rays or other radiation.
In particularly useful embodiments, optical shape sensing (OSS) is employed to detect a position, shape and orientation of an instrument. OSS utilizes special optical fibers which can be integrated into a catheter, guidewire, electrode lead, or other flexible elongated instrument and are connected to an analysis unit outside a body of a patient. The position and the shape of the fiber is measured in real time using modeling and analysis of optical Rayleigh scattering with respect to a reference in the analysis unit connected to one end of the instrument. The position of the instrument along its extent is thereby known in the imaging space. The spatial information on the shape and the position of the instrument during contrast injection and rotational projection acquisition can be used to compute the position of the instrument on the projection images and remove the instrument from the projections using an interpolation method and known instrument geometry and characteristics in imaging.
Shape sensing information, combined with an expert database or library of device characteristics (e.g., based on 3D models or on prior cases or datasets (historic data)) will permit simplified removal of the instrument's footprint or X-ray shadow within an overall projection dataset. The processing overhead related to instrument detection and tracking can be eliminated. The expert database or library can also be augmented by standard machine learning methods for adaptive optimization for the procedures at hand.
It also should be understood that the present invention will be described in terms of medical instruments; however, the teachings of the present invention are much broader and are applicable to any imaging system which may employ shape sensing to remove an instrument or object from an image. In some embodiments, the present principles are employed in tracking or analyzing complex biological or mechanical systems. In particular, the present principles are applicable to internal tracking and imaging procedures of biological systems, procedures in all areas of the body such as the lungs, gastro-intestinal tract, excretory organs, brain, heart, blood vessels, etc. The elements depicted in the FIGS. may be implemented in various combinations of hardware and software and provide functions which may be combined in a single element or multiple elements.
The functions of the various elements shown in the FIGS. can be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions can be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which can be shared. Moreover, explicit use of the term “processor” or “controller” should not be construed to refer exclusively to hardware capable of executing software, and can implicitly include, without limitation, digital signal processor (“DSP”) hardware, read-only memory (“ROM”) for storing software, random access memory (“RAM”), non-volatile storage, etc.
Moreover, all statements herein reciting principles, aspects, and embodiments of the invention, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future (i.e., any elements developed that perform the same function, regardless of structure). Thus, for example, it will be appreciated by those skilled in the art that the block diagrams presented herein represent conceptual views of illustrative system components and/or circuitry embodying the principles of the invention. Similarly, it will be appreciated that any flow charts, flow diagrams and the like represent various processes which may be substantially represented in computer readable storage media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.
Furthermore, embodiments of the present invention can take the form of a computer program product accessible from a computer-usable or computer-readable storage medium providing program code for use by or in connection with a computer or any instruction execution system. For the purposes of this description, a computer-usable or computer readable storage medium can be any apparatus that may include, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk. Current examples of optical disks include compact disk—read only memory (CD-ROM), compact disk—read/write (CD-R/W), Blu-Ray™ and DVD.
Referring now to the drawings in which like numerals represent the same or similar elements and initially to
Workstation 112 may include a display 118 for viewing internal images of a subject provided by an imaging system 110. The imaging system 110 may include, e.g., a magnetic resonance imaging (MRI) system, a fluoroscopy system, a computed tomography (CT) system, ultrasound (US), etc. Display 118 may also permit a user to interact with the workstation 112 and its components and functions. This is further facilitated by an interface 120 which may include a keyboard, mouse, a joystick or any other peripheral or control to permit user interaction with the workstation 112.
The shape sensing system 104 on device 102 includes one or more optical fibers 126 which are coupled to the device 102 in a set pattern or patterns. The optical fibers 126 connect to the workstation 112 through cabling 127. The cabling 127 may include fiber optics, electrical connections, other instrumentation, etc., as needed.
Workstation 112 may include an optical source 106 to provide optical fibers 126 with light when shape sensing 104 includes optical fiber shape sensing. An optical interrogation unit 108 may also be employed to detect light returning from all fibers. This permits the determination of strains or other parameters, which will be used to interpret the shape, orientation, etc. of the interventional device 102. The light signals will be employed as feedback to make adjustments, to access errors, to determine a shape and position of the device 102 and to calibrate the device 102 (or system 100).
Shape sensing device 104 preferably includes one or more fibers 126, which are configured to exploit their geometry for detection and correction/calibration of a shape of the device 102. Shape sensing system 104 with fiber optics may be based on fiber optic Bragg grating sensors. A fiber optic Bragg grating (FBG) is a short segment of optical fiber that reflects particular wavelengths of light and transmits all others. This is achieved by adding a periodic variation of the refractive index in the fiber core, which generates a wavelength-specific dielectric mirror. A fiber Bragg grating can therefore be used as an inline optical filter to block certain wavelengths, or as a wavelength-specific reflector.
A fundamental principle behind the operation of a fiber Bragg grating is Fresnel reflection at each of the interfaces where the refractive index is changing. For some wavelengths, the reflected light of the various periods is in phase so that constructive interference exists for reflection and, consequently, destructive interference for transmission. The Bragg wavelength is sensitive to strain as well as to temperature. This means that Bragg gratings can be used as sensing elements in fiber optical sensors. In an FBG sensor, the measurand (e.g., strain) causes a shift in the Bragg wavelength.
One advantage of this technique is that various sensor elements can be distributed over the length of a fiber. Incorporating three or more cores with various sensors (gauges) along the length of a fiber that is embedded in a structure permits a three dimensional form of such a structure to be precisely determined, typically with better than 1 mm accuracy. Along the length of the fiber, at various positions, a multitude of FBG sensors can be located (e.g., 3 or more fiber sensing cores). From the strain measurement of each FBG, the curvature of the structure can be inferred at that position. From the multitude of measured positions, the total three-dimensional form is determined.
As an alternative to fiber-optic Bragg gratings, the inherent backscatter in conventional optical fiber can be exploited. One such approach is to use Rayleigh scatter in standard single-mode communications fiber. Rayleigh scatter occurs as a result of random fluctuations of the index of refraction in the fiber core. These random fluctuations can be modeled as a Bragg grating with a random variation of amplitude and phase along the grating length. By using this effect in three or more cores running within a single length of multi-core fiber, the 3D shape and dynamics of the surface of interest can be followed. It should be understood that other shape sensing techniques, not limited by those described, may also be employed.
The optical fibers 126 can be integrated into the device 102 (e.g., catheter, guidewire, electrode lead, or other flexible elongated instrument) and connected to the analysis unit 115 outside a body or imaging volume 131 of a patient or subject. The position and the shape of the fiber 126 is measured in real time using modeling and analysis of the optical scattering or back reflection with respect to a reference in the analysis module 115 stored in memory 116. A position of the device 102 along its extent is thereby known in the imaging space of an imaging system 110, e.g., an X-ray system, CT system, etc.
In one embodiment, the imaging system 110 may include a rotational X-ray system, and the device 102 may include a contrast dispensing catheter. The spatial information on the shape and the position of the device 102 during contrast injection and rotational projection acquisition can be used to calculate the position of the catheter 102 in an image 134 (which may be viewed on a display 118). Since the optical sensing module 115 can accurately compute the shape and position of the device 102, an image generator 148 can use this information and other information to pinpoint image artifacts for removal from the image 134. The image generator 148 can identify and remove the device 102 (e.g., catheter) based on the shape sensing information and may employ the shape sensing information along with other information to remove image artifacts. The image generator 148 may employ a suitable interpolation method, such as image inpainting or other image processing technique to alter the pixels of the image to remove the device 102 and/or image artifacts due to the device 102 from the image 134. Knowing catheter/device geometry and characteristics of X-ray imaging, an accurate determination of the image portion which the catheter (or other device) 102 occupies can be accurately determined.
The shape sensing information, combined with an expert database or library 142 of device characteristics may be employed to more confidently identify the device 102 and its artifacts in the image 134. The database or library 142 may include 3D model(s) 132 or stored images of prior cases/historic data 136. Models 132 may be generated based upon images taken before the device 102 has been introduced so that a comparison can be made with images having artifacts or the device 102 present. The historic data 136 may include previously collected image frames so that portions of the device 102 and their earlier trajectories can be determined and employed to predict places where artifacts may occur. The library 142 can also be augmented using a machine learning module 146 or other known learning method for adaptive optimization of images and image comparisons. The optimized images may be employed to more reliably remove artifacts of the shaped sensing tracked device 102 from the image 134 based on a current procedure, a particular patient, a particular circumstance, etc.
For example, design and development of the model 132 and/or historic data 136 may include evolving or recording behavior based on empirical or historic information such as from image data or artifact data. The machine learning module 146 can take advantage of examples (data) to capture characteristics of interest over time. Data can be seen as examples that illustrate relations between observed variables including device shapes and positions. The machine learning module 146 can automatically learn to recognize complex patterns and make intelligent decisions based on data of where instrument projections, instrument images, instrument artifacts, etc. are likely to be in the image 134. The shape sensing data makes the learning much simpler and much more reliable.
Such adaptive optimization will permit for a more simplified removal of the device footprint, artifacts, X-ray shadows, etc. within an overall projection dataset (or image 134). By employing the present principles, processing overhead associated with catheter or device detection and tracking can be eliminated.
In another embodiment, for all rotational angiographic acquisitions of moving structures, e.g., in the heart, the motion information of the catheter or device 102 tracked by the optical shape sensing system 104 can be employed to derive a physiological signal. The physiological signal may include a signal measured by a sensor device 121, e.g., corresponding to an electro-cardiogram (ECG) signal, a signal indicating the displacement of the heart due to breathing motion, or any other signal representing physical dynamic motion. This motion information can be used for gated and/or motion compensated reconstruction. The shape measurements of the shape sensing system 104 of the device 102 are correlated with the projection acquisition in space and time and motion information due to known sources can be accounted for in the image processing in the image generation module 148. Motion or lack thereof (breath hold commands, hyper pacing, and adenosine) may be accounted for or other sensor signals may be employed to collect motion data (e.g., breathing belt, ECG signal).
In an interventional setup, a shape sensing enabled device (102, 104) may be registered to X-ray (CT) imaging space. The registration may be performed using, for example, known phantom-based calibration steps. When a rotational X-ray acquisition is performed for volumetric X-ray imaging, and the shape sensing enabled device is in the X-ray field of view, for each acquired position, there is a spatial correspondence of the device visible in X-ray and its shape from shape sensing. The shape in the X-ray image may however be truncated. Due to system lag of any of the systems, there can however be a temporal discrepancy between the shape (from shape sensing) and what is visible in the X-ray. Temporal correspondence is preferable between shape sensing and the imaging. This can be achieved by time-stamping of both data and calibration of system lags or other methods such as employing a physiological signal to provide a temporal reference.
The shape sensing data and X-ray projections can be synchronized via other external signals (e.g., from extrinsic events in an interventional suite or from physiological streams such as, ECG, hemodynamic, respiratory information, etc.) or via internal clocks which allow for time annotation of continuously acquired multi-modal (shape sensing/X-ray) measurements or for triggered acquisition of these datasets in prospective or retrospective fashion. These synchronized streams permit interleaving of shape sensing data and X-ray measurements and increased accuracy of shape sensing based instrument removal from X-ray projections and subsequent 3D volumetric reconstructions.
In another embodiment, correlation between the X-ray image and the shape sensing data may be performed using scintillating fiber claddings (e.g., fiber cladding visible in fluoroscopic images) to be attached to or integrated in the optical shape sensing fiber (126). When the cladding is inside the X-ray field of view, the cladding image may be employed to temporally correlate the shape signal to the X-ray image. The shape and X-ray information do not have to run on the same clock. Intra-frame motion could even be compensated for using this method.
Referring to
Referring to
In atrial fibrillation (AFIB) or other structural heart disease procedures which employ a rotational angiography to generate 3D information, artifacts due to the bright contrast emitting catheter tip are avoided by erasing the catheter from a sequence of rotational projections 302 as illustrated in
Referring to
During imaging, especially where radiation is employed, reflections, shadows and other artifacts may occur. In block 408, imaging artifacts caused by the instrument in the image volume are detected by employing the shape sensing data from the instrument. The shape sensing data reveals the position and the shape of the instrument in various image projections and angles. These images are reviewed in light of one or more of models, historic data, etc. and the shape sensing data to identify the artifacts.
In block 410, a model stored in memory may be compared with the image with artifacts and medical instrument geometry may also be considered in the comparison for detecting and removing of the artifacts in accordance with information from the shape sensing system. In block 412, historic data of previous events may be compared with the image with artifacts (e.g., the progression of motion of the instrument) for detection and removal of the artifacts in accordance with information from the shape sensing system. In block 414, removal of the image artifacts may be optimized by employing machine learning to better and more accurately identify the artifacts based upon earlier cases, models, etc. In block 420, the imaging artifacts are removed by employing an image interpolation process and the shape sensing data. The image interpolation process may include an inpainting image process or other image process that can adjust pixels to remove artifacts and instrument images from a medical image or the like. In block 422, the image artifacts are preferably contemporaneously removed while collecting image data. This results in real time or near real time image correction (removal of artifacts, etc.).
In block 424, if the instrument is removed from the image (or even if it is not), a simulated image of the medical instrument may be generated for visualization in a generated image. A virtual image of the instrument, free from artifacts and based on shape sensing data, will provide a user with useful information of the shape and position of the instrument in real time or near real time. In block 426, a physiological signal may be correlated with the shape sensing data to account for patient motion. The physiological signal may include an ECG signal, which can be employed to account for heart beats. In other embodiments, a breathing sensor may account for motion due to breathing, a motion sensor may account for muscle movement, etc. In block 428, a procedure or operation continues as needed.
In interpreting the appended claims, it should be understood that:
Having described preferred embodiments for artifact removal using shape sensing (which are intended to be illustrative and not limiting), it is noted that modifications and variations can be made by persons skilled in the art in light of the above teachings. It is therefore to be understood that changes may be made in the particular embodiments of the disclosure disclosed which are within the scope of the embodiments disclosed herein as outlined by the appended claims. Having thus described the details and particularity required by the patent laws, what is claimed and desired protected by Letters Patent is set forth in the appended claims.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/IB2013/052536 | 3/29/2013 | WO | 00 |
| Number | Date | Country | |
|---|---|---|---|
| 61617313 | Apr 2012 | US |
