The present disclosure relates, in general, to the field medical imaging (such as, fluoroscopy, computed tomography [CT], tomosynthesis, low-cost CT, magnetic resonance imaging [MRI], and PET, and the like. In particular, the disclosure presents an apparatus and a method for reconstructing motion 3D objects, which is considered to be four-dimensional (4D) imaging.
An X-ray imaging scanner is useful to diagnose some joint disorders, particularly a 4D X-ray imaging scanner.
For example, refer to the introduction section of the following reference which provides background information: Yoon Seong Choi, et al., “Four-dimensional real-time cine images of wrist joint kinematics using dual source CT with minimal time increment scanning”. Yonsei Medical Journal, 2013. 54(4): p. 1026-1032.
One paragraph of the Choi reference states: “In the past, radiologic studies of joint disorders focused mainly on the static morphologic depiction of joint internal derangements. However, some joint disorders may not show definite abnormalities in a static radiologic study, but will still have dormant abnormalities that are aggravated with joint movement, which triggers the need for radiologic imaging of dynamic joint movement. The wrist joint in particular requires four-dimensional (4D) dynamic joint imaging because the wrist is an exceedingly complex and versatile structure, consisting of a radius, ulna, eight carpals, and five metacarpals all engaged with each other. Each of these carpal bones exhibits multiplanar motion involving significant out-of-plane rotation of bone rows, which is prominent during radio-ulnar deviation. The kinematics of these carpal bones have been not fully elucidated. Thus, studies using 4D wrist imaging were conducted to determine the proper modality and to investigate carpal kinematics.”
Current techniques to obtain 4D images of a moving joint mainly rely on utilizing a multi-detector CT (MDCT) (such as described in the above-mentioned reference). This current technique is considered by some practitioners to have at least two disadvantages. First, it needs a high-end MDCT scanner (e.g., the mentioned reference used a dual source CT scanner, SOMATOM Definition Flash, manufactured by Siemens Medical, Forchheim, Germany). This high-end MDCT has multiple X-ray tubes and fast rotation speed, which can help reconstruct dynamic images with fine temporal resolution. Second, this technique is viewed as inducing excessive radiation dose, and can potentially cause cancer to the patients.
In view of these disadvantages, this disclosure proposes a system and method to reconstruct 4D images.
The discussion above is merely provided for general background information and is not intended to be used as an aid in determining the scope of the claimed subject matter.
Certain embodiments described herein address the need for methods that generate 4D images for diagnostic imaging. Methods of the present disclosure combine aspects of 3D volume imaging from computed tomography (CT) apparatus that employs radiographic imaging methods with surface imaging capabilities provided using structured light imaging or other visible light imaging method.
These aspects are given only by way of illustrative example, and such objects may be exemplary of one or more embodiments of the invention. Other desirable objectives and advantages inherently achieved by the disclosed invention may occur or become apparent to those skilled in the art. The invention is defined by the appended claims.
According to an embodiment of the present disclosure, there is provided a system for reconstructing a 4D image, comprising: a surface acquisition system for generating a 3D surface model of an object; an X-ray imaging system for acquiring at least one 2D X-ray projection image of the object; a controller to control the surface acquisition system and the X-ray imaging system; and a processor to apply a 4D reconstruction algorithm/method to the 3D surface model and the at least one 2D X-ray projection to reconstruct a 4D X-ray volume of the imaged body part in motion.
The foregoing and other objects, features, and advantages of the invention will be apparent from the following more particular description of the embodiments of the invention, as illustrated in the accompanying drawings. The elements of the drawings are not necessarily to scale relative to each other.
The following is a detailed description of the embodiments of the invention, reference being made to the drawings in which the same reference numerals identify the same elements of structure in each of the several figures.
Where they are used in the context of the present disclosure, the terms “first”, “second”, and so on, do not necessarily denote any ordinal, sequential, or priority relation, but are simply used to more clearly distinguish one step, element, or set of elements from another, unless specified otherwise.
As used herein, the term “energizable” relates to a device or set of components that perform an indicated function upon receiving power and, optionally, upon receiving an enabling signal.
In the context of the present disclosure, the phrase “in signal communication” indicates that two or more devices and/or components are capable of communicating with each other via signals that travel over some type of signal path. Signal communication may be wired or wireless. The signals may be communication, power, data, or energy signals. The signal paths may include physical, electrical, magnetic, electromagnetic, optical, wired, and/or wireless connections between the first device and/or component and second device and/or component. The signal paths may also include additional devices and/or components between the first device and/or component and second device and/or component.
In the context of the present disclosure, the term “subject” is used to describe the object that is imaged, such as the “subject patient”, for example.
Radio-opaque materials provide sufficient absorption of X-ray energy so that the materials are distinctly perceptible within the acquired image content. Radio-translucent or transparent materials are imperceptible or only very slightly perceptible in the acquired radiographic image content.
In the context of the present disclosure, “volume image content” describes the reconstructed image data for an imaged subject, generally stored as a set of voxels. Image display utilities use the volume image content in order to display features within the volume, selecting specific voxels that represent the volume content for rendering a particular slice or view of the imaged subject. Thus, volume image content is the body of resource information that is obtained from a radiographic or other volume imaging apparatus such as a CT, CBCT, MDCT, MRI, PET, tomosynthesis, or other volume imaging device that uses a reconstruction process and that can be used to generate depth visualizations of the imaged subject.
Examples given herein that may relate to particular anatomy or imaging modality are considered to be illustrative and non-limiting. Embodiments of the present disclosure can be applied for both 2D radiographic imaging modalities, such as radiography, fluoroscopy, or mammography, for example, and 3D imaging modalities, such as CT, MDCT, CBCT, tomosynthesis, dual energy CT, or spectral CT.
In the context of the present disclosure, the term “volume image” is synonymous with the terms “3 dimensional image” or “3D image”.
In the context of the present disclosure, a radiographic projection image, more simply termed a “projection image” or “x-ray image”, is a 2D image formed from the projection of x-rays through a subject. In conventional radiography, a single projection image of a subject can be obtained and analyzed. In volume imaging such as CT, MDCT, and CBCT imaging, multiple projection images are obtained in series, then processed to combine information from different perspectives in order to form image voxels.
Embodiments of the present disclosure are directed to apparatus and methods that can be particularly useful with volume imaging apparatus such as a CBCT system.
A description of a suitable 4D X-ray imaging scanner is described below. Generally, a 4D X-ray imaging scanner is comprised of two systems: (i) a surface acquisition system and (ii) an X-ray imaging system. In a preferred arrangement, the two systems are calibrated to one coordinate system and are synchronized.
Applicants now describe the imaging process employing a 4D X-ray imaging scanner.
When Applicants' system is employed for medical purposes, the object being imaged is anatomy (body part). For ease of illustration/presentation of the system, an anatomy of a hand is described.
In a FIRST STEP, a surface acquisition system 50 that includes a camera 66 is employed to capture/record/obtain a motion 3D surface model of the body part in the field of view. Refer to
In a SECOND STEP, the X-ray imaging system that includes a source 12 and a detector 20 acquires a series of 2D projection images of the body part. Refer to
Various modalities can be employed for the surface acquisition system and the X-ray imaging system, for example: CT, fluoroscopy, tomosynthesis, and radiography. In a preferred arrangement, the surface acquisition system and the X-ray imaging system are different modalities.
Applicants note that the geometry for the system's X-ray tube and X-ray detector can be either stationary like a radiography/fluoroscopy system (as illustrated in
In a THIRD STEP, after image acquisition, some (or all) of the acquired images are employed to reconstruct the surface of the hand/object. Techniques are known to reconstruct the surface of a moving object. Two examples are referenced, and incorporated herein in their entirety:
(1) Sinha, Ayan, Chiho Choi, and Karthik Ramani. “Deep Hand: Robust Hand Pose Estimation by Completing a Matrix Imputed With Deep Features.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016, pp. 4150-4158 (with video content available at the https://www. address “youtube.com/watch?v=ScXCgC2SNNQ&ab_channel=CdesignLab”.); and
(2) Huang, Chun-Hao, et al. “Volumetric 3D Tracking by Detection.” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 3862-3870 (with video content available at the “https://www. address youtube.com/watch?v=zVavXrcyeYg&ab_channel=Chun-HaoHuang”.)
In a FOURTH STEP, after image acquisition, some (or all) of the acquired images are employed to reconstruct at least one 4D image. This is accomplished by the following sequence of steps, for each 2D X-ray projection image: a) the reconstructed volume is deformed according to the captured motional 3D surface model; b) the reconstructed volume is adjusted according to patient anatomical structures or implants, such that the forward projection of the reconstructed volume can match the acquired 2D X-ray projection image; and c) a reconstruction algorithm (e.g., FBP (filtered back projection) or iterative reconstruction algorithms) is performed/applied to update the volume.
In a FIFTH STEP, after reconstruction, one or more of the 4D images can be displayed, stored, or transmitted.
While the making and use of various embodiments are described, it should be appreciated that the specific embodiments described herein are merely illustrative of specific ways to make and use the system and do not limit the scope of the invention.
Applicants have described a system for reconstructing a 4D image. The system includes: (a) a surface acquisition system for generating a 3D surface model of an object; (b) an X-ray imaging system for acquiring at least one 2D X-ray projection image of the object; (c) a controller to control the surface acquisition system and the X-ray imaging system; and (d) a processor to apply a 4D reconstruction algorithm/method to the 3D surface model and the at least one 2D X-ray projection to reconstruct 4D X-ray volume of the imaged body part in motion.
In one arrangement, the surface acquisition system comprises: (a) one or more light sources adapted to project a known pattern of light grid onto the object using one or more light sources; (b) one or more optical sensors adapted to capture a plurality of 2D digital images of the object; and (c) a surface reconstruction algorithm for reconstructing the 3D surface model of the object using the at least one 2D projection image.
In one arrangement, the light sources and the optical sensors are adapted to be either (i) mounted to a rotational gantry of the X-ray imaging system, (ii) affixed to the bore of the X-ray imaging system, or (iii) placed outside of (separate from) the X-ray imaging system.
In one arrangement, the X-ray imaging system comprises: (a) one or more X-ray sources adapted to controllably emit X-rays; and (b) one or more X-ray detectors including a plurality (optionally: of rows) of X-ray sensors adapted to detect X- rays that are emitted from the X-ray sources and have traversed the object.
In one arrangement, the X-ray sources and X-ray detectors move in a trajectory, wherein the trajectory includes, but is not limited to, a helix (e.g., MDCT), full circle (e.g., dental CBCT CT), incomplete circle (e.g., extremity CBCT), line, sinusoid, and stationary (e.g., low-cost CT), and the like.
In one arrangement, the controller synchronizes the surface imaging system and the X-ray imaging system.
In one arrangement, the 4D reconstruction algorithm/method comprises: (a) a X-ray projection correction process/method/algorithm to generate a corrected 2D X-ray projection; (b) a 3D surface deformation algorithm/method/process to deform each 3D surface model to the next time-adjacent 3D surface model and generate at least one transformation parameter; (c) a 3D volume deformation algorithm/method/process to deform the volume under reconstruction according to the at least one transformation parameter; (d) a 3D volume deformation algorithm/method/process to deform the volume under reconstruction according to the 2D X-ray projection using an anatomical structure or implant; and (e) an analytical form reconstruction algorithm/method/process or an iterative form reconstruction algorithm/method/process.
In one arrangement, the X-ray projection correction process/method/algorithm includes (but is not limited to) a scatter correction, a beam hardening correction, a lag correction, a veiling glare correction, or a metal artifact reduction correction.
In one arrangement, the system further comprises a 3D surface registration algorithm comprising a rigid-object registration algorithm or a deformable registration algorithm.
In one arrangement, the analytical form reconstruction algorithm/method/process includes an FDK (Feldkamp-Davis-Kress) algorithm.
In one arrangement, the iterative form reconstruction algorithm/method/process includes a SART algorithm, a statistical reconstruction algorithm, a total variation reconstruction algorithm, or an iterative FDK algorithm.
In one arrangement, the 4D reconstruction method is applied until a predetermined threshold criterion is met (e.g., for example, a predetermined number of iterations or a maximum error less than a threshold error value).
To more particularly understand the methods of the present disclosure and the problems addressed, it is instructive to review principles and terminology used for 3D volume image capture and reconstruction. Referring to the perspective view of
For a 3D or volume imaging system, the field of view (FOV) of the imaging apparatus is the subject volume that is defined by the portion of the radiation cone or field that impinges on a detector for each projection image. A sequence of projection images of the field of view is obtained in rapid succession at varying angles about the subject, such as one image at each 1-degree angle increment in a 200-degree orbit. X-ray digital radiation (DR) detector 20 is moved to different imaging positions about subject 14 in concert with corresponding movement of radiation source 12.
In order to track patient motion during projection image acquisition, the imaging apparatus needs sufficient data for detecting surface displacement. To obtain this surface modeling information, an embodiment of the present disclosure can employ surface contour acquisition, such as contour acquisition using structured light imaging.
Other methods for obtaining the surface contour can alternately be used. Alternate methods include stereovision technique, structure from motion, and time-of-flight techniques, for example. The surface contour can be expressed as a mesh, using techniques familiar to those skilled in the contour imaging arts.
The surface acquisition system can use a structured light imaging technique, using one or more light sources and one or more light sensors as shown in
Both surface contour characterization and volume image content are used for motion compensation and correction of the present disclosure. This image content can be acquired from previously stored data that can be from the same imaging apparatus or from different apparatus. However, there can be significant advantages in obtaining the surface contour characterization and volume image content from the same apparatus, particularly for simplifying the registration task.
The moving trajectories of the X-ray sources and X-ray detectors can be, for example, helix (e.g., MDCT), full circle (e.g., dental CBCT CT), incomplete circle (e.g., extremity CBCT), line, sinusoidal, and stationary (e.g., low-cost CT), or other suitable movement pattern.
To help reduce motion artifacts in X-ray images, the Applicants propose a motion artifact reduction (MAR) system and method. The motion artifact reduction (MAR) system includes: a surface acquisition or characterization system for generating 3D surface models of a patient; an X-ray volume imaging apparatus for acquiring X-ray projection data of a patient; a controller to synchronize the surface acquisition system and the X-ray imaging apparatus; and a control logic processor (for example, a processor or other computing device that executes a motion reduction algorithm, or the like) that uses the X-ray projection data and 3D surface models to reconstruct a 3D volume, wherein the reconstructed volume has reduced patient motion artifacts.
In some cases, patient motion from a given position can be significant and may not be correctable. This can occur, for example, when the patient coughs or makes some other sudden or irregular movement. In the later reconstruction phase, the control logic processor 28 or controller 38 can suspend acquisition by the X-ray imaging system until the patient can recover the previous position.
The control logic can also analyze the acquired 3D surface image of the patient in real time and perform motion gating acquisition (also termed respiration gating) based on this analysis. With motion gating, surface contour acquisition can be associated with x-ray projection image acquisition and may even be used to momentarily prevent or defer acquisition. Using the controller to monitor and coordinate image acquisition, at least one 3D surface model of the patient can be obtained for each 2D X-ray projection. The acquired 3D surface model can be used for motion reduction in the reconstruction phase.
The schematic diagram of
Embodiments of the present disclosure provide motion compensation methods that characterize patient motion using imaging techniques such as surface contour imaging. A 3D surface model is generated from the acquired surface contour images and is used to generate transformation parameters that modify the volume reconstruction that is formed. Some exemplary transformation parameters include translation, rotation, skew, or other values related to feature visualization. Synchronization of the timing of surface contour imaging data capture with each acquired 2D x-ray projection image allows the correct voxel to be updated where movement has been detected. Because 3D surface contour imaging can be executed at high speeds, it is possible to generate a separate 3D surface contour image corresponding to each projection image 20 (
There are two classic computational approaches used for 3D volume image reconstruction: (i) an analytic approach that offers a direct mathematical solution to the reconstruction process; and (ii) an iterative approach that models the imaging process and uses a process of successive approximation to reduce error according to a cost function or other type of objective function. Each of these approaches has inherent strengths and weaknesses for generating accurate 3D reconstructions.
The logic flow diagram of
Referring to
In pre-processing phase 420 of
A correction step 426 then serves to provide a set of corrected 2D x-ray projections 428 for reconstruction. Correction step 426 can provide a number of functions, including scatter correction, lag correction to compensate for residual signal energy retained by the detector from previous images, beam hardening correction, and metal artifact reduction, for example.
Continuing with the
The
By way of example, images illustrating motion artifacts can be found in Boas, F. Edward, and Dominik Fleischmann. “CT artifacts: causes and reduction techniques. “Imaging in Medicine 4.2 (2012): 229-240.”, incorporated herein in its entirety. Motion artifacts can include blurring and double images, as shown in
Reference is made to Hsieh, Jiang. “Computed tomography: principles, design, artifacts, and recent advances.” Bellingham, Wash.: SPIE, 2009, pages 258-269 of Chapter 7. This reference describes a respiratory motion artifact, as best illustrated in
Accordingly, Applicants have disclosed a system for constructing a 3D volume of an object, comprising: a surface acquisition system for acquiring 3D surface images of the object; an X-ray imaging system for acquiring a plurality of X-ray projection images of the object; a controller to synchronize control the surface acquisition system and the X-ray imaging system to acquire the 3D surface images and X-ray projection images; and a processor to construct a 3D volume using the acquired 3D surface images and X-ray projection images.
Accordingly, Applicants have disclosed a method for reconstructing a 3D volume, comprising: providing a synchronized system comprised of a surface acquisition system and a X-ray imaging system; using the synchronized system, acquiring a plurality of surface images and a plurality of X-ray projection images of a patient; generating a plurality of 3D surface models of the patient using the plurality of surface images; and reconstructing the 3D volume using the plurality of X-ray projection images and the plurality of 3D surface models. In one embodiment, the step of reconstructing the 3D volume employs an analytical form reconstruction algorithm. In another embodiment, the step of reconstructing the 3D volume employs an iterative form reconstruction algorithm.
It can be appreciated that other processing sequences can alternately be executed using the combined contour image and projection image data to compensate for patient motion as described herein.
An embodiment of the present disclosure enables the various types of imaging apparatus 10, 70, 80, 90 shown in
By acquiring radiographic image data and surface contour image data throughout the movement sequence shown in
A few possible intermediate positions are represented in dashed outline in
Using a combination of surface characterization and radiographic projection images obtained at and between index positions allows analysis of joint movement without requiring the significant number of exposures that would otherwise be required to reconstruct full 3D radiographic volume data for each of numerous intermediate positions in a movement sequence such as that shown in
The logic flow diagram of
In the
In iterative processing, the cycle of steps (ii) and (iii) repeats any number of times until the calculated error obtained in (iii) is within acceptable limits or is negligible. The result of this processing is a transformed or reconstructed volume.
It can be appreciated that the basic procedure shown in
After acquisition and processing of images for volume image reconstruction, a display step 170 then executes. Display step 170 can display some portion or all of the movement sequence on display 34, with the transformed volumes 160 generated for each movement position displayed in an ordered sequence. As shown in
A number of different reconstruction tools can be used for generating the reconstructed volume 152 or transformed volumes 160 of
The processing task for generating the transformed volume image can apply any of a number of tools, including using at least one of rigid transformation, non-rigid transformation, 3D-to-3D transformation, surface-based transformation, 3D-to-2D registration, feature-based registration, projection-based registration, and appearance-based transformation, for example.
At each of a succession of time-adjacent intermediate positions, the sequence of
Forward projection through the generated transformed volume helps to reconcile the existing volume deformation with acquired data for an intermediate position 130. A forward projection computed through the transformed volume image data generates a type of digitally reconstructed radiograph (DRR), a synthetic projection image that can be compared against the acquired radiographic projection image as part of iterative reconstruction. Discrepancies can help to correct for positioning error and verify that the movement sequence for a subject patient has been correctly characterized.
According to an alternate embodiment of the present disclosure, the following sequence can be used for updating the volume at each intermediate position 130:
The update of the transformed volume image in d) above can use back projection algorithms, such as filtered back projection (FBP) or may use iterative reconstruction algorithms.
With particular respect to volume transformation methods and 2D-to-3D registration methods, reference is hereby made to the following, by way of example:
It can be appreciated that the sequence shown in
It should also be noted that while the present disclosure describes the use of structured light imaging for surface contour characterization, other methods that employ reflectance imaging or other non-ionizing radiation could alternately be used for surface contour characterization.
According to an embodiment of the present disclosure, information for iterative reconstruction of the transformed image is available from comparison of a forward projection through the initially transformed volume with the actual radiographic projection image. This arrangement is shown in
The corrected image projections can be used to help generate additional projection images for use in volume reconstruction, using methods well known to those skilled in the volume reconstruction arts.
Advantageously, the method of the present invention allows accurate 3D modeling of a motion sequence using fewer radiographic images than conventional methods, with lower radiation dose to the patient. By acquiring radiographic images in conjunction with 3D surface contour imaging content, the method allows the image subject to be accurately transformed with movement from one position to the next, with continuing verification and adjustment of calculated data using acquired image content.
Consistent with an embodiment, the present invention utilizes a computer program with stored instructions that control system functions for image acquisition and image data processing for image data that is stored and accessed from external devices or an electronic memory associated with acquisition devices and corresponding images. As can be appreciated by those skilled in the image processing arts, a computer program of an embodiment of the present invention can be utilized by a suitable, general-purpose computer system, such as a personal computer or workstation that acts as an image processor, when provided with a suitable software program so that the processor operates to acquire, process, transmit, store, and display data as described herein. Many other types of computer systems architectures can be used to execute the computer program of the present invention, including an arrangement of networked processors, for example.
The computer program for performing the method of the present invention may be stored in a computer readable storage medium. This medium may comprise, for example; magnetic storage media such as a magnetic disk such as a hard drive or removable device or magnetic tape; optical storage media such as an optical disc, optical tape, or machine readable optical encoding; solid state electronic storage devices such as random access memory (RAM), or read only memory (ROM); or any other physical device or medium employed to store a computer program. The computer program for performing the method of the present invention may also be stored on computer readable storage medium that is connected to the image processor by way of the internet or other network or communication medium. Those skilled in the image data processing arts will further readily recognize that the equivalent of such a computer program product may also be constructed in hardware.
It is noted that the term “memory”, equivalent to “computer-accessible memory” in the context of the present disclosure, can refer to any type of temporary or more enduring data storage workspace used for storing and operating upon image data and accessible to a computer system, including a database. The memory could be non-volatile, using, for example, a long-term storage medium such as magnetic or optical storage. Alternately, the memory could be of a more volatile nature, using an electronic circuit, such as random-access memory (RAM) that is used as a temporary buffer or workspace by a microprocessor or other control logic processor device. Display data, for example, is typically stored in a temporary storage buffer that is directly associated with a display device and is periodically refreshed as needed in order to provide displayed data. This temporary storage buffer can also be considered to be a memory, as the term is used in the present disclosure. Memory is also used as the data workspace for executing and storing intermediate and final results of calculations and other processing. Computer-accessible memory can be volatile, non-volatile, or a hybrid combination of volatile and non-volatile types.
It is understood that the computer program product of the present invention may make use of various image manipulation algorithms and processes that are well known. It will be further understood that the computer program product embodiment of the present invention may embody algorithms and processes not specifically shown or described herein that are useful for implementation. Such algorithms and processes may include conventional utilities that are within the ordinary skill of the image processing arts. Additional aspects of such algorithms and systems, and hardware and/or software for producing and otherwise processing the images or co-operating with the computer program product of the present invention, are not specifically shown or described herein and may be selected from such algorithms, systems, hardware, components and elements known in the art.
The invention has been described in detail, and may have been described with particular reference to a suitable or presently preferred embodiment, but it will be understood that variations and modifications can be effected within the spirit and scope of the invention. The presently disclosed embodiments are therefore considered in all respects to be illustrative and not restrictive. The scope of the invention is indicated by the appended claims, and all changes that come within the meaning and range of equivalents thereof are intended to be embraced therein.
This application claims the benefit of U.S. Provisional Application No. 62/394,232, filed Sep. 14, 2016, entitled APPARATUS AND METHOD FOR 4D X-RAY IMAGING by Lin et al., which is hereby incorporated by reference in its entirety.
Number | Date | Country | |
---|---|---|---|
62394232 | Sep 2016 | US |