1. Technical Field
The present invention relates to multi-label segmentation and segmenting different organs of the abdomen.
2. Discussion of the Related Art
Image segmentation is the process of partitioning an image into different regions. A goal of image segmentation is to obtain a higher-level description of image content. For instance, in medical imaging, the segmentation of anatomical structures is a key element for computer-aided diagnosis and image-guided therapies.
In an exemplary embodiment of the present invention, a method for segmenting an anatomical image, comprises: receiving a patient anatomical image: receiving a baseline anatomical image having pre-segmented labels, wherein the pre-segmented labels identify regions of interest in the baseline anatomical image; aligning the patient anatomical image with the baseline anatomical image to produce a transformation that when applied to the pre-segmented labels roughly identifies regions of interest in the patient anatomical image that correspond to the regions of interest in the baseline anatomical image; and updating the pre-segmented labels, which have been deformed by application of the transformation, with a new transformation that minimizes the likelihood of intensity distributions within the regions of interest of the patient anatomical image to produce a gradient image that better identifies the regions of interest of the patient anatomical image.
The method further comprises computing the new transformation, wherein computing the new transformation comprises: computing a gradient for all the regions of interest of the patient anatomical image; regularizing the gradient; and generating the new transformation by using the regularized gradient.
The new transformation is applied to the deformed pre-segmented labels by computing a composition of the deformed pre-segmented labels and the new transformation.
Computing the gradient for all the regions of interest of the patient anatomical image comprises: (1) for a region of interest of the patient anatomical image, computing a temporary image for the region of interest; computing an intensity distribution for the region of interest; and computing a gradient for the region of interest; (2) updating the gradient image with the gradient for the region of the interest; and repeating (1) and (2) until the gradient image has been updated with a gradient for all the regions of interest of the patient anatomical image.
The pre-segmented labels are repeatedly updated with new transformations until all the regions of interest of the patient anatomical image are better identified.
The patient anatomical image comprises an abdomen.
The patient anatomical image is a computed tomography (CT) image.
In an exemplary embodiment of the present invention, a system for segmenting an anatomical image, comprises: a memory device for storing a program: a processor in communication with the memory device, the processor operative with the program to: receive a patient anatomical image; receive a baseline anatomical image having pre-segmented labels, wherein the pre-segmented labels identify regions of interest in the baseline anatomical image; align the patient anatomical image with the baseline anatomical image to produce a transformation that when applied to the pre-segmented labels roughly identifies regions of interest in the patient anatomical image that correspond to the regions of interest in the baseline anatomical image; and update the pre-segmented labels, which have been deformed by application of the transformation, with a new transformation that minimizes the likelihood of intensity distributions within the regions of interest of the patient anatomical image to produce a gradient image that better identifies the regions of interest of the patient anatomical image.
The processor is further operative with the program to compute the new transformation, wherein when computing the new transformation the processor is further operative with the program to: compute a gradient for all the regions of interest of the patient anatomical image; regularize the gradient; and generate the new transformation by using the regularized gradient.
The new transformation is applied to the deformed pre-segmented labels by computing a composition of the deformed pre-segmented labels and the new transformation.
When computing the gradient for all the regions of interest of the patient anatomical image the processor is further operative with the program to: (1) for a region of interest of the patient anatomical image, compute a temporary image for the region of interest; compute an intensity distribution for the region of interest; and compute a gradient for the region of interest; (2) update the gradient image with the gradient for the region of the interest; and repeat (1) and (2) until the gradient image has been updated with a gradient for all the regions of interest of the patient anatomical image.
The pre-segmented labels are repeatedly updated with new transformations until all the regions of interest of the patient anatomical image are better identified.
The patient anatomical image comprises an abdomen.
The patient anatomical image is a CT image.
In an exemplary embodiment of the present invention, a computer readable medium tangibly embodying a program of instructions executable by a processor to perform method steps for segmenting an anatomical image is provided, the method steps comprising: receiving a patient anatomical image; receiving a baseline anatomical image having pre-segmented labels, wherein the pre-segmented labels identify regions of interest in the baseline anatomical image; aligning the patient anatomical image with the baseline anatomical image to produce a transformation that when applied to the pre-segmented labels roughly identifies regions of interest in the patient anatomical image that correspond to the regions of interest in the baseline anatomical image; and updating the pre-segmented labels, which have been deformed by application of the transformation, with a new transformation that minimizes the likelihood of intensity distributions within the regions of interest of the patient anatomical image to produce a gradient image that better identifies the regions of interest of the patient anatomical image.
The method steps further comprise computing the new transformation, wherein computing the new transformation comprises: computing a gradient for all the regions of interest of the patient anatomical image; regularizing the gradient; and generating the new transformation by using the regularized gradient.
The new transformation is applied to the deformed pre-segmented labels by computing a composition of the deformed pre-segmented labels and the new transformation.
Computing the gradient for all the regions of interest of the patient anatomical image comprises: (1) for a region of interest of the patient anatomical image, computing a temporary image for the region of interest; computing an intensity distribution for the region of interest; and computing a gradient for the region of interest; (2) updating the gradient image with the gradient for the region of the interest; and repeating (1) and (2) until the gradient image has been updated with a gradient for all the regions of interest of the patient anatomical image.
The pre-segmented labels are repeatedly updated with new transformations until all the regions of interest of the patient anatomical image are better identified.
The patient anatomical image comprises an abdomen.
The patient anatomical image is a CT image.
The foregoing features are of representative embodiments and are presented to assist in understanding the invention. It should be understood that they are not intended to be considered limitations on the invention as defined by the claims, or limitations on equivalents to the claims. Therefore, this summary of features should not be considered dispositive in determining equivalents. Additional features of the invention will become apparent in the following description, from the drawings and from the claims.
A hierarchical multi-label segmentation method based on non-rigid registration techniques to segment an arbitrary number of regions, according to an exemplary embodiment of the present invention, will hereinafter be described. In an exemplary embodiment of the method, first align an image IS, with pre-segmented labels IT
A description of the statistical formulation of region-based segmentation will now be provided.
Let Ω ε Rd be open and bounded, and I:Ω→R be the image to be segmented. Assume that Ω is a partition composed of N independent disjoint regions Ωi. This gives the simplified expression:
where p(I|Ωi) denotes the probability of the image I where Ωi is the region of interest. Assume that values of I at different locations of the same region can be modeled as an independent and identically distributed realization of the same random process. Define pi(I(x)) as the probability density function of a random variable modeling intensity values I(x) in Ωi. Given this model, the optimal partition can be obtained using a maximum likelihood principle, and minimizing the following energy proposed in [Zhu, S. C., Yuille, A. L.: Region competition: Unifying snakes, region growing, and bayes/MDL for multiband image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18(9), 1996, pp. 884-900], the disclosure of which is incorporated by reference herein in its entirety:
In the context of contour evolution, this energy can be expressed as the following energy to minimize:
where Γi represent the contour of the region Ωi, and the parameter ν controls the length of the contours. In particular, this energy is expressed in the context of level sets with a function φi that represents the region Ωi where φi(x)>0 if and only if x ε Ωi:
This formulation does not respect implicitly the condition of disjoint regions, but the minimization of this energy ensures that a pixel is assigned to only one region according to the maximum likelihood principle.
A description of the method for non-rigid registration according to an exemplary embodiment of the present invention will now be provided.
In the following description, given two images I1 and I2, the registration problem is formulated as finding a mapping φ:Ω→Ω that maximizes a similarity measure between the images: S(I1, I2∘φ). First, maximize the local cross correlation between I and IS, SLCC(I,IT
To find the optimal high-dimensional transformation, a sequence of transformations (φk)k=0, . . . ,+∞, is built by composition of small displacements as described in [Chefd'hotel, C., Hermosillo, G., Faugeras, O.: Flows of diffeomorphisms for multimodal image registration. In: Proceedings of IEEE International Symposium on Biomedical Imaging. (2002), pp. 753-756], the disclosure of which is incorporated by reference herein in its entirety,
φk+1=φk∘(φid+ανk), φ0=φid, (5)
where φid is the identity transformation and νk is a velocity vector field that follows the gradient of the cost function to be minimized. Here, νk is obtained by computing the variational gradient of the cost function of the Local Cross-Correlation (LCC) similarity measure, i.e., ∇SLCC(I,IS∘φ) or the ML similarity measure ∇SML(I,IT
The gradient νk is regularized using a fast recursive filtering technique. This approximates a Gaussian smoothing, as described, for example, in [Deriche, R.: Recursively implementing the Gaussian and its derivatives. In: Proceedings of the International Conference on Image Processing, Singapore (September 1992), pp. 263-267], that has proven very efficient in practice. Here, deriving the similarity measure energy according to a high-dimensional transformation results in a vector field ν. To guarantee a well-posed problem, this vector field has to be regularized. For this purpose, different techniques have been proposed. The approach proposed in [Christensen, G. E., Rabbit, R. D., Miller, M. I.: Deformable templates using large deformation kinematics. IEEE Transactions on Image Processing, vol. 5(10), 1996, pp. 1435-1447], the disclosure of which is incorporated by reference herein in its entirety, solves the registration problem using a partial differential equation and has the advantage of capturing large deformations. In the method according to an exemplary embodiment of the present invention, a Gaussian filtering is used that can be seen as a variant of the fluid-approach described in Christensen et al.
The previous iterative scheme (Eq. 5) is repeated until convergence, and can be seen as the discretization (via Taylor expansion) of the transport equation in the Eulerian frame:
where Dφt stands for the Jacobian matrix of φt. Here, large deformations are possible because the regularization is applied to the velocity rather than the deformation described in [Dupuis, P., Grenander, U., Miller, M.: Variational problems on flows of diffeomorphisms for image matching. Quarterly of Applied Mathematics LVI(3), (1998), pp. 587-600], which details the suitable regularity conditions on the velocity field to generate a diffeomorphism.
The method according to an exemplary embodiment of the present invention is embedded in a coarse-to-fine strategy. This reduces the computational cost by working with less data at lower resolutions. This also allows large displacements to be recovered, and helps avoiding local minima. In the method according to an exemplary embodiment of the present invention, five-levels of multi-resolutions are used.
To refine the segmentation, in accordance with an exemplary embodiment of the present invention, a multi-labeled template matching algorithm that recovers local deformations of the shape obtained in the previous section is provided. Consider the registration framework, an image IT
Formulate the problem as finding a transformation φ:Ω→Ω that minimizes the likelihood between the intensity distribution functions of different regions pi according to I and IT
In this equation, IT
The density probability function of different regions is as follows:
With the method according to an exemplary embodiment of the present invention, local shape variations are found by deforming the multi-labeled image IT
Algorithm 1 (show below) describes how to compute the gradient of the similarity measure ∇S(I,IT
A description of experimental results of the multi-label segmentation method according to an exemplary embodiment of the present invention will now be provided.
In image (b) of
The liver segmentation result was compared to a ground-truth using five metrics: volumetric overlap, relative absolute difference, average symmetric absolute surface distance, symmetric RMS surface distance and maximum symmetric absolute surface distance. These metrics were evaluated using by assigning a score as described, for example, in [van Ginneken, B., Heimann, T., Styner, M.: 3d segmentation in the clinic: A grand challenge. In: 3D Segmentation in the Clinic: A Grand Challenge, MICCAI 2007 (2007), pp. 7-15]. Table 1 (shown below) presents the segmentation results.
In
After the images I and IS are input, they are aligned (210). This is done by using the fluid-based technique described by equations 5 and 6 with an LCC similarity measure, for example. The result of this alignment is a mapping/transformation φ*. This mapping/transformation φ* is applied to IT
Now the roughly-initialized (e.g., deformed) pre-segmented labels image TT
Using the image I and the roughly-initialized pre-segmented labels image TT
The left-hand side of
A system in which exemplary embodiments of the present invention may be implemented will now be described with reference to
It is understood that the present invention may be implemented in various forms of hardware, software, firmware, special purpose processors, or a combination thereof. In one embodiment, the present invention may be implemented in software as an application program tangibly embodied on a program storage device (e.g., magnetic floppy disk, RAM, CD ROM, DVD, ROM. and flash memory). The application program may be uploaded to, and executed by, a machine comprising any suitable architecture.
It is also understood that because some of the constituent system components and method steps depicted in the accompanying figures may be implemented in software, the actual connections between the system components (or the process steps) may differ depending on the manner in which the present invention is programmed. Given the teachings of the present invention provided herein, one of ordinary skill in the art will be able to contemplate these and similar implementations or configurations of the present invention.
It is further understood that the above description is only representative of illustrative embodiments. For convenience of the reader, the above description has focused on a representative sample of possible embodiments, a sample that is illustrative of the principles of the invention. The description has not attempted to exhaustively enumerate all possible variations. That alternative embodiments may not have been presented for a specific portion of the invention, or that further undescribed alternatives may be available for a portion, is not to be considered a disclaimer of those alternate embodiments. Other applications and embodiments can be implemented without departing from the spirit and scope of the present invention.
It is therefore intended, that the invention not be limited to the specifically described embodiments, because numerous permutations and combinations of the above and implementations involving non-inventive substitutions for the above can be created, but the invention is to be defined in accordance with the claims that follow. It can be appreciated that many of those undescribed embodiments are within the literal scope of the following claims, and that others are equivalent.
This application claims the benefit of U.S. Provisional Application No. 61/032,237, filed Feb. 28, 2008, the disclosure of which is incorporated by reference herein in its entirety.
Number | Date | Country | |
---|---|---|---|
61032237 | Feb 2008 | US |