1. Field of the Invention
The present invention relates to volumetric image data characterization, and more particularly to a system and method for 3D segmentation of targets in multislice images.
2. Description of Related Art
One goal of the computer-assisted diagnosis with the chest CT scans (chest CAD) is reliable volumetric measurement of the pulmonary nodules. Tumor change qualification based on such volume measurements plays an integral part of the cancer therapy monitoring and post-surgical examinations. There are a number of previous studies addressing the computer-assisted volume measurement of nodules. In these studies, the 2D or 3D tumor segmentation based on voxel intensity thresholding is used as the foundation of their solutions. Although such solutions are sufficient to delineate the well-defined solid nodules with similar average intensity, they provide unreliable segmentation for the part- or non-solid nodules, as shown in FIGS. 1A-B. A recent clinical study has revealed that such nodules occur frequently and have a higher tendency to be malignant, motivating the development of the robust solution for these technically challenging cases.
FIGS. 1A-B illustrate 2D examples of the pulmonary nodule segmentation.
Therefore, a need exists for a system and method for 3D segmentation of targets in multislice images.
According to an embodiment of the present disclosure, a method for three-dimensional segmentation of a target in multislice images of volumetric data comprises determining a center and a spread of the target by a parametric fitting of the volumetric data, and determining a three-dimensional volume by non-parametric segmentation of the volumetric data iteratively refining the center and spread of the target in the volumetric data.
Determining the center and the spread of the target comprises providing a marker in the volumetric data for an initial target location, determining a region around the initial target location, modeling the region around a spatial extremum, and determining the center and spread of the target given the model of the region. Modeling comprises implementing an anisotropic three-dimensional Gaussian intensity model.
Determining the three-dimensional volume comprises determining a set of four-dimensional data points from the volumetric data, determining a bandwidth according to the determined center and spread of the target, and determining successive estimates of the center and spread that converge to a most stable center and spread. The most stable center and spread are determined by a Jensen-Shannon divergence profile.
Determining a three-dimensional volume is performed iteratively for clustering data points in the volumetric data according to spatial and intensity proximities simultaneously.
Determining a three-dimensional volume comprises a mean-shift ascent defining a basin of attraction of the target in a four-dimensional spatial-intensity joint space.
The center is determined according to a given marker, wherein the center is a point in the volumetric data to which the marker converges. The spread is determined as a covariance of the center.
According to an embodiment of the present disclosure, a program storage device is provided readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for three-dimensional segmentation of a target in multislice images of volumetric data. The method steps comprising determining a center and a spread of the target by a parametric fitting of the volumetric data, and determining a three-dimensional volume by non-parametric segmentation of the volumetric data iteratively refining the center and spread of the target in the volumetric data.
Preferred embodiments of the present invention will be described below in more detail, with reference to the accompanying drawings:
FIGS. 1A-B illustrate 2D examples of the pulmonary nodule segmentation;
FIGS. 1C-D illustrate 2D examples of the pulmonary nodule segmentation according to an embodiment of the present disclosure;
FIGS. 5A-H are examples of 3D estimation and segmentation according to an embodiment of the present disclosure.
According to an embodiment of the present disclosure, a robust and accurate method for segmenting the 3D pulmonary nodules in multislice CT scans unifies the parametric Gaussian model fitting of the volumetric data evaluated in Gaussian scale-space and non parametric 3D segmentation based on normalized gradient (mean shift) ascent defining the basis of attraction of the target tumor in the 4D spatial-intensity joint space. This realizes the 3D segmentation according to both spatial and intensity proximities simultaneously. Experimental results show that the system and method reliably segment a variety of nodules including part- or non-solid nodules that poses difficulty for the existing solutions. The system and method also process a 32×32×32-voxel volume-of-interest efficiently by six seconds on average.
Referring to
FIGS. 1C-D illustrate 2D examples of the pulmonary nodule segmentation.
It is to be understood that the present invention may be implemented in various forms of hardware, software, firmware, special purpose processors, or a combination thereof. In one embodiment, the present invention may be implemented in software as an application program tangibly embodied on a program storage device. The application program may be uploaded to, and executed by, a machine comprising any suitable architecture.
Referring to
The computer platform 301 also includes an operating system and microinstruction code. The various processes and functions described herein may either be part of the microinstruction code or part of the application program (or a combination thereof), which is executed via the operating system. In addition, various other peripheral devices may be connected to the computer platform such as an additional data storage device and a printing device.
It is to be further understood that, because some of the constituent system components and method steps depicted in the accompanying figures may be implemented in software, the actual connections between the system components (or the process steps) may differ depending upon the manner in which the present invention is programmed. Given the teachings of the present invention provided herein, one of ordinary skill in the related art will be able to contemplate these and similar implementations or configurations of the present invention.
The volumetric CT data is formalized 402 as a continuous positive function I:R+3→R+ over the data space X=(X1X2,X3). A local region of I(x) around a spatial extremum u 403, expressing a pulmonary tumor, is modeled by the anisotropic 3D Gaussian intensity model,
where α is an amplitude parameter, Σ is a fully-parameterized 3×3 symmetric positive definite covariance matrix, and S is a set of data points in the neighborhood of u, belonging to the basin of attraction of u. The mean u and covariance Σ of Φ describes the location and spread of a target 404, e.g., the tumor, respectively. Thus, the problem can be understood as parametric model fitting or robust estimation of (u,Σ) given I(x). Anisotropy of the tumor can be described by considering the estimation of the fully-parameterized covariance.
Multi-scale analysis is employed for this estimation, given a set of ordered and densely sampled analysis scales (bandwidth) {hk|k=1, . . . ,K}. The model mean and covariance are robustly estimated, by the method described below, for each analysis scale hk, resulting in a set of successive estimates {(uk,Σk)}. A result is given by finding the most stable estimate using a divergence-based stability test. The most stable estimate (u*,Σ*) is defined as the estimate with the scale hk* that assumes a local minimum of the modified Jensen-Shannon divergence profile over the scales. At each scale hk, the divergence is determined over three neighboring scales.
For each analysis bandwidth hk, (uk,Σk) are estimated by scale-space mean shift analysis together with the robust estimation technique based on the basin of attraction. Gaussian scale-space of I(x), or the solution of the diffusion equation ∂hI=½∇2I, is defined by a convolution of I(x) with a set of Gaussian kernels with the analysis scales {h>0}, L(x;H)=f(x)*φ(x;0,H), where H is an isotropic bandwidth matrix of a form H=hI. Its gradient vector is given by,
The vector m(x) is called scale-space mean shift vector and proportional to the gradient vector ∇L(x;H). A convergent iterative method for the normalized gradient ascent in the scale-space xk+1=m(xk)+xk, is used to estimate the tumor center uk, to which the given marker xp converges. To increase the robustness, a set of the gradient ascents are preformed from different initial points samples uniformly around xp. The convergence point of the majority of the initial points defines the center estimate uk.
Given the center estimate, the corresponding covariance Σk, is estimated. Substituting the Gaussian tumor model Eq.(1) to the definition of the scale-space mean shift Eq.(4) reveals that the mean shift can be expressed as a quasi-linear matrix equation, m(x;H)=H(Σ+H)−1(u−x). An over complete set of the linear equations with unknown Σ is constructed by using the mean shift vectors sampled within the basin of attraction of the target tumor. For this, a set of the mean shift iterations are performed from different initial points that are sampled uniformly around the center estimate uk. Nu mean shift ors along convergent trajectories are used for constructing the over complete system,
AΣ=B, ΣεSPD (5)
A=(m(x1; H), . . . , m(xN
B=(b1, . . . ,bN
where SPD denotes a set of all symmetric positive definite matrices in R3×3. A closed-form solution of this constrained system may be given by minimizing an area criterion ∥AY−B−t∥F2 where Y is Cholesky factorization of Σ=YYt and ∥.∥F is the Frobenius matrix norm. The solution may be expressed by a function of symmetric Schur decompositions of P=AtA and {tilde over (Q)}=ΣpUptQUpΣp, given Q≡BtB.
The robustness of the solution is endowed by using the information only within the basin of attraction, which effectively suppresses outliers.
This parametric estimation step yields the estimates of the 3D tumor center and tumor spread in the form of 3D mean vector u* and 3×3 covariance matrix Σ*. Also provided the bandwidth h* that yields the above estimate which are most stable among others. The center and spread estimates can be interpreted as the normal probability distribution g(x) of the center estimate,
According to an embodiment of the present disclosure the non-parametric 3D nodule segmentation is based on defining the basin of attraction of the target nodule in the 4D spatial-intensity joint space (see
Referring to
where N=IId=13=Nd and Nd is the number of voxels along the dimension d. Therefore, in the spatial-intensity joint space, the discretized samples {I(xi)} are interpreted as a set of 4D data points {yi≡(xi,I(xi)} 405. The sample density estimate with normal kernel with a 4×4 bandwidth matrix H (406, see below) is given at a data point y by,
Consequently, the gradient of the density f(y) is given by,
The vector m(y) is the density mean shift in the 4D joint space. A convergent iterative method for the normalized density gradient ascent is obtained by,
yk+1=m(yk)+yk (16)
The iterator Eq.(16) is employed to cluster the data points according to both spatial and intensity proximities simultaneously. The points belonging to the basin of attraction of the target nodule are detected by applying Eq.(16) from a set of initial points, sampled according to the normal prior in Eq.(11), until convergence at ∇f(y)=0. Initial points are sampled with a confidence interval of the 3D normal distribution between plo and pup percentiles 407. The points that converge to the vicinity of (u*,mI) are merged into a cluster that defines the target nodule 408. The points with the probability above pup are also considered to be a part of the nodule. For each point yi in the joint space, there is only one corresponding point xi in the 3D data space. Thus, the cluster membership of yi is directly associated with the data point xi resulting in the segmentation of the tumor and background in the data space x.
This method achieves both robust segmentation without intensity thresholding and insensitivity to variation of the intensity range, in comparison with the global threshold-based approach. However, for achieving the robustness in the segmentation results, it is important that the kernel bandwidth H is set appropriately for a given data {I(x1)}. H is determined by exploiting the normal prior 406. H is formed as a diagonal matrix with the most stable bandwidth h* and the variance estimate of the intensities of σI2.
H=diag(h*, h*, h*, σI2) (17)
where σI2 is given by the sample variance of the intensity values within a q-percentile confidence ellipsoid of the normal distribution g(x),
The sample means of the set of the intensity values and the number of voxels within the confidence ellipsoid are denoted by mI and Nσ, respectively. The parameter c is directly derived from the specific choice of the percentile q. The segmentation procedure using Eq.(16) is carried out using the mean shift vectors computed with the resulting bandwidth matrix.
The segmentation methods have been evaluated with a database of clinical multislice chest CT scans with 1 mm2×1.5 mm slice thickness, containing 77 nodules of 14 patients. The size of the nodules ranges between 3 mm and 25 mm in diameter. The data is also provided with the markers xp and the classification labels for the part- or non-solid nodules given by radiologists. The database includes i) 6 cases of the part- or non-solid nodules, ii) 28 cases of small nodules whose size is less than 5 mm, iii) 20 cases of nodules attached to the pleural surface, iv) 12 cases of largely non-spherical (anisotropic) nodules. An implementation of the above method is instantiated with the following settings. For the scale-space anisotropic intensity model fitting, a set of 25 analysis scales h={0.52,0.752, . . . ,6.52} are used. For 4D joint space segmentation, the confidence limits for sampling the initial points and for estimating the sample intensity variance are set to plo=25%, pup=75%, and q=75%, respectively.
The performance evaluation of the system resulted in the correct parametric fits and non-parametric segmentations for 69 nodules by expert inspection. The 8 failures were due to i) small nodules attached to pleural surface (6 cases), ii) small vascularized nodule (1 case), iii) elongated nodule (1 case). All the part- or non-solid and solitary small nodules were correctly estimated and segmented. The rejection criterion based on chi-square residual analysis Eq.(9) is applied, resulting in successful rejection of all the failure cases attached to pleural surfaces. FIGS. 5A-H illustrate examples of the results. Each image is a 2D dissection of the target volume intersecting the estimated nodule center. The estimation results from the first step are visualized as an intersection of 50%-confidence ellipsoid of the normal prior Eq.(11). The system and method's sensitivity to the initial marker locations is reduced by randomly perturbing the markers within the 50%-confidence limit range, using 36 nodules. The average error of the mean and covariance estimates from total average of perturbation were 1.12 voxel and 8.21 Frobenius matrix norm, respectively. The results show the robustness against the uncertainty of marker location.
The robust and accurate methods for segmenting the 3D pulmonary nodules in multislice CT scans unify the parametric and non-parametric algorithms, realizing accurate and efficient 3D segmentation according to both spatial and intensity proximities simultaneously. Referring again to
Referring to FIGS. 5A-H, examples of the 3D estimation and segmentation results are projected to a 2D plane for visualization. In each figure, row (1) depicts 2D profile of input nodules, row (2) depicts parametric fitting results (“+”:xp, “×”:u*,ellipse:Σ*), and row (3) depicts non-parametric segmentation results. FIGS. 5A-B depict non-solid targets,
Having described embodiments for a system and method for 3D segmentation of targets in multislice images, it is noted that modifications and variations can be made by persons skilled in the art in light of the above teachings. It is therefore to be understood that changes may be made in the particular embodiments of the invention disclosed which are within the scope and spirit of the invention as defined by the appended claims. Having thus described the invention with the details and particularity required by the patent laws, what is claimed and desired protected by Letters Patent is set forth in the appended claims.
This application claims priority to U.S. Provisional Application Ser. No. 60/552,481, filed on Mar. 12, 2004, which is herein incorporated by reference in its entirety.
Number | Date | Country | |
---|---|---|---|
60552481 | Mar 2004 | US |