a)-2(e) illustrates the cross-sectional boundary extraction algorithm step by step on a CE-MRA data used in the method of
a)-3(c) illustrates the mean-shift filtering of a typical edge;
a)-6(f) illustrate the results of artery vein separation algorithm according to the invention on two different MRA data sets on two different MRA data;
c) and 6(f) showing an arterial map;
Like reference symbols in the various drawings indicate like elements.
Referring now to
More particularly, the local vessel modeling; discrete centerline models; and ordered statistical front propagation, Step 102 includes partial vessel segmentation, Step 102A and ordered front propagation, Step 102B.
Partial vessel segmentation, Step 102A includes: segmenting the vessels using local vessel modeling, Step 1021; and developing statistical front propagation and modeling from the local vessel modeling, Step 1022. The segmenting of the vessels using local vessel modeling (Step 1021) comprises: placing seed points on an image of a portion of the patient. The developing statistical front propagation and modeling from the local vessel modeling for the each seed point (Step 1022) comprises: locally estimating vessel and surrounding background statistics by computing vessel orthogonal planes and corresponding cross-sectional boundaries; and segmenting vessels in a limited area partially based on a front propagation algorithm using the estimated statistics.
Next, the method uses ordered front propagation; Step 102B using discrete centerline modeling comprises re-estimating vessel statistics using discrete fronts and surface filling. More particularly, ordered front propagation, Step 102B, includes applying discrete centerline modeling using the developed statistical front propagation and modeling comprising re-estimating vessel statistics using discrete fronts and surface filling, Step 1023. Next, the method determines a measure of accuracy of each front using a discrete centerline model obtained by minimal path detection operating a distance map and re-starts partial segmentation from the front having the highest confidence measure representing the correct vessel, Step 1024. The process iteratively performing the partial segmentation process (i.e., Steps 1021-1022) and Steps 1023-1024 of the ordered front propagation process to segment arterial and venous vessels, independently; each one of the arterial and venous vessels having a separate arterial and venous vessels map, Step 1025.
The method, in Step 104, combines the independently segmented arterial and venous vessels maps into a single map, Step 1041. The arterial and venous vessels in the singe map are then separated by a distance-based watershed transform using discrete centerline models between seeds used as the markers for the watershed transforms. Step 1042.
Here, the radiologist or user places seed points on an image of a portion of the patient; For each seed point, the process locally estimates vessel and surrounding background statistics by computing vessel orthogonal planes and corresponding cross-sectional boundaries, Step 1021.
Local background/foreground separation can be accurately accomplished if the parameters of the normal distribution are estimated correctly. The method uses geometric tubular models to estimate these parameters. (Reference is briefly made to
Most popular approach for computing vessel specific coordinate system is based on the eigenvalue analysis of Hessian matrix, e.g., A. F. Frangi, W. J. Niessen, K. L. Vincken, and M. A. Viergever. Multiscale vessel enhancement filtering. In MICCAI, pages 82-89, 1998; O. Wink, W. J. Niessen, and M. A. Viergever. Multiscale vessel tracking. IEEE Trans. on Medical Imaging, 23(1):130-133, 2004; M. Descoteaux, L. Collins, and K. Siddiqi. Geometric flows for segmenting vasculature in mri: Theory and validation. In Medical Image Conference and Computer Assisted Interventions (MICCAI), 2004; and S. Aylward and E. B. E. Initialization, noise, singularities, and scale in height-ridge traversal for tubular object centerline extraction. IEEE Trans. on Medical Imaging, 21(2):61-75, 2002. Typically, a 3D image is filtered with normalized second order Gaussian derivatives to compute the Hessian matrix at each location.
The eigenvalues, λ1, λ2, λ3, corresponds to the derivatives along the eigenvectors, e1, e2, e3 where the eigenvector e1 corresponding to the smallest eigenvalue is in the direction of tube and the other eigendirections e1, e2 defines the cross-sectional plane, thus forming an orthonormal coordinate system. In this paper, we use the vesselness measure proposed in [9] to find the approximate direction of the vessel of interests. Specifically,
and α, β and γ are the parameters. Since vessels in 3D images can be in different sizes, images are convolved with multi-scale filters and maximum response is taken as a vesselness measure at point.
The intensity distribution of a partial vessel segment is determined from the 2D cross-sectional boundaries. There have been several techniques for computing vessel cross-sectional boundaries (see O. Wink, W. Niessen, and M. A. Viergever. Fast delination and visualization of vessels in 3-D angiographic images. IEEE Trans. on Medical Imaging, 19:337-345, 2000; M. Hernandez-Hoyos, A. Anwander, M. Orkisz, J. P. Roux, and I. E. M. P. Doueck. A deformable vessel model with single point initialization for segmentation, quantification and visualization of blood vessels in 3D MRA. In MICCAI'00, 2000. and H. Tek, A. Ayvaci, and D. Comaniciu. Multi-scale vessel boundary detection. In Workshop of CVBIA, pages 388-398, 2005. Here, the method uses the technique developed by H. Tek, A. Ayvaci, and D. Comaniciu. Multi-scale vessel boundary detection. In Workshop of CVBIA, pages 388-398, 2005) since it separates the intensities of a vessel and its surrounding areas by a multi-scale mean shift filtering, see for example Ser. No. 11/399,164 entitled “Method and Apparatus For Detecting Vessel Boundaries”, filed Apr. 6, 2006, inventor Huseyin Tek, assigned to the same assignee as the present invention (Pub. No. US 2006/0262988 A1).
Specifically, first, edges along a ray are computed in several scales by using mean shift analysis D. Comaniciu and P. Meer. Mean shift: A robust approach toward feature space analysis. IEEE Trans. PAMI, 24(5):603-619, 2002.
Second, incorrect edges obtained from multiple scales are eliminated by the mean-shift based clustering algorithm.
Third, prominent edges are obtained by selecting edges based on their strengths, and the assumption that vessels are not nested structures.
Fourth, smooth and long curve segments are constructed from prominent edges by a local grouping algorithm.
Finally, the curve segments that best describe the vessel boundary are determined from the elliptic shape priors.
The method is capable of segmenting vessel boundaries in great detail even in the extreme conditions.
The normal distribution parameters of vessel N(Vμ,Vσ) and its immediate surroundings (or background) N(Bμ,Bσ) are obtained from the local mode of mean-shift filtering.
Next, in Step 1022, for the each seed point, the method locally estimates vessel and surrounding background statistics by computing vessel orthogonal planes, as shown and described above, for one orthogonal plane 800 in
More particularly, once the parameters of normal distribution are determined, accurate partial vessel segmentation can be easily accomplished by region growing or region competition via watershed transforms, see H. Tek, F. Akova, and A. Ayvaci. Region competition via local watershed operators. In CVPR, pages 361-368, 2005, see for example Ser. No. 10/951,194 entitled “Local Watershed Operators For Image Segmentation”, filed Sep. 27, 2004, inventor Huseyin Tek, assigned to the same assignee as the present invention (Pub. No. US 2005/0201618 A1); and Ser. No. 11/231,424 entitled “Region Competition Via Local Watershed Operation”, filed Sep. 21, 2005, inventor Huseyin Tek, et al., assigned to the same assignee as the present invention (Pub. No. US 2006/0098870 A1).
Let us now illustrate how these partial segmentation are grouped by the process to obtain segmentation of the whole branch.
Next, in Step 1023, vessel statistics are re-estimated using discrete fronts and surface filling. More particularly, the accuracy of vessel segmentation results by continuing the above process heavily depends on where the segmentation re-starts from. Note that the voxels in binary segmentation obtained from the above algorithm are further classified as converged and alive points. Converged points correspond to the voxels which cannot extend the segmentation because there is no vessel point in their neighborhood. Alive points correspond to voxels that are stopped by the distance constraint and their neighborhood contains voxels that can be classified as vessel. If the alive points were not stopped by the distance constraint, they could have segmented vessels and other bright structures. Thus, the alive points must be grouped and classified whether they represent vessels or non-vascular structures. Specifically, voxels from the alive point list are grouped to each other via discrete connectivities, which are called discrete fronts.
The process used to obtain discrete fronts from the alive point list is as follows: First, all alive points are marked as one in an empty map whose voxel values are set to zero. Second, a point from alive list is selected and the voxels that are connected to it are determined by a simply surface filling process. During the surface filling process, the connected voxels are set to empty and they are removed from the alive list. This process determines a single discrete front. The other fronts are determined by repeating this process until there is no more point left in the alive list.
Next, in Step 1024, the process re-estimates vessel statistics using discrete fronts and surface filling. More particularly, at this stage, the process has obtained K discrete fronts from which new partial segmentation needs to start. Since not every discrete front corresponds to the correct vessel, the process first assigns a confidence measure to each front. This confidence measure can be computed from the vesselness measure (see A. F. Frangi, W. J. Niessen, K. L. Vincken, and M. A. Viergever. Multiscale vessel enhancement filtering. In MICCAI, pages 82-89, 1998) the surface area and smoothness of the fronts and the characteristics of the centerlines between the discrete front and the source of partial segmentation, namely the seed point. Here, the process first computes the vesselness measure for a point representing the discrete front. If the vesselness measure is relatively high, a confidence measure based on the centerline models is computed. Observe that voxels representing the fronts must be determined before these algorithms are applied. Here, the voxel representing a front best must be located in the center of the front. Algorithmically, the process first computes the distance map of the segmented area starting from the convergence points and then selects the voxels with highest distance value for the each front. After obtaining a representative point for each front, the process computes the discrete centerlines between them and the source (seed), via a minimal path detection algorithm (based on Dijkstra's method). The cost function (or speed function) this algorithm is computed from the distance transform of segmented vessel and the distance between voxels. Specifically, the cost function for a voxel Vj visited by voxel Vi is given by Cost(Vj)=(1/DT(Vj))*dist(Vj,Vi) where DT is the distance transform of the segmented vessels and dist(Vj,Vi) measures the unsigned distance from Vj to Vi. The DT value forces the front to propagate near the vessel centers. A similar algorithm is presented by Deschamps and Cohen (see T. Deschamps and L. Cohen. Fast extraction of minimal paths in 3d images and applications to virtual endoscopy. Medical Image Analysis, 5(4):281-299, 2001) for finding paths in tubular structures such as the colon. This minimum cost path detection algorithm results in a discrete path consisting of ordered discrete voxel locations.
The process next, in Step 1025, re-starts partial segmentation from the front having the highest confidence measure representing the correct vessel. More particularly, the discrete centerlines between the source and each discrete front have been obtained. Confidence measure for the each front is then determined from the radius function along their centerlines. Ideally, fronts which represent the continuing vessels with the highest confidence measure should have almost constant radius profile assuming that the partial segmentation is applied to the relatively small area i.e., vessel size does not change drastically. On the other hand, if a front is inside another type of vessel or in a non-vascular structure, its radius profile should have abrupt changes. In fact, partial segmentation often leaks into the bright tissues and other vessels from gaps on the boundary, where radius values are very small. Based on these observations, the process assigns each front a confidence measure based on the smoothness of the radius values along its centerline model. The segmentation process re-starts from the front that has the most confidence measure. The center of the front is marked as virtual seed (or source) and the parameters of the normal distribution of the vessel is recomputed by the method described above. This ordered-based partial segmentation continues until all the connections between the user placed seed points are established or all the discrete fronts are propagated.
The process iteratively performs the partial segmentation process to segment arterial and venous vessels, independently; each one of the arterial and venous vessels having a separate arterial and venous vessels map. More particularly, the method combines the independently segmented arterial and venous vessels maps into a single map, Step 1041. The arterial and venous vessels in the singe map are then separated in Step 1042 by a distance-based watershed transform using discrete centerline models between seeds used as the markers for the watershed transforms.
More particularly, the segmentation process described above is applied for the segmentation of arteries and veins, separately (i.e., independently). The resulting discrete artery (or venous) map often includes areas corresponding to veins (or arteries), thus simple combination of these maps does not suffice for an accurate separation. In this paper, we propose to use the distance based watershed transforms (see L. Vincent and P. Souille. Watersheds in digital spaces: an efficient algorithm based on immersion simulations. PAMI, 13(6):583-598, 1991 and H. Tek and H. C. Aras. Local watershed operators for image segmentation. In Medical Image Computing and Computer-Assisted Intervention MICCAI, pages 127-134, 2004 for the separation of these two maps.) See Ser. No. 10/951,194 entitled “Local Watershed Operators For Image Segmentation”, filed Sep. 27, 2004, inventor Huseyin Tek, assigned to the same assignee as the present invention (Pub. No. US 2005/0201618 A1); and Ser. No. 11/231,424 entitled “Region Competition Via Local Watershed Operation”, filed Sep. 21, 2005, inventor Huseyin Tek, et al., assigned to the same assignee as the present invention (Pub. No. US 2006/0098870 A1).
In image segmentation, the gradient map of images is used as a height map in watershed based segmentation algorithms. Here, the process uses a distance-transform of the combined artery-vein segmentation maps. Specifically, the inverse distance map is quite suitable for the separation of two masks since artery and vein maps often overlap in small areas. In addition, here, the method uses the discrete centerline models between the user placed seeds as the markers for the watershed transforms. These discrete centerline models are computed by a minimal path detection algorithm operating the distance of the segmentation results. Since the user placed seed points are very strong clues for the correct labeling, the separation algorithm should use such information as much as possible. Thus, the distance map is further modified by the addition of a potential function created in the vicinity of the user placed seed points. This potential function for each seed forces the discrete centerline pass away from the other types of vessels.
Furthermore, the validity of the centerline models is verified by reconstructing the vessels from them. If the reconstruction from the arteries and veins overlap significantly, the corresponding centerline models are not used in separation. It should also be noted that it is not always possible to separate arteries and veins in certain areas. This is true especially, when the arteries and veins touch each other over a large region. Such region corresponds to a single basin in watershed map. Thus, here a basin-partitioning algorithm based on distance transforms is used. Specifically, if the user placed seed points or centerline models pass through a same basin, this basin is partitioned based the distance transform from these inputs. This basin partitioning property allows the user to be able to correct any kind of errors by placing additional seeds.
a)-6(f) illustrates the results of artery vein separation algorithm according to the invention on two different MRA data sets. These
The method described above may be performed by an appropriately programmed computer. An appropriate computer may be implemented, for example, using well-known computer processors, memory units, storage devices, computer software, and other components. A high-level block diagram of such a computer is shown in
One skilled in the art will recognize that an implementation of an actual computer will contain other components as well, and that
A number of embodiments of the invention have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the invention. Accordingly, other embodiments are within the scope of the following claims.
This application claims priority from U.S. Provisional application Ser. No. 60/793,860 filed on Apr. 21, 2006 which is incorporated herein by reference. This patent application incorporates by reference the following co-pending patent applications: Ser. No. 10/951,194 entitled “Local Watershed Operators For Image Segmentation”, filed Sep. 27, 2004, inventor Huseyin Tek, assigned to the same assignee as the present invention (Pub. No. US 2005/0201618 A1); Ser. No. 11/231,424 entitled “Region Competition Via Local Watershed Operation”, filed Sep. 21, 2005, inventor Huseyin Tek, et al., assigned to the same assignee as the present invention (Pub. No. US 2006/0098870 A1); and Ser. No. 11/399,164 entitled “Method and Apparatus For Detecting Vessel Boundaries”, filed Apr. 6, 2006, inventor Huseyin Tek, assigned to the same assignee as the present invention (Pub. No. US 2006/0262988 A1).
Number | Date | Country | |
---|---|---|---|
60793860 | Apr 2006 | US |