Patent Application
20090060366
Various embodiments of the invention may relate, generally, to the segmentation of objects from images. Further specific embodiments of the invention may relate to the segmentation of bone portions of radiological images.
Computer-aided detection (CAD) solutions for chest images may often suffer from poor specificity. This may be in part due to the presence of bones in the image and lack of spatial reasoning. False positives (FPs) may arise from areas in the chest image where one rib crosses another or crosses another linear feature. Similarly, the clavicle crossing with ribs may be another source of FPs. If the ribs and the clavicle bones are subtracted from the image, it may be possible to reduce the rate of FPs and to increase the sensitivity, via the elimination of interfering structures. Due to the domination of the lung area by the ribs, the probability of a nodule being at least partially overlaid by a rib is high. The profile of the nodule may thus be modified by an overlaying rib, which may make it more difficult to find. Subtracting the rib may result in a far clearer view of the nodule, which may permit a CAD algorithm to more easily find it and reason about it.
The ability to reason spatially may also be a consideration in chest CAD. Delineation of rib and clavicle boundaries may provide important landmarks for spatial reasoning. For example, knowledge of the clavicle boundaries may allow a central line (i.e., spine or mid-line between the two boundaries) of the clavicle to be determined. The clavicle “spine” may be used to provide a reference line or reference point at the intersection point with the rib cage. Similarly, knowledge of the rib boundaries may allow a rib spine to be determined. Knowledge of the rib number along with the rib spine and intersection point with the rib cage may be used to provide a patient-specific point of reference.
Several attempts have been made to, solve the rib segmentation problem. Considering the rib and clavicle subtraction problem, the approach by Kenji Suzuki at University of Chicago may be the most advanced. However, this has been achieved in an academic environment where tuning of the algorithm parameters can be made to fit the characteristics of the sample set. The particular method is based on a direct pixel-driven linear artificial neural net that calculates a subtraction value for each pixel in the image based on the degree of bone density detected by the network. The result can be noisy, and an exemplary implementation only worked for bones near to horizontal in the image.
Various other researchers have anecdotally illustrated techniques for rib segmentation in the open literature. However, in all cases known to the inventors, researchers have noted that the techniques suffer from brittleness, and as a consequence, rib segmentation remains an open area of research. No such applications have yet met the level of performances required for clinical application.
Although clavicle segmentation has been mentioned as potentially useful, no solutions have been proposed.
Various embodiments of the invention will now be described in conjunction with the attached drawings, in which:
As shown in block 2l, an input image may be operated on by various methods to form an image that is normalized with respect to contrast and pixel spacing. Initially all image data may be re-sampled to form a fixed inter-pixel spacing; in some exemplary implementations, this re-sampling may be to 0.7 mm inter-pixel spacing, but the invention is not thus limited. The fixed inter-pixel spacing, may permit subsequent image processing algorithms to employ known optimal kernel scales. To achieve consistent contrast properties across different acquisition systems and acquisition parameters, local contrast enhancement operators may be applied to minimize the effects of global and local biases that may exist in, native image data. Additionally, edge detail may be enhanced (i.e., edge enhancement), which may serve to aid subsequent processes aimed at detecting interfaces, e.g., tissue/air and bone interfaces.
In block 22, a phase symmetry estimate may be computed from the re-sampled, contrast normalized, and edge enhanced image. In a chest image, the phase symmetry image may provide the basis for clavicle and/or rib segmentation. Phase symmetry may generally involve the determination of image features based on determining consistency of pixels across line segments oriented at different angles. Phase symmetry may be used to provide a normalized response from 0 to 1, where one may indicate complete bilateral symmetry for a particular considered scale and orientation. Orientation may provide prior knowledge regarding the orientation of ribs and clavicles and may allow unwanted structures to be suppressed. In addition to employing multi-scale, oriented kernels for selective enhancement, an adaptive noise model may be used to suppress irrelevant responses arising from noise and small-scale structures, such as quasi-linear vessels in the chest.
As shown in block 23, an orientation model may be applied to the output of block 22. Phase symmetry may be helpful in providing both the amplitude and the orientation corresponding to the maximum response. The availability of an orientation estimate may allow a priori orientation models associated with the objects of interest, clavicles and/or ribs, to be exploited. This capability may be exploited to suppress responses that can be attributed to linear structures whose orientation is not consistent with prior models of valid clavicle and rib orientations. In particular, images may be filtered based on orientation to eliminate or attenuate objects that are incorrectly oriented. It is noted that in the case of chest images, one may take advantage of the fact that the orientation models for the left and right lungs are substantial complements of each other.
To further illustrate how embodiments of blocks 21-23 may operate, some exemplary results will now be presented.
Blocks 24-210 of
T1=med(phase-image(find(label_mask˜=0)))+edge_threshold *mad;(phase_image(find(label_mask˜=0)));
edge_mask=hysthresh(phase image, T1, T1/3);
where;
A common issue associated with edge detection processes is the fragmentation of continuous boundaries due to noise and superposition. Although the two-stage threshold defined above may greatly reduce fragmentation, it may not, fully eliminate it. To further reduce fragmentation, edge linking may be employed. Implementation of such edge linking may involve linking the tail and head (or head and tail) of two edges if they are sufficiently close and have consistent orientations to within a specified tolerance.
In an example,
Following the identification of edge objects, generated through the edge thresholding and linking process 24, block 25 may be used to construct object edge models. In one embodiment of block 25, a non-linear least-squares process may be employed to fit an a priori polynomial model to each candidate edge object. Using the extracted polynomial model, the edge may be extrapolated. This may serve to fill gaps and to project edges across the full extent of the lung (in a chest image). In some embodiments, to avoid poorly behaved models, only, those edge objects with adequate normalized extent, which may be defined as object_width/lung_width, may be considered in the fitting process. Furthermore, edge objects at the extreme top or bottom portion of the lung may be, excluded, from consideration, as these regions may be outside the regions of interest for detecting ribs or clavicles. This may be particularly useful for clavicle segmentation, where only the upper third portion of the lung may need to be considered. In the event that estimated coefficients are inconsistent with the a priori model, the edge objects may be considered invalid and may be removed. In those instances when the coefficients are consistent with the prior model, the fitted model may be retained and considered a valid candidate object (e.g., rib or clavicle) border.
A subsequent process that may be used to enhance sensitivity is adaptive correlation of extracted rib models. For a variety of reasons, all boundaries of ribs may not always be detected. Rib boundaries may not be detected, for example, due to fragmentation, poorly behaved modeling of the edges, etc. To improve sensitivity, a two-stage process may be employed. First, an attempt may be made to extract and model the edge directly, as described above. Second, for each patient and each lung, one may select the “best” rib boundary model by computing a suitable error between, the extracted edge and modeled edge. Using the extracted boundary, model, a correlation may be applied. In an exemplary embodiment of the invention, that may be useful in rib and clavicle segmentation, this may be done in the vertical direction across all remaining edge objects. For those vertical positions that generate a sufficiently high correlation, the model may be used, e.g., as a rib boundary location.
The results of block 25 may then be processed in block 26. In chest images, for example, due to the steep and rapid convergence of individual ribs along the rib cage, fitted and/or extrapolated rib boundaries may intersect. This intersection may serve to complicate subsequent processing because the intersection of two rib boundaries may lead to improper labeling. Two rib or clavicle boundaries that intersect may be incorrectly treated as a single object rather than as the upper and lower boundary of a rib or clavicle object. To circumvent this issue, boundary candidates may be analyzed from the center toward the edges of the lung. In the event that two boundary objects that were separated at the center but subsequently intersected, at a point to the left or right of the center, the objects may be assumed to be the upper and lower boundary of a rib. The intersection point may be assumed to be an artifact of the extrapolation process. Therefore, the merged boundaries may be broken apart from the intersection point.
In block 27, invalid boundaries may be pruned. In the case of a chest image, it may be the case that the top-most clavicle and rib boundary should be positive contrast while the bottom-most clavicle and rib boundary should be negative contrast. Erroneous boundaries may be pruned as a precursor to pairing boundaries. After all boundary candidates are selected, the polarity of the edge response may be used to further prune invalid edge objects. Beginning with the top-most extracted boundary, objects may be sequentially removed until a positive contrast boundary is detected. Similarly, as noted above, the last detected boundary should possess a negative contrast. Therefore, beginning with the last extracted boundary, objects may be sequentially removed until a negative contrast boundary is detected. This process may be employed separately for both lung objects and independently when detecting clavicle and rib objects.
Following boundary pruning 27, final boundaries may be selected 28. In block 28, the distances with respect to paired positive and negative contrast edges may be considered. For example, in a chest image, to be considered a valid rib or clavicle boundary pair, two adjacent boundaries may typically have opposite contrast and be separated by a minimum vertical distance (d1) and separated by no more than a maximum vertical distance (d2). Boundary objects not paired with an opposite contrast and a specified distance apart may thus be removed.
In an example corresponding to the example shown in
In order to reconcile the modeled boundaries with the original image, one may need to up-sample the boundaries 29. This may thus form a fall-resolution map of the desired boundaries.
Finally, in some embodiments, it may be desirable to obtain paired vertices that may be used to define objects. In such cases, block 210 may be used to obtain such paired vertices based on the full-resolution boundary delimiters.
If desired, the results may then be used to suppress one or more segmented objects. For example, block 210 may correspond to a bone suppression process, which may be used, e.g., in the case of chest images, to subtract ribs and/or clavicles from the images.
While the illustrations have shown the use of the disclosed techniques in connection with the subtraction of ribs from chest images, such techniques may also be applied to other radiological images in which bone may interfere with observation of soft tissue phenomena. Furthermore, such techniques may also be applicable to non-radiological images in which known structures, which may be similar to bones in radiographic images, may be subtracted.
Various embodiments of the invention may comprise hardware, software, and/or firmware.
Various embodiments of the invention have been presented above. However, the invention is not intended to be limited to the specific embodiments presented, which have been presented for purposes of illustration. Rather, the invention extends to functional equivalents as would be within the scope, of the appended claims. Those skilled in the art, having the benefit of the teachings of this specification, may make numerous modifications without departing from the scope and spirit of the invention in its various aspects.
This application claims the priority of U.S. Provisional Patent Application No. 60/968,1139, filed on Aug. 27, 2007, and incorporated by reference herein.
| Number | Date | Country | |
|---|---|---|---|
| 60968139 | Aug 2007 | US |
