The present invention generally relates to image processing and analysis and computer-aided detection (CAD) and more particularly relates to methods that assess and use data related to the detection of architectural distortion in mammography.
Some believe that indirect signs of malignancy (such as: architectural distortion, bilateral asymmetry, single dilated duct and developing densities) account for almost 20% of detected breast cancer. Architectural distortion is believed to be a sign of nonpalpable breast cancer. Architectural distortion is defined in the Breast Imaging Reporting and Data System (BI-RADS) as follows: “The normal architecture (of the breast) is distorted with no definite mass visible. This includes spiculations radiating from a point and focal retraction or distortion at the edge of the parenchyma. Architectural distortion can also be an associated finding.”
Although most architectural distortions are considered to represent cancer, it is difficult for radiologists to detect this condition because of the nonspecific definition of distortion and due to its subtle nature. This also represents a challenge for computer aided detection. A number of widely available mammography CAD systems showed sensitivity to architectural distortion of less than 50%. Thus, the development of CAD system for detecting architectural distortion is a challenging topic in this field.
Eltonsy et al. in “A concentric morphology model for the detection of masses in mammography” (IEEE Transactions on Medical Imaging, 2007, vol. 26, no. 6, pp. 880-889) developed a method to detect masses and architectural distortion by locating points surrounded by concentric layers of image activity.
Zwiggelear et al. in “Model-based detection of spiculated lesions in mammograms” (Medical Image Analysis, 1999, vol. 3, no. 1, pp. 39-62) proposed a scheme for the detection of spiculated mass lesion.
Rangayyan et al. in “Detection of architectural distortion in prior mammograms of interval-cancer cases with neural networks” (31st Annual international conference of the IEEE EMBS, 2009, Minneapolis, Minn., USA) proposed a method based on Gabor filters and phase portrait analysis to detect initial candidates for sites of architectural distortion.
Baker et al. in “Computer-aided detection (CAD) in screening mammography: sensitivity of commercial CAD systems for detecting architectural distortion” (American Journal of Roentgenology, 2003, vol. 181, pp. 1083-1088) investigated the performance of two commercial CAD systems, including detecting architectural distortion.
While various methods such as those listed may have achieved some level of success in detecting architectural distortion in the mammography image, improvement remains. For example, there is a need for further effort in this area to improve the accuracy of the detection, particularly for commercial systems. Moreover, developments in this area, such as detection of spiculation, may help to improve detection and diagnostic results for other types of conditions. Features used in architectural distortion detection can further benefit the detection of spiculated mass and boost the accuracy for detection of mass in mammography, which can be a significant bottle-neck of current mammographic CAD system.
Overall, advances in the detection of architectural distortion in the mammography image can better assist the radiologist to improve performance in mammography screening and can help to reduce false negatives. This capability helps to boost the detection accuracy of the mammography system and to provide consistent detection of two major breast cancer subtypes, i.e. mass and architectural distortion, thus helping to assist in providing earlier diagnosis and treatment for breast cancer patients.
It is an object of the present invention to advance the art of computer-aided detection for mammography and other tissue imaging. With this object in mind, the present invention provides a method for detecting architectural distortion within mammographic image data, the method executed at least in part on a computer and comprising: identifying breast tissue within the image data; generating an orientation field and a corresponding magnitude field within the identified breast tissue; generating a feature map by processing the orientation field with a phase portrait model at one or more image scales; identifying one or more architectural distortion features according to the generated feature map; and displaying the one or more identified architectural distortion features.
The present invention is suitable for modeling both spiculation and distortion simultaneously. Such an approach helps in the detection of both architectural distortion and spiculated mass in mammography.
It is an advantage of the present invention that it is relatively insensitive to differences in image contrast or other image quality characteristics or to differences due to the specific type of radiology system used for obtaining the image.
These objects are given only by way of illustrative example, and such objects may be exemplary of one or more embodiments of the invention. Other desirable objectives and advantages inherently achieved by the disclosed invention may occur or become apparent to those skilled in the art. The invention is defined by the appended claims.
The foregoing and other objects, features, and advantages of the invention will be apparent from the following more particular description of the embodiments of the invention, as illustrated in the accompanying drawings.
The elements of the drawings are not necessarily to scale relative to each other.
The following is a detailed description of the preferred embodiments of the invention, reference being made to the drawings in which the same reference numerals identify the same elements of structure in each of the several figures.
For the detailed description that follows, the mammographic image is defined as f(X), where X denotes the 2D pixel array and f(x,y) denotes the intensity value for pixel (x,y) in X.
The logic flow diagram of
A segmentation step 1102 defines the outline of the breast tissue in mammography image 1100. Segmentation techniques of various types are well known to those skilled in the diagnostic image processing art. In one embodiment, segmentation of the breast image is provided using a skin line estimation process that defines the contour of the breast tissue (in the CC view) or the breast tissue plus pectoral muscle (in the MLO view). The bounding box of the skin-line contour defines a breast tissue region of interest (ROI). A down-sampling step 1104 then reduces the scale of a breast ROI image 1110 to a more favorable resolution for processing. This helps to make processing more efficient, without loss of accuracy, since it has been found that the effective size of an architectural distortion lesion is statistically larger than that of a regular mass lesion. If the working pixel size used in the detection of architectural distortion is too small, subsequent processing phases for phase portrait template matching may not be able to detect the required patterns and may generate an erroneous feature map.
Still following the processing shown in
Continuing with the
Continuing with
Following segmentation, non breast tissue has been removed from the breast tissue ROI. At this point, the breast tissue ROI employs further enhancement to help highlight high frequency information. This is done by applying a high-pass filter to the original breast tissue ROI image. In one embodiment, the high-pass filter is implemented by the subtracting a Gaussian smoothed version of the original breast tissue ROI from the original breast tissue ROI image:
f
HPF(X)=f(X)−fLPF(X),
where fHPF(X) and fLPF(X) are high-pass and low-pass filtered breast tissue ROIs. High frequency image information allows improved enhancement of underlying image structure.
As the sequence of
The Gabor filter has been used in pattern recognition applications as a preprocessing step to extract orientation related image structure from raw image data. Frequency and orientation representations using Gabor filters are similar to those of the human visual system, and it has been found to be particularly appropriate for texture representation and discrimination. Gabor filters may be used as line detectors that are useful, for example, in fingerprint recognition applications. A bank of Gabor filters, each filter disposed at a different angle, is used for this function.
In diagnostic image processing applications, Gabor filtering has been proposed for use in mass candidate detection, as described in U.S. Patent Application Publication No. 2010/0046814 entitled “Method for Mass Candidate Detection and Segmentation in Digital Mammograms” by Dewaele et al. U.S. Pat. No. 6,137,398 entitled “Gabor Filtering for Improved Microcalcification Detection in Digital Mammograms” by Broussard et al. describes using Gabor filters for detecting false positive microcalcification structures so that they can be eliminated from further processing.
Unlike these earlier approaches, embodiments of the present invention, directed to the task of identifying architectural distortion, employ a bank of Gabor filters as a utility for forming an orientation field or map of breast tissue, as described earlier with reference to
In the example of
A 2D Gabor filter can be conceptualized as a complex plane wave carrier modulated by a 2D Gaussian envelope. A 2D Gabor filter kernel oriented at the angle
can be defined as follows:
Kernels at other angles can be obtained by rotating this kernel. In this embodiment, the parameters in Eq. 1, namely: σx, σy and f are derived from design rules as follows:
In one embodiment, value τ=4 pixels (corresponding to a thickness of 4 cm at a pixel size of 1 cm) and l=8. These values were determined empirically, by observing the typical spicule width and length in mammograms with architectural distortion in a patient database. This is based on a comparative analysis of the Gabor filter with the steerable filter and a 5×5 line detection mask.
The Gabor filter has a nonzero magnitude response at the origin of the frequency plane (DC frequency). Consequently, the low-frequency components of the mammographic image may influence the result of the Gabor filter. Such influence does not affect the computation of the orientation field angle, since the same influence will appear at all angles. However, the nonzero DC response can cause the orientation field magnitude to exhibit values that are affected by low-frequency content of the image. It is thus desirable to reduce the influence of the low-frequency components of the mammographic image in the orientation field magnitude, since the low-frequency components are not related to the presence of oriented structures in the image. For this reason, the mammographic image is high-pass filtered prior to the generation of the orientation field, as has been noted.
The texture orientation at a pixel is estimated as the orientation of the Gabor filter that yields the highest magnitude response at that pixel. The orientation at every pixel is then used to compute the orientation field angle image θ(x,y). The magnitude of the corresponding filter response forms the magnitude image M(x,y). The orientation field thus obtained has the same resolution as the original mammogram (1 cm).
The diagram of
Let fHPF(x,y) be the high-pass-filtered version of the mammogram being processed, high-pass filtered mammography image 1208, and fk(x,y)=(fHPF×GK)(x,y) represent Gabor filtered images 1212 and 1216. Then, the orientation field angle of f(x,y) is given by a step 1218:
The orientation field magnitude M(x,y) is given by step 1218 as:
M(x,y)=|fk
The Gabor filter bank is sensitive to linear structures, such as spicules and fibers. However, the filter bank also recognizes strong edges in the image as oriented features, such as: pectoral muscle edge, the parenchymal tissue edge, and vessel walls, etc. This embodiment focuses predominantly on identifying oriented features as clues for architectural distortion.
Non-Maximum Suppresion (NMS) is used to detect core curvilinear structure that shows the most pronounced textural or structural differences by comparing each pixel in magnitude image M(x,y) with its neighbors along the direction that is perpendicular to the local orientation field angle θ(x,y). If the pixel under investigation has a larger magnitude value than that of its corresponding neighbors, the pixel is considered to be a core curvilinear structure pixel.
Referring to
Let h(x,y) be a Gaussian filter of standard deviation σsmooth, defined as
Define the images
s(x,y)=MNMS(x,y)sin[θ(x,y)] (5)
and
c(x,y)=MNMS(x,y)cos[θ(x,y)] (6)
then, the filtered orientation field angle θsmooth(x,y) smoothed orientation field 1532 (
In
Depending on the size of the defined Gabor filter and how well it fits locally in a different image structure, the orientation field that it produces can be noisy, which may affect subsequent phase portrait matching. As has been shown, the orientation field directly generated by the Gabor filter bank may be further smoothed to provide more continuous orientation information and to focus on major image structures. This orientation field will be further analyzed by phase portrait modeling to find potential architectural distortion lesion locations.
As described previously with respect to the logic flow in
In general, phase portrait technique provides analytical tools to study systems of first-order differential equations. This technique has proved to be useful in characterizing oriented texture: the geometrical patterns in the phase portraits of systems of two, linear, first-order differential equations can be associated with the patterns encountered in an image presenting oriented texture. Phase portrait modeling has been widely used with dynamic systems and in finger print recognition, for example, to detect critical points. Here, it helps to focus on and recognize the main structure in the orientation field in a local manner, without being overly biased by noisy image structure data.
The phase portrait modeling technique uses global optimization to find a best match between a configurable, or deformable, phase portrait template and underlying image structure. Then, based on the properties of the matched phase portrait template, this modeling recognizes whether or not it detects a characteristic architectural distortion pattern (in particular, a node phase portrait pattern, as described in more detail subsequently). The snatching process iterates through each pixel in the orientation field and categorizes and identifies the underlying image structure. The result of such a process is a feature map that has the same size as the input orientation field and that quantifies the probability of architectural distortion in each location.
A phase portrait displays the possible trajectories, in the phase plane, of the state of a dynamical system. Consider the following system of linear first-order differential equations:
where A is a 2×2 matrix and b is a 2×1 column matrix (a vector). The functions p(t) and q(t) represent the state variables of a dynamical system, as a function of time (e.g., the position and the momentum of a particle, or the pressure and the temperature of a gas). Elements {dot over (p)}(t) and {dot over (q)}(t) represent actual transformed values of the system.
In this case, there are three possible types of phase portraits of interest: node, saddle, and spiral. The node type pattern is of particular value for assessing architectural distortion. The chart in
The model in Eq. 8 can thus be used to analyze an orientation field, such as the mammographic orientation field generated using procedures described previously. Consider the following vector field model:
The vector v is an affine function of the coordinates (x,y). As phase portrait modeling is conventionally applied, a particle on the Cartesian (image) plane whose velocity is given by v(x,y) will follow a trajectory that is analogous to the time evolution of the dynamical system in Eq. (8). Therefore, Eq. 10 can be compared to Eq. 8 by associating the vector v with the state velocity ({dot over (p)}(t),{dot over (q)}(t); and the position (x,y) with the state (p(t), q(t). The orientation field generated by Eq. 10 can be defined as:
which is the angle of the vector v with the x axis.
Using the concepts presented above, methods of the present invention qualitatively describe the orientation field of a textured image by locally identifying the type of phase portrait that is most similar to the orientation field, along with the center of the phase portrait, in order to detect and localize architectural distortion.
For phase portrait matching, an analysis window of w×w pixels is moved sequentially, pixel by pixel, to each position in the smoothed orientation field θsmooth(X). At each position of the analysis window, the phase portrait model parameters that best represent the orientation field, matrix A and vector b (in Eq. 8), are estimated. In order to estimate A and b, let Δ(x,y|A,b) be a measure of the error between the smoothed orientation field θsmooth(x,y) and the calculated orientation φ(x,y|A,b) given by the model, at the pixel location (x,y). The error measure is defined as:
Δ(x,y|A,b)=sin[θsmooth(x,y)−φ(x,y|A,b)] (12)
Estimates of Aobt and bopt that minimize Δ(x,y|A,b) are obtained using simulated annealing in one embodiment, applying techniques familiar to those skilled in the image processing art. Alternately, some other suitable method for estimating assessment and adjustment according to error calculation can be used.
As shown in
For detection of architectural distortion, the node map is computed. If the eigenvalues of Aopt are real and of the same sign, the pixel in the node map that corresponds to the fixed point in the phase portrait template is incremented by 1 as the feature map is formed. In order to prevent numerical instabilities in the computation of A−1, results are discarded for node type matching that has a fixed point far away from the center of the phase portrait template.
Architectural distortion lesions vary in size over a large range. In order to detect architectural distortion at different sizes, a multi-scale approach is used. One approach would be to scale the smoothed orientation field θsmooth(X) to two or more resolutions and then to process each orientation field individually; however, this would require considerable processing time and resources. Instead, embodiments of the present invention successively apply phase portrait templates of multiple scales or sizes to the orientation field θsmooth(X).
In the example of
As noted earlier with respect to
The estimates of the fixed point location for a given phase portrait pattern can be somewhat inaccurate, scattered around the true fixed point position due to factors such as the limited precision of the estimation procedure, the presence of multiple overlapping patterns, the availability of limited data within the sliding analysis window, and image noise. A local accumulation of the votes is necessary to diminish the effect of fixed point location errors. In one embodiment, a Gaussian smoothing filter is employed to smooth the resulting feature map for this purpose.
For the purpose of pattern classification, two features in particular are extracted and can be used to characterize each ROI of a suspected architectural distortion lesion: (i) the maximum of the node map and (ii) the entropy of the node map. The maximum value of the node map conveys information about the likelihood that a node phase portrait type is present. The entropy value relates to the uncertainty in the location of the fixed point in the node map. The entropy η of node map n(x,y) is computed as:
Where Sn is the normalization factor and defined as:
Features that have been extracted as part of step 1600 (
Methods of the present invention can be used with systems that can be trained to classify architectural distortion with improved accuracy over time, such as with classification systems employing neural network (NN) classifier logic, for example. Results from a set of training cases as well as from actual patient studies can be used to help train and refine the decision-making process from an NN system, using techniques well known to those skilled in the data analysis arts.
Referring to
It can be appreciated that the overall arrangement of
An operator interface can be provided for allowing adjustment of variable parameters used in processing image data and for display of interim results. Parameters can be adjusted for conditioning the image data, for conditioning orientation field generation, or for conditioning feature map generation.
The invention has been described in detail with particular reference to a presently preferred embodiment, but it will be understood that variations and modifications can be effected within the spirit and scope of the invention. For example, the overall procedure described with reference to