Patent Application
20100284586
The present invention relates to an apparatus and a method for segmenting microcalcifications in a mammographic image, and in particular to the segmentation of microcalcifications in mammographic images for computer-aided diagnosis of breast cancer.
Breast cancer is the most frequent type of cancer for women in the western hemisphere. One in ten women in the western hemisphere develops breast cancer during the course of her lifetime. Nowadays, early detection and diagnosis is still the most effective possibility for reducing the death rate for this type of cancer. Mammographic images are an effective means for enabling early detection of small tumor diseases that cannot be felt. One disadvantage in interpreting mammograms, however, is the difficulty in differentiating between benign and malignant lesions—even for experts in mammography it is a challenge to correctly interpret the form of the lesion.
The early diagnosis of the disease considerably improves the chances of survival of the woman affected. Digital X-ray mammography is the most effective technique for diagnosing breast cancer tumors at an early stage. For simplifying the diagnosis and also for improving the detection rate, frequently, systems for computer-aided diagnosis (CAD) of mammographic images are used. These systems indicate suspicious regions in mammograms to the radiologist by means of automatic analysis of mammographic images by image processing methods and offer support in the diagnosis of the lesions—whether they are benign or malignant tumors.
An important group of breast lesions found in mammographic images are so-called microcalcifications. These are small calcium deposits in the breast tissue, frequently occurring in groups, which indicate early stages of breast tumors. For generating an automatic diagnosis suggestion, the individual calcium particles are usually separated automatically from the background tissue. This separation of calcium particles from the respective background tissue (or background) is also called segmentation. Subsequently, calculation and evaluation of the shape and distribution features of the microcalcifications can be performed. The segmentation results of conventional fully automatic methods for separating individual calcium particles from the background are often insufficient.
In clinical practice, the usage of mammographic images frequently shows a series of weaknesses. Microcalcifications occurring in groups or clusters are, however, an early sign for breast cancer, but the differentiation between benign and malignant clusters of microcalcifications based on their occurrence in mammographic images is often a very difficult task. Hence, when using these conventional methods, it is not surprising that typically only 15% to 20% of breast biopsies carried out due to calcifications confirm malignancy. In the US, for example, a malignant pathology has later been confirmed in only 15-30% of the performed breast biopsies. This low positive prediction value (PPV) for mammographic images with regard to the diagnosis of calcium deposits (occurring in groups) implies many biopsies performed unnecessarily on benign calcifications, i.e. far too many breast biopsies are performed on patients with benign groups of microcalcifications. The unnecessary biopsies cause a large mental and also physical stress for the respective patient.
In the past, options or methods in computer-aided diagnosis of microcalcifications have been suggested with the aim of increasing reliability. These conventional methods use, for example, segmentations of the digital picture of the mammographic images by using so-called wavelets, with the help of which in particular localized structures can easily be detected. These conventional methods that are performed fully automatically use a threshold analysis where a global threshold (for the whole picture or a whole picture region) is determined by the computer, with the help of which microcalcifications are to be differentiated from other structures of the background.
Effective automatic classifications of microcalcifications, i.e. the division into benign and malignant calcifications, are based on a good segmentation of individual calcium deposits or calcium particles. The conventional methods are based on fully automatic segmentation methods for individual calcium deposits. Since, however, none of these fully automatic methods showed an optimum result for segmentation, there is a need for further methods for improving the reliability of classifications of microcalcifications and, hence, the results. It follows that a novel computer-aided diagnosis approach (CADx) is desirable.
According to an embodiment, an apparatus for segmenting microcalcifications in a mammographic image may have: a bandpass filter for bandpass filtering the mammographic image for obtaining the filtered mammographic image; a means for marking image points in the filtered mammographic image which exceed or fall below a predetermined threshold, for marking potential regions of microcalcifications; and a means for individual processing of one of the potential regions of adjacent marked image points for changing an extension of the one potential region for obtaining a segmentation of microcalcifications, wherein the means for individual processing is implemented to manually enlarge or reduce a region of a microcalcification by means of a local change of the predetermined threshold in surroundings of the microcalcification.
According to another embodiment, an apparatus for segmenting microcalcifications in a mammographic image may have: a structural element sampling filter for filtering the mammographic image to obtain a filtered mammographic image; means for marking image points in the filtered mammographic image exceeding or falling below a predetermined threshold for marking potential regions of microcalcifications.
According to another embodiment, a method for segmenting microcalcifications in a mammographic image may have the steps of: bandpass filtering the mammographic image for obtaining a filtered mammographic image; marking image points in the filtered mammographic image exceeding a predetermined threshold as a potential region of a microcalcification; individually processing clusters of adjacent marked image points for changing an extension of clusters such that the clusters represent a segmentation of microcalcifications, wherein the individual processing is performed such that a region of a microcalcification is manually enlarged or reduced by means of a local change of the predetermined threshold in surroundings of the microcalcification.
Another embodiment may have a computer program comprising a program code for performing the method for segmenting microcalcifications in a mammographic image, the method having the steps of: bandpass filtering the mammographic image for obtaining a filtered mammographic image; marking image points in the filtered mammographic image exceeding a predetermined threshold as a potential region of a microcalcification; individually processing clusters of adjacent marked image points for changing an extension of clusters such that the clusters represent a segmentation of microcalcifications, wherein the individual processing is performed such that a region of a microcalcification is manually enlarged or reduced by means of a local change of the predetermined threshold in surroundings of the microcalcification, when the computer program runs on a computer.
According to another embodiment, a method for segmenting microcalcifications in a mammographic image may have the steps of: sampling the mammographic image by means of structural elements, and subtracting a background obtained in this manner from the mammographic image for obtaining a high-pass filtered mammographic image; low-pass filtering the high-pass filtered mammographic image for filtering out high frequent noise portions and for obtaining a filtered mammographic image; and marking image points in the mammographic image exceeding or falling below a predetermined threshold, for marking potential regions of microcalcifications.
Another embodiment may have a computer program comprising a program code for performing the method for segmenting microcalcifications in a mammographic image, the method having the steps of: sampling the mammographic image by means of structural elements, and subtracting a background obtained in this manner from the mammographic image for obtaining a high-pass filtered mammographic image; low-pass filtering the high-pass filtered mammographic image for filtering out high frequent noise portions and for obtaining a filtered mammographic image; and marking image points in the mammographic image exceeding or falling below a predetermined threshold, for marking potential regions of microcalcifications, when the computer program runs on a computer.
The present invention is based on the knowledge that segmentation of microcalcifications can be obtained by first performing bandpass filtering of a picture (mammographic image), and subsequently, individual processing of the bandpass-filtered images can be performed in one or several steps. As a result, segmentation of microcalcifications is obtained, which can, optionally, be further examined in an evaluation unit for predicting the nature of the microcalcifications (benign or malignant) with high probability.
The size of individual calcium particles (microcalcifications) can vary in a range between 0.1 to 1 mm, wherein the calcification particles generally show a high degree of locality. Thereby, the high locality shows in the above-described high edge steepness. Conventional methods for segmentation are based on the fact that microcalcifications appear in the picture as a region with high space-like frequencies. Wavelet-transformations offer the optimum possibility to determine regions with high space-like frequency proportions in a picture, and hence, form the basis for conventional detection mechanisms for microcalcifications.
Correspondingly, in embodiments, the bandpass filter can also be based on wavelet segmentation and be taken over fully automatically by a computer or a computer program, respectively. The subsequent individual processing can be performed by a means where for example, a threshold is selected such that as many calcifications as possible and as little noise as possible appears on the picture. Further, in the means for individual processing, certain microcalcifications can be enlarged or reduced, e.g. when the radiologist assumes that they are insufficiently illustrated in the image. The means for individual processing can also be used in that certain microcalcifications or particles are deleted or even added, or that a particle is divided, so that two separate microcalcifications occur in segmentation. These processing steps, however, can be performed individually by a radiologist or a doctor, such that they are performed both for a whole region with calcium deposits and also for certain sub-regions including microcalcifications.
Hence, embodiments of the present invention describe a semi-automatic method for segmentation of microcalcifications from a background tissue. With the semi-automatic approach necessitating a certain degree of interaction of the radiologist in contrary to the known fully automatic method, the segmentation results can be significantly improved in comparison with fully automatic methods. As has been mentioned, this segmentation of individual calcium particles can be gradually improved based on a fully automatic initial segmentation by intuitive interaction options of the radiologist or doctor. The optimized segmentation obtained in this manner results in an increase of the extracted form and distribution features (e.g. describing the morphology and distribution of the calcium particles), which again results in an improvement of a diagnosis suggestion (benign or malignant) of the CAD device.
Based on a digital database of mammographic images, it is possible to determine the performance of embodiments of the present invention for segmentation by using the respective regions (ROI=regions of interest) including benign or malignant clusters of microcalcifications. This can occur, for example, by using a so-called support vector machine and an ROC analysis (ROC=receiver operating characteristics). The resulting ROC performance is very promising and the semi-automatic segmentation shows a significantly higher performance (detection rate) than is the case in comparable fully automatic systems.
Embodiments of the present invention are based on semi-automatic segmentation of individual calcium deposits or calcium particles, feature extraction and further on clinical data and the classification by using a support vector machine. The performance of embodiments of the present invention can be determined with the help of the digital database for mammographic images (DDSM=digital database for screening mammography) by using a so-called Leaving-One-Out-Sampling and an ROC-curve analysis.
The above and other elements, features, steps, characteristics and advantages of the present invention will become more apparent from the following detailed description of the preferred embodiments with reference to the attached drawings.
Embodiments of the present invention will be detailed subsequently referring to the appended drawings, in which:
Regarding the subsequent description, it should be noted that in the different embodiments equal or similar functional elements or structures have the same or similar reference numerals, and hence the descriptions of these functional elements and the different embodiments are interchangeable.
The means for marking 140 marks an amount of image pixels in the filtered mammographic image 130 as a potential region of microcalcification exceeding or falling below a predetermined threshold S (depending on the definition of the threshold S). The means for individual processing 160 changes, for example, the extension of clusters of adjacent marked image points, such that the clusters of image points represent a segmentation of microcalcifications. Changing the extension can, for example, include enlarging, reducing, or also generating or deleting potential microcalcifications. Here, a differentiation has to be made between the clusters of image points relating to a potential microcalcification and the clusters of microcalcifications. Further, it is possible that individual processing can include dividing a potential microcalcification into two separate microcalcifications.
The intensity distribution I0 in
Accordingly, filtering can be performed such that in the intensity distribution I0 as shown in
In a representation for potential microcalcifications obtained in this manner, it can happen that regions belonging to one microcalcification are illustrated as two separate regions. For example, it can be the case that maximums 230 and 240 do not correspond to two separate microcalcifications, but that they both belong to one microcalcification.
Such artifacts are difficult to detect for a fully automatic program, however, they represent no big challenge for a radiologist or doctor. Accordingly, it is advantageous when the threshold is correspondingly manually optimized. Manual optimization can be performed with regard to the global threshold, but also regionally in certain regions of the mammographic image 110, which are characterized, for example, by particularly strong noise. In this manual readjustment, a trade-off has to be found between detecting as many microcalcifications 105 as possible, but on the other hand, suppressing the noise as much as possible. One example for noise can be regions 115a, 115b, mistakenly assigned to the background.
The change in size of the microcalcification can, for example, be performed for the islands 105b and 105c shown in
Since an individual microcalcification 105 can generally not provide an indication for a tumor disease, it is important to examine whole regions (ROI). In the following figures, this is shown based on a region 300, which is part of the mammographic image 110.
Hence,
Hence,
In the previous embodiments, pre-bandpass filtering has been performed by selecting suitable spectral bands of the wavelet transform.
Applying the rolling ball method to the mammographic image of interest results in a low-pass filtered version of the mammographic image describing a slowly changing background of the image, which will then be subtracted from the mammographic image for reaching a high-pass filtered version where the background has been removed. In particular, the result of applying a rolling ball method to a mammographic image can be imagined as a 3-dimensional area, for example described by the center of the ball 280, when the same is rolled with a diameter D along the 3-dimensional intensity distribution I spanned between column and row direction. Depending on the diameter D of the ball, the result is more or less low-pass filtered.
As can be seen in
Hence, all points not belonging to the microcalcification 105 (i.e. lying outside of the minimum 210) and hence, belonging to background tissue, can be marked. After marking all points that can be sampled by the ball 280 (i.e. across which the ball 280 can roll), or after sampling the area I with the ball 280, the obtained picture describing the curve of the center of the ball can be subtracted from the original picture, possibly by additionally subtracting a remaining steady component or average value, respectively, of the resulting differential picture. As a result, an illustration is obtained where signals are subtracted from the background tissue and hence, microcalcification 105 appear more clearly.
It is obvious that for effectively subtracting the background tissue, the diameter D of the ball 280 should be selected large enough. In particular, the diameter D should be larger than a typical extension Δx or better a maximum extension of a microcalcification 105 (i.e. larger than Δx2 of
Hence, the rolling ball method determines a smooth or continuous background that can be removed from the picture (mammographic image) and marks localized structures, such as microcalcifications 105. As mentioned above, the rolling ball method can be illustrated such that the intensity distribution in
This background image can be subtracted from the original image of the mammographic image 110 such that low-frequency or slowly changing proportions can be removed from a tissue in the mammographic image 110. Instead of a ball, a general structural element can be used, by which slowly varying proportions in the intensity distribution can be sampled, which does not necessarily need to roll on the area corresponding to the picture. Hence, the method can also be referred to as sample filtering by means of a structural element or structural element sample filtering. Since the mammographic image 300 represents a 2-dimensional image, the rolling ball method can, for example, be modified, so that instead of the ball in a 3-dimensional form, a pen, for example, with a semi-ball as a sampling tip can be used.
Optionally, subsequently, low-pass filtering can be performed, such as Gaussian low-pass filtering for removing high-frequency proportions originating, for example, from noise, can be removed from the picture. Hence, overall, bandpass filtering is obtained. As mentioned, the radius of the rolling ball 280 should be selected at least as large as the radius of the largest object in the picture (here microcalcification), which is not part of the background or as large as that microcalcification diameter below which a predetermined percentage, such as 95%, of the microcalcification diameters lies.
Although the just-described form of mammographic image bandpass filtering can also be used without the semi-automatic segmentation described in the above embodiments, such as for subsequent segmentation of microcalcification by means of an automatically determined threshold, the mammographic image bandpass-filtered in this manner can result in a very accurate segmentation together with the above-described user input options. Thereby, adjustability of the bandpass filter boundaries can be provided, for example, by means of the rolling wheel of the computer mouse, for example, by varying the dimensions of the structural element or the diameter of the ball corresponding to the rotation of the wheel, and by showing the updated results until a current adjustment is confirmed by the user, e.g. by means of the left mouse button.
Further specifications of embodiments of the inventive procedure during segmentation and possible classification of microcalcification 105 can also be summarized as follows.
Conventional methods for segmentation are based, as has already been described above, on the fact that microcalcifications appear in the picture as a region with high space-like frequencies. Wavelet transformations offer the optimum possibility of determining regions with high space-like frequency proportions in a picture, and hence, they form the basis for conventional detection mechanisms for microcalcifications. The compactness and high regularity of wavelets of the Daubechies family makes these wavelets an obvious choice in finding calcifications. Hence, it is also advantageous for embodiments of the present invention to use a bandpass filter based on the wavelet transformation by means of Daubechies-6-wavelets. Thereby, first, the area having clusters of microcalcifications is divided into subbands by using wavelet transformation. Subsequently, those bands corresponding to space-like frequencies, which are above or below a certain space-like frequency typical for calcification or calcium particles, are neglected (filtered out). Subsequently, retransformation of the wavelet transformation is performed and as a result, a bandpass filtered version of the original image, i.e. the filtered mammographic image 130 is obtained. The obtained bandpass filtered original image includes not only the microcalcifications, but unfortunately also other picture structures, such as a noise having accidently the same space-like frequencies like a typical calcium particle.
Fully automatic segmentation of calcium particles can be performed by applying a threshold S, which is normally locally adaptive, to the bandpass filtered picture 130. In most cases, however, the fully automatic segmentation is far from perfect, and instead, as has been stated, an interactive segmentation step as an extension for automatic segmentation is useful. Hence, the complete segmentation process comprises an automatic first step and an interactive second step, and is thus semi-automatic.
The above briefly described interactive corrective action can be summarized as follows, or be further specified by using a computer mouse as part of the means for individual processing.
First, in the suggested interactive extension, the threshold S used for segmentation for separating calcifications from a background is optimized by a doctor/radiologist (or another user). Optimization can be performed, for example, by means of a computer or a mouse wheel. By interaction it is possible to find an optimum global threshold that can be applied to the specified region (ROI) for making a diagnosis. For example, by rotating the mouse wheel, the used threshold can be continuously changed until the user has obtained a desired result.
However, in most cases, the global threshold will not be optimum for every particle within a group. Hence, further corrective action is desirable for obtaining a good segmentation result. It has shown that a simple to use and still very effective method for improving the segmentation of individual particles is automatically enlarging or reducing certain regions. In detail, it is possible that individual particles are enlarged or reduced by clicking with the right or left mouse button on the respective particle or the respective region. The exact amount and direction of growth (i.e. in what spatial direction the particle grows) or the reduction of the given segmentation or given region of particles can be controlled, for example, automatically, by using the so-called region growing technique.
By using the two above-described interaction options, the size of the segmentation region of a cluster of particles can be optimized easily and effectively by a doctor. However, two problems remain. On the one hand, the particles actually representing microcalcifications are not illustrated in the image, and on the other hand, it is possible that the noise has mistakenly caused marking of particles (microcalcifications), in particular in the initial segmentation step. Both problems can be solved in that the doctor deletes individual particles—for example by clicking with the right mouse button on the respective particle, which is smaller than a given size. Further, the doctor can generate new calcium particles, for example by clicking on the left mouse button in the region or on the location having no calcium particles.
A further problem, which has also been indicated above, is given by the fact that it sometimes happens that the original segmentation had the result that two particles actually representing separate particles appear in the picture as a uniform combined particle. This problem will be solved by dividing one particle into two particles, for example in that the doctor clicks on the particle with the central mouse button, such that a dividing line results between the two particles that are to occur. A computer program can for example, automatically generate the dividing line. Finally, by automatic analysis, the form or boundaries of the resulting particles can be optimized.
In further embodiments, based on the obtained segmentation 170, features can be extracted, based on which a diagnosis becomes possible whether the calcifications are benign or malignant calcium particles. Many features can be obtained from the segmentation of the individual particle clusters. Thereby, most features comprise the morphology, but the distribution of the individual particles can also be extracted. Regarding the form, for example, a differentiation can be made between round or angular calcium particles, which can further be grouped in a tight manner or arranged along a line. Additionally, further clinical parameters, such as the age of the patient (that are for example included in the DDSM commentary) can be enclosed. Overall, it is possible to extract more than 30 features for a respective ROI region. Since the dimension of the feature vector obtained in this manner can be very high, a self-learning technique can be used for automatic selection of an optimum sub-space of features. An n-dimensional feature vector that can be obtained from a respective ROI region including calcification clusters represents a point in an n-dimensional vector space. Each of these data points belongs to one of two classes corresponding to either benign or malignant microcalcifications. The separation and hence classification of data can be obtained by finding an (n−1) dimensional hyper area optimally (i.e. with a maximum free boundary region) separating the benign from the malignant data points. As in a non-trivial case, often no optimum hyper area can be found, instead, a support vector analysis can be performed (by using a support vector machine). Thereby, the feature space maps into a higher dimensional space where the hyper area can easily be found. This can, for example be performed by a so-called kernel function. For many classification tasks, the support vector machine classifiers form good analysis tools, and also for embodiments of the present invention, a support vector machine represents a good classifier that can differentiate between benign and malignant clusters of calcifications.
For determining the performance of the classification of (a CADx approach for microcalcifications), leaving-one-out-sampling can be performed, which can be supplemented by an ROC-curve analysis. The area under the ROC-curve Az is used as performance metric (measure for performance). It shows that the ROC calculation of the inventive method of the computer-aided diagnosis in fully automatic segmentation provides a value for the area below the ROC curve: Az=0.76±0.01. On the other hand, for a semi-automatic segmentation, the area below the ROC curve is significantly higher, namely Az=0.78±0.01 with a statistic significance of p<0.05.
For performance measurements, computer-aided diagnosis methods use, for example, mammographic images, which can be extracted from the digital databank for mammographic images (DDSM). The DDSM has mammographic images that have been digitalized means of different digital converters. The interesting marked ROI regions include malignant clusters of microcalcifications, which have been extracted from the DDSM database, and have been marked with “cancer—01”, “cancer—02”, “cancer—05”, “cancer—09”, “cancer—15”. All ROI having benign microcalcifications have been marked with “benign—01”, “benign—04”, “benign—06, “benign—13” “benign—14”. All in all, 530 ROI regions are extracted, 224 of which include malignant clusters verified by biopsies and the remaining 306 include benign clusters of microcalcifications.
In summary, embodiments of the present invention provide a computer-aided diagnosis enabling to determine the diagnosis of benign or malignant clusters of microcalcifications. Both the fully automatic and the inventive semi-automatic segmentation can be used for segmenting individual calcium particles from a background tissue. Based on the inventive segmentation, it is possible to extract a large number of features, which depend mostly on the morphology and also on the distribution of the individual calcium particles. Since the feature space has a high dimension, an automatic learning technique can be applied for finding an optimum sub-space of the features. The resulting feature vector is classified by means of a support vector machine. With the aid of a set of ROI regions including microcalcifications, which can be obtained from the DDSM database, the performance of the inventive method can be analyzed very well. The inventive method shows a very good ROC performance. This is particularly astonishing since systems using Bi-Rads attributes (Bi-Rads=breast imaging reporting and data system) show a significantly worse performance by using the same data, than is the case in the embodiments of the present invention. By using the semi-automatic segmentation, the ROC performance is significantly higher than is the case in comparable fully automatic segmentations. This shows that an easy to use interactive segmentation process involving a doctor or a radiologist does not only improve the quality of segmentation of the individual particles, but above this, improves the usefulness of the features that can be extracted based on the segmentation.
In particular, it should be noted that depending on the circumstances, the inventive scheme can also be implemented in software. The implementation can be made on a digital memory medium, in particular a disc or a CD with electronically readable control signals that can cooperate with a programmable computer system such that the respective method is performed. Generally, the invention also consists of a computer program product with a program code for performing the inventive method stored on a machine readable carrier when the computer program product runs on a computer. In other words, the invention can be realized as a computer program with a program code for performing a method when the computer program runs on a computer.
While this invention has been described in terms of several advantageous embodiments, there are alterations, permutations, and equivalents which fall within the scope of this invention. It should also be noted that there are many alternative ways of implementing the methods and compositions of the present invention. It is therefore intended that the following appended claims be interpreted as including all such alterations, permutations, and equivalents as fall within the true spirit and scope of the present invention.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10 2007 058 968.0 | Dec 2007 | DE | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/EP2008/010240 | 12/3/2008 | WO | 00 | 6/30/2010 |
