DETERMINING MAMMOGRAPHIC IMAGE VIEW AND LATERALITY

Abstract

A method for displaying a mammography image. Digital data of the mammography image is obtained. The mammography image is segmented to identify at least a first diagnostically relevant region comprising an image of the breast tissue and a second diagnostically relevant region. A view type is assigned for the image, either cranio-caudal or medio-lateral oblique view, according to a symmetry index calculated from the segmented first diagnostically relevant region. Right or left laterality is assigned to the image according to a laterality feature calculated according to the relative position of at least the second diagnostically relevant region within the image.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

While the specification concludes with claims particularly pointing out and distinctly claiming the subject matter of the present invention, it is believed that the invention will be better understood from the following description when taken in conjunction with the accompanying drawings.

FIG. 1 is a plan view of a typical hanging protocol on a display.

FIG. 2 is a logic flow diagram showing the method of the present invention.

FIGS. 3A and 3B are plan views showing a mammography image with an LMLO view and segmented image.

FIGS. 4A and 4B are plan views showing a mammography image with an LCC view and segmented image.

FIGS. 5A-5D are plan views showing segmented images and their corresponding vertical profiles.

FIGS. 6A-6C are plan views showing one process for detecting laterality of images, according to one embodiment.

FIG. 7 is a table showing probability vectors used for type and laterality assignment, as practiced in one embodiment.

DETAILED DESCRIPTION OF THE INVENTION

The present description is directed to elements forming part of, or cooperating more directly with, apparatus in accordance with the invention. It is to be understood that elements not specifically shown or described may take various forms well known to those skilled in the art.

The method of the present invention uses analysis of segmented images and probability logic to identify type and laterality for mammography images. Using the method of the present invention, a system can accept, as input image data, the standard set of mammography images for a patient and can identify the image view type (MLO or CC) and laterality (R or L). These images can then be provided to a system for display using an appropriate hanging protocol or pattern that meets the needs of the radiology practitioner.

The method of the present invention makes some assumptions about digital mammography images. For example:

• • (i) The images are of the standard image set with four views (RCC, LCC, RMLO, and LMLO), or two views for a breast.
  • (ii) There is a marker, placed by the technician at the time of image capture in the proper position. However, the marker need not be the correct one. (For example, the LMLO marker may have been incorrectly used during capture of the RMLO image).

Referring to FIG. 2, there is shown a block diagram of the basic steps in logic flow for using the method of the present invention.

In an image acquisition step 100, the images of a patient are obtained as digital data. This can be image data generated/captured directly as digital data, such as, for example, from scanned film, computed radiography (CR), or digital radiography (DR).

A segmentation step 110 is executed to segment the radiographic images to identify at least two segmented regions; three basic regions are generally identified by segmentation, as described subsequently. A collimation region (that is, foreground) is the area of the image that is occluded to X-ray collimation during the exposure and normally presents salient borders surrounding the body part. Direct exposure regions (that is, background) are areas that have received direct X-ray exposure. Diagnosis useful regions (that is, anatomy) contain the breast tissue region and the marker region.

There are known segmentation techniques that could be applied in step 110. The method outlined in U.S. Publication No. 2005/0018893 entitled “Method of Segmenting a Radiographic Image into Diagnostically Relevant and Diagnostically Irrelevant Regions” by Wang et al., incorporated herein by reference, can be used. This process typically involves sub-sampling of the original image to generate an image of coarser resolution that can be more easily processed and segmenting the anatomy region from the sub-sampled image. Other segmentation techniques may obtain two thresholds from the image histogram, then segment the image into the foreground, background, and anatomy regions based on these thresholds.

FIGS. 3A, 3B, 4A, and 4B show typical results from applying segmentation techniques, such as described in U.S. Publication No. 2005/0018893 (Wang et al.). In FIGS. 3A and 3B, LMLO image 30 is segmented to provide segmented image 34. In FIGS. 4A and 4B, processing of LCC image 50 yields segmented image 54.

Once an image is segmented, the foreground and background can be removed from the original mammogram image by setting their pixel value to a pre-defined value. As the result, the segmented image only contains the set of diagnostically relevant regions that are useful for further processing, both to determine type and laterality, and to perform the diagnostic assessment. Among these regions, the breast region is the major region in the image, labeled Region 1 in FIG. 4B.

In addition, each of these images has a marker region (shown, for example, at element 12 in FIGS. 3A and 4A, and at element 22 in FIG. 4B) that is one of the diagnostically relevant regions according to the method of the present invention. It is noted that text characters from the marker region are not needed for image identification. In FIG. 4B, marker region 22 is labeled as Region 2. Under some circumstances, a variable collimation region, incorrectly placed markers, or embedded cassette information can appear as other diagnostically relevant regions due to imperfect segmentation, such as is shown as Region 3 in the example of FIG. 4B. Regions 1 (breast region) and 2 (marker region 22) are the minimum set of diagnostically relevant regions needed for the method of the present invention. Additional regions, as exemplified by Region 3 in FIG. 4, may also be diagnostically relevant according to the present invention.

Referring again to FIG. 2, an analysis step 120 is executed on the diagnostically relevant regions, based on segmented images obtained from segmentation step 110. For this step, each region in the segmented image can first be uniquely labeled and assigned an integer, then analyzed. According to one embodiment of the present invention, the label number of each region is associated with the region's area. For example, as shown in FIG. 4B, the region with the largest area is assigned as Region 1, and the second largest region is assigned as Region 2, and so forth.

Region 1 will include the image of breast tissue and is used to determine the type of image, whether CC or MLO. FIGS. 5A and 5C show example segmented images 54 and 34 and illustrates key features used for this procedure. Referring to FIGS. 5B and 5D, for Region 1, a vertical profile 14,16 is first extracted from the image by using the following equation:

$\begin{array}{cc}P\left(y\right)=\sum _x^{}f\left(I\left(x,y\right)\right)f\left(I\left(x,y\right)\right)=\{\begin{array}{cc}1& I\left(x,y\right)\in \mathrm{Region}1\\ 0& I\left(x,y\right)\notin \mathrm{Region}1\end{array}& \left(1\right)\end{array}$

where I(x,y) is the labeled image.

In the present invention, an assumption is used for differentiating the MLO and CC views. That is, it can be assumed that the extracted profile of a CC view image generally includes a self-symmetrical portion, or is at least substantially more symmetrical than is the MLO view. A symmetrical index can be computed as a measure of relative symmetry. In accordance with one embodiment of the present invention, analysis of the profile symmetry obtained in Equation (1) is accomplished by computing the symmetrical index S of the profile, which is obtained by the following equation:

$\begin{array}{cc}S=\frac{1}{\sum _m^{}\sum _i^mP\left(\mathrm{peak}-i\right)-P\left(\mathrm{peak}+i\right)}m=\{\begin{array}{cc}\mathrm{peak}& \mathrm{peak}<\mathrm{rows}-\mathrm{peak}\\ \mathrm{rows}-\mathrm{peak}& \mathrm{peak}\ge \mathrm{row}-\mathrm{peak}\end{array}& \left(2\right)\end{array}$

where:

peak is location of the profile peak, that is, the row with the maximal profile value, as indicated in FIGS. 5B and 5D;

i is the distance from the peak (row number); and

rows represents the total number of rows in the image.

By way of illustration, FIG. 5B shows an example vertical profile 14 for the LCC segmented image 54 of FIGS. 4B and 5A. FIG. 5D shows an example vertical profile 16 corresponding to the LMLO segmented image 34 of FIGS. 3B and 5C. According to Equation (2), the larger the symmetrical index S, the more symmetrical the profile. In other words, the larger this index, the more likely that an image is of CC type, rather than MLO type.

It can be observed that the present invention is not limited to using this embodiment with Equation (2) or using profile symmetry in order to perform analysis for projection recognition. Other suitable algorithms may be known to those skilled in the art and can be employed, provided that they identify the difference between the two projection view types with some degree of accuracy.

Analysis for identifying the laterality of mammography images takes advantage of the other diagnostically relevant regions, that is, the region or regions other than that containing the breast image. For example, in one embodiment, Region 1 is first effectively removed from the segmented image by setting each of its pixels to a predefined value. FIG. 6A shows segmented image 56 having Region 1 removed. Then, the remaining image is split vertically into two equal-sized portions, as shown by top and bottom portions 58a and 58b in FIGS. 6B and 6C.

For each of these top and bottom portions, a laterality feature L is computed using the equation:

$\begin{array}{cc}L=\sum _{x,y}^{}f\left(I\left(x,y\right)\right)f\left(I\left(x,y\right)\right)=\{\begin{array}{cc}1& I\left(x,y\right)\in \mathrm{any}\mathrm{Region}\\ 0& I\left(x,y\right)\notin \mathrm{any}\mathrm{Region}\end{array}& \left(3\right)\end{array}$

where I(x,y) is the labeled image after removing the largest region (that is, Region 1).

If the upper portion has larger laterality feature L value, the image represents the right side of the patient; otherwise, it represents the left side. According to one conventional hanging protocol for screening, the images from the right side of the patient appear on the left, and the images from the left side of the patient appear on the right portion of the display. As with the symmetry index S described earlier, the laterality feature L could alternately be calculated using any of a number of other algorithms, as alternatives to that given by way of example in Equation (3).

A similar approach can be used to identify the correct/incorrect orientation of mammogram images. Generally, if lower portion 58b has a larger L value, the image has incorrect orientation and needs to be rotated.

Referring again to FIG. 2, an identification step 130 is executed. Although the methods described for analysis in step 120 provides an indication of image type and laterality, there can be some ambiguity, due to patient differences, technician practices, and equipment differences. For this reason, identification step 130 of the present invention uses a probability vector for each input mammogram image. This method can use the approach described in U.S. Publication No. 2004/0234125, entitled “System And Method Of Assigning Mammographic View And Laterality To Individual Images In Groups Of Digitized Mammograms” by Menhardt et al., incorporated herein by reference.

As shown in FIG. 7, the probability vector approach has four elements corresponding to the four possibilities for type and laterality (that is, LMLO, RLMO, RCC, and LCC). Each element entered in the table shown in FIG. 7 represents the probability of the input image having the specified view, based on analysis step 120 results. Using this approach, the symmetry index S (Equation 2) and laterality feature L (Equation 3) can be used to generate a probability vector for each image. The final decision on type and laterality is then made by evaluating the four probability vectors of images in the same study.

This method seeks global optimization by maximizing the sum of the probabilities of all mammograms of a patient. In the example of FIG. 7, Image 2 is clearly an LMLO image using this probability data. Images 1 and 3 can be determined with somewhat less confidence. Image 4 is assigned with a lower probability factor than other images. As a result, the sum of probabilities of all images is maximal among all possible sums.

At the conclusion of identification step 130, the image type and laterality of a set of standard mammography images can be automatically determined, along with its orientation. The detected type and laterality can then be assigned to each image and may be displayed along with the image or stored in a file header or in a separate file or other data structure that is associated with the image. Type and laterality assignment could also be displayed in an otherwise unused part of the image background. This assignment would allow the images to be displayed to a practitioner in suitable format, on one or more high-resolution display monitors, without the need for operator intervention or rearrangement. Image data stored with the assigned designation, such as in the image header or in some other manner, would then be available when an image is recovered from storage, such as from a PACS image storage system.

Unlike other approaches, the method of the present invention does not require that an operator use the correct lead marker when performing the image operation. The present invention uses a probabilistic model for decision-making and is thus adaptable to situations where there is somewhat less clarity about image type or there is ambiguity in the recognition results of one or two images in a study. While described primarily with regard to applications in mammography, the method of the present invention could be adapted to other types of diagnostic imaging, where it is a need to classify views taken from different perspectives or on different sides of the body.

The invention has been described in detail with particular reference to certain embodiments thereof, but it will be understood that variations and modifications can be effected within the scope of the invention as described above, and as noted in the appended claims, by a person of ordinary skill in the art without departing from the scope of the invention. For example, various types of image processing algorithms could be applied for segmentation and for determining image type and laterality. Any of a number of alternative approaches can be used for providing a symmetry index or computing a laterality feature.

Thus, what is provided is an apparatus and method for automatic detection of view type and laterality for digital mammographic images.

Parts List

• 10 Display
• 12 Marker
• 14, 16 Vertical profile
• 20 RMLO image
• 22 Segmented marker image
• 30 LMLO image
• 34 Segmented image
• 40 RCC image
• 50 LCC image
• 54 Segmented image
• 56 Segmented image
• 58 a, 58 b Image portion
• 100 Image acquisition step
• 110 Segmentation step
• 120 Analysis step
• 130 Identification step

Claims

1. A method for displaying a mammography image, comprising: obtaining digital data of the mammography image;segmenting the mammography image to identify at least a first diagnostically relevant region comprising an image of the breast tissue and a second diagnostically relevant region;assigning a view type for the mammography image, either cranio-caudal or medio-lateral oblique view, according to a symmetry index calculated from the segmented first diagnostically relevant region; andassigning right or left laterality to the mammography image according to a laterality feature calculated according to the relative position of at least the second diagnostically relevant region within the image.

2. The method of claim 1 further comprising: displaying the mammography image with one or more other mammography images in a relative display position determined by the assigned view type and right or left laterality.

3. The method of claim 1 wherein assigning the view type is accomplished by: forming a vertical profile of the segmented first diagnostically relevant region by identifying a peak row of pixels, wherein the peak row has a peak value corresponding to a maximum image contour along the horizontal; andsumming pixel values for each row above and below the peak row to obtain the symmetry index indicative of symmetry of the segmented first diagnostically relevant region.

4. The method of claim 1 wherein assigning the laterality is accomplished by: dividing the mammography image into upper and lower portions of equal size;removing the first diagnostically relevant region from the mammography image;summing pixel values for each row of the upper and lower portions to determine which portion includes a marker region as its second diagnostically relevant region; andinferring the laterality according to the detected marker region.

5. The method of claim 1 wherein obtaining the digital data of the mammography image data is accomplished by obtaining image data from a computed radiography cassette.

6. The method of claim 1 wherein segmenting the mammography image includes sub-sampling the mammography image at a lower resolution.

7. The method of claim 1 further comprising storing the assigned view type and left or right laterality as data associated with the mammography image.

8. The method of claim 1 wherein the symmetry index yields information for generating a probability vector.

9. The method of claim 1 wherein the laterality feature provides an indicator of incorrect image orientation.

10. A method for displaying a plurality of mammogram images from a patient, comprising: obtaining digital data of the mammography images; andassigning, to each mammography image, a unique combination of view type, either cranio-caudal or medio-lateral oblique view, and left or right laterality, by:segmenting the image to identify at least a first diagnostically relevant region comprising an image of the breast tissue and a second diagnostically relevant region;computing a symmetry index for the first diagnostically relevant region;computing a laterality feature according to the relative position of at least the second diagnostically relevant region within the image; andgenerating a probability vector according to the symmetry index and laterality feature for the image and using the probability vector data to assign the view type and laterality combination to the image.

11. The method according to claim 10 further comprising: displaying the plurality of mammogram images in a pattern according to the view type and laterality combination assigned to each image in the probability vector.

12. The method of claim 10 wherein computing a symmetry index is accomplished by: forming a vertical profile from the first diagnostically relevant region by identifying a peak row of pixels, wherein the peak row has a peak value corresponding to a maximum image contour along the horizontal; andsumming pixel values for each row above and below the peak row to obtain the symmetry index indicative of symmetry of the diagnostically relevant region.

13. The method of claim 10 wherein segmenting further comprises sub-sampling the mammogram image at a lower resolution.

14. The method of claim 10 further comprising storing the assigned view type and left or right laterality as data associated with the mammography images.