Patent Application
20170367677
This application claims the priority benefits of Taiwan application serial no. 105120124, filed on Jun. 27, 2016. The entirety of the above-mentioned patent application is hereby incorporated by reference herein and made a part of specification.
The invention relates to an analysis method and an electronic apparatus using the same, and more particularly, to an analysis method for analyzing breast image and an electronic apparatus using the same.
Mammary carcinoma is a common female malignant tumor, and the main symptom includes breast tumors, abnormal secretions, or shape variation, etc. Screening an aberrant symptom for breast in advance can facilitate in treating the tumor earlier so that deterioration or proliferation of cancer cells can be reduced. Screening methods, such as clinical or self breast detection, biopsy, mammography, ultrasound or magnetic resonance imaging and the like, have been widely used in clinical practice or have become important issues in academic researches.
Traditionally, after a breast image is obtained, medical personnel then examines and verifies whether an aberrant part is included in the breast image. If the aberrant part is found, a different inspection method is used to confirm whether the aberrant part is the malignant tumor. However, when there are a massive number of the breast images, it is very time-consuming and ineffective for medical personnel to examine whether the aberrant part is included in the breast images one by one. On the other hand, mistakes can often be made if the aberrant part in the breast image is filtered only by the human eye.
Accordingly, developing an analysis method and an electronic apparatus using the same to effectively detect, analyze and acquire the aberrant part in the breast image is still one of the major subjects for person skilled in the art.
The invention is directed to an analysis method for breast image and an electronic apparatus using the same, which can be used to detect, analyze and obtain an aberrant part in the breast image.
An embodiment of the invention provides an analysis method for breast image, which includes the following steps. A breast image scanned by an ultrasound device is obtained. Based on rectangular features of the breast image, a region of interest including an aberrant region in the breast image is obtained by applying a detection model. The aberrant region is acquired from the region of interest, and a plurality of feature parameters of the aberrant region are extracted for a property analysis of the aberrant region. The step of obtaining the region of interest includes: dividing the breast image into a plurality of image blocks; calculating the rectangular features for each of the image blocks; determining whether each of the image blocks is a region of interest block by the detection model based on the rectangular features; and when a plurality of said region of interest blocks adjacent to one another are present, merging the plurality of said region of interest blocks as the region of interest.
An embodiment of the invention provides an electronic apparatus suitable for analyzing breast image. The electronic apparatus includes a storage unit and a processor. The storage unit is configured to store a plurality of modules, and the processor coupled to the storage unit is configured to access and execute the modules stored by the storage unit. The modules include an input module, a detection module, an acquisition module and an extraction module. The image input module obtains a breast image scanned by an ultrasound device. The detection module obtains a region of interest including an aberrant region in the breast image by applying a detection model based on a plurality of rectangular features of the breast image. The detection module divides the breast image into a plurality of image blocks, calculates the rectangular features for each of the image blocks, determines whether each of the image blocks is a region of interest block by the detection model based on the rectangular features, and when a plurality of said region of interest blocks adjacent to one another are present, merging the plurality of said region of interest blocks as the region of interest. The acquisition module acquires the aberrant region from the region of interest. The extraction module extracts a plurality of feature parameters of the aberrant region for a property analysis of the aberrant region.
Based on the above, according to the analysis method for breast image and the electronic apparatus using the same as provided in the embodiments of the invention, the region of interest including the aberrant region is obtained from the breast image by applying the detection model based on the rectangular features of the breast image such that the aberrant region can be acquired from the region of interest. With respect to the aberrant region, the feature parameters can be further extracted for the property analysis, and whether multiple aberrant regions belong to the same tumor can also be determined and correspondingly displayed. As a result, the analysis method and the electronic apparatus using the same can be used to identify the aberrant region with the aberrant symptom while providing the related property analysis. On the other hand, the analysis can be completed rapidly, promptly and effectively even in the case of dealing with a massive number of the breast images.
To make the above features and advantages of the invention more comprehensible, several embodiments accompanied with drawings are described in detail as follows.
The accompanying drawings are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.
Reference will now be made in detail to the present preferred embodiments of the invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers are used in the drawings and the description to refer to the same or like parts.
Some embodiments of the invention are described in details below by reference with the accompanying drawings, and as for reference numbers cited in the following description, the same reference numbers in difference drawings are referring to the same or like parts. The embodiments are merely a part of the invention rather than disclosing all possible embodiments of the invention. More specifically, these embodiments are simply examples of devices and methods recited in claims of the invention.
In the analysis method for breast image and the electronic apparatus using the same as proposed in the embodiments of the invention, first of all, a region of interest (ROI) including an aberrant region in a breast image is obtained by applying a detection model based on a plurality of rectangular features of the breast image. The aberrant region refers to a region showing the aberrant symptom in the breast image. More specifically, the region showing the aberrant symptom may be a tumor or a symptom, and a trained detection model can assist in detecting and identifying the aberrant region.
The analysis method and the electronic apparatus using the same can also acquire the aberrant region from the region of interest precisely and extract related feature parameters for a property analysis, so as to improve accuracy in subsequent diagnosis for breast. In addition, the analysis method and the electronic apparatus using the same are also capable of calculating and providing an overall breast density as a diagnostic reference from multiple breast images.
In the embodiment described above, the electronic apparatus 100 is further connected to an ultrasound scanning apparatus, a handheld ultrasound scanner, an automated breast ultrasound system (ABUS) or a magnetic tracker ultrasound scanning system. However, in other embodiments of the invention, the electronic apparatus 100 is directed implemented in from of the ultrasound scanning apparatus, the handheld ultrasound scanner, the automated breast ultrasound system or the magnetic tracker ultrasound scanning system, for example. In other words, the functions of computer aided detection (CADe) and/or computer aided diagnosis (CADx) provided in the present invention may be integrated in a hardware such as the ultrasound scanning apparatus, the handheld ultrasound scanner, the automated breast ultrasound system or the magnetic tracker ultrasound scanning system directly.
In an embodiment of the invention, the storage unit 120 may be a fixed or a movable device in any form, including a random access memory (RAM), a read-only memory (ROM), a flash memory, or similar devices or a combination of the aforementioned devices. In the present embodiment, the storage unit 120 stores a plurality of modules accessible and executable by the processor 140, and the modules include an image input module 122, a detection module 124, an acquisition module 126, an extraction module 128, a training module 130, a density analysis module 132, a tumor determination module 134, a display module 136, etc. On the other hand, the storage unit 120 may also be used to store data related to the breast image, the detection model, parameters, etc., but the invention is not limited thereto.
It should be noted that, the storage unit 120 described in the embodiment above is not limited only to be one single memory device. That is to say, each of the modules may also be separately stored in two or more than two memory devices of the same or different types. In other embodiments of the invention, the modules may also be separately implemented by using a specific circuit structure.
In an embodiment of the invention, the processor 140 is implemented by, for example, a programmable unit such as a central processing unit (CPU), a digital signal processing (DSP) chip, a field programmable gate array (FPGA), a microprocessor, a micro controller, etc., but the invention is not limited thereto. The processor 140 may also be implemented by an independent electronic apparatus or an integrated circuit (IC).
In an embodiment of the invention, the electronic apparatus 100 further includes devices like an input/output interface (not illustrated), a communication interface (not illustrated), etc., but the invention is not limited thereto. Specifically, the input/output interface includes devices for outputting or inputting messages and data, such as a display, a speaker, a keyboard, a mouse, a touch panel, etc. On the other hand, the communication interface supports various communication standards and wireless communication standards so that the electronic apparatus 100 can connect to the other devices.
The analysis method for breast image provided by the embodiments of the invention can be realized by the electronic apparatus 100 illustrated in
In an embodiment of the invention, the breast image is an image obtained by scanning a breast part of the subject with the ultrasound scanning apparatus, the handheld ultrasound scanner, the automated breast ultrasound system or the magnetic tracker ultrasound scanning system. For example, the image input module 122 receives the breast image directly from the ultrasound scanning apparatus, the handheld ultrasound scanner, the automated breast ultrasound system or the magnetic tracker ultrasound scanning system, but the invention is not limited thereto. In another embodiment of the invention, for example, the electronic apparatus 100 stores the obtained breast image in the storage unit 120, so the image input module 122 can then read the breast image from the storage unit 120 for analysis.
With reference to
One of the missions assigned to the detection module 124 is to detect and obtain the region of interest including the aberrant region in the breast image. In general, if there is any tumor or mass in the breast part, usually, a shaded region also correspondingly appears in the breast image obtained by scanning with the ultrasound wave. Therefore, in the present embodiment, the detection module 124 can determine whether there is an abnormal shaded region in the breast image by applying the detection model based on the rectangular features of the breast image. If so, the detection module 124 treats the shaded region as the aberrant region, and obtains the region of interest including the aberrant region. It should be noted that, the rectangular features are Haar-like features.
Next, the detection module 124 determines whether each of the image blocks is a region of interest block by the detection model based on the rectangular features (step S223). More specifically, in an embodiment of the invention, the detection model is used to identify whether at least part of the shaded region caused by the aberrant symptom is included for each of the image blocks, and is trained using the breast images having tumor or mass and the breast images not having tumor or mass. In other words, before the region of interest can be obtained by applying the detection model, a related training must be completed with training breast images.
In an embodiment of the invention, the image input module 122 obtains a plurality of training breast images scanned by the ultrasound device. Then, after calculating the rectangular features for each of the training image blocks, the training module 130 trains a classifier as the detection model based on training image blocks of each of the training breast images. The training breast images are, for example, training breast images having tumor or mass and the training breast images not having tumor or mass, and the classifier is, for example, a binary classifier, such as a support vector machine (SVM), an adaptive boosting (Adaboost) classifier, etc., but the invention is not limited to the above.
With reference to
In other embodiments of the invention, if there is only one region of interest block, the detection module 124 then treats that region of interest block as the region of interest, for example. On the other hand, the breast image may also include a plurality of said region of interest. In other words, the breast image includes multiple groups of the adjacent region of interest blocks.
In an embodiment of the invention, the detection module 124 is, for example, implemented by a computer aided detection (CADe) module, but the invention is not limited thereto.
It should be noted that, while the ultrasound device is used for scanning, in addition to tumor or mass, ribs or other factors may also cause the shaded region in the breast image. In other words, even though the detection module 124 can obtain the region of interest including the aberrant region by applying the rectangular features of the breast image and the detection model, a false detection may still occur owing to ribs or other factors.
In an embodiment of the invention, the detection module 124 further identifies whether the region of interest is the false detection based on a muscle line. In general, a muscle tissue (e.g., pectoralis major) is also included between the breast part, ribs and intercostal of the subject. In other words, said muscle tissue may be used to determine whether the region of interest is located at the breast part, so as to further identify whether the region of interest is the shaded region caused by ribs or other factors.
For instance, the region of interest R in the image 5b located below the muscle line ML indicates that this region of interest R is not located at the breast part. In this case, the subsequent analysis is not performed for the region of interest R of the image 5a. On the other hand, the region of interest R in the image 5d having only a small part protruding above the muscle line ML indicates that this region of interest R is highly possible the shaded region caused by ribs. In this case, the subsequent analysis is not performed for the region of interest R of the image 5c either.
With reference to
In the embodiment described above, the frontground image F of the image 7b mainly corresponds to the aberrant region A of the breast image 7a instead of providing an accurate correspondence relation. After obtaining the frontground image
F and the background image B, the acquisition module 126 further places an unknown region U between the frontground image F and the background image B through dilation and erosion to generate a tri-map 7c.
In the present embodiment, after generating the tri-map 7c, the acquisition module 126 calculates an alpha value for each pixel in the tri-map 7c (step S232).
Specifically, the acquisition module 126 acquires the aberrant region based on the tri-map 7c. Herein, the acquisition module 126 adopts image division methods such as Closed-form Solution and Poisson Matting to determine an image type to which each pixel in the unknown region U belongs based on the frontground image F and the background image B being given.
In general, in the tri-map 7c composed of the frontground image F and the background image B, each pixel may be expressed by one linear combination below.
I
i=αiFi+(1−αi)Bi
Herein, α1 is the alpha value of an pixel, or a proposition of the ith pixel occupied by the frontground image F and the background image B. After calculating the alpha value for each pixel in the unknown region U or the tri-map, the acquisition module 126 further generates an alpha map based on the alpha value of each pixel (step S233). The alpha map is a transparency diagram including the alpha value of each pixel, and a numerical range of each pixel is within a range from 0 to 255. Lastly, the acquisition module 126 acquires the aberrant region from an image 7d generated by binarizing the alpha map (step S234). The images 7b and 7d are edge-enhanced images, and a method for binarizing includes, for example, Otsu's Thresholding or Balanced Histogram Thresholding, but the invention is not limited thereto.
With reference to
For instance, in an embodiment of the invention, the feature parameters of the aberrant region are further inputted to a computer aided diagnosis (CADx) module for the property analysis, but the invention is not limited thereto. The computer aided diagnosis module is, for example, executed by the processor 140 or other devices (systems), and has an aided diagnosis model underwent said training. With the aided diagnosis model of the computer aided diagnosis module, analysis data related to the aberrant region (e.g., benign lesion or malignant lesion, lesion condition) can be obtained and provided to medical personnel as diagnostic reference data.
It should be noted that, in the process of scanning the breast part by the ultrasound device, it is usually required to scan the breast part in multiple directions in order to obtain multiple of said breast images. In an embodiment of the invention, the analysis method is used to separately analyze each of the breast image, and obtain the aberrant region and the feature parameters one by one from each of the breast image.
In another embodiment of the invention, the analysis method and the electronic apparatus 100 using the same may further obtain a breast density of the breast part as another feature parameter for analysis.
In the embodiment described above, when the breast part of the subject is scanned by the ultrasound device, it is required to obtain multiple of the breast images by scanning in different directions. Herein, after calculating the corresponding breast density for each of the breast image, the density analysis module 132 performs a weighted average on the breast densities from all of the breast images so as to further obtain an overall breast density (step S254). The overall breast density includes the breast densities of said multiple of the breast images scanned in various directions, and can therefore be regarded as a substantial breast density of the breast part and served as a reference feature parameter in diagnosis for breast tumor or lesion.
It should be noted that, a plurality of aberrant regions may be acquired from one breast image with the analysis method for breast image described above, and the aberrant regions are possible positions of tumors and masses in the breast part. In an embodiment of the invention, whether the aberrant regions are of the same tumor may be further determined.
In detail, when the electronic apparatus 100 scans and obtains an original image by using, for example, the magnetic tracker ultrasound scanning system, a coordinate of a reference point may be set and a 3D coordinate info nation can be simultaneously recorded for each position in the obtained image according to the coordinate of the reference point. After the aberrant regions are acquired by the acquisition module 126, the tumor determination module 134 can obtain the 3D coordinate information of these aberrant regions, and classify the aberrant regions according to the 3D coordinate information so as to determine the aberrant regions closing to one other as the same tumor. In the present exemplary embodiment, when the distance between one aberrant region (e.g., said first aberrant region) and another aberrant region (e.g., said second aberrant region) among the aberrant regions in terms of 3D coordinate is less than the threshold, the tumor determination module 134 can determine that these two aberrant regions belong to the same tumor (step S903). In particular, the invention is not intended to limit a value of the threshold.
In addition, after determining that the aberrant regions belong to the same tumor, the same tumor can be displayed on the obtained original breast image or a pathway map by the display module 136, wherein the pathway map is generated according to a plurality of images obtained on a scanning path of the ultrasound probe (step S905).
For instance, in an embodiment of the invention, the breast image is the image obtained by scanning the breast part of the subject with the ultrasound scanning apparatus, the handheld ultrasound scanner, the automated breast ultrasound system or the magnetic tracker ultrasound scanning system. In the process of scanning, one scanning path is formed by a moving trajectory of the device or system for scanning on the breast part of the subject. Meanwhile, the device or system for scanning path and combine the images into one pathway map by the processor 140. When the tumor determination module 134 determines that the aberrant regions belong to the same tumor, the display module 136 can display the same tumor on the pathway map to be outputted through an output device (e.g., a screen) for doctors to conduct an instant diagnosis. In other words, an indication (or a prompt) of the tumor may be provided or outputted through the output device immediately (i.e., in real-time) in the process of scanning. For example,
Nonetheless, in an exemplary embodiment, the aberrant region 10a, the aberrant region 10b, the aberrant region 10c and the aberrant region 10d determined as the same tumor may also be displayed superimposingly on the original breast image obtained through the ultrasound wave with the same color (e.g., red) by the display module 136. Similarly, the aberrant region 12a and the aberrant region 12b determined as the same tumor may also be displayed on the original breast image obtained through the ultrasound device with the same color (e.g., blue) by the display module 136. In addition, the aberrant region 14a determined as one independent tumor may also be displayed on the original breast image obtained through the ultrasound wave with one single color (e.g., purple) by the display module 136.
In summary, according to the analysis method for breast image and the electronic apparatus using the same as provided in the embodiments of the invention, the region of interest including the aberrant region is obtained from the breast image by applying the detection model based on the rectangular features of the breast image such that the aberrant region can be acquired from the region of interest. With respect to the aberrant region, the feature parameters can be further extracted for the property analysis, and whether multiple aberrant regions belong to the same tumor can also be determined and correspondingly displayed. As a result, the analysis method and the electronic apparatus using the same can be used to identify the aberrant region with the aberrant symptom while providing the related property analysis. On the other hand, the analysis can be completed rapidly, promptly and effectively even in the case of dealing with a massive number of the breast images.
It will be apparent to those skilled in the art that various modifications and variations can be made to the structure of the present invention without departing from the scope or spirit of the invention. In view of the foregoing, it is intended that the present invention cover modifications and variations of this invention provided they fall within the scope of the following claims and their equivalents.
| Number | Date | Country | Kind |
|---|---|---|---|
| 105120124 | Jun 2016 | TW | national |
