Patent Application
20100111398
This application claims priority from Indian Provisional Application Serial No. 2660/CHE/2008 filed on Oct. 31, 2008, entitled “Characterize the epithelium at tissue level and cellular level to categorize cancerous from normal oral mucosa”, which is incorporated herein by reference in its entirety.
Embodiments of the disclosure relate to analyzing of an image of an oral sample.
Oral cancer is a cancerous tissue growth in oral cavity. Oral Submucous Fibrosis (OSF) is a progressive pre-cancerous condition that indicates presence of pre-malignant epithelial cells in the oral cavity that leads to oral cancer. Often, early diagnosis of the OSF prevents the pre-malignant epithelial cells from developing into oral cancer.
Detection of the OSF is done through analysis of tissue samples collected from oral mucosa. The oral mucosa is mucous membrane epithelium of the oral cavity. Existing techniques to detect the OSF rely on biopsy and microscopic examination of the tissue samples by a pathologist. The detection of the OSF is dependent on expertise or intelligence of the pathologist analyzing the tissue samples.
An example of a method for analyzing an image of an oral sample includes receiving the image of the oral sample. The method also includes converting the image to a gray-scale image. Further, the method includes de-noising the gray-scale image. Furthermore, the method includes enhancing epithelial region in the gray-scale image. The method includes generating a binary image from the gray-scale image. Further, the method includes detecting boundary of the epithelial region in the binary image. Furthermore, the method includes extracting the boundary of the epithelial region. The method also includes extracting basal cell nuclei in the epithelial region. Further, the method includes determining one or more parameters of the epithelial region and the basal cell nuclei to enable detection of the oral sample as one of pre-malignant and non-malignant.
Another example of a method for analyzing an image of an oral sample by an image processing unit includes receiving the image of the oral sample. The method includes converting the image to a gray-scale image. The method also includes detecting at least one of thickness of the epithelial region, visual texture of the epithelial region, number of basal cell nuclei per unit length, size of the basal cell nuclei, and shape of the basal cell nuclei from the gray-scale image. Further, the method includes classifying the oral sample as one of pre-malignant and non-malignant based on the detection.
An example of an image processing unit (IPU) for analyzing an image of an oral sample includes an image and video acquisition module that electronically receives the image. The IPU also includes a digital signal processor that detects at least one of thickness of epithelial region, visual texture of the epithelial region, number of basal cell nuclei per unit length, size of the basal cell nuclei, and shape of the basal cell nuclei from the image to enable detection of the oral sample as one of pre-malignant and non-malignant.
In the accompanying figures, similar reference numerals may refer to identical or functionally similar elements. These reference numerals are used in the detailed description to illustrate various embodiments and to explain various aspects and advantages of the disclosure.
Referring to
In some embodiments, the oral sample can be obtained using one or more techniques, for example incisional biopsy, punch biopsy, shave biopsy, excisional biopsy, or curettage biopsy. The incisional biopsy can be defined as a process of removing tissues using a blade, for example a scalpel blade or a curved razor blade. The incisional biopsy can include a lesion or part of affected skin and part of the normal skin. The oral sample obtained by incisional biopsy is then subjected to on one or more clinical procedures. The clinical procedure includes, but is not limited to paraffinization, de-paraffinization, treatment using alcohol, and treatment using xylene. The oral sample is also stained by treating the oral sample with Harris hematoxylin solution for few minutes, for example 8 minutes and with eosin-phloxine B solution or eosin Y solution for another few minutes, for example 1 minute. The staining can be referred to as haematoxylin & Eosin (H&E) staining and the oral sample obtained after staining can be referred to as H&E stained oral sample. After staining of the oral sample, color of nuclei when observed under the microscope 105 appears as blue and the color of cytoplasm when observed under the microscope 105 appears as pink or red.
The microscope 105 can be coupled to an image sensor, for example a digital camera 120. The coupling can be performed using an opto-mechanical coupler 125. The digital camera 120 acquires an image of the oral sample. The image of the oral sample can be acquired under 10×, 20×, 40×, 100× of primary magnification provided by the microscope 105. The magnification 10× is used for examination of epithelial region. The magnifications of 20× or 40× is used for examination of collagen fibre region. In some embodiments, both the magnifications of 20× and 40× are used for examination of the collagen fibre region. The magnification 100× is used for examination of basal cell nuclei. In one example, the digital camera 120 is capable of outputting the image having at least 1024×768 pixel resolution. In another embodiment, the digital camera 120 is capable of outputting the image having 1400×1328 pixel resolution.
The digital camera 120 can be coupled to an image processing unit (IPU) 130. The IPU can be a digital signal processor (DSP) based system. The digital camera 120 can be coupled to the IPU 130 through a network 145. In one example, the digital camera 120 is coupled to the IPU 130 via a direct link. Examples of direct link between camera and IPU 130 include, but are not limited to, BT656 and Y/C, universal serial bus port, and IEEE ports. The digital camera 120 can also be coupled to a computer which in turn is coupled to the network 145. Examples of the network 145 include, but are not limited to, internet, wired networks and wireless networks. The IPU 130 receives the image acquired by the digital camera 120 and processes the image.
In some embodiments, the IPU 130 can be embedded in the microscope 105 or in the digital camera 120. The IPU 130 processes the image to detect whether the oral sample is pre-malignant or non-malignant. The IPU 130 can be coupled to one or more devices for outputting result of processing. Examples of the devices include, but are not limited to, a storage device 135 and a display 140.
The IPU 130 can also be coupled to an input device, for example a keyboard, through which a user can provide an input. The IPU 130 includes one or more elements to analyze the image and is explained in conjunction with
Referring to
An image and video acquisition module 210 electronically receives the image from an image sensor, for example the digital camera. In one example, the image and video acquisition module 210 can be a video processing subsystem (VPSS). The VPSS includes a front end module and a back end module. The front end module can include a video interface for receiving the image. The back end module can include a video encoder for encoding the image. The IPU 130 includes a digital signal processor (DSP) 205, coupled to the switched central resource 215, that receives the image of the oral sample and processes the image. The DSP 205 converts the image to a gray-scale image. Further, the DSP 205 determines one or more parameters of the epithelial region as well as parameters of the basal cell nuclei in the oral sample to enable detection of the oral sample as one of pre-malignant and non-malignant.
In some embodiments, the IPU 130 also includes a classifier that compares the parameters with a predefined set of values corresponding to a type of cancer. If the parameters match the predefined set of values then the classifier determines the oral sample to be pre-malignant else as non-malignant. The classifier also generates abnormality marked image, based on comparison, which can then be displayed, transmitted or stored, and observed.
In some embodiments, the DSP 205 also includes a classifier that compares the parameters with a predefined set of values corresponding to a type of cancer. If the parameters match the predefined set of values then the classifier determines the oral sample to be pre-malignant else as non-malignant. The classifier also generates abnormality marked image, based on comparison, which can then be displayed, transmitted or stored, and observed. The abnormalities marked image based on the plurality of parameters is displayed on the display 140 using a display controller 240.
Referring to
At step 305, an image of an oral sample is received. The IPU receives the image from the source.
At step 310, the image is converted to a gray-scale image. The image can be a color image. The color image and the gray-scale image are made of picture elements, hereafter referred to as pixels.
At step 315, the gray-scale image is de-noised.
The gray-scale image can be processed using a weighted median filter to remove speckle noise and salt-pepper noise. The weighted median filter can be referred to as non-linear digital filtering technique and can be used to prevent edge blurring. A median of neighboring pixels values can be calculated. The median can be calculated by repeating following steps for each pixel in the image.
Various other techniques can also be used for removing noises. Examples of the techniques include, but are not limited to, a mean filter technique described in “Digital Image Processing” by R. C. Gonzalez and R. E. Woods, 2e, pp. 253-255, which is incorporated herein by reference in its entirety.
At step 320, epithelial region in the gray-scale image is enhanced using histogram stretching technique. The histogram stretching technique is an image enhancement technique that improves contrast in an image by modifying the range of gray-scale values the image contains. In some embodiments, the epithelial region can be enhanced based on histogram equalization. The histogram stretching technique is a linear scaling technique, whereas the histogram equalization is a non-linear scaling technique. The histogram stretching technique as described in a book titled “Digital Image Processing” by R. C. Gonzalez and R. E. Woods, 2e, pp. 107-108, is incorporated herein by reference in its entirety.
At step 325, a binary image from the gray-scale image is generated. The binary image can be defined as an image having two values for each pixel. For example, two colors used for the binary image can be black and white. Various techniques can be used for generating the binary image, for example Otsu auto-thresholding. The technique is described in a publication titled “A threshold selection method from gray-level histograms” by N Otsu published in IEEE Trans. Systems Man Cyber., vol. 9, pp. 62-66, 1979, which is incorporated herein by reference in its entirety.
At step 325, alternatively an entropy based approach for image thresholding can be used as described in publications titled “A new method for gray-level picture thresholding using the entropy of the histogram” by J. N. Kapur, P. K. Sahoo, and A. K C. Wong, published in J. Comput. Vision Graphics Image Process., vol. 29, pp. 273-285, 1985 and “Picture thresholding using an iterative selection method”, by T. Ridler and S. Calvard, published in IEEE Trans. Systems Man, Cyber., vol. 8, pp. 630-632, 1978, which are incorporated herein by reference in its entirety.
At step 330, boundary of the epithelial region is extracted using morphological boundary extraction technique as described in a book titled “Digital Image Processing” by R. C. Gonzalez and R. E. Woods, second edition, pp. 556-557, which is incorporated herein by reference in its entirety.
At step 335, non-epithelial region pixels detected in the step 330 are removed using connected component labeling technique and the boundary of the epithelial region is extracted as described in “Digital Image Processing” by R. C. Gonzalez and R. E. Woods, second edition, pp. 558-561, which is incorporated herein by reference in its entirety.
At step 340, basal cell nuclei in the epithelial region are extracted. Extracting the basal cell nuclei in the epithelial region is based on a parabola curve fitting technique, a watershed segmentation technique, thresholding, and a connected component labeling technique. The techniques for extracting the basal cell nuclei are described in a publication titled “Fitting nature's basic functions part i: polynomials and linear least squares” Rust, B. W, computing in science and engineering, 84-89, 2001” and Digital Image Processing” by R. C. Gonzalez and R. E. Woods, second edition, pp. 644-646, which is incorporated herein by reference in its entirety.
At step 345, one or more parameters of the epithelial region as well as parameters of the basal cell nuclei are determined to detect oral sample as pre-malignant or non-malignant. The parameters include thickness of epithelial region, visual texture of the epithelial region, number of basal cell nuclei per unit length, size of the basal cell nuclei, and shape of the basal cell nuclei.
In one embodiment, the thickness of the epithelial region is based on length of epithelium contour. Generally, the epithelial region is not a closed contour and the length of the epithelium contour is always higher than other unwanted edges. Further, techniques including edge linking and connected component labeling technique are used to extract the contours of the epithelium. Further, euclidean distance of the epithelial region is determined after hotelling transform as described in “Digital Image Processing” by R. C. Gonzalez and R. E. Woods, second edition, pp. 700-701, which is incorporated herein by reference in its entirety. The hotelling transform is performed to rotate the epithelium contour to make it horizontal. Further, mean distance, median distance, maximum distance, minimum distance, and standard deviation can be computed to determine thickness of the epithelial region.
The visual texture of the epithelial region is based on variations in gray-scale intensities of the pixels in the gray-scale image. The visual texture is determined based on fractal dimension of the epithelial region. To measure variation in the grayscale image the technique described in “Fractal dimension estimation for texture images: A parallel approach” Biswas, M., Ghose, T., Guha, S., & Biswas, P. Pattern Recognition letters, vol. 19, pp. 309-313, 1998, is incorporated herein by reference in its entirety.
The number of basal cell nuclei per unit length is calculated by performing a parabola curve fitting technique. The parabola curve fitting technique is done with the extracted contour of epithelio-mesenchymal junction for defining the basal layer boundary. The epithelio-mesenchymal junction can be defined as a region where connective tissue of lamina propria meets overlying oral epithelium. The lamina propria is a constituent of the oral mucosa. The color image is super-imposed between the parabola curve and the extracted contour to separate the basal layer. The separated basal layer image is converted into the gray scale image. Further, local variation within cells is diminished after applying an averaging filter. Furthermore, thresholding is performed followed by morphological closing operation with a disk of diameter, for example, 6 pixels. Further the watershed segmentation technique as described in “Digital Image Processing” by R. C. Gonzalez and R. E. Woods, second edition, pp. 644-646, Pearson-Prentice Hall, India”, incorporated herein by reference in its entirety, is used to separate the individual nuclei. Further, the nuclei are labeled based on the connected component labeling technique. The number of nucleus present in the basal layer can be determined based on the labeling. The length of the lower contour of the epithelium is used for counting the number of pixels in that contour and is multiplied with a conversion factor of microscope, for example 0.06 μm to find the length of epithelial contour. The number of cells present in the contour will give the no of cell per unit length.
The size of the basal cell nuclei is based on area of the basal cell nuclei. The area of the basal cell nuclei is measured by counting the number of pixels on interior boundary of the basal cell nuclei and adding one half of the pixels on the perimeter, to correct for the error caused by digitization as described in “Cancer diagnosis via linear programming” Mangasarian and W. H. Wolberg, SIAM News, vol. 23, no. 5, September 1990, pp 1-18, which is incorporated herein by reference in its entirety.
The shape of the basal cell nuclei is based on at least one of the area of basal cell nuclei, perimeter of the basal cell nuclei, compactness of the basal cell nuclei and eccentricity of the basal cell nuclei. The area of the basal cell nuclei is measured by counting the number of pixels on the interior boundary of the basal cell nuclei and adding one half of the pixels on the perimeter, to correct for the error caused by digitization. The perimeter is measured as the sum of the distances between consecutive boundary points. The compactness is measured based on the perimeter and area to give a measure of the compactness of the cell nuclei and is calculated as shown in equation 1 given below:
Eccentricity is the ratio of the length of minor (u) to major (v) axis of the ellipse approximation of the basal cell nuclei and is calculated as shown in equation 2 given below:
At step 405, an image of the oral sample is received.
At step 410, the image is converted to a gray-scale image.
At step 415, at least one of thickness of the epithelial region, visual texture of the epithelial region, number of basal cell nuclei per unit length, size of the basal cell nuclei, and shape of the basal cell nuclei from the image is detected.
At step 420, the oral sample is classified as pre-malignant or non-malignant based on the detection.
The parameters enable detection of the oral sample as one of pre-malignant and non-malignant. Pre-malignant can be defined as pre-cancerous. Non-malignant can be defined as being non-cancerous, for example being benign. In some embodiments, a subset of the parameters can be used for detection based on accuracy desired. Reduction in number of parameters being processed helps in reducing computational power of the IPU.
In some embodiments, the oral sample can be classified as one of pre-malignant and non-malignant based on at least one of the perimeter, the area, the compactness, the eccentricity of the basal cell nuclei and the texture of the epithelial region. The classification can be done by comparing the parameters with a predefined set of values for different grades of cancers. For example, cancers can be differentiated based on degrees. The predefined set of values can be different for different grades of cancers. A cancer can be detected when the parameters satisfy the predefined set of values. Each predefined value can be a number or a range.
Various techniques can be used for classification, for example a Bayesian classifier technique as described in “Pattern Classification”, Duda R. 0., Hart P. E., and Stork D. G, pp. 20-23, Wiley, 2005, can be used and is incorporated herein by reference in its entirety.
In some embodiments, an abnormalities marked image can be generated based on the parameters.
In some embodiments, at least one of transmitting the abnormalities marked image, storing the abnormalities marked image, and displaying the abnormalities marked image can be performed. The abnormalities marked image can then be used by doctors and experts.
An example of the thickness values for non-malignant and the OSF affected epithelial region is illustrated in Table 1.
An example of the fractal dimension values for non-malignant and the OSF affected epithelial region is illustrated in Table 2.
An example of the number of basal cell nuclei per unit length values for non-malignant and the OSF affected epithelial region is illustrated in Table 3.
A plurality of parameters, for example area, perimeter, compactness, and eccentricity of the cell nucleus is determined. An example of the values of the parameters is illustrated in Table 4.
In the foregoing discussion, the term “coupled or connected” refers to either a direct electrical connection or mechanical connection between the devices connected or an indirect connection through intermediary devices.
The foregoing description sets forth numerous specific details to convey a thorough understanding of embodiments of the disclosure. However, it will be apparent to one skilled in the art that embodiments of the disclosure may be practiced without these specific details. Some well-known features are not described in detail in order to avoid obscuring the disclosure. Other variations and embodiments are possible in light of above teachings, and it is thus intended that the scope of disclosure not be limited by this Detailed Description, but only by the Claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2660/CHE/2008 | Oct 2008 | IN | national |
