1. Field of the Invention
The present invention relates to image processing methods and apparatus for generating processed versions of input images by mapping pixel values of images on to pixel values of the processed versions of the images. In one example, the images may be produced from parts of the body captured, for example, from an endoscope during surgery on the human or animal body and displayed for review by a surgeon.
2. Description of the Prior Art
It is known to process images in some way so as to make features of interest which appear in those images clearer or to some extent more easily recognisable. In one example, the images can be medical images, such as X-rays or images produced by endoscopes. Endoscopes provide one example of generating medical images inside the body for diagnosing and treating a human or animal patient. However, generally when faced with an image from inside the body, the surgeon may not immediately recognise areas of interest which may require surgery and/or treatment of some kind.
WO 96/13805 discloses an image processing apparatus in which enhancements are made by spatial histogram analysis of images generated for example, from X-rays. The enhancement is made by compressing the tonal range and in some cases expanding the contrast range of an area of interest to reveal more significant information depending upon the purpose of the image. In some examples, the image is segmented in order to perform a histogram analysis. In one example, the segmentation is performed by applying a k-means clustering algorithm.
A technical problem is concerned with providing an improvement in processing images so that features of those images can be viewed more clearly.
According to the present invention there is provided an image processing apparatus operable to generate at least one processed image from an input image. The image processing apparatus is operable to receive the input image represented as a plurality of pixels each of which includes red, green and blue component values and to identify k local mean for each of the red, green and blue component values of the pixels, where k is an integer. The image processing apparatus is operable to identify for each of the k local mean, for each of the red, green and blue components, a candidate range of component values, and a mapping function for mapping the candidate range of the component values of the onto an available dynamic range of possible component values for representing the image. The image processing apparatus is operable to apply for each of the red, green and blue components of each pixel of the input image the mapping function identified for each of the k local mean, to form for each of the k-local mean a processed image for display.
In one example, the k local mean of the component values of the pixels of the image are identified using a k means clustering algorithm.
Embodiments of the present invention can provide a system which can be used, in one application to assist surgeons during visible light endoscopic examinations. The system can be used by surgeons to detect and analyse lesions in operating theatres, thereby reducing a need for histologies and repeat procedures. The system applies a contrast adjustment method to images in order to highlight areas or features of interest, such as lesions and the detailed structure on their surface, within endoscopic images. The system processes the image produced to use, as much of the dynamic range of the display device as possible, to display the area of the image containing the lesion. The distribution of pixel values in the input image is analysed and candidate ranges selected which include the feature or area of interest which may contain, for example, lesions. These candidate ranges can then be used to generate candidate images which contain an enhanced view of the lesion. In some examples, the candidate image is selected for displaying the lesion with the highest amount of detail. This can be acceptable in an operating theatre. Once the candidate image has been selected the system can display the result in real-time alongside the original endoscopic video image.
The system can be applied in real-time to video data from standard definition or high definition visible light endoscopes. The system can also provide still image and video capture functions to enable peer review and future DICOM compliance. The system can be used to enhance the diagnostic capabilities of existing endoscopic equipment.
Various further aspects and features of the present invention are defined in the appended claims.
The above and another objects, features and advantages of the invention will be apparent from the following detailed description of illustrative embodiments which is to be read in connection with the accompanying drawings, in which:
Embodiments of the present invention will now be described with reference to the accompanying drawings where like parts have corresponding alphanumeric references, and in which;
a illustrates a generation of a single processed image for K=1 of the K means clustering algorithm,
In
As shown in
According to the K means clustering algorithm an initial mean is chosen for each of the K local mean values. A Euclidean distance is then calculated between the initial chosen mean and each of the pixels which are nearest to that “local” mean. The mean value is then adjusted by recalculating the mean in order to minimise the Euclidean distance between the local mean and the pixels which are nearest to the local mean. This process is then repeated until a local mean for pixels nearest to that local mean is determined. Thus, as shown in
As a next stage in the processing of the pixel values of the image, the dynamic range of the pixels within a variance range of the mean, referred to as a candidate range, are mapped onto a total dynamic range which is available for presenting the image. This is performed, for example, for several values of K, providing assumptions of the number of features which are of interest in the image. Thus, K for example, is performed for K=1, 2 and 3. For K=1 then one processed image is produced, for K=2 then two processed images are produced and for K=3 then three processed images are produced. This is because for each value K, a local mean is identified and the pixels within a variance value of that local mean have their dynamic ranged mapped on the total dynamic range available for displaying the image. Therefore the candidate ranges are calculated as follows:
1. Calculate the mean (μ) and standard deviation (σ) of the pixels within the image. A range [Rlow, Rhigh] is calculated from the mean and standard deviation.
R
low=μ−σ a.
R
high=μ+σ b.
v′=(v−Rlow)/(Rhigh−Rlow)*255, if Rlow<v<Rhigh d.
2. Five further ranges are obtained by performing K-means clustering with K=2 and with K=3 and using the means and standard deviations of the resulting clusters.
3. Two of the ranges with the smallest standard deviation are eliminated as these typically correspond to pixels from shadow areas or very bright areas.
4. The corresponding candidate images for the three remaining ranges are generated in the same way as for step 1.
One example is illustrated in
A corresponding example for K=3 is shown in
As explained above, for K=1 then one image is generated, for K=2 two images are generated and for K=3 three processed images are generated. Thus in total there are six processed images produced for each of the values K=1, K=2, K=3. This is illustrated in
The automatic method of enhancing the contrast in endoscopic images coupled with a small manual step for selecting the appropriate image is a good way to improve the endoscopic examination without having to explicitly detect lesions.
A summary of the operations performed by the image processor 12 is provided by the flow diagram in
S1—The image which is to be processed is received from a camera, for example, from an endoscope in a form which provides pixel values having RGB components. Alternatively, the RGB components of the pixels can be calculated by the image processing device.
S4—For each pixel of the image, a feature factor is formed from the RGB values for use in the K means clustering algorithm.
S6—The K means clustering algorithm is then applied to the pixels using the feature vectors to identify K local mean values for each of K=1, K=2 and K=3. Of course, other values of K could be used.
S8—For each of the K local mean values, the mean value is taken and a variance either side of that value is applied to identify a local dynamic range of features of interest.
S10—For each of the K local mean values produced by applying the K means clustering algorithm for each value of K, a mapping function is calculated to map the RGB pixel values of the input image I onto a processed version of the input image to stretch the dynamic range of the candidate range of pixels onto the total dynamic range available such as 0 to 255.
S12—For each of the K local mean values the mapping function is applied to each of the red, green and blue components to produce pixels for each corresponding processed image. The processed images are then displayed.
Various aspects and features of the embodiments described above may be changed and adapted whilst still falling within the scope of the present invention as defined in the appended claims. For example, any value of K could be used to determine the K local mean. Furthermore other ways of determining the local mean other than the K means clustering algorithm could be used. In addition, whilst the embodiment has been described with reference to medical imaging using an endoscope, it would be appreciated that the invention is not limited to medical applications or medical images and could find application in other areas such a topographical processing of images, archaeology and geographical mapping.
K-means (MacQueen. 1967) is one of the simplest unsupervised learning algorithms that solve the well known clustering problem. The procedure follows a simple and easy way to classify a given data set through a certain number of clusters (assume k clusters) fixed a priori. The main idea is to define k centroids, one for each cluster. These centroids should be placed in a cunning way because of different location causes different result. So, the better choice is to place them as much as possible far away from each other. The next step is to take each point belonging to a given data set and associate it to the nearest centroid. When no point is pending, the first step is completed and an early grouping is done. At this point we need to re-calculate k new centroids as barycenters of the clusters resulting from the previous step. After we have these k new centroids, a new binding has to be done between the same data set points and the nearest new centroid. A loop has been generated. As a result of this loop we may notice that the k centroids change their location step by step until no more changes are done. In other words centroids do not move any more. Finally, this algorithm aims at minimizing an objective function, in this case a squared error function. The objective function
where ∥xi(j)−cj∥2 is a chosen distance measure between a data point xi(j) and the cluster centre cj, is an indicator of the distance of the n data points from their respective cluster centres.
The algorithm is composed of the following steps:
Although it can be proved that the procedure will always terminate, the k-means algorithm does not necessarily find the most optimal configuration, corresponding to the global objective function minimum. The algorithm is also significantly sensitive to the initial randomly selected cluster centres. The k-means algorithm can be run multiple times to reduce this effect.
K-means is a simple algorithm that has been adapted to many problem domains. As we are going to see, it is a good candidate for extension to work with fuzzy feature vectors. More information can be found at:
http://www.elet.polimi.it/upload/matteucc/Clustering/tutorial_html/kmeans.html
Although illustrative embodiments of the invention have been described in detail herein with reference to the accompanying drawings, it is to be understood that the invention is not limited to those precise embodiments, it is to be understood that the invention is not limited to those precise embodiments, and that various changes and modifications can be effected therein by one skilled in the art without departing from the scope and spirit of the invention as defined by the appended claims.
Number | Date | Country | Kind |
---|---|---|---|
0700352.8 | Jan 2007 | GB | national |