METHOD AND APPARATUS FOR SEED POINT DETERMINATION

Abstract

Embodiments of the present invention provide a computer-implemented method of processing image data for determining seed points with respect to the image data, comprising determining a voting map, wherein said voting map is based on voting locations determined for at least some pixels within the image data, determining a density map, wherein determining said density map comprises determining a weighting for at least some pixels within the voting map, wherein the weighting is determined as a non-linear function based on a value of each of the at least some pixels of the voting map, and determining the seed points in the image data on the density map.

Description

BACKGROUND

It is sometimes desired to segment an image. Segmenting an image manually may be very difficult, or even virtually impossible, where the image includes large numbers of features to be segmented. An example is an image of biological cells may contain thousands of cells which would be extremely time consuming to manually segment.

Methods of automatically segmenting images have been devised. In order to automatically segment an image some methods require an identification of seed points within the image. A seed point is associated with a feature of interest and provides a starting point for segmenting that feature from other features, such as cells, within the image. To be most effective, segmentation typically requires exactly one seed point per feature of interest. It will be appreciated that seed point identification may be useful in other situations than segmentation. However accurate determination of seed points may be problematic, particularly where the features in the image are widely varying in, for example, shape.

It is an object of embodiments of the invention to at least mitigate one or more of the problems of the prior art.

STATEMENTS OF INVENTION

According to aspects of the present invention, there is provided methods and apparatus as set forth in the appended claims.

According to an aspect of the present invention, there is provided a method of processing image data for determining seed points with respect to the image data, comprising determining a voting map based on voting locations determined for at least some pixels within the image data, determining a density map, wherein determining said density map comprises determining a weighting for at least some pixels within the voting map, and determining the seed points in the image data based on the density map. In some embodiments, the weighting is optionally determined as a non-linear function based on a value of each of the at least some pixels of the voting map.

According to an aspect of the present invention, there is provided a method of processing image data for determining seed points with respect to the image data, comprising determining a voting map, wherein said voting map is based on voting locations determined for at least some pixels within the image data, determining a density map, wherein determining said density map comprises determining a weighting for at least some pixels within the voting map, wherein the weighting is determined as a non-linear function based on a value of each of the at least some pixels of the voting map, and determining the seed points in the image data based on the density map.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention will now be described by way of example only, with reference to the accompanying figures, in which:

FIG. 1 shows an image of biological cells;

FIG. 2 shows a method of determining seed points according to an embodiment of the invention;

FIG. 3 shows a further method of determining a voting map according to an embodiment of the invention;

FIG. 4 shows an illustration of a feature of a vote according to an embodiment of the invention; and

FIG. 5 shows an illustration of a density map according to an embodiment of the invention;

FIG. 6 shows an illustration of potential seed points according to an embodiment of the invention;

FIG. 7 illustrates an apparatus according to an embodiment of the invention; and

FIG. 8 is a graph illustrating performance of an embodiment of the invention.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

Embodiments of the present invention determine one or more seed points in an image. The image may comprise one, or more often, a plurality of features. The features may have a degree of similarity, such as biological cells. It will, however, be appreciated that embodiments of the present invention are not restricted to being utilised with images of cells. FIG. 1 shows an example image 100 including a plurality of features, which in this example are cells.

The image is formed by a plurality of pixels, such as arranged in a grid arrangement.

The image may be formed by image data comprising m×n pixels with each pixel having one or more associated values, such as colour values i.e. RGB. The image may be a greyscale image i.e. where each pixel has only one value indicative of luminance or brightness. Each pixel may assume a value between predetermined minimum and maximum values, such as 0 and 255 although other values may be used. The value of each pixel may be indicative of an amount of radiation (such as although not exclusively visible light) reflected by, or transmitted through, the features present in the image.

In some embodiments of the invention the image is a phase image. In such a phase image a value of each pixel is indicative of a phase change imparted by a portion of a feature at the location corresponding to the pixel. Methods of determining phase images may be known as ptychographic methods. The method of determining the phase image may be a process performed on data provided by a detector, wherein the process may be an iterative phase-determination process. Example methods of determining phase images are disclosed in WO 2005/106531 and WO 2010/064051, which are incorporated herein for all purposes. It will be appreciated that other methods for determining phase images are known and embodiments of the present invention are not limited to the method by which the phase image is produced. The example image in FIG. 1 is a phase image.

The image data may be based upon data output by an imaging device, such as a camera or a charge-coupled device (CCD), comprising a plurality of pixels which each output a portion of the image data. Other imaging devices may be used. The data output by the imaging device may be processed, such as described above, to determine the phase image.

FIG. 2 illustrates a method 200 of determining seed points within an image according to an embodiment of the invention.

An input to the method 200 is a voting map 205. The voting map 205 is based on the image. The voting map 205 may have dimensions corresponding to the image on which it is based i.e. an equal number of pixels. That is, the voting map may comprise m×n pixels. Each pixel of the voting map 205 has a value indicative of a number of votes for that location in the voting map corresponding to a centre of a feature in the image. The voting map is a map of voting locations. A location of each vote in the map is indicative of a corresponding pixel's estimate of the centre of the feature corresponding to the pixel. That is, each vote is a corresponding pixel's vote for the centre of the feature with which the pixel is associated. In the case that each feature is circular and has a radius which is known a priori, the votes should correspond to the centre of the feature, such as the centre of the cell. However where the features have varying or irregular shapes and/or sizes, the voting map 205 may comprise votes indicative of a plurality of centres for at least some features.

Before describing the method 200 further, a method 300 of determining the voting map 205 according to an aspect of the present invention will be described with reference to FIG. 3. It will be appreciated that other methods of determining the voting map 205 may be envisaged.

In step 310 an image is provided. As noted above, in some embodiments, the image may be provided from a detector, such as a pixelated detector, for example a CCD, recording an intensity of radiation falling on the detector from one or more objects irradiated with radiation. In some embodiments, prior to the method 300 the image may be generated, such as according to a ptychographic or other phase determination method or process, or the image may be output by the detector and provided to the method 300.

The one or more objects may be retained during imaging by a retainment means such as a microscope slide, although it will be appreciated that other arrangements of retainment means or retainment device may be used.

As discussed above, the image comprises a plurality of pixels each having at least one value which may correspond to an intensity of radiation falling a corresponding pixel of the detector. However, in other embodiments, the image may be a phase image where the value of each pixel is indicative of a phase change imparted to radiation by a portion of each feature corresponding to the respective pixel. In step 310 the image may be received from the detector or stored in a memory.

In step 320 the image is processed to provide a gradient map 325. Step 320 may comprise flatting the image. Step 320 may comprise one or both of smoothing and blurring the image. The image may be flatted by application of a background subtraction algorithm, such as a rolling ball algorithm, although other image flattening techniques may be used. Such a background subtraction algorithm may be particularly useful when background values of the image vary across the image. The flattened image has a background with more uniform values. The image may be smoothed by application of a filter, such as a Gaussian filter. The smoothing removes or reduces noise present in the image.

In step 320 the gradient map 325 is determined. The gradient map 325 may be determined by application of one or more Sobel filters to the image, although other methods may be used to determine the gradient map 325. The gradient map 325 is indicative of directional changes in pixel values of the image, where the pixel values may be indicative of intensity or phase, as discussed above. Each pixel may be associated with at least one value indicative of a local direction of the gradient. In some embodiments, each pixel of the image may be associated with a vector indicative of the direction and a magnitude of the gradient at the respective pixel location in the image. In some embodiments, a first filter may be used to determine a gradient in a corresponding first direction and a second filter may be used to determine a gradient in a corresponding second direction. The first direction may be horizontal and the second direction may be vertical. Based on an output of the first and second filters a gradient magnitude and direction is determined for each pixel in the image. Thus, as a result of step 320, the gradient map 325 is determined.

In step 330 a voting process is performed to determine a voting map 335. In step 330 at least some pixels in the image ‘vote’ on the location of a seed point. In some embodiments, every pixel in the image votes on the location of a seed point, although it will be realised that this is not necessary in every embodiment. The pixels vote for the location of the seed point for the feature with which the pixel is associated in the image.

FIG. 4 illustrates an example portion of image data 400 indicative of an edge 405 or peripheral boundary of a feature such as a cell. For a pixel in the image data 400 the gradient map 325 indicates a direction 410 of the local gradient at the pixel. A predetermined constant R_E 420 is used indicative of an estimated distance between the edge 405 and a centre of the feature, which may be an estimated radius of the feature where the feature is expected to be circular. The estimated distance R_E 420 is used to determine a distance 420 between the pixel and a location of the pixel's vote 440. The distance R_E 420 may be determined before the method 300 is performed, such as based on a user's input. In some embodiments the vote is determined as a vote distribution having a magnitude and variance. The magnitude of the vote distribution may be based upon the magnitude of the gradient for the pixel in the image data. The variance may be based on a predetermined variance V_E 430, as illustrated in FIG. 4, such as may be input by a user prior to the method. The vote distribution may be a Gaussian distribution in embodiments of the method 300, although it will be appreciated that other distributions may be used. In some embodiments the variance V_E 430 may be a predetermined number of pixels, such as 3 although other values may be used. Thus, for every pixel in the image for which a vote is determined, a corresponding distribution is located with respect to the pixel having a magnitude and variance indicative of that pixel's vote for the centre of the respective feature.

In the voting map 335 the votes from all of the pixels may be summed. Thus pixels in the voting map 335 are associated with a sum of magnitudes corresponding to a plurality of votes. The voting map 335 may have a size, such as m×n pixels, corresponding to the image data, or may be cropped in step 330 to the size of the image data. The voting map 335 may be provided as an input 205 to the method illustrated in FIG. 2.

Where the features in the image data are uniformly circular and have a radius corresponding to the estimated radius R_E, the voting map 335 may reliably indicate the centre of features in the image data. However, where the image contains features of one or both of varying sizes and shape, the voting map 335 may comprise more than one peak associated with each feature.

Referring again to FIG. 2, the method 200 comprises a step 210 of applying a voting threshold to the voting map 205. The voting threshold eliminates or removes any votes in the voting map 205 having values below the voting threshold. In some embodiments, each pixel in the voting map is assigned a weighting. Pixels having a number of votes below the voting threshold may be weighted by a low value, such as 0, thereby eliminating them from further consideration. The voting threshold may be a predetermined value. However in other embodiments of the invention the voting threshold is based upon a number of votes in the voting map 205, or a magnitude of votes in the voting map 205. In some embodiments, the voting threshold is based upon an average number of votes per pixel in the voting map 205. In some embodiments the voting threshold (voting_threshold) is determined as:

$\mathrm{voting\_threshold}=\frac{\mathrm{mean}\left(\mathrm{votes}\right)}{I}$

where mean(votes) is a mean number of votes per pixel of the voting map 205 and I has a value of between a predetermined minimum and maximum values. In some embodiments the minimum value is 1. The minimum and maximum may be between 5 and 15 in some embodiments. In some embodiments I has a value of 10, although other values may be used.

In step 220 a weighting is applied to at least some pixels in the voting map 205. The weighting may be applied, in some embodiments, to those pixels not eliminated in step 210, such as by already being assigned a weight of 0. The weighting is determined as a non-linear function based on a value of the pixel. In some embodiments of the invention the weighting may be determined as:

$\mathrm{weighting}=\lfloor \sqrt{\frac{\mathrm{voting\_value}}{\mathrm{voting\_threshold}}}\rfloor$

where voting_value is a vote value for a pixel. It will be appreciated that └ ┘ denotes a “floor” function determining a greatest integer that is less than or equal to x (rounding towards-∞). As a result of step 220 a weighted voting map is determined.

A density map 225 is determined based on the weighted voting map. The density map 225 is determined by storing in the density map 225 a distribution for each weighted vote in the weighted voting map. A magnitude of each distribution is based upon the weighted vote value for each pixel in the weighted voting map. In some embodiments, the magnitude of the distribution is equal to the weighted vote value for the corresponding pixel. The distribution is centred in the density map 225 at the location of the pixel in the weighted voting map. A variance of the distribution may be based upon the constant R_E 420 which, as noted above, is indicative of an estimated distance between the edge 405 and the centre of the feature, which is an estimated radius of the feature where the feature is expected to be circular. In some embodiments the variance may be determined as:

$\mathrm{variance}=\frac{R_E}{C}$

where C is a constant which, in some embodiments, may be up to 5 or may be selected as 3, although other values may be used. The distribution may be a Gaussian distribution in embodiments of the method 300, although it will be appreciated that other distributions may be used.

Density values arising from each weighted vote in the weighted voting map are summed in the density map 225. That is, a density value for each pixel in the density map 225 may correspond to a sum of density values arising from a plurality of pixels in the weighted voting map.

In step 240 local maxima in the density map 225 are determined. The local maxima are determined in step 240 by identifying in the density map 225 pixels having a largest value within a predetermined pixel range. The one or more local maxima may be stored in a data structure, such as a maxima list.

FIG. 5 illustrates a portion of density map 500 which is formed by a plurality of pixels arranged in columns and rows, as will be appreciated. For a pixel 510 of the density map 500 the density value of the pixel 510 is compared against density values of neighbouring pixels within the pixel range 520, 530. In FIG. 5 the pixel range is illustrated as being a predetermined number of pixels horizontally 520 and vertically 530. In some embodiments, such as illustrated in FIG. 5, the number of horizontal pixels 520 is equal to the number of vertical pixels 530. The pixel range 520, 530 may be three pixels horizontally and vertically 520, 530 although it will be appreciated that the pixel range may be different in other embodiments. Thus the density value of pixel 510 is compared against the eight values of surrounding pixels within the pixel range 520, 530. The pixel 510 is determined to be a local maximum if it has a density value greater than all of the pixels within the pixel range 520, 530. Whilst the neighbouring pixels in FIG. 5 are square shaped, it will be realised that in other embodiments the shape may be otherwise shaped, such as by considering neighbouring pixels within a predetermined pixel radius of the pixel 510. The local maximum is found for each pixel in the density map 225. Thus as a result of step 240 some pixels of the density map are identified as being local maxima in the density map 225.

In step 250 it is determined whether the local maxima values determined in step 240 are greater than a maxima threshold value. If, in step 250 a value of a local maximum is less than or equal to the maxima threshold, it is removed, i.e. eliminated from further consideration, in step 260. Step 250 is repeated for each local maximum value identified in step 240. Thus, as a result of steps 250 and 260, only local maxima with values greater than the maxima threshold remain under consideration. In some embodiments, only local maxima with values greater than the maxima threshold remain in the maxima list determined in 240. The output of step 250 may be used as seed points for the image. However, for some features in the image, two or more seed points may be associated with each feature. That is, some features in the image may be associated with a plurality of seed points corresponding to more than one local maxima above the maxima threshold. This particular occurs when the features have a range of size and shapes, such as for naturally occurring features.

In order to ameliorate the problem of multiple seed points per feature, in step 270 the potential seed points are consolidated in step 270. The potential seed points are consolidated by reducing a number of potential seed points per feature. In embodiments of the invention, potential seed points are removed based upon a continuity of pixels in the density map 225 between potential seed points having density values above a seed threshold 650.

The potential seed points may be stored in a data structure, such as a list, according to the density value of each potential seed point. That is, the list in some embodiments may be ordered according to the density value of each potential seed point.

FIG. 6 illustrates a portion of a density map 600 for which potential seed points have been identified in steps 240-260 associated with two pixels 610, 620. In other words, pixels 610, 620 have been identified as local maxima of density values. Line 650 represents the seed threshold. Step 270 of the method 200 consolidates the potential seed points 610, 620 by determining if a path exists in the density map 600 between the two pixels 610, 620 associated with potential seed points. The path is determined to exist if a continuum of other pixels is located between the two pixels 610, 620 having density values above the seed threshold 650. For example, a third pixel 630 is adjacent to the second pixel 620 having a density value above the seed threshold 650, as illustrated in FIG. 6. It will be observed that the density value of third pixel 630 is lower than that of the second pixel 620, but above threshold 650. A fourth pixel 640 exists between the third and first pixels 630, 610 having a density value below the seed threshold 650. Thus the fourth pixel 640 represents a break in the path between the first and second pixels 610, 620. It will be appreciated that in embodiments of the invention, the path may only be formed by horizontal and vertically adjacent pixels, whilst in other embodiments the path may be formed including diagonally-adjacent pixels. If a path exists between the potential seed points associated with the first and second pixels, 610, 620 then the potential seed point having the lowest density value i.e. second pixel 620, is removed from the data structure thereby consolidating the potential seed points. The value of the seed threshold 650 may be determined based upon one or both of the density values associated with potential seed points 610, 620. In one embodiment, the seed threshold 650 value may be determined based upon a density value of the larger of the potential seed points i.e. density value of first pixel 610. The seed threshold 650 value be determined as a proportion or percentage of the larger of density values associated with the potential seed points i.e. the first pixel 610.

In embodiments of the invention the consolidation in step 270 may be performed by selecting a first potential seed point in the data structure or list, such as the potential seed point associated with pixel 610. The first potential seed point may be a seed point having a greatest density value. The seed threshold 650 may be set based on a value of the selected seed point's density, such as 50%, 60% or 70% of the density value of the potential seed point, although other values or percentages may be used. A flood-fill is then performed using pixels in the density map having a value less than the seed threshold 650 as walls or barriers to restrain the flood fill. If pixels associated with other potential seed points in the list are encapsulated within the fill, then the potential seed points associated with these pixels are removed from the list. That is, if it is determined that a path exists between the first potential seed point 610 and pixels associated with other potential seed points within the flood-filled area, then the pixels associated with the other seed points are removed from the data structure. By repeating for all the potential seed points in the data structure the number of potential seed points is consolidated in step 270.

As a result of step 270 (and preceding steps) seed points are output from the method 200 indicative of seed points associated with features in the image.

FIG. 7 illustrates an apparatus 700 according to an embodiment of the invention. The apparatus 700 may be arranged to perform a method according to an embodiment of the invention, such as the method 200 illustrated in and described with reference to FIG. 2.

The apparatus comprises a processing means in the form of a processing unit 710. The processing unit 710 is communicably coupled to a memory 720. The processing means 710 may comprise one or more processing devices which operatively execute software instructions. The software instructions case the processing means 710 to perform a method such as the method 200. The software instructions may be stored in a computer readable medium, such as in the memory 720.

The apparatus 700 comprises an interface means which may be an interface unit 720 for operatively receiving data. The interface unit 720 is arranged to receive one or both of image data corresponding to the image or the voting map 205. The image data may be provided from a detector 750 in some embodiments, as illustrated in dotted lines in FIG. 7. Alternatively the image data may be received via a communications network, such as an IP-based network for example over the Internet, although other communication network technologies may be used. Particularly, although not exclusively, where the image is a phase image, the phase image may be received from another computing device (not shown) arranged to perform a method of phase determination, such as disclosed in WO 2005/106531 or WO 2010/064051, although embodiments of the invention are not limited in this respect. The image data and/or voting map may be stored in the memory 720.

Testing of an embodiment of the present invention against a prior art method of seed point determination, and the results are provided in the table below.

		Prior Art	Present Invention
	Per Cell	88.3%	96.3%
	Per Image	89.8%	97.7%

It will be understood that “per cell” is a percentage of cells correctly detected of a plurality of images tested containing 1857 cells, whilst “per image” is a mean detection rate over 21 images studied. Thus it can be appreciated that embodiments of the present invention provide improved determination of seed points for features within images.

FIG. 8 illustrates a performance of an embodiment of the invention in relation to a prior art method for 21 images. As can be appreciated, whilst a relative performance of the embodiment of invention in relation to a small number of images is equal, for most images the performance of the embodiment of the invention exceeds that of the prior art method.

It will be appreciated that embodiments of the present invention can be realised in the form of hardware, software or a combination of hardware and software. Any such software may be stored in the form of volatile or non-volatile storage such as, for example, a storage device like a ROM, whether erasable or rewritable or not, or in the form of memory such as, for example, RAM, memory chips, device or integrated circuits or on an optically or magnetically readable medium such as, for example, a CD, DVD, magnetic disk or magnetic tape. It will be appreciated that the storage devices and storage media are embodiments of machine-readable storage that are suitable for storing a program or programs that, when executed, implement embodiments of the present invention. Accordingly, embodiments provide a program comprising code for implementing a system or method as claimed in any preceding claim and a machine readable storage storing such a program. Still further, embodiments of the present invention may be conveyed electronically via any medium such as a communication signal carried over a wired or wireless connection and embodiments suitably encompass the same.

All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and/or all of the steps of any method or process so disclosed, may be combined in any combination, except combinations where at least some of such features and/or steps are mutually exclusive.

Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise. Thus, unless expressly stated otherwise, each feature disclosed is one example only of a generic series of equivalent or similar features.

The invention is not restricted to the details of any foregoing embodiments. The invention extends to any novel one, or any novel combination, of the features disclosed in this specification (including any accompanying claims, abstract and drawings), or to any novel one, or any novel combination, of the steps of any method or process so disclosed. The claims should not be construed to cover merely the foregoing embodiments, but also any embodiments which fall within the scope of the claims.

Claims

1. A computer-implemented method of processing image data for determining seed points with respect to the image data, comprising: determining a voting map, wherein said voting map is based on voting locations determined for at least some pixels within the image data;determining a density map, wherein determining said density map comprises determining a weighting for at least some pixels within the voting map, wherein the weighting is determined as a non-linear function based on a value of each of the at least some pixels of the voting map; anddetermining the seed points in the image data based on the density map.

2. The method of claim 1, wherein the weighting value for the at least some pixels of the density map is determined is based upon the value of the corresponding pixel of the voting map and a voting threshold.

3. The method of claim 2, wherein the weighting value for the at least some pixels of the density map is based upon a square root of a ratio between the value of the corresponding pixel of the voting map and the voting threshold.

4. The method of claim 2, wherein the weighting value for the at least some pixels of the density map is determined as:

5. The method of claim 1, further comprising eliminating voting values in the voting map less than a voting threshold value.

6. The method of claim 5, wherein the voting threshold value is based upon an average number of votes per pixel in the voting map.

7. The method of claim 5, wherein the voting threshold value is determined as:

8. The method of claim 7, wherein I has a value of between 5 and 15.

9. The method of claim 5, wherein the eliminating voting values in the voting map comprises weighting the voting values to a predetermined value; optionally the predetermined value is 0.

10. The method of claim 1, comprising determining local maxima in the density map, wherein the seed points are determined based on the local maxima in the density map.

11. The method of claim 10, wherein the determining the local maxima comprises selecting a pixel of the density map and determining whether a value of the pixel is a greatest value of pixels within a pixel range.

12. The method of claim 11, wherein the pixel range is a first number of pixels surrounding the selected pixel.

13. The method of claim 11, comprising selecting the pixel of the density map as a local maxima if said pixel has a greater value than all pixels within the pixel range.

14. The method of claim 10, wherein the determining the seed points comprises determining whether a pixel path exists between first and second local maxima in the density map, the pixels in the pixel path having values above a seed threshold.

15. The method of claim 14, wherein the seed threshold is based on a value of one or both of the first and second local maxima.

16. The method of claim 15, wherein the seed threshold is based upon a greatest value of the first and second local maxima.

17. The method of claim 16, wherein the seed threshold is a predetermined percentage of the greatest value of the first and second local maxima.

18. The method of claim 14, comprising eliminating a lower of the first and second local maxima if said path exists.

19. The method of claim 18, wherein the seed points are determined as remaining local maxima.

20. (canceled)

21. The method of claim 1, wherein the image data comprises value indicative of phase.

22. An apparatus, comprising: a memory for storing data therein;a processor, wherein the processor is arranged, in use, to perform a method comprising the steps of:determining a voting map, wherein said voting map is based on voting locations determined for at least some pixels within image data stored in the memory;determining a density map, wherein determining said density map comprises determining a weighting for at least some pixels within the voting map, wherein the weighting is determined as a non-linear function based on a value of each of the at least some pixels of the voting map; anddetermining the seed points in the image data based on the density map.

23. The apparatus of claim 22, wherein the image data is based upon data output by an imaging device.

24. (canceled)

25. A non-transitory computer readable medium having tangibly stored thereon computer software which, when executed by a computer, is arranged to perform a method according to claim 1.

26. (canceled)