Patent Application
20050017972
The invention relates to the setting of visualization parameter boundaries, such as color and opacity boundaries, for displaying images, in particular two-dimensional (2D) projections from three-dimensional (3D) data sets.
When displaying an image, such as in medical imaging applications, it is known to associate particular signal values with particular colors and opacities (known as visualization parameters) to assist visualization. This mapping is done when using data from a 3D data set (voxel data set) to compute a 2D data set (pixel data set) representing a 2D projection of the voxel data set for display on a computer screen or other conventional 2D display apparatus. This process is known as rendering.
The 2D data set is more amenable to user interpretation if different colors and opacities are allocated to different signal values in the 3D data set. The details of the mapping of signal values to colors and opacities are stored in a look-up table which is often referred to as the RGBA color table (R, G, B and A referring to red, green, blue and alpha (for opacity) respectively). The color table can be defined such that an entire color and opacity range is uniformly distributed between the minimum and maximum signal values in the voxel data set, as in a gray scale. Alternatively, the color table can be defined by attributing different discrete colors and opacities to different signal value ranges. In more sophisticated approaches, different sub-ranges are ascribed different colors (e.g. red) and the shade of the color is smoothly varied across each sub-range (e.g. crimson to scarlet).
When displaying data such as in medical imaging, the signal values comprising the data set do not usually correspond to what would normally be regarded as visual properties, such as color or intensity, but instead correspond to detected signal values from the measuring system used, such as computer-assisted tomography (CT) scanners, magnetic resonance (MR) scanners, ultrasound scanners and positron-emission-tomography (PET) systems. As an example, signal values from CT scanning will represent tissue opacity, i.e. X-ray attenuation. In order to improve the ease of interpretation of such images it is known to map different colors and opacities to different ranges of display value such that particular features, e.g. bone (which will generally have a relatively high opacity) can be more clearly distinguished from soft tissue (which will generally have a relatively low opacity).
When displaying a 2D projection of a 3D data set, in addition to attributing distinct ranges of color to voxels having particular signal value ranges, voxels within the 3D data set may also be selected for removal from the projected 2D image to reveal other more interesting features. The choice of which voxels are to be removed, or sculpted, from the projected image can also be based on the signal value associated with particular voxels. For example, those voxels having signal values which correspond to soft tissue can be sculpted, i.e. not rendered and therefore “invisible”, thereby revealing those voxels having signal values corresponding to bone which would otherwise be visually obscured by the soft tissue.
The determination of the most appropriate color table (known in the art as a preset) to apply to an image derived from a particular 3D data set is not trivial and is dependent on many features of the 3D data set. For example, the details of a suitable color table will depend on the subject, what type of data is being represented, whether (and if so, how) the data are calibrated and what particular features of the 3D data set the user might wish to highlight, which will depend on the clinical application. It can therefore be a difficult and laborious task to produce a displayed image that is clinically useful. Furthermore, there is inevitably an element of user-subjectivity in manually defining a color table and this can create difficulties in comparing and interpreting images created by different users, or even supposedly similar images created by a single user. In addition, the user will generally base the choice of color table on a specific 2D projection of the 3D data set rather than on characteristics of the overall 3D data set. A color table chosen for application to one particular projected image will not necessarily be appropriate to another projection of the same 3D data set. A color table which is objectively based on characteristics of the 3D data set rather than a single projection would be preferred.
Accordingly, there is a need in the art for a method of automatically determining appropriate color table presets when displaying medical image data.
According to the invention there is provided a method of setting visualization parameter boundaries for displaying an image from a 3D data set comprising a plurality of voxels, each with an associated signal value, comprising: selecting a volume of interest (VOI) within the 3D data set; generating a histogram of signal values from voxels that are within the VOI; applying a numerical analysis method to the histogram to determine a visualization threshold; and setting at least one of a plurality of boundaries for a visualization parameter according to the visualization threshold.
By restricting the histogram to voxels taken from the VOI, a numerical analysis method can be applied to the histogram which is sensitive to subtle variations in signal value and can reliably identify significant boundaries within the 3D data set for visualization. This allows the visualization parameter boundaries to be set automatically, which is especially useful for 3D data sets for which the signal values have no calibration, as is the case for MR scans.
In some embodiments, a first visualization parameter boundary is set at the visualization threshold. In other embodiments, first and second visualization parameter boundaries are set either side of the visualization threshold. This latter approach can be advantageous if an opacity curve interpolation algorithm is used to calculate an opacity curve between the visualization parameter boundaries.
The numerical analysis method may be applied once to determine only one visualization threshold. Remaining visualization parameter boundaries can then be set manually. Alternatively, the numerical analysis method can be applied iteratively to the histogram to determine a plurality of visualization thresholds and corresponding visualization parameter boundaries.
A significance test may be applied to visualization thresholds and, according to the outcome of the significance test, a significance marker can be ascribed for those ones of the voxels having signal values at or adjacent the visualization threshold, wherein the significance marker indicates significance or insignificance of the visualization threshold.
If two visualization parameter boundaries are set, one each side of the visualization threshold, and the visualization threshold is determined to be significant, then it is convenient to mark as significant only the voxels having signal values at one of the two visualization parameter boundaries. In one example, if a visualization threshold is calculated by the numerical analysis method to lie at a signal value of 54, and visualization parameter boundaries are set at 54±3, i.e. at 51 and 57, then the voxels with signal values of 57 can be marked as significant, and the voxels with signal values of 51 as insignificant.
The significance test can be used to distinguish between visualization parameter boundaries used as enhancements to visualizations of a single tissue type (known as cosmetic boundaries) and those used to identify different tissue-types for the purpose of segmentation (known as significant boundaries). Accordingly, the method may further comprise applying a selection tool to the 3D data set, wherein the selection tool is sensitive to the significance markers. One or more of the selection tools can be designed to ignore voxels that have been marked as insignificant.
The rate of change of a visualization parameter across a visualization parameter boundary may also be modified based on the significance of the visualization parameter boundary. A sharpness parameter can be calculated for determining what rate of change of the visualization parameter to apply at a boundary.
In some embodiments of the invention, the sharpness parameter is the same as the significance marker. The sharpness need not simply be a binary operand, but can adopt a range of integer values, for example from 0 to 100. A sharpness of zero indicates an insignificant boundary, which is referred to as a cosmetic boundary in view of its irrelevance to selection tools. A sharpness of 100 indicates a boundary that has the maximum degree of significance. Intermediate values are used to indicate intermediate significance. In addition to affecting the blending of visualization parameters, the non-zero values may be used for filtering by the selection tools so that boundaries with a significance value of, for example, 5 are significant to some but not all selection tools, a boundary with a significance value of 50 is significant for a greater subset of the selection tools, and a boundary with the maximum significance value of 100 is significant to all selection tools. Alternatively, the non-zero significance values may be used by selection tools to resolve conflicts between different marked boundaries, with boundaries having higher significance values taking precedence. Examples of selection tools are tools for marking objects in a set of connected or unconnected voxels with a visualization parameter (e.g. color or opacity) between two significant visualization parameter boundaries, multiple groups of connected or unconnected voxels above a significant boundary or multiple bands of connected or unconnected voxels below a significant boundary. Marked voxels could then, for example, be sculpted. Sculpting is a well known term of art used to describe voxels that are marked to be transparent from view irrespective of their signal values.
In the best mode of the invention, the numerical analysis method comprises: forming a convex hull of a plurality of segments around the histogram; determining which perpendicular from the segments to the histogram has the greatest length; and taking the signal value at the intersection between the histogram and the perpendicular as the visualization threshold. The sharpness value and the significance test can then be based on the length of the perpendicular determined to have the greatest length. For example, the visualization threshold can be determined to be insignificant if the ratio of the length of the perpendicular to a parameter derived from the signal value range and/or the frequency range of the histogram is below a minimum score.
For some automatic presets, the numerical analysis method is applied to the histogram within a predetermined restricted range of signal values to search for a visualization threshold within that restricted range. This will be particularly useful for 3D data sets with calibrated signal values, such as X-ray data sets calibrated in Hounsfield units. Accordingly, the restricted range may be defined in terms of Hounsfield units.
To provide the user with information about the nature of the automatically calculated thresholds, the histogram and its visualization parameter boundaries can be displayed to the user together with the image created from the 3D data set, thus making the user aware of the visualization parameter boundaries determined by the automatic preset.
The method of the invention is particularly powerful in that it can take account of sculpting performed on the 3D data set prior to automatic preset determination according to the invention. A common example of sculpting will be when a plane is defined through a 3D data set and all voxels to one side of the plane are not rendered, irrespective of their signal values. Another example of sculpting will be the removal of a given set of connected voxels with signal values in a specified range, thus restricting the range of signal values to be visualized prior to determining an automatic preset. Sculpting can be taken account of by restricting the histogram to unsculpted voxels in the VOI.
It has been recognized that voxels with the highest and lowest signal values often constitute bad data which can skew the results of the numerical analysis of the histogram. Accordingly, it is preferred that voxels with the highest and/or the lowest signal values are excluded from the numerical analysis method. For example, the voxels with the lowest and highest 0.1% of the signal values can be excluded. Other proportions could also be envisaged.
In some implementations the method may operate interactively. In such cases, if a user re-defines the VOI, the method of setting visualization parameter boundaries is automatically reapplied to continuously provide the most appropriate visualization parameter boundaries.
The invention further provides a computer program product bearing computer readable instructions for performing the method of the invention.
The invention also provides a computer apparatus loaded with computer readable instructions for performing the method of the invention.
According to a further aspect of the invention there is provided a method of numerically processing a medical image data set comprising voxels, the method comprising: receiving user input to positively and negatively select voxels that are and are not of a tissue type of interest; determining a distinguishing function that discriminates between the positively and negatively selected voxels on the basis of one or more characterizing parameters of the voxels; and classifying further voxels in the medical image data set on the basis of the distinguishing function. This method thus applies supervised pattern recognition to classify the voxels.
By receiving input in response to a user specifying both positive examples of voxels (i.e. those which do correspond to the tissue type of interest) and negative examples of voxels (i.e. those which do not correspond to the tissue type of interest), the method is able to objectively classify further voxels in the data set. Because of this, the method provides for an easy and intuitive to use technique for allowing users to select regions of interest for further examination or removal from the data set.
The method may include presenting a representative (2D) image derived from the (3D) medical image data set to a user, such as a sagittal, coronal or transverse section view, whereby the user selects voxels by positioning a pointer at appropriate locations in the example image. An example voxel may then be taken to be a voxel whose coordinates in the medical image data set map to the location of the pointer in the example image. Alternatively, for a single positioning of the pointer, a number of example voxels may be selected, for example those in a region surrounding a voxel whose coordinates in the data set map to the location of the pointer in the example image may be taken as being selected. Selecting multiple voxels with a single positioning of the cursor allows for a more statistically significant sample of example voxels to be provided with little additional user input.
At least one of the one or more characterizing parameters of a voxel may be a function of surrounding voxels. For example, a local average, a local standard deviation, gradient magnitude, Laplacian, minimum value, maximum value or any other parameterization may be used. This allows voxels to be classified on the basis of characteristics of their surroundings, rather than simply on the basis of their voxel value. This means that similar tissue types can be properly classified more accurately than with conventional classification methods based on voxel value alone. This is because subtle difference in “texture” in the vicinity of a voxel can help to distinguish it from other voxels having otherwise similar voxel values. It is also noted that for some modalities such as MR there may be multiple voxel values, such as T1 and T2 in multi-spectral MR, which could each be used to define a separate characterizing parameter. These could be used collectively in combination to set the distinguishing function.
Moreover, the user input may additionally include clinical information, such as specification of tissue type or anatomical feature, regarding either the positively or negatively selected voxels, or both. Following this user input, the distinguishing function can then determined from the characterizing parameters having regard to the clinical information input by the user.
Once the voxels have been classified, an image of the data set may be rendered which takes account of the classification of voxels. The rendered image may then be displayed to the user. For example, the positively selected voxels may be tinted with a color in a monochrome gray scale rendering.
In some examples, a binary classification may be used whereby voxels are classified as either corresponding to the tissue type of interest or not corresponding to the tissue type of interest. In these cases, voxels classified as not corresponding to the tissue type of interest may be rendered as transparent or semi-transparent in a displayed image. The general practice of rendering features that are not of interest as semi-transparent is sometimes referred to as “dimming” in the art. Alternatively, voxels which are classified as corresponding to the tissue type of interest may be rendered as transparent, or voxels classified as corresponding to the tissue type of interest may be rendered to be displayed in one range of displayable colors and voxels classified as not corresponding to the tissue type of interest being rendered to be displayed in another range of displayable colors.
An image based on rendering a volume data set representing the value of the distinguishing function of the voxels can also be made.
In other examples, rather than using a binary classification, voxels may be classified according to a calculated probability that they correspond to the tissue type of interest. In these cases, an image may be generated by rendering of a volume data set representing the probability that the voxels correspond to the tissue type of interest, rather than rendering based on voxel values themselves. For example, the probability can be mapped onto opacity of the rendered material instead of taking a threshold. Another approach would be to render as transparent any voxels having a probability of corresponding to the tissue type of interest of less than a certain value.
Where the classification provides an estimated probability for each voxel, the probabilities per voxel may themselves be considered as voxel values in a medical image data set which may be re-classified according in a subsequent iteration of the method. This implements a form of relaxation labeling.
Further, it will be appreciated that the user input can be prompted by displaying an image to a user from a 3D data set comprising a plurality of voxels, each with an associated signal value, for example by selecting a volume of interest (VOI) within the 3D data set; generating a histogram of signal values from voxels that are within the VOI; applying a numerical analysis method to the histogram to determine a visualization threshold; and setting at least one of a plurality of boundaries for a visualization parameter according to the visualization threshold.
According to a further aspect of the invention there is provided an apparatus for numerically processing a medical image data set comprising voxels, the apparatus comprising: storage from which a medical image data set may be retrieved; a user input device configured to receive user input to positively and negatively select voxels that are and are not of a tissue type of interest; and a processor configured to determine a distinguishing function that discriminates between the positively and negatively selected voxels on the basis of one or more characterizing parameters of the voxels; and to classify further voxels in the medical image data set on the basis of the distinguishing function.
The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.
For a better understanding of the invention and to show how the same may be carried into effect reference is now made by way of example to the accompanying drawings in which:
a shows a 2D projection of a 3D data set with tissue opacity values being represented by a linear gray-scale;
b schematically shows a graphical representation of the color and opacity curve mappings used in generating the 2D image shown in
c shows a 2D projection of a 3D data set with ranges of tissue opacity values being represented by ranges of a gray-scale defined by presets;
d schematically shows a graphical representation of the color and opacity curve mappings used in generating the 2D image shown in
e shows a 2D projection of a 3D data set with ranges of tissue opacity values being represented by ranges of colors defined by presets;
a shows a histogram of data values within a VOI within a 3D data set and to which a first convex hull has been applied to determine a first visualization threshold;
b shows a histogram of data values within a VOI within a 3D data set and to which a second convex hull has been applied to determine a second visualization threshold;
c shows a histogram of data values within a VOI within a 3D data set and to which a third convex hull has been applied to determine a third visualization threshold;
d shows a histogram of data values within a VOI within a 3D data set and to which a fourth convex hull has been applied to determine a fourth visualization threshold;
a shows a visualization state tool loaded with a VOI from a 3D data set for which color boundaries have been determined according to an automatic preset according to a first example of the invention, referred to as “Active MR”;
b shows an example image displayed according to the automatic preset of
a shows a visualization state tool loaded with a VOI from a 3D data set for which color boundaries have been determined according to an automatic preset according to a second example of the invention, referred to as “Active Bone (CT)”;
b shows an example image displayed according to the automatic preset of
a shows a visualization state tool loaded with a VOI from a 3D data set for which color boundaries have been determined according to an automatic preset according to a third example of the invention, referred to as “Active Angio (CT)”;
b shows an example image displayed according to the automatic preset of
a, 2c and 2e show example 2D images of the same projection from a 3D CT data set but with different presets.
a shows an example 2D image which is a projection of a 3D data set obtained from a CT scanner. A VOI within the 3D data set has been selected for display. The material surrounding the VOI is not rendered in the projection. The image is displayed with a uniform gray-scale ranging from black to white.
b schematically shows a graphical representation of the color and opacity curve mappings used in generating the 2D image shown in
There are three tissue types in the projected image. These are a region of bone 26, a region of soft tissue 30 and a barely visible network of blood vessels 28. The high X-ray stopping power of bone, compared to that of blood and soft tissue, makes the region of bone 26 easily identifiable in the image due to the high opacity associated with the associated voxels. However, since the opacities of blood and soft tissue are more similar, they are not as clearly distinguished. In particular, it is difficult to see the blood vessel network.
c shows a 2D image of the same projection as
d schematically shows a graphical representation of the color and opacity curve mappings used in generating the 2D image shown in
In
e shows a 2D image of the same projection as
The range of displayable colors and opacities to which the voxel signal values are to be mapped (i.e. the entries in the color table) will in general depend on the specific application. For example, in one application images might be represented using five color ranges with the color ranges being black, red, orange, yellow and white. The opacity level from black to white might range from 0% to 100% opacity with the colors mixing in relation to the opacity curve. This would allow for a smooth transition between bands, with the color and opacity values at the upper edge or boundary of one range matching the color and opacity values at the lower edge of boundary of the adjacent range. In this way, the five ranges blend together at their boundaries to form a smoothly varying and continuous spectrum. The colors at the bottom boundary of the first range and the top boundary of the fifth range are black and white respectively.
Each of these five color ranges can be mapped to the voxel signal values which represent different tissue types to distinctly identify the different tissues. For example, bone might be represented in shades of cream, blood in shades of yellow, a kidney in shades of orange and so on, all with varying opacities. The task of attributing color and opacity to different tissue types becomes one of determining suitable signal values, hereafter referred to as visualization thresholds, for defining boundaries between the ranges.
Sub-ranges of signal values between color boundaries may be taken to represent different tissue types. The shades of color available within the color range associated with each tissue type can then be appropriately mapped to the sub-range of signal values. For instance, in one example there are 32 shades of cream available for coloring bone, and bone is associated with signal values between 154 and 217 (in arbitrary units). A color look-up table could then associate the first shade of cream with signal values 154 and 155, the second shade of cream with signal values 156 and 157, the third with 158 and 159 and so on through to associating the thirty-second shade of cream with signal values 216 and 217.
The signal values D of the voxels within the selected VOI are distributed between a minimum value S and a maximum value E. Within this overall range of signal values, four distinct voxel value sub-ranges are evident. A first voxel value sub-range I has a narrow peak at relatively high signal values, a second voxel value sub-range II has a relatively broad peak, a third voxel value sub-range III has a shoulder on the lower signal value side of the second voxel value sub-range II and a fourth voxel value sub-range IV has a shoulder on the lower signal value side of the third voxel value sub-range III.
The different voxel value sub-ranges I-IV identified in the histogram are likely to relate to different tissue types in the 3D data set and so would benefit from being displayed with different color ranges. For example, one might reasonably infer that sub-range I of high X-ray attenuation corresponds to bone, sub-range II corresponds to blood, sub-range III corresponds to soft tissue and sub-range IV represents the background tissue type or air.
It is not necessary when displaying the images to pre-associate voxel value sub-ranges in the histogram with particular tissue types. In fact, if the signal values in the data set are un-calibrated it may not even be possible to do so from the signal values alone. Nonetheless, if distinct voxel value sub-ranges in the histogram can be identified by a numerical analysis, derived images can be shown with different tissue types clearly and consistently displayed without the need for user-driven post-display processing.
In a next step 35, an iteration parameter n is given an initial value of 1. The iteration parameter is indicative of how many visualization thresholds for determining visualization parameters have already being determined, at this stage of the flow chart, a value of n means that n−1 visualization thresholds have previously been found. In a next step 36, an nth convex hull of the histogram is determined. The nth convex hull is defined to be a curve spanning the signal value range L to U and which comprises that series of line segments which combine to form the shortest possible single curve drawn between the histogram values at L and U subject to the condition that at any given signal value within the range, the curve must be greater than or equal to the value of the histogram, and further subject to the condition that the nth convex hull must also meet the histogram profile at all previously identified color boundaries.
a shows the histogram previously shown in
In a next step 37, an nth visualization threshold Tn is found by determining the point on the histogram profile for which the nth convex hull has the maximum nearest distance. This is the point on the histogram from which the longest possible line can be drawn to meet the nth convex hull perpendicularly. This point, which must intersect the nth convex hull on one of its extended straight line portions, can be determined, to a finite accuracy, by a number of known techniques. The longest perpendicular which can be drawn between the first convex hull and the histogram profile shown in
In a next step 38 shown in
In a next step 39, one or more color boundaries are set based on the nth visualization threshold Tn. In this example, a single color boundary threshold is set at the data value matching the visualization threshold Tn. In other examples, depending on clinical application, and/or the requirements of subsequent visualization tools, it may be preferable to associate two color boundaries with a single visualization threshold. For instance, by setting first and second color boundaries at signal values slightly displaced to the lower and higher signal value side of the signal value of an nth visualization threshold, for example at data values Tn+/−3 in arbitrary units, a rapid change in colors and opacities allotted to signal values in the vicinity of the visualization threshold occurs which can help to highlight features of the boundary in a subsequently displayed 2D image.
In a next step 40, a sharpness value is set for each of the one or more color boundaries associated with the nth visualization threshold Tn. The sharpness value may be based on the significance parameter of nth visualization threshold Tn and can be used to assist in displaying images. The sharpness value may, for example, range from 0 to 100 and be used to determine a level of color blending between the colors near to a color boundary. Increased color blending can be set to occur at boundaries with relatively low sharpness values. This ensures that low significance boundaries appear less harsh in a displayed image. Conversely, little or no color blending is applied to boundaries with relatively high sharpness values. This ensures high significance boundaries appear well defined in a displayed image. If multiple color boundaries are set according to a single visualization threshold, it may be convenient to associate a sharpness value based on the significance parameter of the visualization threshold with only a single one of the multiple color boundaries, and to set a fixed sharpness value for the other of the multiple color boundaries. Since the sharpness values are based on the significance parameter, sharpness values can also be taken as a measure of the significance of a visualization threshold and associated boundaries. The significance of a boundary may play a role in further processing as discussed further below.
In a next step 41, a test is performed to determine whether additional color boundaries are required. The iteration parameter n, which at this stage of the flow chart indicates how many visualization thresholds have been determined, is compared with the total number (Ntotal) of visualization thresholds required. Ntotal will depend on the number of displayable color ranges and how many color boundaries have been set for each of the visualization thresholds. For example, in this case, where all visualization thresholds provide a single color boundary, and if there are five displayable color ranges (hence four color boundaries), four visualization thresholds are required and Ntotal=4. However, in another example, a particular application might require a color boundary to be set either side of the first visualization threshold and single color boundaries to be set at each subsequent visualization threshold. In such a case the four color boundaries associated with five displayable color ranges would be set after determining only three visualization thresholds since the first visualization threshold sets two color boundaries, accordingly Ntotal=3.
If it is determined that no further color boundaries are required (i.e. n=Ntotal) the flow chart follows the N-branch from step 41 to step 42 where the preset determination is complete. If further color boundaries are required (i.e. n<Ntotal), the flow chart follows the Y-branch from step 41 to a step 42 where the iteration parameter n is incremented, and then returns to step 36 to continue as described above.
b shows the histogram previously shown in
The longest perpendicular which can be drawn between the second convex hull and the histogram profile shown in
c again shows the histogram previously shown in
As noted above, in the example shown in
If there are P available color ranges for display and the VOI contains P or more than P distinct tissue types, the method outlined above automatically identifies P voxel value sub-ranges from the histogram of the voxels contained in the VOI. For a given type of data, an appropriate value of P (and hence number of visualization thresholds to be identified) can be selected based on the expected characteristics of the data set.
If, on the other hand, the VOI contains fewer than P distinct tissue types, the allotment of P color ranges will cause more than one color range to be allotted to at least one of the tissue types. A color boundary which is placed within a signal value range representing a single tissue type may appear confusing in the display, especially if the user is unaware of it. As noted above, color blending based on a sharpness value derived from the significance parameter for each visualization threshold can be used to de-emphasize such boundaries.
In addition to setting the sharpness value, the significance parameter may also form the basis of a significance test for determining the significance of a visualization threshold. The significance parameter may, for example, derive from the length of the determined longest perpendicular between the nth convex hull and the histogram. The significance test may require that this longest perpendicular is at least a pre-defined fraction of the histogram's characteristic dimensions. For instance, the significance test may require that the longest perpendicular be at least 5% of the geometric height or width of the histogram. In another example, the significance test may require that the longest perpendicular be at least 10% of the value of the appropriately normalized height and width of the histogram added in quadrature. The height and width may be differently normalized to provide different weighting. Whilst a default fraction, such as 5% or 10%, may be used, the fraction may also be changed to better suit a particular application and expected histogram characteristics. A determined visualization threshold which lies within a signal value range representing a single tissue type will fail an appropriately configured significance test. Boundaries associated with these visualization thresholds will be noted as cosmetic boundaries and will be ignored by selection tools.
d shows the histogram previously shown in
Where a visualization threshold is deemed not to be significant, for example with reference to a model of a histogram's expected characteristics or by comparison with a pre-determined minimum length of longest perpendicular, it is can either be noted as such to provide a cosmetic color boundary in the color table (and given an appropriate sharpness value of, for example, 0), or it can be discarded. The concept of a cosmetic boundary is a boundary which is defined in the color table and thus relevant for display purposes, but which is ignored by other tools in the graphics system which are boundary sensitive, such as tools used for selecting objects that contain algorithms that automatically search for and mark boundaries. One or more cosmetic boundaries may be determined iteratively in the manner described above until sufficient number are determined to satisfy the requirements of the number of available displayable color ranges (i.e. j−1 color boundaries for j displayable color ranges).
If color boundaries associated with visualization thresholds which fail the significance test are to be discarded, rather than kept but marked as cosmetic, the iterative search for further visualization thresholds may cease after the first significance-test-failing visualization threshold is identified. In these circumstances there will be fewer identified voxel value sub-ranges in the histogram (which nominally correspond to fewer distinct tissue types within the VOI) than there are displayable color ranges. Individual color ranges may be allotted to the individual signal value ranges defined by the identified color boundaries and the remaining color ranges unused, or cosmetic boundaries can be artificially defined. For example, a cosmetic boundary could be generated by defining a color boundary mid-way between the two signal values of the most widely separated significant color boundaries. If multiple cosmetic boundaries are to be defined, the signal values with which to associate them can be determined serially, i.e. one after another using the above criterion, or in parallel such that they collectively divide the widest signal value range in to equal sections.
The significance of a given color boundary may be a continuous parameter and need not be a purely binary—e.g. a color boundary need not simply be significant or non-significant (i.e. non-cosmetic or cosmetic). As noted above, sharpness values derived form a significance parameter may be used as a direct measure of a boundary's significance such that in many cases the sharpness value itself will be used to directly indicate a boundary's significance. Accordingly, significance may be given an insignificant level (e.g. 0) and one or more levels of significance (e.g. integers between 1 and 100). This facility may be used by other graphics tools within the image processing system, for example to determine the probability of a boundary being significant when attempting to resolve conflicts when determining the bounds of a topological entity. A zero level of sharpness (i.e. significance) is set for boundaries which fail the significance test.
As indicated above, the visualization threshold's significance could be based on the length of the appropriately normalized length of the perpendicular between the nth convex hull and the histogram. By indicating to a user the significance of the identified color boundaries used in the display, the interpretation of the displayed image can be aided. The significance of each of the color boundaries may also play a role in the appropriate use of connectivity algorithms used to define surfaces or volumes within the 3D data set which are associated with features identified by a user in the projected image.
The image data could take a variety of forms, but the technique is particularly well suited to embodiments in which the image data comprises a collection of 2D images resulting from CT scanning, MRI scanning, ultrasound scanning or PET that are combined to synthesize a 3D object using known techniques. The aided visualization of distinct features within such images can be of significant benefit in the interpretation of those images when they are subsequently projected into 2D representations along arbitrarily selected directions that allow a user to view the synthesized 3D object from any particular angle they choose.
Having described the principles of a method for automatically determining presets, specific examples of its application will now be given. In some applications, presets may be required to satisfy further conditions before they are accepted for defining color range boundaries (or other visualization parameters). For example, in applications where bone visualization within a CT data set is of prime interest, preset thresholds should not be defined between those signal values representing soft tissue and blood vessels since both of these should be made transparent in the displayed image. Instead, less significant but more appropriate thresholds within the range of signal values representing bone may be preferred.
Magnetic resonance imaging (MR) data sets are generally un-calibrated and display a wide range of data-values, dependent on, for example, acquisition parameter values or the position of the VOI with respect to a scanner's detector coils during scanning. Accordingly, it is not usually possible to pre-estimate suitable signal values with which to attribute color range presets and this makes the present invention especially useful for application to MR data sets.
a schematically shows the appearance of a visualization state tool displayed on the display 144 shown in
The data display window shows a logarithmically scaled histogram of the signal values for the entire 3D data set overlaid with dashed and solid vertical lines marking the cosmetic and significant boundary positions respectively. The boundary lines may also be marked differently, for example based upon their relative significance. The histogram is colored to represent the color table and opacity mapping at each signal value indicated by the scale along the top of the data display window. The frequency distribution shown here is that of the entire image data set and is not restricted to the VOI on which the preset determination is based. In some cases it may aid a user's interpretation if the histogram of the selected VOI is shown. This might be in place of the histogram of the entire data set or indicated separately, such as in a separate window or drawn as a curve overlaying the entire image data set histogram. A curve is plotted overlaying the frequency distribution which shows the opacity curve. The opacity curve is formed by interpolation between the opacity values set for each of the color boundary positions taking into account the sharpness at each boundary, in this example 0, 0, 73, 90 and 100 for voxel values 11, 19, 102, 158 and the maximum voxel value respectively. The display modification buttons 82 allow a user to pan along the histogram shown in the display window and also allows for repositioning color boundaries if required.
The color boundaries shown in the display of boundary positions 78 in
When applied to an MR data set, the active preset determination method of this example first tries to find a candidate threshold, using the technique described above, and which further satisfies the condition that 60% (+/−30%) of the volume is transparent.
In this example, a suitable threshold is found to be at signal value 15. However, the design of the “Color/Opacity Settings” interpolation used by the particular display software used in this example operates best if two boundaries are placed a small distance on either side of the computed background threshold in order to provide a rapid rise in the opacity curve as usually desired. In this example, these two boundaries are placed at positions ±4 signal value units either side of the visualization threshold at signal values of 11 and 19 respectively. The boundaries may also be placed at other positions, for example, at positions ±3 signal value units either side of the visualization threshold.
Since in this example there are five color ranges to allot (requiring four color boundaries), the method looks for up to two more candidate visualization thresholds above the background level, at which to place the two remaining color boundaries. If two significant visualization thresholds cannot be found, then the missing color boundaries are placed in the center of the largest gap, but, as noted above, the associated sharpness (i.e. significance) is set to 0, indicating a cosmetic boundary with no significance to selection.
In this example, a second visualization threshold at signal value position 102 is determined to be significant and a single color boundary with a sharpness set to 5 is defined at signal value position 102. No further significant visualization thresholds are found and the remaining color boundary, between yellow and white, is placed at signal value position 158. It should be noted that, whilst not immediately apparent from the histogram shown in the data display window 80, this cosmetic boundary is in the middle of the widest gap between significant color boundaries. There are two reasons why it is perhaps not immediately apparent. Firstly, the histogram upon which the analysis is made differs from the histogram shown in the data display window since the former is restricted to the VOI and un-sculpted domain whereas the latter represents the entire 3D data set. Secondly, as noted above, in order to prevent the analysis from being thrown off by spurious values in the tails of the distribution, the lowest and highest 0.1% of voxels are excluded from the numerical analysis. Because the histogram shown in the dialog has a logarithmic vertical scale the voxels at the extremes of the voxel value range can appear more significant.
b shows an example image displayed according to the automatic preset of
In this example, the same preset type is used as in the first example. This may be useful for CT data sets in which a user wants to visualize soft tissue.
This example is for use on CT data sets for the purpose of visualizing and selecting bone.
a schematically shows the appearance of a visualization state tool presenting an example of use of the “Active Bone (CT)” Preset. The different fields within the visualization state tool shown in
The “Active Bone (CT) Preset” operates by determining a first significant visualization threshold within the signal value range 70 HU to 270 HU. In the example, a visualization threshold value of 182 HU is determined. If no such visualization threshold is found then 170 HU is used. This first visualization threshold is used to set the background level in the display software by setting two boundary positions at ±45 HU from the first visualization threshold. The boundary positions in this example are accordingly at 137 HU and 227 HU. With the five available ranges of color indicated in
b shows an example image displayed according to the automatic preset of
This preset assumes the data are in correctly calibrated Hounsfield units. The purpose of this preset is to visualize angio-tissue.
a schematically shows the appearance of a visualization state tool presenting an example of use of the “Active Angio (CT)” Preset. The different fields within the visualization state tool shown in
The yellow/white boundary is placed at a position determined by the histogram analysis within the range 550 HU±200 HU, and this boundary is given a sharpness of 5 so that is significant to selection. In the example a boundary position of 550 HU is determined. Thus, in good data sets, the selection tools can be used in conjunction with this preset to discriminate bone from contrast enhanced vasculature.
The other boundary positions are fixed at values −500 HU, 105 HU and 195 HU.
b shows an example image displayed according to the automatic preset of
Summary
It should be clear from the above examples that the preset determination method of the present invention can be specifically tailored in any number of ways to apply to data sets with known specific characteristics. The method can also be used as an entirely general tool with no prior knowledge of the data set, such as in the MR example described above. In addition, users may themselves customize the preset determination to suit the requirements of a particular study. This might be done, for example, where CT calibrated data are used and a user requires features with a particular X-ray attenuation to be identified. This might also be done to distinguish between two tissue types of similar X-ray attenuations. Similarly, with un-calibrated data, a user might modify the preset determination method based on the appearance of a single 2D projection so that the preset is applied consistently to all 2D projections generated from that voxel data set. By storing parameters associated with a user's personal customizations to the method (such as the significance test stringency, the typical fraction of the data set the user wants to appear as transparent, or specific signal value ranges in which thresholds should occur), the modified method could be consistently applied to further data sets.
While by way of example the foregoing description is directed mainly towards determining visualization thresholds which are used to define color boundaries, it will be appreciated that determined visualization thresholds are equally suitable for defining boundaries for visualization parameters other than color, such as opacity, that are relevant for rendering. Furthermore, it is often clinically useful for the placement of boundaries in an opacity mapping to be positioned at the same signal values as the boundaries in a color mapping. Other visualization parameters for which the invention could be used include rate of change of color with signal value, rate of change of opacity with signal value, and segmentation information.
The preset determination is also not limited to finding any particular number of boundaries, and the associated number of visualization thresholds, but is extendable to determining any number of boundaries or visualization thresholds. In some applications it may be appropriate to determine more visualization thresholds than there are distinct boundaries required to allow less significant thresholds to play a role in defining the specific allocation of available color shades or transitions between colors within one or more of the determined ranges, for example.
Although in the above examples, the visualization parameter boundaries are determined automatically, in other cases, some level of user input can assist in determining the most appropriate conditions for displaying an image. This is because once an automatic preset has been determined it may be desirable to make an assumption regarding what aspects of the data a user is interested in seeing in a displayed image. For example, in the histogram of CT data shown in
In addition to employing user supplied examples of tissue types which are and which are not of interest to assist in displaying images in conjunction with the above described automatic preset determination, such techniques can also be applied more generally to classify different tissue types in medical image volume data.
The technique can be particularly useful where different tissue types appear very similar in the data, for example because they have similar X-ray stopping powers for CT data. In the histogram shown in
A user viewing the display shown in
In this example the method is performed by a suitably programmed general purpose computer, such as that shown in
In Step 111 of
It will be appreciated that other schemes for allowing a user to identify voxels can also be used. For example, rather than “click” on an individual pixel in one of the section views, a range of pixels could be identified by a user “clicking” twice to identify opposite corners of a rectangle, or a centre and circumference point of circle, or by defining a shape in some other way. Voxels corresponding to pixels within the perimeter of the shape may then all be deemed to have been identified.
In Step 112, input is received from the user which identifies (selects) voxels not corresponding to the tissue type of interest. Step 112 may be performed in a manner which is similar to Step 111 described above, but in which the user positions the cursor over pixels in the sagittal, coronal and/or transverse sections which do not correspond to the tissue type of interest. The user may indicate his selection by “clicking” a different mouse button to that used to identify the positively selected voxels. Alternatively, the same mouse button might be used in parallel with the pressing of a key on the keyboard 148.
To allow subtly different tissue types to be distinguished, the user should identify voxels which are most similar to the tissue type of interest, but which he wants to exclude nonetheless. This is because voxels which differ more significantly from voxels corresponding to the tissue type of interest are easier to classify as not being of interest. In this case, where the tissue type of interest is an anomalous region of brain which appears slightly different from its surroundings in the sagittal section view 102, the user should identify voxels by selecting pixels in the area surrounding the anomalous region. However, if there are other regions which also appear similar to the tissue type of interest, but which are not necessarily in close proximity to it, the user may also identify some voxels corresponding to these regions. For example, five or so voxels not corresponding to the tissue type of interest might be identified. However, as few as one or many more than five may also be chosen. For example, if there are a number of regions in the data appearing only slightly different from the tissue type of interest, the user may choose to identify a number of voxels in each of these regions. The voxels identified in Step 112 will be referred to as negatively selected voxels, and the process of identifying them will be referred to as making a negative selection.
In Step 113 one or more characterizing parameters are computed for each of the voxels selected in Steps 111 and 112. In this example implementation four characterizing parameters, namely voxel value V, a local average A, a local standard deviation σ and maximum Sobel edge filter response S over all orientations, are determined for each voxel. In another embodiment, instead of maximum Sobel edge filter response, gradient magnitude is used. In this case the local average and standard deviation are computed for a 5×5×5 cube of voxels centered on the particular voxel at hand. However, other regions may also be used. For example, a smaller regions may be considered for faster performance. Furthermore, the regions need not be three-dimensional, a 5×5 square of voxels, or other region, in an arbitrarily chosen or pre-determined plane may equally be used.
In Step 114 the distribution of computed characterizing parameters are analyzed to determine which of them may be used to distinguish between the positively selected and the negatively selected voxels.
It can be seen from
It can be seen from
However, it can be seen from
It can be seen from
From these distributions of the computed characterizing parameters for the positively and negatively selected voxels, it is apparent that local standard deviation σ is a characterizing parameter which distinguishes well between positively and negatively selected voxels, and as such is considered to be a distinguishing parameter. In this example implementation only one distinguishing parameter is sought and is chosen on the basis of it being the most able of the computed characterizing parameters to discriminate between the positively and negatively selected voxels. The ability of a given characterizing parameter to discriminate is referred to as its discrimination power and may be parameterized using conventional statistical analysis. In this example, this is done by separately calculating the average and the standard deviation of each characterizing parameter for the positively and the negatively selected voxels. The discriminating power of a given characterizing parameter is then taken to be the difference in the average for the positively and negatively selected voxels divided by the quadrature sum of their standard deviations. The charactering parameter having the greatest discriminating power is then taken to be the distinguishing parameter. As will be seen further below, in other examples multiple distinguishing parameters may be used, for example all characterizing parameters having a discriminating power greater than a certain level or a fixed number of characterizing parameters having the highest discriminating powers may be used.
In Step 115, the distinguishing parameter (i.e. local standard deviation σ in this case) is calculated for other voxels in the data. Although this may be done for all of the voxels, it may be more efficient to restrict the calculation to only a subset of voxels. For example, a conventional segmentation algorithm may first be applied to the data to identify which voxels belong to significantly different tissue types (e.g. bone or brain). Once this is done, the local standard deviation σ may then be calculated only for those voxels which have been classified by the conventional segmentation algorithm as corresponding to brain. This is because there would be no need to perform the computation for voxels which have already been distinguished from the tissue type of interest by the conventional segmentation algorithm. Alternatively, the calculation may only be made for voxels in a VOI identified by the user.
In step 116, the distinguishing parameter, i.e. the local standard deviation for the example characterizing parameter distributions seen in FIGS. 12A-D, is used to classify each of the other voxels. This is performed in this example by defining a critical local standard deviation σc (marked in
It will be appreciated that although in this particular example the computed value of one of the characterizing parameters (local standard deviation) is itself identified as being able to distinguish between the tissue type of interest and surrounding tissue, this is a special example of the more general case in which a distinguishing functional relationship between characterizing parameters is identified. For example, for a particular tissue type interest it might be found the ratio of two different characterizing parameters has a greater discriminating power between positively and negatively selected voxels than either of the characterizing parameters themselves. A numerical example of how this can arise is if values generally between 2.5 and 3.5 (arbitrary unit) are found for one characterizing parameter for both positively and negatively selected voxels and values generally between 5 and 7 (arbitrary units) are found for another characterizing parameter, again for both positively and negatively selected voxels. Because of this, neither characterizing parameter alone is able to discriminate properly between positive and negatively selected voxels. However, if for the tissue type of interest the second characterizing parameter is always close to twice the value of the first, whereas for the negatively selected voxels, the two parameters are unrelated, a distinguishing function based on the ratio of the two parameters can be identified.
Depending on clinical application, additional requirements may be imposed on which voxels are to be considered to correspond to the tissue type of interest. For example, a requirement that the tissue type of interest forms a single volume may be made by applying a connectivity requirement. This would mean voxels which are not linked to the positively selected voxels by a chain of voxels classified as corresponding to the tissue type of interest will be classified as not corresponding to this tissue type, even if their distinguishing parameters are such that they would otherwise be considered to do so.
Once the voxels have been classified, the user may proceed to examine those corresponding to the tissue type of interest as desired. For example, the user may render an image showing only the tissue type of interest. In another example, the tissue type of interest may be shown in one color and other tissue types in other colors, that is to say the method shown in
In some examples, an image based on the distinguishing parameter itself (or a function thereof) may be rendered (e.g. using the distinguishing parameter as the imaged parameter in the rendering rather than voxel value). In the above described situation, rather than rendering an image based on voxel values in the image data set (i.e. X-ray stopping power for CT data), an image based on the local standard deviation for each of the voxels may be rendered instead. Ranges of color and/or opacity may be associated with different values of local standard deviation and an image rendered accordingly. Visualization presets for the rendered image may be calculated as previously described, for example. This approach can provide for a displayed image in which a user can easily distinguish the tissue type of interest from surrounding tissue because characteristics of the tissue type of interest which differentiates it from its surroundings are used as the basis for rendering the image.
Rather than display an image, the classification may be used in conjunction with conventional analysis techniques, for example to calculate the volume of the anomalous region corresponding to the tissue type of interest. It will of course be appreciated that in some cases a region of interest might be of interest merely because the user wishes to identify it so it can be discarded from subsequent display or analysis.
It is not necessary for the steps shown in
Furthermore, the process may return to earlier steps during execution. For example, a user may be alerted at Step 114 if there are no characterizing parameters having a discriminating power above a predetermined level. In response to this, the user may choose to return to Step 111 and/or Step 112 to provide more examples. Alternatively, in such a circumstance the user may instead indicate that additional characterizing parameters should be determined and their discriminating powers examined, or may simply choose to proceed with the classification nonetheless.
The method shown in
Furthermore, more than one distinguishing parameter may be used for the classification. For example, if multiple parameters are identified in Step 114 as being capable of distinguishing between the positively and negatively selected voxels, these multiple distinguishing parameters may each then be computed for the other voxels in Step 115. The classification in Step 116 could then be based on a conventional multi-dimensional expectation maximization (EM) algorithm or other cluster recognition process which takes the distinguishing parameters computed for the positively and negatively selected voxels as seeds for defining for the populations of voxels (i.e. the population of voxels corresponding to the tissue type of interest and the population of voxels not corresponding to the tissue type of interest). Example classification schemes when the distinguishing function has two or more characterizing parameters are multivariant Gaussian maximum likelihood and k-nn (nearest neighbors).
The EM algorithm provides the distributions for the positive and negative cases which then allows, for each voxel, a probability to be determined that the voxel is a member of the population exemplified by the positively selected voxels, that is to say a probability that the voxel corresponds to the tissue type of interest. The EM algorithm may also provide an estimate of the overall fraction of voxels which are members of the population exemplified by the positively selected voxels. This information allows an image of the tissue type of interest to be rendered from the volume data in a number of ways.
One way is to render all voxels having a probability of corresponding to the tissue type of interest lower than a threshold level as transparent, and render the remaining voxels using conventional techniques based on their voxel values (e.g. opacity to X-rays for CT data). The threshold level may be selected arbitrarily, for example at 50%, or may be selected such that the total number of voxels falling above the threshold level corresponds to the overall fraction of voxels which are members of the population exemplified by the positively selected voxels predicted by the EM algorithm.
Another way of generating an image showing the tissue type of interest would be to again render all voxels having a probability of corresponding to the tissue type of interest lower than a threshold level as transparent, but to then render the remaining voxels based on their probability of corresponding to the tissue type of interest, rather than their voxel values. This provides a form of probability image from which a user can immediately identify the likelihood of individual areas being correctly classified as corresponding to the tissue type of interest.
In either case, where an image based on rendering of probabilities is displayed, the user may be presented with the opportunity of manually altering the threshold level. This allows the user to determine an appropriate compromise between including too many false negatives (i.e. voxels which do not correspond to the tissue type of interest) and excluding too many true positives (i.e. voxels which do correspond to the tissue type of interest).
It will be appreciated that in addition to the example characterizing parameters shown in
While the above description relates to a situation where a user is interested in further examining only a single tissue type, it will be understood that the method may equally be employed where a user wishes to identify multiple tissue types. This can be achieved by a user making positive selections for each of the different tissue types of interest in Step 111 shown in
The method may also be applied in an iterative manner. For example, following execution of the method shown in
In some implementations of the method, the computation of the distinguishing features may include additional analysis techniques to assist in the proper classification of voxels. For example, partial volume effects might cause a boundary between two types of tissue which are not of interest to be wrongly classified. If this is a concern in a particular situation, techniques such a partial volume filtering as described in WO 02/084594 [1] may be employed when computing the distinguishing parameters.
In cases where a user considers that the classification has not performed adequately, for example should one of the negatively selected voxels be attributed a high probability of being a member of the population exemplified by the positively selected voxels, further segmentation analysis techniques may be applied. For example conventional morphological segmentation algorithms may be applied to volume data representing the probability of each voxel comprising the volume data of corresponding to the tissue type of interest.
Additional user input may also be used to assist the classification process. In particular, the user input may additionally include clinical information, such as specification of tissue type or anatomical feature of interest. For example, the user input may adopt the paradigm “want that gray matter—don't want that white matter”, or “want that liver—don't want that other (unspecified) tissue”, or “want that (liver) tumor—don't want that healthy (liver) tissue”, or “want that (unspecified) tissue—don't want that fat tissue”. This user input can be done by appropriate pointer selection in combination with filling out a text label or selection from a drop down menu of options. Following this user input, the distinguishing function can then determined from the characterizing parameters having regard to the clinical information input by the user. For example, if the positively selected voxels are indicated as belonging to a tumor, local standard deviation may be preferentially selected as the distinguishing function, since this will be sensitive to the enhanced granularity that is an attribute of tumors.
In some clinical studies multiple volume data sets of a single patient may be available, for example from different imaging modalities or from the same imaging modality but taken at different times. If the images can be appropriately registered with one another, it is possible to classify voxels in one of these volume data sets on the basis of positively and negatively selected voxels in another. Distinguishing parameters may even be based on an analysis of voxels in one data set yet be used to classify voxels in another data set. This can help because with more information made available, it is more likely that a good distinguishing parameter can be found.
It will be appreciated that although particular embodiments of the invention have been described, many modifications/additions and/or substitutions may be made within the spirit and scope of the present invention.
Thus, for example, although the described embodiments employ a computer program operating on a general purpose computer, for example a conventional computer workstation, in other embodiments special purpose hardware could be used. For example, at least some of the functionality could be effected using special purpose circuits, for example a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC) or in the form of a graphics processing unit (GPU). Also, multi-thread processing or parallel computing hardware could be used for at least some of the processing. For example, different threads or processing stages could be used to calculate respective characterizing parameters.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 10212363 | Aug 2002 | US |
| Child | 10726280 | Dec 2003 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 10726280 | Dec 2003 | US |
| Child | 10922700 | Aug 2004 | US |
