Patent Application
20080246770
The present invention relates generally to a method for generating a 2-D image of a 3-D object using a modified direct volume rendering technique.
Visualization of the structure of 3-D objects on a 2-D image is an important topic in the field of computer graphics and finds application in many industries for example the medical industry. A CT scanner produces thousands of parallel 2-D image slices of a patient's body, each slice including a 2-D array of data values, each value representing an attribute of the body at a given location, for example, density. All the slices are stacked together to form an image volume or a volumetric dataset of the patient's body. The display on the monitor of a medical workstation of interactive 2-D images showing 3-D characteristics of a body provides a powerful diagnosis tool.
Two important rendering methods for visualizing 3D data sets are Direct Volume Rendering (DVR) and Iso-Surface Rendering (ISR). In state-of-the-art medical workstations, two distinct algorithms for both methods are implemented, which differ considerably.
In DVR, the scalar attribute or voxel at any point within the image volume is associated with a plurality of optical properties, such as colour or opacity, defined by a user specified transfer function and stored as a grey scale. The 2-D image plane consists of a regularly spaced grid of picture elements or pixels, each pixel having red, green, and blue colour components. A plurality of notional rays are cast through the volume and they are attenuated or reflected by the volume. The amount of attenuated or reflected ray energy of each ray is indicative of the 3-D characteristics of objects embedded within the image volume, and determines a pixel value on the 2-D image plane in accordance with the opacity and colour mapping of the volume along the corresponding ray path.
The volume data is typically sampled at equidistant locations along each ray path. The pixel values associated with the plurality of ray origins on the 2-D image plane form an image that can be rendered on a computer monitor. In principle, every sample point along a ray path may contribute to the final pixel colour, and the sample points are usually processed in the order they are located along the ray.
In iso-surface rendering, the surface defined by a given threshold value is determined and visualized. Iso-surface algorithms based on ray casting perform an efficient search for the surface and determine the exact surface point inside a voxel at a high accuracy. This can be done, for example, by a binary search or by using Newton iteration as explained in detail in “M. K. Bosma: “Iso-Surface Volume Rendering: Speed and Accuracy for Medical Application, Ph.D. thesis Twente University, PrintPartners Ipskamp, Enschede, 2000”. When using binary search, n computational steps are sufficient to locate the surface along the viewing ray at an accuracy of 2−n voxels. Such an iterative approach is possible as only one point along each viewing ray contributes to the output image, and only this point has to be determined. The order in which the sampled points are visited are not necessarily in the order they are located along the ray.
Theoretically, DVR algorithms may also be used to display iso-surfaces by selecting an appropriate classification function assigning an opacity of 0 or 1 to grey values below or above the threshold, respectively. However, compared to iterative iso-surface rendering this leads to a visible loss in image quality. DVR is useful to display data sets where more than a single iso-surface is relevant, for example, to display multiple iso-surfaces, noisy data in a smooth way or regions that cannot be visualized by an iso-surface at all. However, achieving the same image quality as obtained by iso-surface rendering would require decreasing the step distance to 2−n, where n may be 10, which is computationally infeasible.
Embodiments of the present invention aim to alleviate the above-mentioned problems.
Embodiments of the present invention aim to combine the advantages of iso-surface rendering (high image quality) with the flexibility of direct volume rendering. On the one hand, the embodiments of the present invention aim to allow efficient generation high-quality iso-surface images. On the other hand, the full flexibility of direct volume rendering is maintained, and in the same image semi-transparent and differently coloured objects can be displayed.
In consequence, only one visualization tool is required in a medical workstation. This may improve both the maintainability of the software and the usability of the workstation: The user interface can be simplified such that the user does not have to choose between different rendering modes, or—if the mode switch is done automatically—he does not observe potentially distracting changes in the image appearance.
According to the present invention there is provided a method for generating a 2-D image of a 3-D object represented by a volume data set comprising a multiplicity of data points each having an object parameter value, the method comprising: casting a plurality of notional rays through the 3-D object and for each ray, dividing a ray path into a plurality of base sampling intervals defined by data points on the path and if it is determined that a difference in parameter values across a base sampling interval is greater than a pre-determined value, generating within the base sampling interval smaller sampling intervals between data points until it is determined that a difference in parameter values across each generated smaller sampling interval in the base interval is less than the pre-determined threshold and wherein values indicative of an interaction between the ray and the 3-D object in the sampling intervals along the path are accumulated to determine a pixel value in the 2-D image.
An embodiment of the invention will now be described by way of example only with reference to the accompanying drawing in which:
Referring first to
The processing arrangement 101 may be a suitably programmed processor of the workstation 100, or a special purpose processor having circuit means such as filters, logic operators and memories, that are arranged to perform the functions of the method steps according to the invention. The system 100 may use a computer program product having program instructions to be executed by the processing arrangement 101 in order to carry out the method steps. The data processing system, display and/or storage means may be located remotely from the data acquisition means of the system.
Different rays correspond to different pixels on a 2-D image screen, for example, the display 102 of the workstation 100. Each pixel value is determined by the interaction between at least one ray and the image volume 112.
To calculate an individual pixel value for each sample point along a ray, the following steps are performed:
a) A sample grey value is taken from the volume data.
b) Depending on the sample grey value, optical parameters (e.g. attenuation, color) are determined for that sample. The transfer function (sometimes also called classification) is the function that maps grey values to optical values.
c) Depending on these optical values, the color of the pixel is updated using a “ShadeAndAccumulate” procedure as is known in the art. This happens in two steps: First, an effective color for the current sample is determined. The color returned by the transfer function can be seen as a material color. This color is modified according to a light model, typically a Phong-Blinn model, depending on the lighting and the surface normal at the sample point. Secondly, the visibility of the sample according to the attenuation of the ray accumulated so far is determined and the shaded sample color is added to the final pixel color with that weight.
Details and further information on light models for volume rendering can be found in, Max Nelson: “Optical Models for Direct Volume Rendering”, IEEE Transactions on Visualization and Computer Graphics”, 1995, pp. 99-108
The direct volume rendering method of the present invention applies the following recursive procedure to each base interval along a ray's path.
The procedure estimates for each base interval, whether or not the difference between any two values of the relevant property of the volume data set, for example its opacity, between the two sample points can exceed a pre-determined threshold.
If, according to estimation, the difference in opacity values does not exceed the threshold, the base interval is processed as per normal by a ‘ShadeAndAccumulate’ procedure that determines the color contribution of the ray interval [P0,P1] and adds it to the current pixel color as per a standard DVR algorithm.
If, according to estimation, the difference in opacity values does exceed the threshold, the base interval is divided into two smaller new intervals [P0,H] and [H,P1], both preferably of equal size, and both new intervals are recursively processed by the same procedure using the same criteria. That is to say, each new interval is divided into smaller intervals until it is determined that the difference in opacity values across an interval does not exceed the pre-determined threshold or the geometrical extent of an interval is fallen below another threshold value. Once this has been determined for an interval, the ‘ShadeAndAccumulate’ procedure is applied to the interval.
The estimation function ‘OpacityDifference’ for the expected opacity difference can, for example, be implemented by evaluating the classification function at the two points defining an interval, P0 and P1, if the interval is a base interval. More generally, the maximum change can be determined along the opacity function for the complete interval defined by the two grey values sampled at the two points defining an interval, again P0 and P1 if the interval is a base interval. It should be noted that the estimation function may also depend on other classification parameters than the opacity, e.g. the color or other optical parameters that may also change quickly along the ray.
The requirement that a recursion can only happen if the distance between sample points is greater than a pre-determine threshold prevents an endless recursion loop.
A further example is described with respect to
The opacity in a first point (P1) is low whilst that at a second point (P2) is high. Hence, the difference in opacity values at the first point (P1) and the second point (P2) exceeds the threshold value, and so the interval defined by first point (P1) and the second point (P2) is divided into two new intervals defined respectively by the first point (P1) and a third point P3 and by the third point P3 and the second point P2.
The new interval defined by the first point P1 and the third point P3 has a constant opacity of 0. Hence, no further division into new intervals is necessary, and this interval is processed in a single ‘ShadeAndAccumulate’ step. Preferably, a properly implemented algorithm detects that this interval is fully transparent, so that no expensive shading operation is performed here.
The difference in opacity values at the third point (P3) and the second point (P2) still exceeds the threshold value, and so the new interval defined by these two points is divided into two new intervals defined respectively by the third point (P3) and a fourth point P4 and by the fourth point P4 and the second point P2.
The new interval defined by the fourth point P4 and the second point P2 has a constant opacity of 1. Hence, no further division into new intervals is necessary.
The difference in opacity values at the third point (P3) and the fourth point (P4) still exceeds the threshold value, and so the new interval defined by these two points, is divided into two new intervals defined respectively by the third point (P3) and a fifth point P5 and by the fifth point P5 and the fourth point P4.
The new interval defined by the fifth point P5 and the fourth point P4 has a constant opacity of 1 and no further subdivision is necessary.
The difference in opacity values at the third point (P3) and the fifth point (P5) is still high. However, the distance between P3 and P5 is less than the threshold value for the sampling distance and so the interval defined by these points is processed in a single non-trivial ‘ShadeAndAccumulate’ step.
Now, the algorithm can stop processing as the accumulated opacity along the ray is 1 (=fully opaque), and additional sample points would not have any impact on the result. Hence, the intervals [P5, P4] and [P4, P2] do not need to be processed further.
Altogether, exactly one non-trivial ‘ShadeAndAccumulate’ step has been performed, and the algorithm has effectively performed a binary search for the surface. Hence, the image quality is the same as for an iso-surface algorithm, and the computational complexity is also in the same order.
A more general example is shown in
The opacity difference between the third point P3 and the second point P2, both in the opaque region 401c is zero, and so this interval is processed in a single ‘ShadeAndAccumulate’ step.
The opacity difference between the third point P3 and the first point P exceeds the threshold, and so this interval is divided into two new regions defined respectively by the first point P1 and a fourth point P4 in the transparent region 401a, and the fourth point P4 and the third point P3.
The opacity difference between the first point P1 and the fourth point P4, both in the transparent region 401a is zero, and so this interval is processed a single ‘ShadeAndAccumulate’ step.
The opacity difference between the fourth point P4 and the third point P3 exceeds the threshold, and so this interval is divided into two new regions defined respectively by the fourth point P4 and a fifth point P5 in the semi-transparent region 401b, and the fifth point P5 and the third point P3.
Finally, the opacity difference between the fourth point P4 and the fifth point P5, and the opacity difference between the fifth point P5 and third point P3 are each less than the threshold so that the recursion stops. Both these intervals are separately processed with ‘ShadeAndAccumulate’ steps. Altogether, the sampling distance has adapted to the opacity changes along the ray.
Methods embodying the present invention allow the generation of images that can neither be computed by iso-surface rendering nor by standard direct volume rendering. Examples are illustrated in
In
The segment 600a describes an arbitrarily small grey level interval related to a very thin opaque area at the skin of the hand phantom. Using an embodiment of the present invention, the skin in
Using a standard DVR algorithm it is a matter of chance whether a sample point is located inside the skin area or not, resulting in a fractured surface in
It will be appreciated that embodiments of the invention may be used in numerous other imaging modalities as well as medical applications, which require interactive 3D data visualization at high image quality for the structures-of-interest with different rendering modes.
Having thus described the present invention by reference to a preferred embodiment it is to be well understood that the embodiment in question is exemplary only and that modifications and variations such as will occur to those possessed of appropriate knowledge and skills may be made without departure from the spirit and scope of the invention as set forth in the appended claims and equivalents thereof. In the claims, any reference signs placed in parentheses shall not be construed as limiting the claims. The word “comprising” and “comprises”, and the like, does not exclude the presence of elements or steps other than those listed in any claim or the specification as a whole. The singular reference of an element does not exclude the plural reference of such elements.
| Number | Date | Country | Kind |
|---|---|---|---|
| 05300698.7 | Aug 2005 | EP | regional |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/IB2006/052911 | 8/23/2006 | WO | 00 | 2/25/2008 |
