1. Field of Invention
This invention relates to the field of object illumination rendering.
2. Background of the Invention
The realism of rendered graphics of an object often depends on the accuracy of the illumination effects associated with the object that is being modeled. These illumination effects typically require intensive processing, especially for real-time graphics rendering. Many of the available rendering methods, however, do not effectively account for illumination effects. For a non-limiting example, some low-order methods for calculating the illumination effects only use a limited number of lighting coefficients and thus are limited to low-frequency lighting and produce only soft shadows for the rendered object.
Recent computer graphics research has explored the use of spherical harmonics (SH) for global illumination to generate realistic real-time rendering of 3D scenes of an object, taking into account subtle effects through different types of diffuse lighting models, which include but are not limited to, un-shadowed, shadowed, and inter-reflected. The goal is to render the global illumination of the object lit by any type of light sources. The lighting environment and the local visibility of the object can be independently pre-computed as SH coefficients thus permitting near-instantaneous relighting of objects. However, the pre-computation of the SH coefficients is often quite expensive in terms of computing resources consumed.
A scattered spherical harmonic (SH) approximation method is proposed for pre-computation of SH coefficients at sampling vertices of an object to be rendered. The approach sparsely populates the object with a plurality of SH sampling vertices and uses the least squares approximation to calculate the SH coefficients at any (other) arbitrary point of the object by extrapolating from the computed SH coefficients at the SH sampling vertices.
a)-(b) illustrate exemplary renderings of a cloud using dense and scattered SH sampling approximation, respectively.
The invention is illustrated by way of example and not by way of limitation in the figures of the accompanying drawings in which like references indicate similar elements. It should be noted that references to “an” or “one” or “some” embodiment(s) in this disclosure are not necessarily to the same embodiment, and such references mean at least one.
Spherical Harmonics and its Coefficients
Spherical harmonics (SH) are basis functions defined over the surface of a unit sphere that can be used to reconstruct any two-dimensional functions. Spherical harmonics provide ways to compress the amount of incoming lights through a hemisphere dramatically so that such data on lights can be easily stored for each point on an object, while supporting very fast computations. The term “basis function” comes from the fact that this type of function can be scaled and combined to approximate any mathematical function. The scaling factors used to combine the basis functions are usually referred to as coefficients. For a non-limiting example, a function ƒ(x) can be approximated by the family of basis functions Bi(x), and the coefficients ci can be calculated through integration of these basis functions Bi(x) with the function ƒ(x) over ƒ(x)'s domain, as shown by the equations below.
For a two-dimensional function ƒ(s) defined over the unit sphere 110, the coefficients ci become clm, the spherical harmonics coefficients, and the basis function Bi becomes ylm (s), the spherical harmonics. The nth order approximated function of ƒ, called {tilde under (ƒ)}, can then be reconstructed by:
Where the clm coefficients are given by:
Here, the {tilde under (θ)} function is a band-limited approximation of the original function ƒ, which in turn can be fully reconstructed by:
Since spherical coordinates over the unit sphere of domain s are defined by
the ci coefficients can be calculated by:
ci=∫02π∫0πƒ(θ,φ)yi(θ,φ)sin θdθφ
The use of more coefficients to represent the source lighting when rendering an object enables handling of high frequency lighting that does not limit the source to only low frequency lighting. For a non-limiting example, the approach described above is capable of handling lighting that is represented by more than 1000 basis functions (i.e. high frequency lighting).
In some embodiments, Monte Carlo integration can be applied to the integration of any function, wherein Monte Carlo integration can numerically resolve any integral without actually resolving the equation itself. Monte Carlo integration is based on stochastic sampling, meaning that the unit sphere needs to be divided into a regular grid of n×n cells so that a number of samples evenly distributed on the unit sphere can be taken at a random point inside each cell.
Once a sampling scheme is chosen, the integral in the equation above can be resolved numerically. The important thing to Monte Carlo integration is the Monte Carlo estimator, which allows estimating the integral of any function ƒ by summing the product of the value of that function at sample points by the value of the probability density function (PDF), p, at the same sample points:
If the distribution of the sample points is uniform over the set of integration, the inverse of the PDF (also known as the weighting function) can be taken out and the approximation of the integral of the function is just the product of the function evaluated at these sample points, divided by a weight.
Since the set of integration is over the surface of the unit sphere, the sample points need to be uniformly distributed over the unit sphere. Conventionally, the sample points can be distributed on an n×n square grid, jittered, and mapped into spherical coordinates, by using the following equation:
θ=2 cos−1(√{square root over (1−x)})
φ=2πy
As a result, the weighting function is constant and is the value of the surface of the unit sphere (i.e., 4π):
The SH coefficients can then be calculated as a simple sum of the product of the function to project and the spherical harmonic basis function evaluated at sample points:
The following summarizes the basic steps to calculate SH coefficients at a given vertex of an object to be rendered:
In computer graphics, a rendering equation can describe the complete distribution of light in a scene and has been used to implement all sorts of global illumination approaches. The common formulation of the rendering equation without participating media is as follows:
Lr(x,
This equation simply says that the outgoing radiance at point x on a surface in the outgoing direction ω is the sum of the emitted radiance and the radiance reflected from this point in this direction.
The reflectance of the object as a function of illumination geometry and viewing geometry may be represented by a bidirectional reflectance distribution function (BRDF). The BRDF may depend on various parameters, which include but are not limited to, the characteristics of the source light, the structural and optical properties of the surface, the shadow-casting, multiple scattering, mutual shadowing, transmission, reflection, absorption, emission by the surface elements, facet orientation distribution, and facet density. The reflected radiance can be expressed as the integral over the hemisphere of incoming directions at point x of the product of the BRDF and the incident radiance (in other words, the incoming light):
L0(x,
By using differential solid angle, the rendering equation can be rewritten as follows:
L0(x,
This equation expresses the reflected radiance as an integral over the set of all points in the scene S of the incident radiance leaving x′ toward x and V, the visibility function between the x and x′, and G, the geometry term. V is a simple test that returns 1 if x and x′ can see each other, or 0 if not. The geometry term depends on the geometric relationship between the surfaces at point x and x′.
In some embodiments, the rendering equation used to calculate the amount of light at different points on the surfaces of the scene can be simplified. For non-limiting examples, the emitted radiance can be taken out if self-emitting objects is not considered. Also, if only Lambertian diffuse lighting—light is equally scattered in any direction, whatever the incident direction—is considered, the BRDF becomes just a constant (the ratio of the reflectance and π) and can be taken out of the integral:
It becomes clear from the discussion above that the intensity of light leaving a given point on a surface can be expressed as an integral. Such integral, however, cannot be calculated in real time nor in symbolic terms (at least for the time being). The idea behind spherical harmonic lighting is to compute this integral as a preprocessing step and project the different functions on the spherical harmonic basis. Thanks to orthonormality of the spherical harmonics, i.e.,
integrating the product of two band-limited functions, say ƒ and g, over the unit sphere s is the same as calculating the dot product of the spherical harmonics coefficients of these functions, i.e.,
In other words, a complex integral of two functions over the unit sphere can be reduced to the simple addition and multiplication of numbers. The equations above will be applied at all vertices to evaluate the rendering equation with the pre-calculated SH coefficients of the different functions involved at run time. Consequently, the preprocessing steps can be decomposed as follows:
After both the incoming light and the transfer function are SH projected (SH coefficients pre-computed) using the Monte Carlo integration technique explained above during a preprocessing step, The real-time integral evaluation is then simply performed by computing the dot product of the pre-computed SH coefficients of the transfer function and those of the incoming light at any given vertex of an object or volume.
Scattered Spherical Harmonic Approximation
Rendering the illumination effect on an object (volume) requires a dense set of SH coefficients to be pre-computed at each vertex or grid point of the object based on the process discussed above, which can be expensive both in computation and in storage. In reality, density variation in the interior of a volume, which for non-limiting examples can be a cloud or smoke 201 as shown in
In data regression analysis, various kinds of techniques for smooth interpolation of sparse, scattered 3D data can be utilized. For a non-limiting example, least squares, also known as ordinary least squares analysis, is a technique well known to one skilled in the art for linear regression. Least squares approach determines the values of unknown quantities in a statistical model by minimizing the sum of the residuals (difference between the predicted and observed values) squared. The objective consists of adjusting an approximate function y to best fit a known set of data. The data set consist of n data d({right arrow over (x)}i) at sample points {right arrow over (x)}i with i=1, 2, . . . , n. The approximate function has the form of ƒ({right arrow over (x)})=āp({right arrow over (x)})=a0+a1x+a2x2 . . . anxn, where p({right arrow over (x)})=(1, x, . . . , xn) is the basis vector for the least-squares polynomials, and {right arrow over (a)}=(a0, a1, . . . , an) is the vector of the adjustable parameters for basis vector. The goal of least squares fitting is to find the parameter values such that the approximate function best fits the data according to a defined error criterion by minimizing the sum square error function with respect to the adjustable parameters, i.e.,
Alternatively, radial basis functions can be utilized in place of least squares to achieve the same goal.
A scattered spherical harmonic approximation method is proposed for SH computation, which sparsely populates (selects from) a volume with a plurality SH sampling vertices and uses least squares approximation to extrapolate and compute the SH coefficients at any (other) arbitrary point of the volume. The objective is to find the coefficients of an n-degree polynomial ƒ({right arrow over (x)}) which approximates the values of the SH coefficients computed at sampling vertices of the volume. The minimization problem for each SH coefficient cj can be stated as:
where {right arrow over (x)}i is the location of sampling vertex i, cj({right arrow over (x)}i) is computed SH coefficient cj at {right arrow over (x)}i. As well-known to one skilled in the art, the minimization problem can be solved with the following:
where {right arrow over (a)} is the vector of unknown parameters for the least-squares polynomials. Note that the matrix on left hand side contains only position information, thus using a solution method like LU de-composition permits reuse of factorization result when computing {right arrow over (a)} for each SH coefficient.
Referring to
Referring to
In some embodiments, the underlying system used for modeling the volume or object can suggest locations for the sparsely populated SH sampling vertices. For a non-limiting example, if a collection of sphere primitives are used to fill a voxel grid, then SH sampling at the center of each sphere can encode the visibility in the local region. For the general case, sampling vertices can be distributed on a coarse isosurface around the density variation of the volume. A subset of these sampling vertices can be driven inside the surface, and some of them can be placed out-side the surface to suppress oscillations from the fitting polynomial.
With scattered SH approximation method discussed above, the memory use is not determined by the grid resolution, i.e., the number of points used for rendering the object, but the desired quality of output. For a non-limiting example, the cloud in
One embodiment may be implemented using a conventional general purpose or a specialized digital computer or microprocessor(s) programmed according to the teachings of the present disclosure, as will be apparent to those skilled in the computer art. Appropriate software coding can readily be prepared by skilled programmers based on the teachings of the present disclosure, as will be apparent to those skilled in the software art. The invention may also be implemented by the preparation of integrated circuits or by interconnecting an appropriate network of conventional component circuits, as will be readily apparent to those skilled in the art.
One embodiment includes a computer program product which is a machine readable medium (media) having instructions stored thereon/in which can be used to program one or more computing devices to perform any of the features presented herein. The machine readable medium can include, but is not limited to, one or more types of disks including floppy disks, optical discs, DVD, CD-ROMs, micro drive, and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic or optical cards, nanosystems (including molecular memory ICs), or any type of media or device suitable for storing instructions and/or data. Stored on any one of the computer readable medium (media), the present invention includes software for controlling both the hardware of the general purpose/specialized computer or microprocessor, and for enabling the computer or microprocessor to interact with a human viewer or other mechanism utilizing the results of the present invention. Such software may include, but is not limited to, device drivers, operating systems, execution environments/containers, and applications.
The foregoing description of the preferred embodiments of the present invention has been provided for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations will be apparent to the practitioner skilled in the art. Particularly, while the concept of “module” is used in the embodiments of the systems and methods described above, it will be evident that such concept can be interchangeably used with equivalent concepts such as, class, method, type, interface, bean, component, object model, and other suitable concepts. Embodiments were chosen and described in order to best describe the principles of the invention and its practical application, thereby enabling others skilled in the art to understand the invention, the various embodiments and with various modifications that are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the following claims and their equivalents.
This application claims priority to U.S. Provisional Patent Application No. 60/962,120, filed Jul. 25, 2007, and entitled “Method and system for scattered spherical harmonic approximation,” by Nafees Zafar et al., and is hereby incorporated herein by reference.
Number | Name | Date | Kind |
---|---|---|---|
7212207 | Green et al. | May 2007 | B2 |
7663623 | Zhou et al. | Feb 2010 | B2 |
20050035965 | Sloan et al. | Feb 2005 | A1 |
20050080602 | Snyder et al. | Apr 2005 | A1 |
20050104883 | Snyder et al. | May 2005 | A1 |
20060028468 | Chen et al. | Feb 2006 | A1 |
20060214931 | Snyder et al. | Sep 2006 | A1 |
20060279570 | Zhou et al. | Dec 2006 | A1 |
20070018996 | Wang et al. | Jan 2007 | A1 |
20070229502 | Tong et al. | Oct 2007 | A1 |
20080018647 | Bunnell | Jan 2008 | A1 |
Entry |
---|
Kristensen et al., Precomputed Local Radiance Transfer for Real-Time lighting Design, Jul. 2005, ACM SIGGRAPH 2005, ACM Transactions on Graphics, vol. 24, Issue 3, pp. 1208-1215. |
Arikan, et al. “Fast and Detailed Approximate Global Illumination by Irradiance Decomposition” ACM SIGGRAPH, 2005 pp. 1108-1114. |
Chris Oat, “Adding Spherical Harmonic Lighting to the Sushi Engine” 3D Application Research Group ATI Research, Inc., Game Developers Conference pp. 1-59. |
Sloan, et al. “Precomputed Radiance Transfer for Real-Time Rendering in Dynamic, Low-Frequency Lighting Environments”, in SIGGRAPH 2002, Computer Graphics Proceedings, pp. 527-536. |
Number | Date | Country | |
---|---|---|---|
20090027390 A1 | Jan 2009 | US |
Number | Date | Country | |
---|---|---|---|
60962120 | Jul 2007 | US |