Patent Grant
7443394
The present invention relates to computer animation. More particularly, the present invention relates to techniques and apparatus for efficient rendering of objects that are specified to include translucent materials.
Throughout the years, movie makers have often tried to tell stories involving make-believe creatures, far away places, and fantastic things. To do so, they have often relied on animation techniques to bring the make-believe to “life.” Two of the major paths in animation have traditionally included, drawing-based animation techniques and physical animation techniques.
Drawing-based animation techniques were refined in the twentieth century, by movie makers such as Walt Disney and used in movies such as “Snow White and the Seven Dwarfs” (1937) and “Fantasia” (1940). This animation technique typically required artists to hand-draw (or paint) animated images onto a transparent media or cels. After painting, each cel would then be captured or recorded onto film as one or more frames in a movie.
Physical-based animation techniques typically required the construction of miniature sets, props, and characters. The filmmakers would construct the sets, add props, and position the miniature characters in a pose. After the animator was happy with how everything was arranged, one or more frames of film would be taken of that specific arrangement. Physical animation techniques were developed by movie makers such as Willis O'Brien for movies such as “King Kong” (1933). Subsequently, these techniques were refined by animators such as Ray Harryhausen for movies including “Mighty Joe Young” (1948) and Clash Of The Titans (1981).
With the wide-spread availability of computers in the later part of the twentieth century, animators began to rely upon computers to assist in the animation process. This included using computers to facilitate drawing-based animation, for example, by painting images, by generating in-between images (“tweening”), and the like. This also included using computers to augment physical animation techniques. For example, physical models could be represented by virtual models in computer memory, and manipulated.
One of the pioneering companies in the computer animation (CA) industry was Pixar. Pixar is more widely known as Pixar Animation Studios, the creators of animated features such as “Toy Story” (1995) and “Toy Story 2” (1999), “A Bugs Life” (1998), “Monsters, Inc.” (2001), “Finding Nemo” (2003), “The Incredibles” (2004), and others. In addition to creating animated features, Pixar developed computing platforms specially designed for CA, and CA software now known as RenderMan®. RenderMan® was particularly well received in the animation industry and recognized with two Academy Awards®.
RenderMan® software is used to convert graphical specifications of objects and convert them into one or more images. This technique is known in the industry as rendering. One specific portion of the rendering process is known as surface shading. In the surface shading process the software determines how much light is directed towards the viewer from the surface of objects in an image in response to the applied light sources.
For different types of object materials, however, the amount of light directed towards the viewer is more than the light reflected from the surface. Such types of materials include materials that are at least partially translucent in nature, such as plastic, skin, milk, and the like. Translucent materials may also include materials that are non-uniformly translucent, such as marble, jade, and the like, and may have sub-surface features. Compensation for subsurface scattering and other effects of translucency are deemed important for the realism of computer-generated imagery.
Techniques to compensate and to render translucent objects have previously been proposed and generally fall into two categories: surface based approaches, and stochastic ray tracing methods. Each of these approaches has significant shortcomings. Surface based approaches suffer from technical difficulties in the parametric representation of sample locations. Additionally, such approaches do not address the fundamental problem of representing an internal structure of the scattering media.
Stochastic ray tracing approaches can be used to solve the full physical problem, however such approaches are generally very time consuming. Another approach includes a finite element analysis through a hierarchical evaluation of a radiative transfer equation. Again such approaches are very computationally intensive. Because the present solutions to the problem of rendering objects of translucent materials is slow, animators using such rendering techniques cannot quickly see how the animation will look. Accordingly, the animators must often wait overnight to see the rendering results.
In light of the above, what is needed are efficient techniques to render translucent objects.
The present invention relates to the field of computer animation. More particularly, the present invention relates to techniques and apparatus for efficient rendering of objects composed of translucent materials.
The techniques described below are based on an explicit representation of the subsurface illumination fluxes on a 3 dimensional grid. This representation allows the efficient spatial indexing of illumination samples from the unstructured geometry of the object. This representation allows the rendering engine to quickly solve the transport of diffuse illumination inside the translucent material with full representation of the internal structure.
The process described below includes the following phases: 1) evaluation of surface illumination on object, collection of surface illumination (diffuse reflection), determining of area weighting, and scattering properties on sample grids; 2) efficient determination of the diffuse radiative transfer on the sample grids; and 3) sampling the diffuse illumination during final rendering pass.
In this embodiment, the first phase is the illumination sampling pass. In this phase, the relevant light sources are evaluated on the surface of an object and the surface shading model is evaluated to determine the quantity of diffuse illumination (diffuse reflection) for each color channel. The diffuse illumination (diffuse reflection) at the surface are transmitted to the translucency solver, described below. Additionally, Cartesian solver grids are constructed which cover the bounding volume of the object sample points. The diffuse illumination/diffuse reflection values are inserted onto the solver grids where they are merged with other values. The merge process is accomplished by adding the representative sample area for the sample into one grid and simultaneously adding the product of the sample area and the illumination value are added into a second grid. Scattering properties such as attenuation and scattering length are also mapped onto grid structures using the same technique. For internal structures with varying scattering properties additional samples of these properties are added. During this phase the samples may be offset from the actual sample position using the surface normal to simulate the effects of an initial forward or an initial reverse scattering process before the illumination enters the diffuse solver.
The second phase of the process is the solution of the scattering pass itself. In this embodiment, the illumination and scattering properties are normalized by processing all of the grids with a short range dealiasing filter and diving the area weighted illumination and scattering property grids by the area grid value. The diffuse illumination compensation values are then solved by solving the elliptic radiative transfer equation (1) on the grid values. This process can be done efficiently in several ways. For example, in one approach the process is performed via a recursive low pass filter whose filter coefficients vary with scattering parameters across the grid in each direction for each dimension. The filter parameters are chosen so that the bidirectional filter impulse response approximates the Green's function of the scattering operator with the local scattering parameters. In another approach, the variable coefficient elliptic scattering equation is solved using a fast elliptic solver such as a fast convolution FFT solver or multi-grid solver.
The third phase of the process is to take the completed grid containing the solved diffuse illumination compensation values and point sample the grid using an interpolation kernel from the surface shaders. This may occur during a final rendering pass for the surface. All of the sample processing for this phase is done in global 3-space so that no surface coordinate parameterizations are needed and smoothness across multiple geometry elements is provided.
According to on aspect of the invention, a method for rendering an object is disclosed. Various techniques includes receiving a specification of a first component of the object and a second component of the object, wherein the first component is associated with a first plurality of surface points on the object and wherein the second component is associated with a second plurality of surface points on the object, determining diffuse illumination values for surface points on the object, wherein the diffuse illumination values includes a first plurality of diffuse illumination values associated with the first plurality of surface points, and a second plurality of diffuse illumination values associated with the second plurality of surface points, and associating a first three-dimensional grid with the first component, wherein the first three-dimensional grid includes a first plurality of vertices and a first plurality of voxels, and wherein each voxel includes at least a first portion of the object. Additional techniques includes associating a second three-dimensional grid with the second component, wherein the second three-dimensional grid includes a second plurality of vertices and a second plurality of voxels, and wherein each voxel includes at least a second portion of the object, mapping the first plurality of surface points on the object to a first set of voxels from the first plurality of voxels, and mapping the second plurality of surface points on the object to a second set of voxels from the second plurality of voxels. Other steps may include determining first diffuse illumination values for a plurality of vertices associated with the first plurality of voxels in response to the first plurality of diffuse illumination values for the first plurality of surface points on the object, determining second diffuse illumination values for a plurality of vertices associated with the second plurality of voxels in response to the second plurality of diffuse illumination values for the second plurality of surface points on the object, performing one or more low pass filter operations on the first diffuse illumination values to determine first diffuse illumination compensation values, and performing one or more low pass filter operations on the second diffuse illumination values to determine second diffuse illumination compensation values. Further processes may include determining first diffuse illumination compensation values for the first plurality surface points in response to the first diffuse illumination compensation values, determining second diffuse illumination compensation values for the second plurality surface points in response to the second diffuse illumination compensation values, determining first compensated diffuse illumination values for the first plurality of surface points on the object in response to the first diffuse illumination values for the first plurality surface points and in response to the first diffuse illumination compensation values, and determining second compensated diffuse illumination values for the second plurality of surface points on the object in response to the second diffuse illumination values for the second plurality surface points and in response to the second diffuse illumination compensation values.
According to another aspect of the invention, a computer system for rendering an object is disclosed. One apparatus includes a memory configured to store a specification of an object, including a first portion and a second portion, wherein the first portion is associated with a first plurality of surface points, and wherein the second portion is associated with a second plurality of surface points, wherein the memory is configured to store a first plurality of illumination values associated with the first plurality of surface points, and configured to store a second plurality of illumination values associated with the second plurality of surface points. The first portion and the second portion are often adjacent, and sometimes the first plurality of surface points and the second plurality of surface points overlap. Another component may include a processor coupled to the memory, wherein the processor is configured to associate a first three-dimensional grid including a first plurality of vertices and a first plurality of voxels with the first plurality of surface points on the first component, wherein the processor is configured to associate a second three-dimensional grid including a second plurality of vertices and a second plurality of voxels with the second plurality of surface points on the second component, wherein the processor is configured to determine first illumination values for the first plurality of vertices in response to the first plurality of illumination values associated with the first plurality of surface points, wherein the processor is configured to determine second illumination values for the second plurality of vertices in response to the second plurality of illumination values associated with the second plurality of surface points; wherein the processor is configured to perform one or more low pass filter operations on the first illumination values to determine first subsurface values associated with the first plurality of vertices, wherein the processor is configured to perform one or more low pass filter operations on the second illumination values to determine second subsurface values associated with the second plurality of vertices, wherein the processor is also configured to determine a first plurality of subsurface values associated with the first plurality of surface points in response to the first subsurface values, and wherein the processor is also configured to determine a second plurality of subsurface values associated with the second plurality of surface points in response to the second subsurface values.
According to yet another aspect of the invention, a computer program product for a computer system including a processor is described. Various computer program product include code that directs the processor to receive a specification of an object, including a first portion and a second portion, wherein the first portion is associated with a first plurality of surface points, and wherein the second portion is associated with a second plurality of surface points, wherein the a first plurality of illumination values is associated with the first plurality of surface points, and wherein a second plurality of illumination values is associated with the second plurality of surface points, code that directs the processor to associate a first volumetric grid with the first plurality of surface points on the first component, and code that directs the processor to associate a second volumetric grid with the second plurality of surface points on the second component. Other computer program product may include code that directs the processor to determine first illumination values for the first volumetric grid in response to the first plurality of illumination values, code that directs the processor to determine second illumination values for the first volumetric grid in response to the second plurality of illumination values, and code that directs the processor to perform low pass filtering on the first illumination values for the first volumetric grid to determine first scattering values for the first volumetric grid. Additional computer program product may include code that directs the processor to perform low pass filtering on the second illumination values for the second volumetric grid to determine second scattering values for the second volumetric grid, code that directs the processor to determine first scattering values for the first plurality of surface points in response to the first scattering values for the first volumetric grid, and code that directs the processor to determine second scattering values for the second plurality of surface points in response to the second scattering values for the second volumetric grid. The codes typically reside on a tangible media such as optical media, magnetic media, semiconductor media, organic media, or the like.
In order to more fully understand the present invention, reference is made to the accompanying drawings. Understanding that these drawings are not to be considered limitations in the scope of the invention, the presently described embodiments and the presently understood best mode of the invention are described with additional detail through use of the accompanying drawings in which:
In the present embodiment, computer system 100 typically includes a monitor 110, computer 120, a keyboard 130, a user input device 140, a network interface 150, and the like.
In the present embodiment, user input device 140 is typically embodied as a computer mouse, a trackball, a track pad, wireless remote, and the like. User input device 140 typically allows a user to select objects, icons, text and the like that appear on the monitor 110.
Embodiments of network interface 150 typically include an Ethernet card, a modem (telephone, satellite, cable, ISDN), (asynchronous) digital subscriber line (DSL) unit, and the like. Network interface 150 are typically coupled to a computer network as shown. In other embodiments, network interface 150 may be physically integrated on the motherboard of computer 120, may be a software program, such as soft DSL, or the like.
Computer 120 typically includes familiar computer components such as a processor 160, and memory storage devices, such as a random access memory (RAM) 170, disk drives 180, and system bus 190 interconnecting the above components.
In one embodiment, computer 120 is a PC compatible computer having multiple microprocessors such as Xeon™ microprocessor from Intel Corporation. Further, in the present embodiment, computer 120 typically includes a UNIX-based operating system.
RAM 170 and disk drive 180 are examples of tangible media for storage of data, audio/video files, computer programs, applet interpreters or compilers, virtual machines, embodiments of the herein described invention including geometric scene data, object data files, shader descriptors, a rendering engine, output image files, texture maps, displacement maps, scattering lengths and absorption data of object materials, and the like. Other types of tangible media include floppy disks, removable hard disks, optical storage media such as CD-ROMS and bar codes, semiconductor memories such as flash memories, read-only-memories (ROMS), battery-backed volatile memories, networked storage devices, and the like.
In the present embodiment, computer system 100 may also include software that enables communications over a network such as the HTTP, TCP/IP, RTP/RTSP protocols, and the like. In alternative embodiments of the present invention, other communications software and transfer protocols may also be used, for example IPX, UDP or the like.
In one embodiment of the present invention, the process begins with the specification of an image to be rendered, step 200. In this example, image specification includes the specification of one or more scene backgrounds, the specifications of objects and textures in the images, the specification of lighting to be applied to the images, the specification of object placement data, the specification of animation parameters for objects in the scene, displacement maps, and the like. Typically, images are specified by one or more animators using conventional three-dimensional modeling techniques and software currently available. For example, software such as Alias/Wavefront's Maya™ Software, and the like may be used to specify the scene.
In the present examples, a specification of a background typically includes a geometric specification of a location or a “set” into which objects, described below, are placed. The geometric specification may include geometric boundaries, one or more “environment” maps, and the like. These environment maps may be static or may vary with time. In examples of the present invention, a set may represent any location where the objects are placed, such as a room, an interior location, an exterior location, or the like.
The objects are placed or positioned by the animator into the set. These objects may be in inanimate objects within a scene such as furniture, plants, vehicles, and the like. Additionally, these objects may be objects that move or are moved within the scene, for example, characters, chairs, doors, and the like. In the present embodiment, objects may be simple to complex, with some objects being formed from one or more smaller objects. In some embodiments, only portions of objects may be retrieved from a memory, and the remaining portions of the objects are dynamically created before being rendered. In such cases, the dynamically created objects are termed procedurally generated objects.
In the present embodiment, objects are typically stored separately from each other. That is, each object may be stored in its own file and be retrieved by a computer file system. In other embodiments of the present invention, data for more than one object may be included into a single file.
Each scene typically also includes a specification of light sources and lighting parameters. Lighting data may include light sources, light types (e.g. diffuse, point), lighting locations, lighting directions, and the like. This lighting data may be stored within a single file or multiple files.
Next, an object to be rendered is identified, retrieved from disk memory and then stored in program memory, step 310. In one embodiment, streaming technology is used so that the rendering engine can begin processing the object, although the entire data file has not yet been completely retrieved by memory. In other embodiments, the rendering engine begins processing the object after it is retrieved from memory.
In the present embodiment, once the object is retrieved from disk memory, the surface of the object is shaded, step 320. Any number of conventional surface shading programs may used for this step, for example, RenderMan® software may be used to determining the surface shading for each light source. In one embodiment, the shading of the surface is determined for all lighting sources at the same time. In an alternative embodiment, the surface of the object is shaded individually for each light source. For example, if there are two light sources, there may be two separate shading values for each location on the surface of the object.
In this embodiment, a volumetric grid is then constructed to bound the entire object, step 330. In one example, the volumetric grid is Cartesian grid with approximately 200 to 500 vertices per side; accordingly the Cartesian grid may include from 8 million to 125 million voxels. The number of grid locations per side may be unequal and may vary, for example, a grid may be constructed to have 110 voxels in an x direction, 400 in a y direction and 500 in a z direction; thus the grid may include approximately 22 million voxels. In light of the present patent application, one of ordinary skill in the art would understand that the resolution of the volumetric grid in any direction and total number of voxels defined will vary according to specific engineering requirements and resources. In one embodiment, the voxels may be cubes, however in other embodiments, the voxels may have unequal dimensions and be irregular in shape.
In various embodiments of the present invention, multiple scattering grids may be used to bound an object. Use of a single rectangular volume can be inefficient to bound large and complex objects. For example, for humanoid characters, a single bounding box would include a large number of voxels that include empty space. As an example, areas between the legs, or between the arms and body would typically be empty. These empty voxels are defined and maintained in memory, however are not used. Additionally, depending upon how the object is composed of sub-components, the subsurface scattering effects of the different sub-components may be effectively independent. For example, diffuse illumination incident on the right hand of an object do not typically impact the subsurface scattering illumination on the left hand.
In various embodiments of the present invention, the multiple grids may be specific to the pose of the object. As an example, in a first pose an arm of a person is straight, and a single bounding box may be defined for the arm sub-component. However, in a second pose of the arm, the arm is bent at the elbow at 90 degrees. In such a case, if a single bounding box were used, many empty voxels would be allocated between the hand and the shoulder. Accordingly, in this example, two or more sub-grids may be allocated, including one sub-grid for the hand and forearm and one sub-grid for the upper arm.
In various embodiments of the present invention, to reduce the amount of unused voxels computed, objects may first be separated into sub-components and then bounding boxes for the sub-components may be computed. Voxels in the smaller bounding boxes for the sub-components are then more likely to be used for scattering calculations, described below. An example, limbs of a humanoid may be bound within their own bounding boxes, and a torso may be bound within its own bounding box. According, the empty space between the arms and the body are not allocated. Additionally, such embodiments enable selective rendering of sub-components of an object. For example, the subsurface scattering illumination determinations may be performed on the head, but not the torso or legs for one or more scenes.
One issue that may arise as a result of the use of smaller bounding boxes is the potential of boundary mismatch or inconsistency at the boundaries. In various embodiments of the present invention, to address this issue, grids of sub-components are intentionally overlapped between boundaries with other sub-components. In various embodiments, the computed grid values in the transition area between the sub-components are smoothly (e.g. linearly) interpolated, or the like to reduce the mismatch.
In various embodiments of the present invention, the inventors have determined that often internal geometric elements in a object model interfere with sub-surface scattering techniques, such as that described below. More particularly, characters are often modeled with eye sockets or mouth “bag” voids in a model. In operation these internal geometric elements skew the sub-surface scattering computations. This skewing translates into unexpected local darkening or glowing of the surface, sometimes known as sparkles and pops.
In various embodiments of the present invention, these voids are ignored from the geometric model thus resulting in a less complex geometric model. For example, in one embodiment, a mouth “bag” and eye sockets may be removed from a head model, thus the geometry of the head model will become mostly convex. In the present embodiments, when mapping voxels to the geometric model, some voxels (“void voxels”) will be associated with regions of the object where the voids were previously located. In another embodiment, the voids are tagged and ignored during the irradiance gathering step.
In various embodiments of the present invention, the diffuse illumination compensating values (e.g. subsurface illumination contributions), determined below, for these “void voxels” are then simply discarded and not used for shading purposes. Accordingly, the undesirable artifacts due to these voids are reduced.
In the present example, not all voxels will include a portion of the object or the surface of the object. For this embodiment, such voxels are ignored for the following calculations or discarded.
Next, the voxels that bound any portion of the surface of the object (surface voxels) are identified, step 340.
In the present embodiment, the vertices of each surface voxel that includes a surface portion of the object are associated with diffuse illumination values. The surface portions typically include shading surfaces (e.g. triangles) defined by shading values (diffuse illumination values) and associated shading areas. In this embodiment, the shading values typically include red, green, blue (RGB) shading components; and the shading areas typically represent an area of a respective shading surface.
In this embodiment, the size of shading areas are smaller than the grid spacing, accordingly, an area weighting operation is performed for each shading value, step 350.
Next, the area weighting values for each shading value are distributed to each of the 8 vertices in a voxel, step 360. In this embodiment, after each of the area-weighted shading values are distributed to the vertices, the contributions are added-up, step 365. In one embodiment, a typical vertex is associated with contributions of shading surfaces from different voxels.
In this example, voxels 460, 470 and 480 are illustrated from the top and from the side in profile view in 490. In this example, the diffuse illumination value of shading surface 420 is assumed to be at the centroid 495 of shading surface 420. Further, as can be seen, centroid is location within voxel 470 ¼ the way from the closest corner 425, ⅕ the way in another direction, and ¼ in the third direction. Further, in this example, the diffuse illumination value is determined to be 40, and the surface area for shading surface 420 is 5 units.
In this example the area-weighted illumination value is (40×5=200)200. The area-weighted illumination is then divided in a three to 1 ratio to the bottom comers. Thus the bottom comers divide 150 and the top comers divide 50. Next, based upon the position of centroid 495, these illumination values are further divided in approximately the way illustrated in
In other embodiments of the present invention, any other conventional method for distributing shading values to vertices of voxels is contemplated. For example, the area-weighted contribution for all shading surfaces may be determined once for the voxel, and that value is distributed to the eight vertices in the voxel, depending upon position.
In this example, voxels 530, 540 and 550 are illustrated including the shading surfaces and shading values shown. Also illustrated is a profile view 560 illustrating a location of the shading surfaces within voxels 530, 540, and 550. In this example, using area weighting calculations, the average illumination value in voxel 530 is illustrated as 32; the average illumination value in voxel 540 is 32, and the average illumination value in voxel 550 is 32. From the top view, the average illumination values are associated with the centers of the respective voxels, i.e. from the top, the average illumination values are assumed to be equidistant to the four respective comers of the voxels. From profile view 560, the average illumination values for voxels 530 and 550 associated with the center of the voxels, and are equidistant from the top and bottom comers, but the average illumination values for voxel 540 is associated with a position about 1.7 times closer to the bottom comers than the top comers.
As illustrated in
In the present embodiment, the above process is then typically repeated to distribute material properties values of the object onto the vertices. In the present embodiment, the material properties include attenuation factors and scattering lengths for the material bounded by respective voxels, step 370. Because the attenuation factors and scattering lengths are parameters of the material within the object, and not just on the surface, the process is performed for all voxels that include a portion of the object. In this embodiment, as a result, the material properties are mapped onto vertices of the voxels as a volume-weighted scattering length grid, and a volume-weighted attenuation factoring grid.
At this stage, a number of grids may be allocated and populated, for example, one grid to store weighted diffuse illumination values, one grid to store weighted attenuation factors, and one grid to store weighted scattering lengths. In other embodiments, additional grids may be allocated to stored additional values. In still another embodiment, only a single grid may be defined, and the weighted diffuse illumination values and weighted material properties are merely associated with that grid.
In the present embodiment, the diffuse illumination values, attenuation factors, and scattering lengths are color dependent. In this example, diffuse illumination values include a red component, a green component, and a blue component, however other color system may also be used. In one embodiment, the attenuation factors and scattering lengths are color dependent. As an example, for a green jade material, the scattering lengths of red and blue may be shorter in the material than the scattering length of green in the material. Accordingly, in the present embodiment, nine volumetric grids may be created and populated. In other embodiments, a single grid having vertices associated with multiple values may also be used.
The following steps are performed to efficiently simulate a scattering process of the translucent material. In the present embodiment, the process begins by processing the area-weighted diffuse illumination grid values, the volume-weighted scattering length grid values, and the volume-weighted attenuation factor grid values with a short range dealiasing filter per color step 380. In various embodiments of the present invention, the dealiasing filter may be a blur filter, or the like.
Next, in this embodiment, the area weighted diffuse illumination values, the volume-weighted scattering lengths, and volume-weighted attenuation factors are normalized, step 385. In one example, the area weighted diffuse illumination grid values are divided by a grid storing the areas of the respective voxels; and the volume-weighted scattering length grid values and the volume-weighted attenuation factor grid values are both divided by a grid storing the volumes of the respective voxels. In other embodiments, other types of normalizing filters may be used.
Embodiments of the present invention can simulate non-uniform materials within the object, such as bones, cartilage, metal, and the like with the variance across the grid. Very few organic elements are of homogeneous nature. Many translucent objects comport internal parts that occlude the light scattering inside. In the case of skin for instance, the underlying bone and muscle structure plays an important part in the subsurface scattering. Spatially varying the scatter length across the model surface however is one way to simulate this internal complexity. To this end, the irradiance gathering process also records a spatially varying tricolor (RGB) scatter length that is being provided by the surface shader.
In various embodiments of the present invention, the scattering length triad can be generated by any conventional shading methods, such as procedural texture, painted maps or a combination of both. As examples, a user may retrieve one or more two-dimensional views of a three-dimensional bone, a face, or the like in a two-dimensional paint program such as Adobe Photoshop® or the like. Next, the user paints the desired scattering length values in an overlay layer on top of the two-dimensional views. In various embodiments, the color and or intensity of the color is used to signify the desired scattering length values. For example, the scatter length is painted such that the “depth” of tissue is thinner in a forehead area of a face, and such that the “depth” of tissue is thicker on the cheeks of the face, and the like.
In such embodiments, the painted overlay layer becomes associated with the two-dimensional view of the bone, the face, or the like. In various embodiments of the present invention, at a later time, three-dimensional paint-projection techniques are subsequently used to map the overlay layer pixel values back to the three-dimensional object. Feathering techniques may also be used so that the scattering length values smoothly transition about the edges of the bone, face, or the like. In various embodiments of the present invention, multiple two-dimensional views of a three-dimensional object can be “painted” upon at the same time, as described in Application No. PCT/US04/16946 filed Mar. 23, 2004. That application is incorporated by reference for all purposes.
In various embodiments of the present invention, objects that make-up an object have scattering-lengths defined, as described above. As an example, the forearm includes skin and flesh with a first scattering length, and underlying bone with a second scattering length. These respective scattering lengths are then mapped to vertices of the volumetric illumination grids, described above.
Next, the diffuse illumination values are determined by solving the elliptic radiative transfer equation on the grid values:
μ∇2D=I−αD
In this equation, μ is a local scattering coefficient, α is the local extinction coefficient, I is the source illumination from the surface, and D is the diffuse radiosity. In the present embodiment, μ and α (an extinction value) are derived from the attenuation factor grid values and the scattering length grid values; and I is derived from the diffuse illumination grid values. In this equation, αD represents the amount of illumination I absorbed, and μ∇2D represents the illumination flux.
As a side effect of increasing the scattering length of red 1305, the inventors discovered that in region 1330, the amount of scattering of blue 1310 and green 1320 exceed the intensity of red 1305. Because of this, within region 1330, blue 1310 and green 1320 become the predominant color over red 1305. However, in embodiments of the present invention, it is typically desired that the color of light having the longest scattering length should also remain the predominant color through out the range of the object.
In various embodiments of the present invention, to address this issue, the extinction coefficients of the undesired colors are increased relative to the desired predominant color. As is known, extinction is commonly a sum of absorption and scattering. Because increasing the amount of scattering (decreasing scattering lengths) of blue 1310 and green 1320 would not change the fact that there would be a region where blue and green have higher scattering than red, the inventors adjust the absorption of blue 1310 and green 1320 of the material. As a result, as shown in
In the present embodiment, the diffuse illumination values are determined by running a recursive low pass filter on the illumination grid values, step 390. In one embodiment, the recursive filter may be similar to the following equation:
Y(n)=I(n)+α(n)Y(n−1)
In this equation, Y(n) represents a diffuse illumination compensation value at point n; I(n) represents the diffuse illumination value at point n; α(n) represents an extinction value at point n; and Y(n−1) represents the diffuse illumination compensation value at point n−1. In this embodiment, α(n) (derived from the scattering length and absorption values) varies across the grid.
In this embodiment, the recursive low pass filter is run upon the illumination grid values in six directions, for example, +x, −x, +y, −y, +z, −z directions. By doing this, the illumination values are effectively spread within the material to approximate the Green's function of the scattering operator (μ∇2), while also compensating for material having different scattering lengths. In this embodiment, the values for α are selected to simulate Green's function. More particularly, both the recursive low pass filter and Green's functions are both exponential, thus α is selected such that the low pass filter has approximately the same decay characteristics of Green's function. In the present embodiment, the change is α for adjacent samples is assumed to be small compared to the value of α.
In other embodiments, other numerical methods for determining the diffuse illumination grid are contemplated. For example, a fast elliptic solver such as a fast convolution FFT solver or a multi-grid solver can be applied using the illumination grid values and scattering grid values, described above. In light of the present disclosure, one of ordinary skill in the art will recognize that other methods for solving for the diffuse illumination values can be used.
In the present embodiment, a result of this process is a diffuse illumination grid where the vertices are populated with diffuse illumination compensation values (scattering illumination values) resulting from the spreading of light within an object. More particularly, the diffuse illumination compensation values of surface voxels represent an amount of illumination to be added to the surface diffuse illumination values determined by the surface shader. For example, if an translucent object is thin and is back-lit, the surface shader may specify a low diffuse illumination value at surface. However, using the process described above, the total diffuse illumination value at the surface may be increased on account of the diffuse illumination compensation values determined above.
In the present example, the surface voxels determined above are again identified, and the diffuse illumination compensation values for vertices in the surface voxels are interpolated to determine diffuse illumination compensation factors for each surface sample, step 400. Any conventional interpolation process may be implemented. Next, the diffuse illumination compensation values are combined with the surface shading values (diffuse illumination values) determined by the shader, step 410. In various embodiments of the present invention, the combination may be any linear or non-linear combination of the two values. In the present embodiment, the resulting surface illumination values represent the appearance of the object illuminated by lighting sources compensated for the object material.
In various embodiments of the present invention, the inventors of the present invention have discovered that performing subsurface scattering of light in the method described in the above-referenced application may unintentionally soften fine geometric details on the surface of the object such as wrinkles, creases, bumps, and the like. To reduce such effects, weighted combinations of traditional surface shading values and subsurface scattering values are used to determine the final value for the surface.
In various embodiments, the respective combination of the diffuse illumination compensation factor (scattered light) and the diffuse illumination value (reflected light) for each surface sample, is based upon a weighting factor. In some embodiments, the weighting factor may be the same for all surface samples. As an example, the combined diffuse illumination values for all surface points are based upon 60% of the diffuse illumination values and 40% of the diffuse illumination compensation values, 75% and 25%, or any other proportion. In other embodiments, the weighting factors may be the same or different for the surface samples. For example, the combined diffuse illumination values for a first and a second surface point may be based upon a ratio of 3:1 of the diffuse illumination value to diffuse illumination compensation value, whereas a third surface point may be based upon a ratio of 1:5. In other examples, other proportions are contemplated.
In various embodiments, the weighting factors may be automatically (procedurally) determined, determined based upon input from a user, or a combination of these methods. In various implementations, the inventors of the present invention have discovered that in areas with fine geometric details, such as a corner of a mouth, crease of an eye, or the like, the diffuse illumination value should have a higher weight. Further, in areas with relatively little geometric fluctuations, such as a cheek or forehead, the sub-surface scattering (diffuse illumination compensation values) values may have a higher weight. In one embodiment, the system may automatically determine areas of the object with high and low geometric complexity. Then in areas of high geometric complexity, the system assigns a smaller weight to the surface points compared to surface points within areas of low geometric complexity. As a result, in areas of high geometric complexity, the percentage or ratio of the diffuse illumination compensation values compared to the diffuse illumination values are small compared to surface points within areas of low geometric complexity.
In one embodiment, manual methods may also be used by a user to set weights for surface points.
Next, in various embodiments, the two-dimensional image is projected back into three-dimensional space, step 720. Specifically, the pixel values in the two-dimensional image are projected back to surface positions of the object. Feathering techniques may be used to smooth the projection. Additionally, more than one two-dimensional view may be used for the paint-projection process. Further details regarding two-dimensional paint-based methods are disclosed in PCT/US04/08993 filed Mar. 23, 2004, incorporated by reference for all purposes.
In various embodiments, the pixel values in the two-dimensional image are then used to represent weighting factors for the surface positions, as discussed above in step 410. For example, in this method, the pixel values relate to the amount of the diffuse reflection and the amount of the sub-surface scattering values for the surface position. For instance a pixel value of 255 (selected from {0 . . . 255})indicates that the surface position includes only the diffuse reflection; a pixel value of 0 indicates that the surface position includes only the sub-surface scattering values; a pixel value of 128 indicates that the surface position includes approximately 50% of the diffuse reflection and 50% of the diffuse illumination compensation value, and the like.
According to the embodiment above, the paint values in the overlay layer are then mapped back to the three-dimensional model. Accordingly, paint values of geometric object 850 and 860 are then mapped back to areas 810 and 820, respectively; and paint values of geometric object 870 are mapped back to thinner area 830.
In other embodiments of the present invention, the inventors have discovered that in some regions of an object, the sub-surface scattering values may unintentionally “leak” light because of the geometry of the object. In such cases, instead of modifying and re-running the sub-surface scattering process, or providing an updated description of the geometric object and running the sub-surface scattering process, it is simpler to “ignore” the light leaks, by weighting the diffuse reflection values much higher than the sub-surface scattering values (e.g. 90% versus 10%), or the like in those areas.
In additional embodiments of the present invention, the inventors have recognized that the use of wavelength dependant scattering typically alters the hue of surface colors. However, this sometimes conflicts with the surface color and surface texture of an object that are painstakingly determined by one or more object modelers, artistic directors, or the like. Accordingly, the inventors have recognized that the use of sub-surface scattering illumination on surfaces, without regard to the surface color may conflict with the determined surface color. Although, using sub-surface scattering illumination as described above may result in a more technically accurate result, the inventors have determined that attempting to maintain the surface color is more important in various embodiments.
According to one embodiment of the present invention, the diffuse illumination value (0.77, 0.36, 0.36) is used to perform the sub-surface scattering operation to determine diffuse illumination compensation values. In this example, at surface point 950, the diffuse illumination compensation value is (0.39, 0.01, 0.01). Then, assuming a weighted combination of 75% diffuse illumination value and 25% diffuse illumination compensation value at surface point 950, the compensated diffuse illumination value is approximately (0.17, 0.09, 0.01). As can be determined, this value is more orange than yellow in color, although second surface should be more yellowish in color.
In various embodiments of the present invention, in step 320 in
Then, in various embodiments, prior to steps 350-400, the incident illumination at the surface positions are scaled by the respective material colors of the surface positions to determine a color for the scattering illumination calculations. In one specific embodiment, the incident illumination is multiplied by the square root of the input surface color and then scaled. As an example, using color values from 0 to 255, if the incident light is gray light (e.g. (r,g,b)=(196, 196, 196)), and the input surface color is bluish (e.g. 144, 144, 225), the incident light (196, 196, 196) is multiplied by a square root of the input surface color (e.g. 12, 12, 15) and then scaled. For example, if the scaling factor is 15, the resulting color for the scattering illumination calculation, the scattering illumination color is approximately (156,156, 196), which is approximately blue in color. As discussed above, in various embodiments, steps 350-400 may then be individually performed for the separate component colors (e.g. r,g,b) because the object material may absorb or scatter the component colors at different rates.
In various embodiments of the present invention, after determining the illumination values at surface positions due to the subsurface scattering, the illumination values are scaled by the respective material colors of the output surface positions to determine a color for the diffuse illumination compensation values. Similar to the above, in one specific embodiment, the scattering illumination is multiplied by the square root of the output surface color and then scaled. As an example, if the value of the subsurface scattering at a surface point is yellowish in color (e.g. 100, 100, 10), and the material surface is greenish (9, 244, 100). In this example, the subsurface scattering value of (100 100,10) is multiplied by a greenish value (e.g. 3,12,10) and then scaled. For example, if the scaling factor is 12, the resulting color for the illumination compensation value is approximately (25,100,83), which is greenish in color, matching the color of the material surface.
The above is merely an example of a process according to embodiments of the present invention. In one implementation the color values are scaled from 0 to 1. In such implementations, the resulting color values are different than when the color values are scaled from 0 to 255, or the like.
According to one embodiment of the present invention, illumination source 1030 (0.77, 0.77, 0.77) is a multiplied with the square root of the orange-colored material: (0.95, 0.69, 0.69) to obtain the illumination value (0.73, 0.50, 0.50). After the sub-surface scattering operation, the subsurface scattering value at point 1050 is (0.37. 0.13, 0.01). According to the above embodiment, this value is then multiplied by the square root of the yellow-colored material: (0.56, 0.56, 0.56), to obtain the diffuse illumination compensating value is (0.28, 0.10, 0.01).
Next, assuming a weighted combination of 75% diffuse illumination value and 25% diffuse illumination compensation value at surface point 1050, the compensated diffuse illumination value is approximately (0.13, 0.10, 0.01). In the present example, the value of (0.13, 0.10, 0.01) is more yellow than the value of (0.17, 0.09, 0.01) determined for surface point 950, in
In the foregoing specification, the invention has been described with reference to specific exemplary embodiments thereof. Many changes or modifications are readily envisioned. For example, some embodiments of the present invention allow the user to simulate scattering of light in a material in a non-uniform manner. This is also known as anisotropic scattering. Typically, the irradiance gathering stage discards the directionality of incident light intensities, accordingly, scattering directionality is impossible. However, it is possible to achieve some amount of scattering directionality by recording the irradiance samples at surface locations and moving the values along the vector defined by the incident light sources to the surface locations.
In the various embodiments, to simulate this process, the diffuse illumination values are modified within the illumination grid prior to the scattering phase, described above. More specifically, the diffuse illumination values determined at voxels bounding surface points may be moved forward into the object or moved backward outside the object between steps 360 and 365.
In
In
In various embodiments of the present invention, after compensating for forward scattering for all light sources, as shown, for example in
In various embodiments of the present invention, the inventors have determined that advantages of the present schema may be applied to various rendering pipelines. For example, some embodiments may be combined with the existing RenderMan® Reyes rendering architecture. Other embodiments may also be combined with the shot rendering techniques described in the above-referenced application. Further, it is envisioned that a great number of shaders currently compatible with RenderMan® could be used in various embodiments of the present invention.
In some embodiments of the present invention, the subsurface scattering effects discussed above, may or may not be desired for all renderings of a scene. For example, in some embodiments, the generation of these effects may be more time consuming than desired by a user, i.e. they may not be real-time processes. Accordingly, in some embodiments, one or more switches may be included in the rendering pipeline that bypasses the techniques described above.
In various embodiments of the present invention, switching on and off of the subsurface scattering effects described above may be made on different levels of granularity. For example, scattering effects may be switched on or off for different parts of object models, different object models, different scenes, different shots, and the like. Additionally, the switches may allow the user to globally determine a percentage of scattered illumination contributions and diffuse illumination contributions.
In some embodiments, a simpler simulation of the subsurface scattering process may be used to provide an approximation of the subsurface scattering, when the techniques described above are turned off. For example, a diffuse reflection in the illumination model may be used. As another example, the techniques described in U.S. patent application Ser. No. 10/810064 filed Mar. 25, 2004 may be used.
Embodiments of the present invention may be applied to any number of rendering platforms and for a variety of purposes. For example, embodiments may be use on engineering workstations for development purposes, on visualization systems for artists and animators, in a rendering farm machine for final production, and the like. Accordingly, the concepts disclosed above are extremely valuable in a variety of applications.
In the above disclosure, any resolution for the grids may be used. The inventor has determined that it is advantageous that the grid be larger than the smallest shaded surface. Further, the grid may be much greater in size. In one embodiment, a typical voxel may include two shading surface values to nine shading surface values. In other embodiments, voxels may include a smaller number of shading surface values, e.g. 0.5 or greater, and voxels may include a larger number of shading surface values, e.g. 10 to 25, 20 to 40, 30 or greater, and the like.
Further embodiments can be envisioned to one of ordinary skill in the art after reading the attached documents. In other embodiments, combinations or sub-combinations of the above disclosed embodiments can be advantageously made. The block diagrams of the architecture and flow charts are grouped for ease of understanding. However it should be understood that combinations of blocks, additions of new blocks, re-arrangement of blocks, and the like are contemplated in alternative embodiments of the present invention.
The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense. It will, however, be evident that various modifications and changes may be made thereunto without departing from the broader spirit and scope of the invention as set forth in the claims.
The present application is a continuation-in-part of and incorporates by reference for all purposes: application Ser. No. 10/428,322 filed Apr. 30, 2003. The present application also claims priority to and incorporates by reference for all purposes: Provisional Application No. 60/572,253 filed May 17, 2004 and Provisional Application No. 60/624,705 filed Nov. 2, 2004. The present application also incorporates by reference for all purposes: Application No. PCT/US04/08993 filed Mar. 23, 2004.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5936629 | Brown et al. | Aug 1999 | A |
| 6226005 | La Ferriere | May 2001 | B1 |
| 6281904 | Reinhardt et al. | Aug 2001 | B1 |
| 6313846 | Fenney et al. | Nov 2001 | B1 |
| 6330000 | Fenney et al. | Dec 2001 | B1 |
| 6456405 | Horikoshi et al. | Sep 2002 | B2 |
| 6512517 | Knittel et al. | Jan 2003 | B1 |
| 6624812 | Collodi | Sep 2003 | B1 |
| 6744435 | Zwicker et al. | Jun 2004 | B2 |
| 6867774 | Halmshaw et al. | Mar 2005 | B1 |
| 7019744 | Anderson | Mar 2006 | B2 |
| 7098910 | Petrovic et al. | Aug 2006 | B2 |
| 7129940 | Cook et al. | Oct 2006 | B2 |
| 7176918 | Anderson et al. | Feb 2007 | B2 |
| 7184043 | Anderson et al. | Feb 2007 | B2 |
| 20040217957 | Anderson et al. | Nov 2004 | A1 |
| 20040217961 | Cook et al. | Nov 2004 | A1 |
| 20040227757 | Petrovic et al. | Nov 2004 | A1 |
| 20040263511 | West et al. | Dec 2004 | A1 |
| 20050237334 | Chuter | Oct 2005 | A1 |
| 20050253842 | Petrovic et al. | Nov 2005 | A1 |
| 20060274065 | Buyanovskiy | Dec 2006 | A1 |
| Number | Date | Country | |
|---|---|---|---|
| 20050212795 A1 | Sep 2005 | US |
| Number | Date | Country | |
|---|---|---|---|
| 60572253 | May 2004 | US | |
| 60624705 | Nov 2004 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 10428322 | Apr 2003 | US |
| Child | 11084697 | US |
