Patent Application
20060066624
The invention relates generally to computers, and more particularly to images rendered by computers.
Many modern computers use displays intensively to display images. An image is a graphical representation that is rendered by a renderer, such as a graphics card. Numerical precision errors, algorithmic mistakes, and other factors may cause an image to be displayed differently on different renderers. In the past, a rendered image was considered “correct” if the rendered image had at least a selected percentage of pixels that were identical with a reference image rendered by a reference renderer. This measure of “correctness” measures the differences between a reference image and a rendered image, not whether the rendered image was rendered correctly. What is needed is a better method and system for measuring the correctness of rendered images.
Briefly, the present invention provides a method and system for measuring the correctness of rendered images. A reference renderer generates a reference image from data describing the image. A tolerance buffer is generated that includes a tolerance for a pixel representing a portion of the reference image. If a renderer renders the pixel outside the tolerance for the pixel, the renderer is deemed to have rendered the pixel incorrectly. If the renderer renders the pixel within the tolerance, the renderer is deemed to have rendered the pixel correctly.
In one aspect, the tolerance buffer may include a tolerance for each pixel comprising the reference image. If the renderer renders each pixel in the reference image within the tolerance associated therewith, the renderer is deemed to have rendered the image correctly. If the renderer renders one or more pixels outside the tolerances associated therewith, the renderer is deemed to have rendered the image incorrectly.
In another aspect, the tolerance for each pixel may be generated by determining the maximum error bound associated with each pixel. This maximum error bound may be affected by any of a number of numerical precision and other errors.
Other aspects will become apparent from the following detailed description when taken in conjunction with the drawings, in which:
Exemplary Operating Environment
The invention is operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known computing systems, environments, and/or configurations that may be suitable for use with the invention include, but are not limited to, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microcontroller-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
The invention may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and so forth, which perform particular tasks or implement particular abstract data types. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
With reference to
Computer 110 typically includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by the computer 110 and includes both volatile and nonvolatile media, and removable and non-removable media. By way of example, and not limitation, computer-readable media may comprise computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by the computer 110. Communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the any of the above should also be included within the scope of computer-readable media.
The system memory 130 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 131 and random access memory (RAM) 132. A basic input/output system 133 (BIOS), containing the basic routines that help to transfer information between elements within computer 110, such as during start-up, is typically stored in ROM 131. RAM 132 typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 120. By way of example, and not limitation,
The computer 110 may also include other removable/non-removable, volatile/nonvolatile computer storage media. By way of example only,
The drives and their associated computer storage media, discussed above and illustrated in
The computer 110 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 180. The remote computer 180 may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer 110, although only a memory storage device 181 has been illustrated in
When used in a LAN networking environment, the computer 110 is connected to the LAN 171 through a network interface or adapter 170. When used in a WAN networking environment, the computer 110 typically includes a modem 172 or other means for establishing communications over the WAN 173, such as the Internet. The modem 172, which may be internal or external, may be connected to the system bus 121 via the user input interface 160 or other appropriate mechanism. In a networked environment, program modules depicted relative to the computer 110, or portions thereof, may be stored in the remote memory storage device. By way of example, and not limitation,
Measuring Correctness in Rendered Images
To display the triangle 210, the display may cause pixels included within the triangle 210 to display one or more colors. Although the triangle 210 is shown having darkened circles on each of its vertices, these darkened circles may not be displayed when the triangle is rendered. Likewise, the borders of the triangle 210 may not be displayed as continuous lines but rather by causing discrete pixels that lie inside the border of the triangle 210 to display colors.
Many errors may occur when an image is rendered by a particular renderer. These errors may be related to lighting, texturing, and transformation. Some of these errors are a result of numerical precision errors such as rounding or truncation that occurs as the renderer renders the image. For example, a rounding error may cause a particular pixel to have a certain color because it is determined that the pixel is part of a shape while full accuracy would dictate that the pixel have another color because it is not part of the shape.
Other errors may occur as a result of an algorithmic mistake in which the algorithm a renderer uses is flawed in some manner or another. For example, a renderer may improperly implement a line drawing algorithm or may cause certain pixels to have certain values, but only if the pixels are in certain locations on the display.
Some errors are tolerable. For example, a pixel that lies on the edge of a white triangle having a black background may be white or black depending on whether it is determined that the pixel lies within or without the triangle. In this case, it may be equally acceptable to have the pixel display black or white because of the numerical precision of the renderers which will render the image. There may be, on the other hand, no tolerance for error for a rendering a pixel that lies in the center of the triangle as the pixel should display white on every renderer. If such a pixel displays anything but white, this may indicate that a renderer is unacceptably flawed and needs to be fixed.
An error in rendering a shape may occur in determining whether a pixel, or more accurately the point at which the pixel is sampled, lies within the mathematical boundary defined by the shape. Frequently (and as assumed below for clarity), pixels are sampled at their center; so it is this point which is used to determine whether each pixel is inside a shape. It will be understood, however, that pixels may be sampled at a variety of other points which may then be used to determine whether each pixel is inside a shape without departing from the spirit or scope of the invention.
When a shape, such as a triangle 305, encloses a pixel center, such as a pixel center 315, a pixel (i.e., pixel 320) associated with the pixel center 315 may be caused to display a color associated with the shape. If the pixel center 315 is determined not to lie within the triangle 305, the pixel may be caused to display a default color (unless the pixel is included in another shape that is to be displayed).
This type of error may be referred to as a per-pixel error. Other per-pixel errors include, for example, errors due to geometry transformation imprecision, per-pixel lighting errors, and “z-buffer aliasing” artifacts. Geometry transformation imprecision occurs when a pixel (or the shape including the pixel) is transformed through some operation from one location to another location. For example, a triangle may be rotated or viewed from a different perspective which may cause a per-pixel error of various pixels through numerical precision.
Per-pixel lighting errors may result from imprecise lighting calculations. For example, referring to
Even though a display typically shows images on a two-dimensional surface, displaying the objects in the image may involve three-dimensional analysis in which objects may overlap. Each pixel from each object may be placed in a z ordering that indicates which pixel should be displayed if the pixel overlaps a pixel from another object. A z-buffer aliasing artifact may occur when two shapes are at almost the same distance from a point of view. For example, if two triangles lie almost on top of each other, it is possible that portions of both triangles may be determined by a renderer as being closer to the point of view. If one triangle is black and another triangle is white, a renderer may render an image that includes portions from both triangles that sharply contrast with each other.
In each of the examples described in
In an embodiment of the invention, there may be a tolerance allotted for each pixel in an image. The tolerance may be calculated based on allowable truncating and rounding errors. If a rendered pixel is within a tolerance from its associated reference pixel, the rendered pixel may be considered acceptable. If a rendered pixel is not within a tolerance from the reference pixel, the rendered pixel may be considered unacceptable.
To determine acceptable and unacceptable image renderings, a tolerance buffer may be associated with a frame buffer (sometimes called a bitmap of an image) wherein there is a one-to-one correspondence between representations of pixels in the frame buffer (sometimes referred to simply as pixels) and tolerances for pixels in the tolerance buffer. A pixel in the frame buffer may have a set of values (sometimes referred to as a pixel vector) associated with different attributes of the pixel such as color and transparency channels. For example, in one implementation, a pixel in the frame buffer may have values that range between 0 and 255 for each attribute of transparency, red, green, and blue.
Likewise, a tolerance vector in the tolerance buffer may have a value associated with each value of a pixel vector for a pixel in the frame buffer. If a reference pixel in the frame buffer of a reference image has a pixel vector having attributes [transparency, red, green, blue] of CREF=[A,R,G,B] and an associated tolerance vector of an associated tolerance buffer has values of T0=[εA, εR, εG, εB], then a rendered pixel, CREN that corresponds to the reference pixel of the rendered image is correct if for each value of the associated vectors, CREF−T0≦CREN≦CREF+T0. Another way of representing this condition is that if the absolute value of (CREN−CREF)≦T0, for each value of the associated vectors, then the rendered pixel is correct.
Note that if any of the values of a rendered pixel vector is outside of the tolerance of its associated value of the tolerance vector, then the pixel is incorrect. In other words, if the absolute difference between a value of the rendered pixel vector and its associated value in the reference pixel vector is greater than the associated value of the tolerance vector, then the rendered pixel is incorrect. For example, if CREF=[255,20,30,40], T0=[0,5,3,2], and CREN=[255,20,30,37], then CREN is incorrect as the difference between the blue values (i.e., the absolute value of 40−37=3) is greater than the blue tolerance value (i.e., 2), even though all of the other values match.
Further note that if T0 is set to [0,0,0,0] for a particular pixel that a rendered pixel for that pixel will be incorrect if its pixel vector is any different from the pixel vector of the corresponding pixel in the reference image. In pixel representations in which the maximum any value of a pixel vector may be is 255, if T0 is set to [255,255,255,255], then a rendered pixel is essentially ignored with respect to correctness. That is, if T0 is set to a maximum value for a particular pixel, it does not matter how a renderer renders the particular pixel.
A pixel may be associated with a tolerance vector having a maximum tolerance, if it is substantially equally likely that the pixel may be rendered in any one of two ways. For example, a pixel (e.g., pixel 320) with a pixel center that is right by a border of the triangle 305 may be included or not included in the triangle depending on how accurate the rendering engine is. Furthermore, it may be determined that either including or not including the pixel is a correct rendering as the pixel center 315 is so close to the border of the triangle 305. If, however, the triangle is white, C=[255,255,255,255], and the pixel will be left at a default value of black, i.e., C=[255,0,0,0] if it is not included in the triangle, then there is a stark difference between the outcome of determining whether the pixel center is within the triangle or without.
Furthermore, a pixel may be associated with a tolerance vector that has a maximum tolerance for fewer than all of the values of the tolerance vector. For example, a red triangle may have a reference color CREF=[255,255,0,0] and may have a side very close to a center of a pixel. To allow the pixel to be colored red or not at all, a tolerance vector for the pixel may be set to [0,255,0,0]. Then, an un-tolerated rendering error occurs if the pixel is rendered in any color other than red. Likewise, if the triangle consists of a combination of two colors, the tolerance vector of a pixel close to the edge of the triangle may have two color values with maximum tolerance.
A tolerance vector for a pixel may accumulate tolerances from many sources. For example, three cones of light may each be close to encircling a pixel. One cone may have a color of [255,20,0,0], another cone may have a color of [255,0,30,0], and another cone may have a color of [255,0,0,40]. To account for the possibility that acceptable rounding or truncating errors may cause the pixel to be encircled by all three lights or none of the lights a tolerance vector associated with the pixel may be set to [0,20,30,40].
If the same pixel was also close to the boundary of a triangle having a color of [255,10,30,80], the tolerance vector may be set to [0,20,30,80] to tolerate the possibility that the pixel is encircled by all three light sources and is part of the triangle.
In general, a tolerance vector may represent an upper bound of the error that may be expected at a rendered pixel associated with the pixel. If the rendered pixel is outside this bound, either the renderer has made an unacceptable mistake or an error that was not accounted for in the tolerance vector occurred. If the former condition exists, the mistake of the renderer may be found and fixed. If the later condition exists, the error may be added to the tolerance vector.
Another set of errors that may affect the tolerance of a pixel or set of pixels are per-vertex errors. In general, the tolerance vector of these pixels may be computed in the same manner in which colors are modified in the reference image. For example, if an operation related to per-vertex errors changes color values in a pixel by certain percentages, the color values in the tolerance vector may also be modified by the percentages.
For example, vertex 215 of triangle 210 of
When, however, the triangle 210 is viewed from a perspective other than the one described above, error may be distributed to tolerance vectors associated with pixels included in the triangle by interpolating the error between each pair of vertices in a perspective-correct manner. In this perspective-correct manner, the error vector for each pixel included in the triangle may be modified in the same way the pixels are modified when doing perspective-correct Gourad shading.
In other words, the tolerance buffer may be treated as an image that is modified in the same way that a frame buffer of a reference image is modified. If a pixel in the frame buffer is overwritten, its associated tolerance vector in the tolerance buffer is overwritten. If a pixel is blended in the frame buffer (e.g., through transparency effects), the values in an associated tolerance vector in the tolerance buffer may be blended in the same way. If a vertex having a pixel vector of [255,40,60,80] and an associated tolerance vector of [0,10,20,30] causes a pixel to have a pixel vector of [255,20,30,40] though perspective-correct shading, then the associated tolerance vector may be modified in the same manner to [0,5,10,15]. The tolerance buffer may be viewed as another image wherein the color values of each pixel represents tolerance. The same transformations that apply to the frame buffer may be applied to the tolerance buffer.
Another per-vertex error is per-vertex lighting error. This error may also be caused by numerical precision issues in calculating the cone of a spot-light, the range of a local light, or the transformation of a vertex normal. In any case, the tolerance vector at a vertex may then be blended in a perspective correct way across the tolerance vectors associated with pixels on adjacent faces to the vertex. Once the error is blended across adjacent faces it can be added to the per-pixel error of the pixels which make up those faces.
If a pixel is partially or totally transparent, one or more pixels that lie beneath the pixel (in the z ordering) may influence the final display values of the displayed pixel. For example, in one implementation, a transparency value of 255 may represent that a pixel is not transparent (i.e., opaque) while a transparency value of 0 may represent that a pixel is completely transparent. If a pixel is partially transparent, the tolerance vector associated with the pixel may be comprised of tolerances associated with the pixel and any pixels that show through the pixel. For example, if a pixel has a pixel vector of [127,50,30,20] and a pixel that shows through the pixel has a pixel vector of [255,0,0,0], the resulting pixel may have a pixel vector of [255,25,15,10]. Similarly, if the foreground pixel would generate a tolerance vector of [0,40,20,10] and a pixel behind the pixel would generate a tolerance vector of [0,0,0,0], the resulting tolerance vector may be [0,20,10,5].
Some renderers may perform operations typically performed on a per-vertex basis on a per-pixel basis. For example, lighting may be performed on a per-pixel basis rather than a per-vertex basis. In this case, the per-vertex corrections described above for lighting may not be done. Rather, the tolerance may be computed or adjusted as each pixel is changed based on a per-pixel operation. Similarly, in other typical per-vertex operations that are instead done on a per-pixel basis, the tolerance may be computed on a per-pixel basis.
Some errors may result from texture that is placed on an image. Texture allows pictures to be placed on triangles or other shapes used to draw the image. The texture may have sharp boundaries that lie close to pixel centers. As a result, on different renderers, small numerical imprecision may result in drastically different color values at a given pixel location. To deal with this error, a low-pass filter may be applied to the texture before applying the texture to the image and computing the tolerance buffer. By removing the high frequencies from the textures, the tolerance buffer may exclude errors caused by texturing inaccuracy.
Some renderers may have programmable logic that allows a programmer to determine how objects are rendered. If it is determined that steps within the programmable logic introduce error that could cause multiple outcomes based on numerical precision of the renderer, the tolerance buffer may take into account these errors by having specified tolerances related to the steps.
The error sources described above are exemplary and are not intended to limit the error sources that may be used to compute a tolerance buffer. Indeed, any number of sources of errors may adjust the tolerance vector associated with particular pixel without departing from the spirit or scope of the invention.
Turning to
At block 530, the tolerance buffer may be updated for any other errors. For example, a particular renderer may render pixels on the edge of a display with color values of two less than they should be. If a pixel on the edge of the display is set to black, no adjustment may be needed. Otherwise, to avoid indicating that the renderer is flawed, the values of tolerance vectors associated with pixels on the edge of the display may be increased by two.
At block 535, the process returns to the calling process.
Turning to
At block 620, a determination is made as to whether the pixel replaces another already-rendered pixel. For example, a pixel from an object that is further from a point of view may be drawn before a pixel that is from an object that is closer to the point of view. The two pixels may also map to the same location on the display or frame buffer. If so, the pixel from the closer object overwrites the further away pixel. Similarly, the tolerance vector associated with the pixel is overwritten by the tolerances computed for the new pixel as indicated in block 625.
At block 630, a determination is made as to whether the rendering will affect one or more values of any existing pixel. For example, the rendering of a pixel for an object that is beneath a partially transparent pixel of another object may adjust values of a pixel vector as described previously. If this occurs, the tolerance vector associated with the existing pixel is changed in the same manner (e.g., by the same percentage or percentages) as how the values of the pixel vector are changed in the frame buffer as indicated in block 635.
At block 640, a tolerance vector is updated based on the rendered pixel. Each time the pixel is affected by yet another object, the tolerance vector may again be updated to include the tolerance associated with the rendering of a pixel associated with the other object. For example, if multiple light sources may encompass a pixel center, the tolerance vector associated with the pixel may be updated as each light source is rendered.
At block 645, a determination is made as to whether another pixel is in the object. If so, processing branches to block 650; otherwise, processing branches to block 655 at which the process returns to the calling process.
Turning to
While examples have been shown with pixel vectors having four attributes (i.e., transparency and red, green, and blue channels), it will be recognized that aspects of the invention may be applied to pixel vectors having more or fewer attributes without departing from the spirit or scope of the invention. Furthermore, it will be recognized that aspects of the invention may be applied to attributes other than the four attributes mentioned above. In addition, while aspects of the invention have been described in vectors that may be represented with 8 bits per attribute, it will be readily recognized that more or less than 8 bits per attribute may be employed without departing from the spirit or scope of the invention.
It will be recognized that there are advantages associated with various aspects of the present invention. For example, whereas some rendering detection techniques are only able to detect whether a certain number of pixels are incorrect, aspects of the present invention may be used to detect whether each pixel is rendered within tolerance.
As can be seen from the foregoing detailed description, there is provided an improved method and system for measuring the correctness of rendered images. While the invention is susceptible to various modifications and alternative constructions, certain illustrated embodiments thereof are shown in the drawings and have been described above in detail. It should be understood, however, that there is no intention to limit the invention to the specific forms disclosed, but on the contrary, the intention is to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope of the invention.
