Patent Application
20050275760
This disclosure is related to modifying a rasterized surface, such as for graphics and/or video processing, for example.
Computer graphics is an extensive field in which a significant amount of hardware and software development has taken place over the last twenty years or so. See, for example, Computer Graphics: Principles and Practice, by Foley, Van Dam, Feiner, and Hughes, published by Addison-Wesley, 1997. Typically, in a computer platform or other similar computing device, dedicated graphics hardware is employed in order to render graphical images, such as those used in connection with computer games, for example. For such systems, dedicated graphics hardware may be limited in a number of respects that have the potential to affect the quality of the graphics, including hardware flexibility and/or its rendering capability.
One issue that relates to graphics quality is the rendering of trimmed surfaces. In one approach, trimmed Non-uniform Rational B-spline (NURB) surfaces are rendered with Adaptive Forward Differencing. See “Rendering Trimmed NURBS with Adaptive Forward Differencing,” by Shantz and Chang, Computer Graphics, Vol. 22, No. 4, August 1988, pp 189-198. In this approach, adaptive forward differencing is extended to higher order, the basis matrix for each scan is computed, the shading approximation function for rational surfaces is calculated, and the NURB surfaces are trimmed and image mapped. Trimming is accomplished by using AFD to scan convert the trimming curves in parameter space, producing the intersection points between the trim curves and an isoparametric curve along the surface. A winding rule is used to determine the regions bounded by the curve which are then rendered with AFD. In another approach, all trimmed surfaces are converted into individual Bezier patches with trimming regions defined by closed loops of Bezier or piecewise linear curves. Step sizes are calculated in parameter space for each curve and surface which guarantee the size of facets in screen space will not exceed a user specified tolerance. All points on the trimming curves where the tangents are parallel to the u or v axes are discovered, here, the local minima and maxima. Using the extremes, the trimming region of the patch is divided into u,v-monotone regions. Each region is defined by a closes loop of curves. Using the calculated step sizes, each u,v-monotone region is uniformly tessellated into a grid of rectangles connected by triangles to points evaluated along the curves. The polygons defined in u,v parameter space are transformed into facets in object space by evaluating their vertices with the surface factions. Surface normals are also calculated. Each facet is transformed to screen space, clipped, lighted, smooth shaded and z-buffered using 3D graphics hardware. See “Real-Time Rendering of Trimmed Surfaces,” by Rockwood, Heaton, and Davis, Computer Graphics, Vol. 23, No. 3, July 1989, pp 107-116.
However, higher quality graphics continues to be desirable as the technology and the marketplace continues to evolve. Thus, signal processing and/or other techniques to extend the capability of existing hardware in terms of the quality graphics that may be produced continues to be an area of investigation.
Subject matter is particularly pointed out and distinctly claimed in the concluding portion of the specification. The claimed subject matter, however, both as to organization and method of operation, together with objects, features, and advantages thereof, may best be understood by reference of the following detailed description when read with the accompanying drawings in which:
Embodiments of methods, apparatuses, devices, and/or systems for modifying a rasterized surface, such as for graphics and/or video processing, for example, are described. For example, in accordance with one embodiment, a method of modifying a rasterized surface using dedicated graphics hardware is as follows. One or more trim regions are loaded in texture memory in a parameter space of the surface. A surface is rasterizied using said dedicated graphics hardware. Portions of the rasterized surface are modified based at least in part on the one or more trim regions.
In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the claimed subject matter. However, it will be understood by those skilled in the art that the claimed subject matter may be practiced without these specific details. In other instances, well-known methods, procedures, components and/or circuits have not been described in detail so as not to obscure the claimed subject matter.
Computer graphics is an extensive field in which a significant amount of hardware and software development has taken place over the last twenty years or so. See, for example, Computer Graphics: Principles and Practice, by Foley, Van Dam, Feiner, and Hughes, published by Addison-Wesley, 1997. Typically, in a computer platform or other similar computing device, dedicated graphics hardware is employed in order to render graphical images, such as those used in connection with computer games, for example. For such systems, dedicated graphics hardware may be limited in a number of respects that have the potential to affect the quality of the graphics, including hardware flexibility and/or its rendering capability. However, higher quality graphics continues to be desirable as the technology and the marketplace continues to evolve. Thus, signal processing and/or other techniques to extend the capability of existing hardware in terms of the quality graphics that may be produced continues to be an area of investigation.
As previously discussed, dedicated graphics hardware may be limited in its capabilities, such as its graphics rendering capabilities and/or its flexibility. This may be due at least in part, for example, to the cost of hardware providing improved abilities relative to the demand for such hardware. Despite this, however, in recent years, the capabilities of dedicated graphics hardware provided on state-of-the-art computer platforms and/or similar computing systems have improved and continue to improve. For example, fixed function pipelines have been replaced with programmable vertex and fragment processing stages. As recently as 6 years ago, most consumer three-dimensional (3D) graphics operations were principally calculated on a CPU and the graphics card primarily displayed the result as a frame buffer. However, dedicated graphics hardware has evolving into a graphics pipeline comprising tens of millions of transistors. Today, a programmable graphics processing unit (GPU) is capable of more than simply feed-forward triangle rendering. State-of-the art graphics chips, such as the NVIDIA GeForce FX and the ATI Radon 9000, for example, replace fixed-function vertex and fragment processing stages with programmable stages, as described in more detail hereinafter. These programmable vertex and fragment processing stages have the capability to execute programs allowing control over shading and/or texturing calculations, as described in more detail hereinafter.
Similar to CPU architectures, a GPU may be broken down into pipeline stages. However, whereas a CPU embodies a general purpose design used to execute arbitrary programs, a GPU is architected to process raw geometry data and eventually represent that information as pixels on a display, such as a monitor, for example.
Typically, for an object to be drawn, the following operations are executed by such a pipeline:
1. An application executing on a CPU may instruct a GPU where to find vertex data, such as 105, within a portion of memory.
2. Vertex stage 110 may transform the vertex data from model space to clip space and may perform lighting calculations, etc.
3. Vertex stage 110 may generate texture coordinates from mathematical formulae.
4. Primitives, such as triangle, points, quadrangles, and the like, may be rasterized into fragments.
5. Fragment color may be determined by processing fragments through fragment processing stage 180, which may also perform, among other operations, texture memory look-ups.
6. Some tests may be performed to determine if fragments should be discarded.
7. Pixel color may be calculated based at least in part on fragment color and other operations typically involving fragments' or pixels' alpha channel.
8. Pixel information may be provided to frame buffer 160.
9. Pixels may be displayed, such as by display 170.
As illustrated by block 115 of
As illustrated by block 120 and previously suggested, a graphics pipeline typically will perform transform and lighting (T & L) operations and the like. Block 120 depicts a fixed-function unit; however, these operations are being replaced more and more by programmable vertex units, such as 130, also referred to as vertex shaders. Vertex shader 130 applies a vertex program to a stream of vertices. Therefore, the program processes data at the vertex level. Most operations are performed in one cycle, although this restriction need not apply. A typical vertex program is on the order of a hundred or more instructions.
As with the vertex stage, the fragment processing stage has undergone an evolution from a fixed function unit, such as illustrated by block 140, to a programmable unit, such as illustrated by block 150. Thus, previously, texturing, filtering and blending were performed using fixed function state machines or similar hardware. As with vertex shaders, a pixel shader, such as 150, also referred to as a programmable fragment processing stage, permits customized programming control. Therefore, on a per pixel basis, a programmer is able to compute color and the like to produce desired customized visual effects.
These trends in programmability of the graphics pipeline have transformed the graphics processing unit (GPU) and its potential applications. Thus, one potential application of such a processor or processing unit is to accomplish high quality graphics processing, such as may be desirable for a variety of different situations, such as for creating animation and the like, for example. More specifically, in recent years, the performance of graphics hardware has increased more rapidly than that of central processing units (CPUs). As previously indicated, CPU designs are typically intended for high performance processing on sequential code. It is, therefore, becoming increasingly more challenging to use additional transistors to improve processing performance. In contrast, as just illustrated, programmable graphics hardware is designed for parallel processing of vertex and fragment stage code. As a result, GPUs are able to use additional transistors more effectively than CPUs to produce processing performance improvements. Thus, GPUs offer the potential to sustain processing performance improvements as semiconductor fabrication technology continues to advance.
Of course, programmability is a relatively recent innovation. Furthermore, a range of differing capabilities are included within the context of “programmability.” For the discussion of this particular embodiment, focus will be placed upon the fragment processing stage of the GPU rather than the vertex stage, although, of course, the claimed subject matter is not limited in scope in this respect. Thus, in one embodiment, a programmable GPU may comprise a fragment processing stage that has a simple instruction set. Fragment program data types may primarily comprise fixed point input textures. Output frame buffer colors may typically comprise eight bits per color component. Likewise, a stage typically may have a limited number of data input elements and data output elements, a limited number of active textures, and a limited number of dependent textures. Furthermore, the number of registers and the number of instructions for a single program may be relatively short. The hardware may only permit certain instructions for computing texture addresses only at certain points within the program. The hardware may only permit a single color value to be written to the frame buffer for a given pass, and programs may not loop or execute conditional branching instructions. In this context, an embodiment of a GPU with this level of capability or a similar level of capability shall be referred to as a fixed point programmable GPU.
In contrast, more advanced dedicated graphics processors or dedicated graphics hardware may comprise more enhanced features. The fragment processing stage may be programmable with floating point instructions and/or registers, for example. Likewise, floating point texture frame buffer formats may be available. Fragment programs may be formed from a set of assembly language level instructions capable of executing a variety of manipulations. Such programs may be relatively long, such as on the order of hundreds of instructions or more. Texture lookups may be permitted within a fragment program, and there may, in some embodiments, be no limits on the number of texture fetches or the number of levels of texture dependencies within a program. The fragment program may have the capability to write directly to texture memory and/or a stencil buffer and may have the capability to write a floating point vector to the frame buffer, such as RGBA, for example. In this context, an embodiment of a GPU with this level of capability or a similar level of capability may be referred to as a floating point programmable GPU.
Likewise, a third embodiment or instantiation of dedicated graphics hardware shall be referred to here as a programmable streaming processor. A programmable streaming processor comprises a processor in which a data stream is applied to the processor and the processor executes similar computations or processing on the elements of the data stream. The system may execute, therefore, a program or kernel by applying it to the elements of the stream and by providing the processing results in an output stream. In this context, likewise, a programmable streaming processor which focuses primarily on processing streams of fragments comprises a programmable streaming fragment processor. In such a processor, a complete instruction set and larger data types may be provided. It is noted, however, that even in a streaming processor, loops and conditional branching are typically not capable of being executed without intervention originating external to the dedicated graphics hardware, such as from a CPU, for example. Again, an embodiment of a GPU with this level of capability or a similar level comprises a programmable streaming processor in this context.
In this particular embodiment, GPU 210 may comprise any instantiation of a programmable GPU, such as, for example, one of the three previously described embodiments, although for the purposes of this discussion, it is assumed that GPU 210 comprises a programmable floating point GPU. Likewise, it is, of course, appreciated that the claimed subject matter is not limited in scope to only the three types of GPUs previously described. These three are merely provided as illustrations of typical programmable GPUs. All other types of programmable GPUs currently known or to be developed later are included within the scope of the claimed subject matter. For example, while
Likewise, for this simplified embodiment, system 200 comprises a CPU 230 and a GPU 210. In this particular embodiment, memory 240 comprises random access memory or RAM, although the claimed subject matter is not limited in scope in this respect. Any one of a variety of types of memory currently known or to be developed may be employed. It is noted that memory 240 includes frame buffer 250 in this particular embodiment, although, again, the claimed subject matter is not limited in scope in this respect. For example,
It is worth repeating that
As previously suggested and as shall be discussed in more detail, in this particular embodiment, a three-dimensional (3D) surface is rasterized using dedicated graphics hardware. Likewise, one or more trim regions are rasterized in a parametric space of the particular surface. These trim regions are loaded in texture memory of the dedicated graphics hardware, such as memory 540 illustrated in
Although the claimed subject matter is not limited in scope to method embodiment 300 illustrated in
Referring now to block 310 of
At block 340, GPU 210 then uses the one or more trim regions, contained in texture memory, to trim portions of the rasterized surface or patch. In one particular embodiment, although the claimed subject matter is not limited in scope in this respect, the GPU may employ fragment shading, e.g., a technique to produce shading via a fragment program, to modulate alpha and/or color at least in part based upon the loaded one or more trim regions. Fragment shading by the GPU is illustrated for this particular embodiment schematically in
Thus, the opacity or transparency of the patch may be modulated, for example, at corresponding patch locations based at least in part on the trim regions. Of course, the claimed subject matter is not limited in scope to this particular approach. For example, in alternative embodiments, rather than modulating opacity, for example, instead, the appropriate pixel values may be discarded or otherwise processed by the fragment stage so that the trim regions portions of the patch will no longer be visibly apparent when the object is displayed, thereby producing a trimmed surface. For example, the fragment program may “kill” the fragment if appropriate portions of the one or more trim regions have corresponding patch locations in the rasterization of the surface. Of course, in alternative embodiments within the scope of the claimed subject matter, the surface may also be modified in a manner so that the trim region portions of the patch remain at least partially visible. The resulting three-deimensional patch using the trim regions to modulate opacity, for this particular embodiment, is illustrated conceptually at subfigure (c) of
Referring again to
Although the claimed subject matter is not limited in scope in this respect, one approach to determining whether a resolution is sufficiently fine for rasterizing a trim region in the u-v parameter space of the patch or surface may be based, at least in part, on the size of the patch when rasterized on a display. It may be desirable, for example, to choose a resolution sufficiently fine so that texels in the trim region will have a sub-pixel size when displayed. Although the claimed subject matter is not limited in scope in this respect, such an approach is similar to the choice of tessellation rates, for example, employed in Reyes-like rendering. See, for example, “The Reyes Image Rendering Architecture,” by R. L. Cook, L. Carpender, and E. Catmull, SIGGRAPH 87, 95-102.
Although the claimed subject matter is not limited in scope to this particular embodiment, it has the potential to provide a number of advantages. As previously discussed, in one potential embodiment, resolution for one or more trim regions may be adjusted. Adjusting resolution allows quality graphics to be achieved. Additionally, the previously described embodiment is fast when compared with alternate approaches. Therefore, in addition to improving quality, such an approach may be suitable for real time processing, such as for computer graphics and/or computer games, as previously indicated. As was suggested, graphics pipelines have been developed to have the ability to quickly and efficiently perform particular types of computations and calculations. Such computations and calculations include the rasterization of trim regions previously described. By way of contrast, if a CPU, rather than a GPU, were to attempt these types of computations, it would likely be more time consuming. Thus, in this particular embodiment, the ability of a GPU to rasterize curves and/or lines, and perform additional filtering, shading and the like quickly and efficiently has, in this context, been leveraged. Furthermore, the approach of this particular embodiment, previously discussed, in which a patch or surface is rendered without trim regions allows a high quality representation of the patch to be rendered quickly and efficiently before attempting modification of the surface. In contrast, other approaches involving tessellation of the trim regions via the CPU are likely to degrade quality and speed. Of course, while these may be particular advantages, as previously indicated, the claimed subject matter is not limited in scope to this embodiment or to any particular embodiment. Likewise, therefore, the claimed subject matter is not limit to achieving these particular advantages.
It is, of course, now appreciated, based at least in part on the foregoing disclosure, that software may be produced capable of producing the desired graphics processing. It will, of course, also be understood that, although particular embodiments have just been described, the claimed subject matter is not limited in scope to a particular embodiment or implementation. For example, one embodiment may be in hardware, such as implemented to operate on a device or combination of devices as previously described, for example, whereas another embodiment may be in software. Likewise, an embodiment may be implemented in firmware, or with any combination of hardware, software, and/or firmware, for example. Likewise, although the claimed subject matter is not limited in scope in this respect, one embodiment may comprise one or more articles, such as a storage medium or storage media. This storage media, such as, one or more CD-ROMs and/or disks, for example, may have stored thereon instructions, that when executed by a system, such as a computer system, a computing platform, a GPU, a CPU, another device or system, or combinations thereof, for example, may result in an embodiment of a method in accordance with the claimed subject matter being executed, such as one of the embodiments previously described, for example. As one potential example, a computing platform may include one or more processing units or processors, one or more input/output devices, such as a display, a keyboard and/or a mouse, and/or one or more memories, such as static random access memory, dynamic random access memory, flash memory, and/or a hard drive, although, again, the claimed subject matter is not limited in scope to this example.
In the preceding description, various aspects of the claimed subject matter have been described. For purposes of explanation, specific numbers, systems and/or configurations were set forth to provide a thorough understanding of the claimed subject matter. However, it should be apparent to one skilled in the art having the benefit of this disclosure that the claimed subject matter may be practiced without the specific details. In other instances, well-known features were omitted and/or simplified so as not to obscure the claimed subject matter. While certain features have been illustrated and/or described herein, many modifications, substitutions, changes and/or equivalents will now occur to those skilled in the art. It is, therefore, to be understood that the appended claims are intended to cover all such modifications and/or changes as fall within the true spirit of the claimed subject matter.
