Patent Application
20060106591
The present invention relates generally to systems running applications that benefit from the incorporation of physics-based animations or physics data. More particularly, the present invention relates to Physics Processing Unit (PPU) enhanced systems incorporating a Graphics Processing Unit (GPU).
PPU enhanced systems have recently been developed. Systems readily adapted to PPU enhancement include, as examples, Personal Computers (PCs) and game consoles. Generally speaking, the performance capabilities of a conventional Central Processing Unit (CPU), such as a Pentium® or similar microprocessor, is enhanced by the additional provision of a PPU. Several exemplary embodiments of basic, PPU-enhanced system are disclosed in above referenced U.S. patent application.
The term “PPU-enhanced” generally describes any system incorporating a PPU to generate physics data for consumption by a main application running on a CPU. “Physics data” comprises any data or data structure related to a mathematical algorithm or logical/mathematical expression adapted to solve a physics problem or express a physics relationship.
Any set of logical computations or algorithms operating upon physics data is termed a physics “simulation.” A simulation generally runs on the PPU in cooperation with the CPU to generate a body of physics data that accurately defines the movement and/or interaction of objects and features in an animated scene displayed by a peripheral device associated with the system. So, in one sense the physics simulation run primarily on the PPU can be said to visually enhance the animation of a scene generated by the main application running on the CPU.
Such computationally derived physical animations form an increasingly important aspect of numerous applications. Computer games are an excellent example of applications that benefit from the added realism of animations derived from a defined set of physics-based inputs, parameters, and data. The term “animation” is used here to generally describe any visual representation of an event. The term “physics-based animation” refers to any animation derived, at least in part, from data provided by a physics simulation. A simulation is often said to drive the resulting animation. However, the direct relationship between simulation and animation, and the fact that the underlying simulation is not apparent to the system user typically results in an alternative use for the terms animation and simulation.
Cutting edge applications generally demand that physics-based animations, and the underlying simulations run in real-time. This requirement poses a significant problem for conventional systems. For example, conventional PCs are able to resolve only a limited amount of physics data in the time allowed by real time animation frame rates. This disability arises from structural limitations in the CPU architecture, data transfer bandwidth limitations, and the computational workload placed upon the CPU by other processes inherent in the execution of the main application.
For clarity of reference, the term “system” subsumes the term “host system.” A system may include a PPU, whereas the term “host system” generally refers to the combination of at least a CPU and an associated main memory. This combination of “host system” elements interacts with the PPU in a system.
Host systems almost always include a GPU. That is, many contemporary applications include significant graphics content and are intended to run with the aid of a separate GPU. GPUs are well know in the industry and are specifically designed to run in cooperation with a CPU to create (or “render”) animations having a three dimensional (3-D) quality.
PPU enhancement of a system implicates a number of changes in the way data is transferred between various system components. Non-PPU-enhanced systems have the luxury of a single computational platform (e.g., the CPU) generating data for consumption by a GPU. This is not the case in a PPU-enhanced system. Thus, new approaches to overall system architecture are required. Related data transfers must also be carefully considered.
In one aspect, the present invention provides a PPU-enhanced system in which animation data is efficiently generated using the computational resources provided by either the CPU or the PPU. As a result, CPU resources are made available for other tasks associated with the execution of an application.
In a related aspect, data is transferred in a system comprising a CPU and a main memory connected within the system through a chip set. A GPU and PPU may be connected within the system through a local bus, such as a PCI bus.
Animation data used by the GPU to render a scene on a display device is preferably stored in a GPU memory adapted to receive animation data from the main memory, or optionally, directly from PPU memory.
In yet another related aspect, the system provided by the present invention may connect the PPU and GPU though a single local bus or multiple buses. In one preferred embodiment the GPU is connected through a high bandwidth bus, such as an AGP bus, and the PPU is connected through a PCI or PCI express bus.
In another aspect, the present invention provides a system comprising a processor. The processor comprises a CPU and a GPU connected to an L2 cache and a bus interface, a Input/Output (I/O) interface connected to the bus interface, and a memory controller connected to the bus interface. The system also comprises a main memory connected to the processor through the memory controller, a chip set connecting a local bus to the processor through the I/O interface, and a PPU connected within the system through the local bus.
In still another aspect, the present invention provides a system comprising a processor like the one described above, except for the addition of another data processing element (or core) implementing the PPU. That is, the PPU is pulled into the processor as a separate processing core logically grouped with the CPU and GPU processing cores. In this preferred embodiment, the CPU, GPU, and PPU receive data from a common L2 cache and/or a main system memory.
The present invention further provides a method of rendering an animation in a system like any ones described herein. The method preferably comprises storing application data in the main memory, transferring simulation data from the main memory to a memory associated with the PPU, running a physics simulation on the PPU in relation to the simulation data, returning physics data from PPU memory to the main memory, transferring animation data from the main memory to a memory associated with the GPU, and rendering the animation in relation to the animation data stored in GPU memory.
The present invention still further provides a method of rendering an animation on a display device associated with a system like the ones described herein in which the CPU generates a first portion of the animation while the PPU is generating physics data. At least the first portion of the animation data is stored in memory and the animation is rendered by the GPU using the animation data. The physics data may be a second portion of the animation data, or be used by the CPU to generate a second portion of the animation data.
The drawings, taken together with the foregoing discussion, the incorporated references, and the detailed description that follows describe several preferred embodiments of the present invention. The drawings include the following:
The present invention extends the former recognition that conventional software-based solutions to physics simulations have limits that affect their practical usefulness within the execution of emerging applications. Graphics-intensive applications, such as PC and console games, benefit greatly from the real-time incorporation of many more active objects and features (hereafter “object/features”) and related forces than can be reasonably simulated using specialty software run on a general purpose CPU.
Accordingly, the present invention approaches the problem of generating complex, visually realistic animations, and therefore executing the underlying software components—both graphics related and physics related—from an entirely different perspective. Unlike conventional solutions, the present invention preferably relies on a hardware-based Physics Processing Unit (PPU). This additional hardware may be provided by a separate processing unit connected within the system via a local bus, or by a separate processing core provided in a multi-core processing unit. However configured, and in cooperation with the CPU provided by a host system, the PPU provides the enormous, additional, and highly specialized processing capabilities required to implement complex, real-time, physics-enhanced applications and render the associated graphics.
The additional computation capabilities provided by the PPU may be applied to the generation of “animation data” as well as “physics data.” The general computation and transfer of physics data, as well as the instruction programming necessary to execute related physics subroutines within a PPU-enhanced system, have been described in various exemplary embodiments contain in commonly assigned U.S. Patent Applications [Attorney Docket No. AGEIA-009] and [AGEIA-010] concurrently filed on Nov. 8, 2004. The subject matter of these applications is hereby incorporated by reference.
The terms “animation data” and “physics data” are used to distinguish potential differences inherent in the generation and transfer of various data types within the system. In some circumstances, physics data and animation data are directly related one to another. In other circumstances, physics data and animation data are completely unrelated. Thus, the distinction between these two data types is meant to be descriptive rather than limiting.
Generally speaking, animation data defines the visual representation of object/features in a scene as rendered by the GPU. Animation data includes, for example, data defining the position, orientation, and/or movement of object/features in the scene. Animation data also includes data defining the shape of deformable object/features (e.g., soft objects, cloth, fluid, smoke, vegetation, etc.). Several aspects related to the generation and use of animation data within a PPU-enhanced system will be described in the context of a “fluid” example. The fluid object/feature may take the form of an animated body of water for instance.
Like many deformable bodies, the shape of a fluid object/feature is conventionally defined by a mesh of geometric shapes, such as triangles or polygons. A triangle mesh is assumed for purposes of this description. As is conventionally understood, the triangle mesh is defined by an array of vertex data, each vertex data element corresponding to the position of one point of a constituent triangle in the mesh, and/or data identifying vertices that form the individual triangles. These two forms of data are generally referred to as the “vertex buffer” and “index buffer”, respectively. The body of “vertex data” defines a field of (visually) small, interconnected triangles which define the frame-by-frame shape of the fluid object/feature. Vertex data is an excellent example of animation data, since such data structures are conventionally used by the GPU to render the associated object/feature in an animation scene.
In a conventional system, CPU resources are used to calculate the movement of fluid object/feature on a frame-by-frame basis. Movement of the object/feature is really determined by calculating the re-positioning and/or re-orientation of the individual vertexes forming the vertex mesh defining fluid object/feature. As can be imagined, calculating the movement of the fluid object/feature requires a considerable number of calculations. In conventional systems, this vast multiplicity of computations must compete with every other computational requirement placed on the CPU by the main application. However, as presently preferred in a PPU-enhanced system, the great bulk of these computations are exported from the host system to the PPU.
As will be described in some additional detail below, physics data generated by the PPU may be used, either directly or indirectly (i.e., with some intervening additional processing provided by the host system), to generate some forms of animation data. As a result, much of the burden associated with the generation of animation data may be shifted from the host system to the PPU. Indeed, the PPU is generally much better adapted to the execution of the numerous computations required to generate certain forms of animation data. By using the PPU to generate certain types of animation data, such data is more rapidly and efficiently generated within the system, and CPU resources are freed up to address other demands made of the application.
A principal object of many emerging applications is the real-time rendering of highly realistic animations responsive to a user's input or direction. PC and console games are ready examples of this broad class of applications. The visual display of 3-D graphics is accomplished by means of three (3) related processing stages: tessellation, geometry, and rendering. These processing stages are implemented in a sequence of pipelined instruction executions and resulting data transfers. In the tessellation stage, objects/features are decomposed into geometric primitives (e.g., triangles and/or polygons) for simplified processing. During the geometry stage tessellated object/features are transformed. Transformation is a generic term describing a process typically characterized by a sequence of complex mathematical operations, such as translating, rotating, and projecting. The rendering stage rasterizes the geometrically manipulated primitives into display pixels and applies certain visual effects to the pixel data, such as texture mapping, Z-buffering, depth cueing, anti-aliasing, and MIP mapping.
Conventional 3-D graphics pipelines are implemented using the combined resources of a CPU and GPU. That is, the computational resources (e.g., the floating-point units) provided by the CPU are generally applied to the tessellation and geometry stages. Rendering has traditionally been accomplished using GPU resources, although recent trends have seen the operations associated with the geometry stage exported from the CPU to the GPU.
In a PPU-enhanced system, PPU resources may be used to execute the many, if not all, of underlying computational operations typically required in the tessellation and geometry stages of an animation process. The actual use or involvement of the PPU in the tessellation and geometry stages of an animation process will of coarse vary by design and in accordance with system priorities. However, certain presently preferred PPU uses and related guidelines for PPU uses are described below.
The use of a PPU in the tessellation and geometry stages of an animation process is likely to vary by object/feature type. For example, nearly all of the computational operations associated with the tessellation and geometry stages of a deformable object/feature animation are preferably performed using PPU resources. Whereas, the geometry stage-related operations of translation, rotation, and projection, as applied to rigid bodies, are preferably performed using GPU and/or CPU resources. This having been said, the forces and torques acting upon rigid bodies are preferably calculated using PPU resources. Once such “force-related” physics data is calculated by the PPU, the PPU then calculates the new translation and rotation of rigid bodies. Then, the translation and rotation data is transferred to the host system and used by the CPU or GPU to translate and rotate rigid bodies associated with a current animated frame.
Before continuing with a discussion of PPU involvement with the generation and transfer of animation data several exemplary hardware embodiments will be described. A first exemplary embodiment of a hardware architecture defined in relation to the present invention is shown in
Data from a main application is generally provided to the system shown in
In a conventional system, some portion of the application data transferred to main memory 30 is subsequently transferred from main memory 30 and loaded into L1 cache 11 and/or L2 cache 12 preparatory to the execution of instructions and operations by CPU 10. In a conventional system, the CPU is the sole computational platform for generating physics data, animation data, etc.
However, in a PPU-enhanced system, some portion of the application data is physics simulation data, (e.g., data related to computational operations performed by the PPU). Simulation data is transferred by chip set 20 from main memory 30 via the local bus 22 to a PPU memory 45 associated with PPU 44. (Hereafter, PPU memory 45 will not be shown in the drawings, but is assumed as part of an overall “PPU system” generally indicated as PPU 44). PPU 44 thereafter performs computational operation(s) related to the physics data.
As has been noted above and will be described in greater detail hereafter, the PPU is capable of generating both physics data and animation data. Animation data is data ready for consumption by the GPU. All other data returned by the PPU to the host system is generally termed “physics data.” The “generation” of both physics data and animation data is accomplished by the execution of one or more logical and/or mathematical operations. These logical and mathematical operations generally calculate the changing relationships of animated object/features in a time-wise progression of frames. The GPU renders the animation on a display device on a frame-by frame basis in response to animation data derived directly from the PPU, animation data derived solely within the host system, and/or animation data derived in the host system based, wholly or in part, upon physics data derived from the PPU.
Accordingly, physics data and/or animation data resulting from the PPU-executed computational operations is typically returned to main memory 30. Animation data stored in main memory 30 is subsequently transferred from main memory to GPU memory 41. Direct Memory Access (DMA) operations are preferably used to transfer: application data from a peripheral to main memory 30; simulation data from main memory 30 to PPU memory 45; and, animation data from main memory 30 to GPU (video) memory 41.
In the example shown in
However, emerging systems often benefit from the incorporation of one or more additional data transfer mechanisms.
Further improvements in the system's overall computational efficiency are provided by an improved CPU architecture, such as that provided by Pentium® II type processors. Within
There are a number of difficult issues that must be addressed in both of the foregoing architectures. One issue relates to the efficient use of memory resources and the transfer of data between the plurality of memories within the system. That is, the system of
In systems where direct data transfers between PPU memory 45 and GPU memory 41 are not available, animation data stored in PPU memory 45 following the completion of PPU computations may be rapidly prefetched using main memory 30 as a transfer point. That is, animation data is transferred from PPU memory 45 to main memory 30 and thereafter transferred from main memory 30 to GPU memory 41. Alternatively, however, the delay and inefficiency inherent in such two-stop data transfers may be eliminated by a direct transfer of animation data from PPU memory 45 to GPU memory 41.
There are certainly instances where direct animation data transfers from the PPU memory 45 to GPU memory 41 are not possible. As noted above, some physics data generated by the PPU requires additional processing by the CPU in order to generate animation data. In such instances, some or all of the PPU generated physics data must first be transferred to either main memory 30 or a data cache associated with CPU 10.
The dedicated AGP bus provided by the embodiment shown in
Recent attempts to address data bottlenecking and to reduce the need for prefetching animation data to a GPU memory have resulted in the design of highly integrated, multimedia processors, preferably formed on a unitary silicon die. An exemplary block level diagram of such a processor 8 is shown in
Chip set 20 supplies a local (e.g., PCI) connections to system peripherals, including, for example, disk drive 43 and PPU 44. A sound card, a LAN controller, a fast serial link (e.g., a IEEE 1394 “firewire” bus), and/or relatively slow I/O ports for peripherals such as a keyboard and mouse may also be conventionally connected to processor 8 through chip set 20.
Memory controller 60 bridges processor 8 to main memory 30 and may provide data compression/decompression to reduce the bandwidth of traffic crossing over memory bus 31. Main memory 30 and memory controller 60 preferably implement one or more RAMBUS, fast SDRAM, DDR, of GDDR type protocol(s). GPU 40 preferably provides at least TFT, DSTN, RGB, or similar type video output to an external display device.
The system of
Within the context of a PPU-enhanced system like the one shown in
With three separate processing cores accessing data from one or more common data caches and/or main memory 30 considerable attention must be paid to potential problems of bus contention and data integrity. For example, a hierarchal cache configuration is almost certainly necessary with this architecture. The hierarchal cache configuration preferably includes a cache controller and shared bus 17 connecting CPU 10, GPU 40, and PPU 44 through L2 cache 12 with bus interface 50. In conjunction with memory controller 60, bus interface 50 fills L2 cache 12 from main memory 30.
The L2 cache will preferably include several designated portions respectively assigned to the exclusive use of a corresponding processing unit (e.g., the CPU, GPU or PPU). Conventional sand-box partitioning, set associated swap areas, or similar techniques, together with conventional bus contention resolution techniques may be used to implement and control the hierarchical cache configuration.
An exemplary flow of data between system elements will now be described with reference to the system shown in
When executing the PC game using system resources, the CPU essentially runs a game engine, a graphics API, a PPU API, and related device drivers. The game engine comprises game specific software and typically includes packages directed to the implementation of AI, user input (e.g., keyboard, joystick, mouse, etc.) processing, sound processing, scene rendering, and general animation. Animation software manages the playback of artist scripted key-frame animations. The game engine software interpolates motion (e.g., positions and rotations) of object/features between defined key-frames, and blends multiple key-framed sequences as need.
The graphics API provides a standard software interface for the game engine to access the functionality provided by the GPU card. Conventional graphics APIs include D3D (by Microsoft, Inc.) and OpenGL. The device driver software manages communication between the graphics API and the GPU hardware. The animation data typically sent to the GPU through use of the graphics API and related device drivers includes, as examples, vertex buffers, triangle strips, texture maps, and a command stream containing various commands to control the rendering of the scene by the GPU.
Much of the animation data sent to the GPU, especially large data structures such as vertex buffers and texture maps, are generally stored in main memory 30. Main memory 30 often comprises a specialized (i.e., a specifically designated) section called an AGP aperture that enables data transfers from main memory 30 to GPU memory 41.
The physics API provides a software interface for the game engine to access various physics simulation functionality provided by PPU 44. Typical simulation data transferred from the host system to the PPU through the physics API and related PPU device drivers includes: static data, inputs, parameters, updates to persistent state data, and a stream of commands to the PPU. The commands are used to control the physics simulation, or to query various elements of state data. For example, the application running on the host system CPU may need to know the current position, velocity and/or orientation of an object/feature in the simulation. Other commands may instruct the PPU to send, as returned physics data, selected portions of the simulation state data to the host after selected simulation time-steps. In response to these types of commands, the PPU writes physics data, generally via a DMA transfer, from PPU memory 45 to main memory 30. Once stored in main memory 30, the physics data may be accessed by host system components under the direction of the game engine and through the physics API.
Within a PPU-enhanced system, the GPU will typically require access to at least some portion of the simulation state data (e.g., data defining the current position and orientation of a rigid body, or a vertex buffer) stored in PPU memory in order to properly render a scene. Since the physics data to-be-returned may be quite voluminous, and since this data must sometimes be updated in main memory during each animation frame, it is important to transfer the required physics data as quickly as possible. This is true whether the physics data requires additional processing or is ready for immediate consumption by the GPU, as animation data. Where the main memory contains an AGP aperture (or similar designated memory portion), the PPU preferably returns physics data directly to the AGP aperture. Once the PPU-derived animation data is loaded into main memory 30, the CPU can issue a command to the GPU to render the scene using the data in main memory.
The flowchart shown in
The flowcharts shown in
Once the rigid body position data is updated in the PPU memory, it is transferred to the host system (203) where it is made available to the GPU (204) and used to render the rigid body within the scene (205). Of note, the rigid body position data derived by the PPU may be considered either physics data or animation data depending on its nature and the particular data input capabilities of the GPU. For example, where the rigid body position data is expressed as a mesh data structure, it may well be used as animation data suitable for direct consumption by the GPU. Where, however, the rigid body position data is expressed as a primitive, further CPU processing is normally required since conventional GPUs are not adapted to the direct input of such primitive data structures. Even where the rigid body position data is expressed as a mesh data structure otherwise acceptable as an input to the GPU, further CPU processing may be desirable. For example, the CPU may provide a higher resolution mesh for the rigid body and use the PPU-derived translation and rotation data to transform the vertices of the higher resolution mesh.
The flowchart of
A smooth particle hydrodynamics (SPH) subroutine being run on the PPU computes physics data related to the fluid from the fluid inputs received from the host system (302). For example, the SPH subroutine computes the forces acting on the fluid particles. Taking these forces into account, the (updated) acceleration, velocity, and position for each fluid particle is calculated. Taking the fluid particle position data into account, a mesh data structure is generated to represent the surface of the fluid. The mesh data structure generally consists of an array of vertex position data (the “vertex buffer”), and data identifying the vertices of the triangles forming the mesh (the “index buffer”). The PPU may also compute for each respective vertex, a vector representing the surface normal direction at the vertex (the “normal data”).
Taken together, the vertex data, index data, and normal data form “fluid animation data” typically requiring no additional processing by the CPU. As a result, the fluid animation data is transferred to the GPU (303) and used to render the fluid in the scene (304). Transfer of the fluid animation data may occur through the main memory or directly between the PPU memory and the GPU memory.
The fluid animation data may be used for purposes within the PPU. For example, the fluid data may also be used to determine collisions between the fluid and other object/features in the scene.
The foregoing embodiments and related methods are teaching examples of the present invention. Many specific adaptations and modifications may be made to these embodiments. For example, as has been noted above with respect to
As previously noted the GPU and PPU memories shown in
In sum, many different system architectures may be used to one or more methods rendering an animation on a display device associated with a system. Within these method(s) some (first) portion of the animation data may be generated using CPU resources. While the CPU is generating the first portion of the animation data, the PPU is simultaneously and in parallel, at least in part, generating physics data. The physics data may be, in fact, another (second) portion of the animation data, or it may serve as an input to the CPU generation of the second portion of the animation data.
Whether generated in the CPU, the PPU, or a combination of the CPU and PPU, all portions of the animation data are ultimately stored in memory (e.g., L2 cache, GPU memory and/or main memory) before being rendering using the GPU resources.
A myriad of implementation adaptations and modifications to the working example presented herein will be readily apparent to those of ordinary skill in the art. However, these adaptations and modifications fall within the scope of the present invention which is defined by the attached claims.
This application is related to commonly-assigned U.S. patent application Ser. No. 10/715,440 filed Nov. 19, 2003. The subject matter of this related application is hereby incorporated by reference.
