For a detailed description of exemplary embodiments of the invention, reference will now be made to the accompanying drawings in which:
Certain terms are used throughout the following description and claims to refer to particular system components. As one skilled in the art will appreciate, different companies may refer to a component by different names. This document does not intend to distinguish between components that differ in name but not function. In the following discussion and in the claims, the terms “including” and “comprising” are used in an open-ended fashion, and thus should be interpreted to mean “including, but not limited to . . . .” Also, the term “couple” or “couples” is intended to mean either an indirect or direct electrical connection. Thus, if a first device couples to a second device, that connection may be through a direct electrical connection, or through an indirect electrical connection via other devices and connections. Additionally, the term “software” refers to any executable code capable of running on a processor, regardless of the media used to store the software. Thus, code stored in non-volatile memory, and sometimes referred to as “embedded firmware,” is within the definition of software. Further, the term “system” refers to a collection of two or more hardware and/or software components, and may be used to refer to an electronic device, such as a computer system or a portion of a computer system.
The following discussion is directed to various embodiments of the invention. Although one or more of these embodiments may be preferred, the embodiments disclosed should not be interpreted, or otherwise used, as limiting the scope of the disclosure, including the claims. In addition, one skilled in the art will understand that the following description has broad application, and the discussion of any embodiment is meant only to be exemplary of that embodiment, and not intended to intimate that the scope of the disclosure, including the claims, is limited to that embodiment.
The present disclosure describes systems and methods for efficiently utilizing memory in the context of compressed images and video streams. While the following embodiments are discussed with regard to a complete image and video picture application, one of ordinary skill in the art will appreciate that the embodiments are equally applicable to image and video processing subsystem applications, such as, for example, MPEG, QuickTime®, H264, and H263. Such subsystem applications may further include three-dimensional (3D) or two-dimensional (2D) graphics applications for processing video and still images that require access to a 2D frame in a raster-scan order organization and for which small 2D objects are usually accessed within that frame. All such subsystem applications are intended to be within the scope of the present disclosure.
A primary goal of image and video compression, encoding, and decoding is to represent an image with as few bits as possible. This is often done by detecting temporal and spatial redundancies in the graphical data and compressing its digital representation to reduce these redundancies. As a result, many image encoding and decoding standards involve grouping pixels of an image into two dimensional blocks that contain a predetermined number of pixels. This grouping of pixels is often performed during the image encoding process. In the case of the MPEG video encoding standard, for example, these macroblocks are sixteen pixels tall by sixteen pixels wide, composed of four, eight-by-eight sub-blocks At least some of the preferred embodiments disclosed in the present disclosure partition a display of graphical data into two-dimensional user configured quantities called “tiles,” and these tiles form the basis for determining the organization of the image data stored in memory. Since graphical data is encoded in a raster scan fashion, the tiled version of the graphical data stored in memory comprises a pattern of parallel horizontal rows of tiles. By choosing the size and shape of these tiles and implementing other features, performance improvements in memory access speed and reductions in the power consumption of the memory are possible.
The memory subsystem 110 comprises memory controller 115 coupled to memory array 120. In the preferred embodiment of
In at least some embodiments, memory array 120 is arranged into sub-arrays of rows and columns of transistors that each stores a single bit. The bits from individual rows of multiple sub-arrays are further grouped into bytes and/or words that each represents a single pixel represented on display 111. Pixels are then further organized into the tiles described above, with the size of the tiles (in bytes) preferably set equal to or less than the capacity of a row within a sub-array of memory array 120. Thus, when a single data row, which is distributed over the grouped sub-arrays, is read from or written to memory array 120, pixels spanning multiple scan lines of a localized area of display 111 may be concurrently accessed. The data row in memory represents a two-dimensional area of the visual image or video displayed, rather than a single-dimensional scan line of the image or video.
For a given pixels-per-tile size, a variety of tile dimensions are possible. The dimensions of the tiles of the preferred embodiments will depend on the context defined for specific graphical data, and on the objects represented and processed. For example, if the 2D image objects accessed by a given application have horizontal dimensions that, overall, are significantly larger than the vertical dimensions of the objects, there is a higher statistical probability that the objects will be accessed in the horizontal direction than in the vertical direction. The tile dimensions may be set larger in the horizontal direction than the vertical direction to take advantage of this higher statistical probability. Similarly, if the accessed 2D objects are more likely to be accessed in the vertical direction rather than the horizontal direction (e.g., if the objects are, overall, tall and narrow), then the tile dimensions may be set larger in the vertical direction. Although the tile size and dimensions are usually set by and for a particular application being executed, the tile size and dimensions may be reconfigured to accommodate different applications concurrently. Additionally, memory array 120 may include multiple sub-arrays of different sizes and with different access distribution characteristics, each optimized for different contexts and selectable by an application executing on a particular initiator. Many different organizations and contexts will become apparent to those skilled in the art, and all such organizations and optimizations are intended to be within the scope of the present disclosure.
In the preferred embodiment of
Continuing to refer to
It should be noted that although the address transformation logic 130 of the preferred embodiment of
The mapping between the virtual address and the physical address is variable and depends upon which of a plurality of contexts within the memory controller is utilized. Each context refers to a different memory mapping function, and each performs a different address transformation. For example, assume that the memory array 120 is 64 MB and the user desires to use 16 MB for a first image from CPU 105a with 1 byte per pixel and the user also desires to use 16 MB for a second image from the image sensor with 2 bytes per pixel, where the first and second image resolutions are different. In this example, the user may define two different contexts within address transformation logic 130, wherein each image is virtually addressed by its respective hardware initiator using two different base addresses and different tile configuration settings.
As will be appreciated by one of ordinary skill in the art, because address transformation logic 130 of the preferred embodiments of
Under the pixel addressing scheme described above, pixels are sequentially addressed within the virtual memory address space in the same order as they are displayed on display 111. Thus, each of the hardware initiators 105 accesses memory subsystem 110 as if the pixels of image 250 are directly mapped, one-for-one, to sequential locations within the virtual memory address space, regardless of the actual location of the pixel data within memory array 120 of
To perform the aforementioned transformation, pixel information for each tile is stored within a single memory row by combining the base address, horizontal and vertical tile positions, and horizontal and vertical pixel offsets within a tile to generate a physical address within memory array 120. In at least some illustrative embodiments, this transformation is represented mathematically by equation 1:
PA=PA
Base+(PVert*2(Y+N))*W+(PHoriz*2(X+Y+N))+(YOffset*2(X+N))+(XOffset*2(N)) (1)
wherein PA is the resulting physical address, PABase is the physical base address of the image; PVert is the vertical position of the tile addressed (referenced to the image origin at the top, left-hand corner of the image and increasing in value from top to bottom); PHoriz is the horizontal position of the tile addressed (referenced to the image origin and increasing from left to right); YOffset is the vertical offset of the pixel addressed within the tile (referenced to the tile origin at the top, left-hand corner of the tile and increasing in value from top to bottom); XOffset is the horizontal offset of the pixel addressed within the tile (referenced to the tile origin and increasing in value from left to right); Y is the number of bits allocated to Y-Offset field 215; W is the number of pixels per scan line of a mapped image; X is the number of bits allocated to X-Offset field 220; and N is one less than the number of bits allocated per pixel.
By using a transformation such as the one in equation (1), each tile is mapped into a single memory row within memory array 120. Each increment in either vertical position 205 or horizontal position 210 causes a new physical memory row to be addressed, while any change in value for either Y-Offset 215 or X-Offset 220 results in the same single row being accessed for a given value of vertical position 205 and horizontal position 210.
In addition to storing graphical data in a tiled fashion, the preferred embodiment described also implements bank interleaving of memory array 120.
The partially-interleaved memory array 305 of
A fully-interleaved memory array 310 comprises the same number of banks and rows as the non-interleaved array 300, however, banks 0-3 are interleaved such that sequential tile accesses occur in rows of different banks and tiles are mapped across all four banks. This reduces the latency associated with waiting for banks to close even further, as compared to partially-interleaved memory array 305.
In at least some preferred embodiments, partial array self-refresh (PASR) is employed, wherein memory banks that include rows that have not been accessed within the required refresh interval are automatically refreshed internally by circuitry within or coupled to memory array 120. Because the refresh is performed as needed on a bank-by-bank basis, the amount of power consumed during the refresh process may be reduced as compared to refreshing the entire array (also referred to as total array self-refresh or TASR). Such a partial refresh capability may be particularly useful in battery-powered handheld electronic devises where power consumption is a chief concern. PASR is preferably used with partially-interleaved array 305, since refreshes of fully-interleaved array 310 require a refresh of all of the banks, given the deliberate distribution of tile data across all banks. Thus, if a self-refresh is required, a total array self-refresh has to be executed across all banks within fully-interleaved array 310. A partial refresh of only some banks could cause the data in the remaining banks (not refreshed within the required refresh interval) to become corrupted.
Tiling memory contexts in the non-interleaved array 300 can result in significant performance improvements and power consumption reductions during memory accesses over non-tiled memory contexts. In addition to tiling the memory contexts, some embodiments may implement other features to improve memory accesses. Referring again to
The above discussion is meant to be illustrative of the principles and various embodiments of the present invention. Numerous variations and modifications will become apparent to those skilled in the art once the above disclosure is fully appreciated. For example, although hardware initiators are discussed herein, other embodiments may employ one or more software initiators that interact with memory subsystem 110 in the same manner as hardware initiators 105. Further, the embodiments disclosed may be useful in memory devices and configurations where the sequence of memory address accesses may impact the overall performance of memory subsystem 110 (e.g., DRAM, flash memory, synchronous DRAM (SDRAM), mobile DRAM, and/or magnetic RAM). Also, although the embodiments described illustrate the mapping of pixel data, other 2D and 3D graphical data associated with a graphical object may also be mapped as described, such as Z-Buffer data, light rendering data, mask stencil data, and pre-computed light map data, just to name a few examples. It is intended that the following claims be interpreted to embrace all such variations and modifications.
Number | Date | Country | Kind |
---|---|---|---|
06291383.5 | Aug 2006 | EP | regional |