The invention relates to digital image processing. More particularly, the invention is directed to an overlay circuit and method for reducing memory bandwidth and power consumption requirements when displaying a main image overlaid by an overlay image.
Digital images are comprised of arrays of pixels. A pixel is a discrete element in a display screen that can be illuminated with a particular color and degree of brightness. The term pixel is also used to refer to the datum that define the color and intensity of the physical pixels in display and hard copy rendering devices.
Graphics display systems typically employ a graphics controller for controlling the flow of image data from a host processor or a peripheral device, such as a camera, to a display device such as an LCD. Commonly, graphics display systems are employed in all types of computer systems, including portable devices, such as cellular telephones, personal digital assistants, music players, and other battery powered devices.
Image data are typically stored in a memory, which may be embedded in the graphics controller. The graphics controller refreshes the display device with image data at around 60 times per second, depending on the particular device, by fetching image data from the memory and writing it to the display device. The graphics controller refreshes the pixels in the display device in raster sequence. A raster sequence proceeds from left to right and top to bottom.
The memory in graphics display systems is commonly accessed by other devices and a predetermined amount of time is required to read or write a byte of data. The term memory bandwidth is used in this specification to refer to the memory's capacity for data transfer. Memory bandwidth is fixed. And when access to memory is required by many devices or large data transfers are required, the memory bandwidth may be insufficient to accommodate all demands in a timely manner. Accordingly, reducing demand for memory bandwidth is generally always desirable.
More than one image may be rendered on the display device simultaneously. Often when two or more images are displayed, one image overlays another. Each of the images may be stored separately in memory. For example, main image data defining a main image may be stored in a first portion of the memory and overlay image data defining an overlay image may be stored in a second portion. Typically, the main image is displayed so that it appears to lay under the overlay image. Because the overlay image is positioned over the main image at particular display locations, it can be defined by an overlay window, which generally is rectangular. The main image data corresponding to the overlay window are masked by the overlay image and cannot be seen.
All of the main image data are fetched from the memory in each refresh cycle. However, not all of the main image data are transmitted to the display. Main image data are transmitted for all pixel locations in the display except for the pixel locations where the overlay window is displayed. For overlay window pixel locations, overlay image data are transmitted to the display.
It is a problem of no small significance that main image data are fetched from memory for pixel locations in the display for which overlay image data is displayed. It is a problem because image data are fetched, but not used. Memory bandwidth and power are required to fetch data from memory. A graphics display system which fetches data, but does not use it wastes memory bandwidth and power. Such systems may be acceptable if the simultaneous display of a main and overlay image is infrequent. But where main and overlay images are frequently rendered together on a display, the amount of memory bandwidth wasted can be significant. Further, a significant amount of power can be wasted. And in battery powered devices, power conservation is critical.
Accordingly, there is a need for an overlay control circuit and method that reduces memory bandwidth requirements and conserves power.
The invention is directed to an overlay control circuit and method. Preferably, the overlay control circuit and method is employed in a graphics controller, and the principles of the invention are illustrated in this context in this specification. However, control circuits and methods according to the invention may be used in other contexts as desired.
With respect to a preferred overlay control circuit, a graphics controller provides main image data and overlay image data to a display device. The graphics controller preferably includes a memory for storing the main image data and the overlay image data, an overlay indicator circuit for producing an indicating signal, and an overlay control circuit.
The overlay control circuit causes main image data to be fetched from memory. The indicating signal, when asserted, indicates that a datum of the overlay image data is to be represented at a particular pixel location in the display device. The overlay control circuit stops fetching main image data when the indicating signal is asserted.
Preferably, the memory comprises a first portion for storing the main image data and a second portion for storing the overlay image data. A main image display pipe is provided for transmitting the main image data from the first portion of the memory to the display device. In addition, an overlay image display pipe is provided for transmitting the overlay image data from the second portion of the memory to the display device. The overlay control circuit writes the main image data fetched from the first portion of the memory to the main image display pipe and writes the overlay image data fetched from the second portion of the memory to the overlay image display pipe.
In one preferred embodiment, the invention is directed to a computer system incorporating the principles of the invention.
In another embodiment, the invention is directed an overlay control method according to the principles of the invention.
In yet another embodiment, the invention is directed to machine readable media embodying a program of instructions according to the principles of the invention.
This summary is provided as a means of generally determining what follows in the drawings and detailed description and is not intended to limit the scope of the invention. Objects, features and advantages of the invention will be more readily understood upon consideration of the following detailed description taken in conjunction with the accompanying drawings.
The invention is directed to an overlay circuit and method for reducing memory bandwidth and power consumption requirements when displaying a main image overlaid by an overlay image. This specification describes the present preferred embodiments of the invention, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numbers are used in the drawings and specification to refer to the same or like parts, blocks, and steps.
Beginning at the left end of the row, the pixels M1, M2, M3, . . . MN are mapped into sequential pixel locations of the row L1. These pixels correspond to N pixels of main image data. Specifically, the pixels are mapped into pixel locations in the row L1 of the display device beginning with the first (left-most) column and continue up to the shown column start column. No overlay pixels are mapped into these pixel locations.
Continuing sequentially in the row L1 and beginning at the column start pixel location, the pixels P1, P2, P3, . . . PQ are mapped into sequential pixel locations of the row L1. These pixels correspond to Q pixels of overlay image data and are mapped into the pixel locations up to the shown column stop pixel location.
In addition, the pixels MN+1, MN+2, MN+3, . . . MN+Q are also mapped into the same pixel locations in the row L1 as the pixels P1, P2, P3, . . . PQ, corresponding to Q pixels of main image data. These main image pixels, however, cannot be seen in the display device 22 because they lie underneath the overlay image 26.
Beginning in the pixel location after the column stop pixel location, the pixels MN+Q+1, MN+Q+2, MN+Q+3, . . . MN+Q+S are mapped into sequential pixel locations of the row L1. These pixels correspond to S pixels of main image data and are mapped into pixel locations up to the right end of the row L1. No overlay pixels are mapped into these pixel locations.
The preceding paragraphs describe how pixels are mapped to pixel locations in a row of the display device 22. As mentioned, the image data are typically written on the display device 22 in raster order. Thus, the pixels are typically written to the display device in the order in which they are sequentially mapped. In other words, the pixels for the row L1 are typically written to the display device 22 in the following order: M1, M2, M3, . . . MN, P1, P2, P3, . . . PQ, MN+Q+1, MN+Q+2, MN+Q+3, . . . MN+Q+S. Further, the overlay image pixels: MN+1, MN+2, MN+3, . . . MN+Q are not written to the display device 22
As will be readily appreciated, image data may be written to the display in some order other than raster order. In embodiments other than those described in this specification, the principles of the invention may be adapted to accommodate image data written to a display device in other than raster order.
Image data are mapped in a similar fashion with respect to the vertical dimension. Main image data are mapped to all of the pixel locations of a column, and overlay image data are mapped to the column between a row start and row stop. In the vertical dimension, when both main image data and overlay image are mapped to the same location, the main image data are not displayed.
The memory 34 is commonly accessed by devices other than circuitry for fetching pixels for the display 22. To prevent the display device 22 from being starved of image data while another device is accessing the memory 34, portions of the image data are read from the memory and written to a buffer, which in turn provides the data to the display device.
The buffer is typically a FIFO, though this is not essential. In this specification, the buffer is referred to as a “display pipe.” Preferably, the graphics controller 32 includes two display pipes. A main image display pipe 36 transmits each of the main image datum from the memory 34 to the display 22. An overlay image display pipe 38 transmits each of the overlay image datum from the memory 34 to the display 22. The main and overlay image pipes are operated in parallel.
The memory 34 preferably includes a first portion 34a for storing main image data and a second portion 34b for storing overlay image data. Main image data are fetched from the first portion 34a and written to the main image display pipe 36. Overlay image data are fetched from the second portion 34b and written to the overlay image display pipe 38.
Each of the display pipes is divided into two stages. A first stage 36a of the display pipe 36 is in connected in series with a second stage 36b, and a first stage 38a of the display pipe 38 is connected in series with a second stage 38b. Additional display pipes may be provided for additional overlay images.
The first stages 36a, 38a are clocked at a rate (“MCLK”) appropriate for reading from the memory. The second stages 36b, 38b are clocked at a rate (“PCLK”) appropriate for writing data to the display. Typically, the memory clock rate MCLK is a higher frequency than the display clock PCLK. Image data are fetched from the memory 34 and written to fill the first stages of the display pipes when these stages reach a predetermined level of fullness. Image data is written to a display pipe 36, 38 until the first stage is filled. The dual frequency data transfer capability of the display pipes 36, 38 allows the pipes to be rapidly filled with image data and emptied at a relatively slow rate. While the display pipes 36, 38 are being emptied, other devices may access the memory. But when a display pipe starts getting near empty, it is rapidly refilled at the MCLK rate. By using the display pipes 36, 38 to transfer data to the display device 22, the display device is never starved of image data.
The outputs of the second stages 36b, 38b of the display pipes are coupled to a multiplexer 40. The output of the multiplexer 40 is coupled to the display device 22. A main/overlay select signal is used to select one of the inputs of the multiplexer 40. The signal selects a pixel for each particular pixel location in the display screen 20. The signal selects either a main image pixel from the main image display pipe 36 or an overlay image pixel from the overlay image display pipe 38. The main/overlay select signal is generated by a main/overlay active circuit (not shown).
When the main/overlay select signal causes an overlay image pixel to be displayed at particular pixel location, the corresponding main image pixel is not displayed. The main image data for this pixel is not used, even though the data was fetched from the memory 30 and transmitted through the main image display pipe 36. Not displaying the main image pixel is a significant problem.
The graphics controller 32 includes an overlay control circuit 40 according to the invention, which provides a solution to this problem. The overlay control circuit 42 includes a memory access controller 44 adapted to fetch data from the memory 34 and cause the image data fetched from the memory to be written to the respective pipes 36, 38.
Referring to
In operation, the memory access controller 44a causes main image data to be fetched from the memory portion 34a and written to the first stage of the main image display pipe 36a. The memory access controller 44b causes overlay image data to be fetched from the memory portion 34b and written to the first stage of the overlay image display pipe 38a.
The memory access controller 44a is coupled to an overlay indicator circuit 46 that generates an overlay active signal 48. The overlay active signal 48 provides an indication as to where in the display screen 20 the overlay image data are to be mapped. Particularly, for each pixel location on the display device, the overlay indicator circuit 46 sends an overlay-active signal 48. If the signal is asserted with respect to a particular location, it is “active” and it indicates mapping an overlay pixel. If the signal is not asserted with respect to a particular location, it is not active and it indicates mapping a main image pixel.
In one embodiment, the overlay-active signal 48 is used to control fetches from the memory 34. More particularly, for a pixel destined for a particular location in the display device 22 for which the overlay-active signal 48 is active, the memory access controller portion 44a is adapted to prevent further access to the portion 34a of the memory 34 containing main image data. The flow of main image data into the main image display pipe 36 is stopped so that main image data that are to be overwritten by overlay image data are not fetched from the memory. This provides the outstanding advantages of increasing memory bandwidth and decreasing power consumption. Accesses to the memory portion 34a are resumed when the overlay-active signal 48 is no longer active. Further, while the overlay-active signal 48 is active, the memory access controller portion 44b is adapted to cause overlay pixels to be fetched from the portion 34a of the memory 34 and written to the overlay display pipe 38; and when the overlay-active signal 48 is no longer active, the memory access controller 44b is adapted to stop the flow of overlay image data into the overlay image display pipe 38.
For example, referring back to
As can be seen from
In general, the resolution of the overlay image data and the main image data may differ. As the term is used in this specification, resolution refers to the depth of the image data (“pixel depth”). For example, in a bi-level image, each pixel is represented by a single bit of data. In a gray-scale image, each pixel is typically represented by a 30 bit data word. In a color image, each pixel may be represented by an 8, 16, 24, or 36 bit data word. Where an 8-bit data word may represent one of 256 different colors, a 16-bit data word may represent one of 65,536 different colors, and so on. The pixel depth is the number of bits needed to define a pixel. Pixel depth is related to resolution in that pixel depth describes how accurately the image will be rendered.
Where the resolution of overlay image data and main image data differ, as indicated in
To accommodate differences in resolution between the two types of image data,a scaling circuit 66 is therefore preferably provided for the memory access controller 44a. The scaling circuit 66 assists the memory access controller 44a in determining the amount the address must be incremented where the resolution of the pixels is different. The host 30 may inform the scaling circuit of the anticipated difference in resolution, or this information may be provided to the scaling circuit from some other source. The scaling circuit 66 may reside in the memory access controller 44a or may reside outside the controller 44, functioning as a co-processor.
In a preferred embodiment of the invention, the overlay-active signal 48 is used by the memory access controller 44 to turn the main and overlay image display pipes 36, 38 off to further reduce power consumption. The overlay control circuit 42 preferably includes a pipe enable controller 68 to enable or disable the first stages of the pipes from receiving more data.
The pipe enable controller 68 may comprise a separate main image display pipe enable controller 68a and an overlay image display pipe controller 68b as shown in
As will be readily appreciated by persons of ordinary skill, the overlay-active signal 48 should be synchronized with the flow of data through the two display pipes to achieve the desired output on the screen of the display device.
Overlay control circuits and methods according to the present invention may be implemented in hardware or software, or both, and may employ machine readable media embodying one or more programs of instruction executed by the machine as will be readily apparent to persons of ordinary skill in the art.
It should be recognized that, while a specific overlay control circuit and method has been shown and described as preferred, other configurations and methods could be utilized, in addition to those already mentioned, without departing from the principles of the invention.
The terms and expressions which have been employed in the foregoing specification are used therein as terms of description and not of limitation, and there is no intention in the use of such terms and expressions to exclude equivalents of the features shown and described or portions thereof, it being recognized that the scope of the invention is defined and limited only by the claims that follow.