Patent Grant
8749565
1. Field of the Invention
This invention is related to the field of graphical information processing, more particularly, to conversion from one color space to another.
2. Description of the Related Art
Part of the operation of many computer systems, including portable digital devices such as mobile phones, notebook computers and the like, is the use of some type of display device, such as a liquid crystal display (LCD), to display images, video information/streams, and data. Accordingly, these systems typically incorporate functionality for generating images and data, including video information, which are subsequently output to the display device. Such devices typically include video graphics circuitry to process images and video information for subsequent display.
In digital imaging, the smallest item of information in an image is called a “picture element”, more generally referred to as a “pixel”. For convenience, pixels are generally arranged in a regular two-dimensional grid. By using this arrangement, many common operations can be implemented by uniformly applying the same operation to each pixel independently. Since each pixel is an elemental part of a digital image, a greater number of pixels can provide a more accurate representation of the digital image. The intensity of each pixel can vary, and in color systems each pixel has typically three or four components such as red, green, blue, and black.
Most images and video information displayed on display devices such as LCD screens are interpreted as a succession of image frames, or frames for short. While generally a frame is one of the many still images that make up a complete moving picture or video stream, a frame can also be interpreted more broadly as simply a still image displayed on a digital (discrete, or progressive scan) display. A frame typically is made up of a specified number of pixels according to the resolution of the image/video frame. Information associated with a frame typically includes color values for every pixel to be displayed on the screen. Color values are commonly stored in 1-bit monochrome, 4-bit palletized, 8-bit palletized, 16-bit high color and 24-bit true color formats. An additional alpha channel is oftentimes used to retain information about pixel transparency. The color values can represent information corresponding to any one of a number of color spaces. Oftentimes, pixels that have been processed and/or rendered undergo error checking prior to being sent to a display, to ensure picture accuracy. One possible error checking mechanism is a cyclic redundancy check (CRC).
CRC can be used as a hash function to detect accidental changes to pixel data (pixel values) at the output of a display pipeline. A CRC calculation typically yields a short, fixed-length binary sequence—CRC code—for each specified set of pixel data. The CRC code can be transmitted or stored together with the specified set of pixel data. When a set of pixel data is read or received, the calculation may be repeated if the new CRC does not match the one calculated earlier, in which case the set contains a data error, requiring corrective action, which may include rereading the pixel data or requesting the set of pixel data to be sent again. If the CRC matches the one calculated earlier, the data is assumed to be error free. The check (data verification) code is a redundancy in that it does not increase the information content of the message, and the algorithm is based on cyclic codes. CRC may refer to the check code or the function that calculates it, typically accepting data streams of any length as input while outputting a fixed-length code. CRC error checking is generally simple to implement in binary hardware, is particularly effective at detecting common errors that result from transmission channel noise, and produces codes that are easy to analyze mathematically. Typically, an n-bit CRC, applied to a data set of arbitrary length, detects any single error burst that is not longer than n bits, and detects a fraction 1-2(−n) of all longer error bursts.
While CRC offers an effective and relatively simple error checking mechanism, many systems require the error checking mechanism to be integrated without adversely affecting system performance, while also minimizing the duration of required tests/simulations that need to be performed on these systems. Other corresponding issues related to the prior art will become apparent to one skilled in the art after comparing such prior art with the present invention as described herein.
In one set of embodiments, display pipes in a video system may terminate with an output FIFO (first-in first-out) buffer from which pixels are provided to a display controller coupled to a graphics/video display. The graphics/video display may display video/images represented by the pixels. The display pipes may frequently fill the FIFO buffer at a much higher rate than at which the display controller fetches the pixels from the FIFO buffer. An error-checking block, e.g., a CRC (cyclic redundancy check) block may be connected in front of the FIFOs to receive the pixels processed by the display pipes, in a manner similar to the FIFOs receiving the processed pixels. The display pipes may support an error-checking-only (ECO) mode of operation during which pixels may be generated as fast as the display pipes are capable of processing them. In this mode of operation, the output FIFOs may be disabled. As a result, the pixels are not written to the FIFO, which therefore does not fill up, but are instead processed directly by the error-checking block. Accordingly, the length of test/simulation time required to perform a test may be determined by the rate at which pixels are generated rather than the rate at which the display controller displays the pixels. Furthermore, this mode makes it possible to perform testing in environments where a display is not supported or is not available. The results generated by the error-checking may be read and compared to an expected value to detect test pass/fail conditions. Consequently, there is no need to connect a display controller/display to the FIFOs for testing and/or simulation purposes.
In one embodiment, a display pipe includes one or more processing blocks to process pixels and produce output pixels from the processed pixels, and further includes a buffer to store the output pixels for reading by a display controller during a first mode of operation. The buffer can be disabled to not store the output pixels during a second mode of operation. The display pipe further includes an error-checking block to receive the output pixels during the second mode of operation, and compute an error-checking value corresponding to the output pixels at a rate commensurate with a rate at which the one or more processing blocks process the pixels. The buffer may be a FIFO buffer, and the error-checking block may perform CRC calculations. The error-checking block may also compare the error-checking value to an expected value to detect test pass/fail conditions, where the pixels correspond to one or more image and/or video frames.
A video system may include a display pipe to generate pixels at a first clock rate, the generated pixels representing a frame. The video system may also include a FIFO buffer to receive and store the generated pixels when the FIFO buffer is enabled, and may further include a display controller to retrieve the stored generated pixels from the FIFO buffer at a second clock rate when the FIFO buffer is enabled. An error checking circuit in the video system may receive the generated pixels at the first clock rate when the FIFO buffer is disabled, and may compute an error checking value corresponding to the received generated pixels. In one set of embodiments, the display pipe may generate sets of pixels, each set of pixels representing a respective image/video frame. The error checking circuit may receive each set of pixels at the first clock rate when the FIFO buffer is disabled, and compute a respective error checking value corresponding to each received generated set of pixels. When the buffer is enabled, the display controller may provide the pixels it has retrieved from the FIFO buffer to a display device at the second clock rate, to display the retrieved pixels on the display device. The video system may also include a processing unit to retrieve the error-checking value from the error-checking circuit, and compare the error checking-value with an expected value to determine pass/fail conditions of the display pipe. The processing unit may also be used to enable and disable the FIFO buffer.
In one set of embodiments, a system may include a system memory to store visual information represented by a set of pixels. A display pipe may fetch the set of pixels from the system memory, process the set of pixels to generate a stream of pixels, and output the stream of pixels. A FIFO buffer may receive and store the stream of pixels output by the display pipe, while a display controller may read the stream of pixels from the FIFO buffer and provide them to a display device configured to display the visual information represented by the stream of pixels. The display controller may read the stream of pixels from the FIFO buffer at a rate commensurate with a refresh rate of the display device. The system may also include an error checking circuit to receive the stream of pixels output by the display pipe at a rate at which the display pipe processes the set of pixels, and compute an error-checking value based on the received stream of pixels. A processing unit coupled to the display pipe may be used to disable the FIFO buffer to allow the display pipe to output the stream of pixels at the rate at which the display pipe processes the set of pixels. The processing unit may also be used to provide an expected value to the error-checking unit, and the error-checking unit comparing the error-checking value with the expected value to determine a pass/fail condition of the display pipe.
The following detailed description makes reference to the accompanying drawings, which are now briefly described.
While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that the drawings and detailed description thereto are not intended to limit the invention to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope of the present invention as defined by the appended claims. The headings used herein are for organizational purposes only and are not meant to be used to limit the scope of the description. As used throughout this application, the word “may” is used in a permissive sense (i.e., meaning having the potential to), rather than the mandatory sense (i.e., meaning must). Similarly, the words “include”, “including”, and “includes” mean including, but not limited to.
Various units, circuits, or other components may be described as “configured to” perform a task or tasks. In such contexts, “configured to” is a broad recitation of structure generally meaning “having circuitry that” performs the task or tasks during operation. As such, the unit/circuit/component can be configured to perform the task even when the unit/circuit/component is not currently on. In general, the circuitry that forms the structure corresponding to “configured to” may include hardware circuits and/or memory storing program instructions executable to implement the operation. The memory can include volatile memory such as static or dynamic random access memory and/or nonvolatile memory such as optical or magnetic disk storage, flash memory, programmable read-only memories, etc. Similarly, various units/circuits/components may be described as performing a task or tasks, for convenience in the description. Such descriptions should be interpreted as including the phrase “configured to.” Reciting a unit/circuit/component that is configured to perform one or more tasks is expressly intended not to invoke 35 U.S.C. §112, paragraph six interpretation for that unit/circuit/component.
Turning now to
SIU 106 may be an interconnect over which the memory controller 104, peripheral components NIC 110 and VPU 116, processor 114 (through L2 cache 112), L2 cache 112, and CDMA controller 124 may communicate. SIU 106 may implement any type of interconnect (e.g. a bus, a packet interface, point to point links, etc.). SIU 106 may be a hierarchy of interconnects, in some embodiments. CDMA controller 124 may be configured to perform DMA operations between memory 102 and/or various peripheral components 126-132. NIC 110 and VPU 116 may be coupled to SIU 106 directly and may perform their own data transfers to/from memory 102, as needed. NIC 110 and VPU 116 may include their own DMA controllers, for example. In other embodiments, NIC 110 and VPU 116 may also perform transfers through CDMA controller 124. Various embodiments may include any number of peripheral components coupled through the CDMA controller 124 and/or directly to the SIU 106. DCU 118 may include a display control unit (CLDC) 120 and buffers/registers 122. CLDC 120 may provide image/video data to a display, such as a liquid crystal display (LCD), for example. DCU 118 may receive the image/video data from VPU 116, which may obtain image/video frame information from memory 102 as required, to produce the image/video data for display, provided to DCU 118.
Processor 114 (and more particularly, instructions executed by processor 114) may program CDMA controller 124 to perform DMA operations. Various embodiments may program CDMA controller 124 in various ways. For example, DMA descriptors may be written to the memory 102, describing the DMA operations to be performed, and CDMA controller 124 may include registers that are programmable to locate the DMA descriptors in the memory 102. The DMA descriptors may include data indicating the source and target of the DMA operation, where the DMA operation transfers data from the source to the target. The size of the DMA transfer (e.g. number of bytes) may be indicated in the descriptor. Termination handling (e.g. interrupt the processor, write the descriptor to indicate termination, etc.) may be specified in the descriptor. Multiple descriptors may be created for a DMA channel, and the DMA operations described in the descriptors may be performed as specified. Alternatively, the CDMA controller 124 may include registers that are programmable to describe the DMA operations to be performed, and programming the CDMA controller 124 may include writing the registers.
Generally, a DMA operation may be a transfer of data from a source to a target that is performed by hardware separate from a processor that executes instructions. The hardware may be programmed using instructions executed by the processor, but the transfer itself is performed by the hardware independent of instruction execution in the processor. At least one of the source and target may be a memory. The memory may be the system memory (e.g. the memory 102), or may be an internal memory in the integrated circuit 103, in some embodiments. For example, a peripheral component 126-132 may include a memory that may be a source or target. In the illustrated embodiment, peripheral component 132 includes the ROM 142 that may be a source of a DMA operation. Some DMA operations may have memory as a source and a target (e.g. a first memory region in memory 102 may store the data to be transferred and a second memory region may be the target to which the data may be transferred). Such DMA operations may be referred to as “memory-to-memory” DMA operations or copy operations. Other DMA operations may have a peripheral component as a source or target. The peripheral component may be coupled to an external interface on which the DMA data is to be transferred or on which the DMA data is to be received. For example, peripheral components 130 and 132 may be coupled to interfaces onto which DMA data is to be transferred or on which the DMA data is to be received.
CDMA controller 124 may support multiple DMA channels. Each DMA channel may be programmable to perform a DMA via a descriptor, and the DMA operations on the DMA channels may proceed in parallel. Generally, a DMA channel may be a logical transfer path from a source to a target. Each channel may be logically independent of other DMA channels. That is, the transfer of data on one channel may not logically depend on the transfer of data on another channel. If two or more DMA channels are programmed with DMA operations, CDMA controller 124 may be configured to perform the transfers concurrently. For example, CDMA controller 124 may alternate reading portions of the data from the source of each DMA operation and writing the portions to the targets. CDMA controller 124 may transfer a cache block of data at a time, alternating channels between cache blocks, or may transfer other sizes such as a word (e.g. 4 bytes or 8 bytes) at a time and alternate between words. Any mechanism for supporting multiple DMA operations proceeding concurrently may be used.
CDMA controller 124 may include buffers to store data that is being transferred from a source to a destination, although the buffers may only be used for transitory storage. Thus, a DMA operation may include CDMA controller 124 reading data from the source and writing data to the destination. The data may thus flow through the CDMA controller 124 as part of the DMA operation. Particularly, DMA data for a DMA read from memory 124 may flow through memory controller 104, over SIU 106, through CDMA controller 124, to peripheral components 126-132, NIC 110, and VPU 116 (and possibly on the interface to which the peripheral component is coupled, if applicable). Data for a DMA write to memory may flow in the opposite direction. DMA read/write operations to internal memories may flow from peripheral components 126-132, NIC 110, and VPU 116 over SIU 106 as needed, through CDMA controller 124, to the other peripheral components (including NIC 110 and VPU 116) that may be involved in the DMA operation.
In one embodiment, instructions executed by the processor 114 may also communicate with one or more of peripheral components 126-132, NIC 110, VPU 116, and/or the various memories such as memory 102, or ROM 142 using read and/or write operations referred to as programmed input/output (PIO) operations. The PIO operations may have an address that is mapped by integrated circuit 103 to a peripheral component 126-132, NIC 110, or VPU 116 (and more particularly, to a register or other readable/writeable resource, such as ROM 142 or Registers 138 in the component, for example). It should also be noted, that while not explicitly shown in
In one embodiment, PIO operations may use the same interconnect as CDMA controller 124, and may flow through CDMA controller 124, for peripheral components that are coupled to CDMA controller 124. Thus, a PIO operation may be issued by processor 114 onto SIU 106 (through L2 cache 112, in this embodiment), to CDMA controller 124, and to the targeted peripheral component. Alternatively, the peripheral components 126-132 may be coupled to SIU 106 (much like NIC 110 and VPU 116) for PIO communications. PIO operations to peripheral components 126-132 may flow to the components directly from SIU 106 (i.e. not through CDMA controller 124) in one embodiment.
Generally, a peripheral component may comprise any desired circuitry to be included on integrated circuit 103 with the processor. A peripheral component may have a defined functionality and interface by which other components of integrated circuit 103 may communicate with the peripheral component. For example, a peripheral component such as VPU 116 may include video components such as a display pipe, which may include graphics processors, and a peripheral such as DCU 118 may include other video components such as display controller circuitry. NIC 110 may include networking components such as an Ethernet media access controller (MAC) or a wireless fidelity (WiFi) controller. Other peripherals may include audio components such as digital signal processors, mixers, etc., controllers to communicate on various interfaces such as universal serial bus (USB), peripheral component interconnect (PCI) or its variants such as PCI express (PCIe), serial peripheral interface (SPI), flash memory interface, etc.
As mentioned previously, one or more of the peripheral components 126-132, NIC 110 and VPU 116 may include registers (e.g. registers 138-140 as shown, but also registers, not shown, in NIC 110 and/or within VPU 116) that may be addressable via PIO operations. The registers may include configuration registers that configure programmable options of the peripheral components (e.g. programmable options for video and image processing in VPU 116), status registers that may be read to indicate status of the peripheral components, etc. Similarly, peripheral components may include memories such as ROM 142. ROMs may store data used by the peripheral that does not change, code to be executed by an embedded processor within the peripheral component 126-132, etc.
Memory controller 104 may be configured to receive memory requests from system interface unit 106. Memory controller 104 may be configured to access memory to complete the requests (writing received data to the memory for a write request, or providing data from memory 102 in response to a read request) using the interface defined the attached memory 102. Memory controller 104 may be configured to interface with any type of memory 102, such as dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate (DDR, DDR2, DDR3, etc.) SDRAM, Low Power DDR2 (LPDDR2) SDRAM, RAMBUS DRAM (RDRAM), static RAM (SRAM), etc. The memory may be arranged as multiple banks of memory, such as dual inline memory modules (DIMMs), single inline memory modules (SIMMs), etc. In one embodiment, one or more memory chips are attached to the integrated circuit 10 in a package on package (POP) or chip-on-chip (COC) configuration.
It is noted that other embodiments may include other combinations of components, including subsets or supersets of the components shown in
Turning now to
In one set of embodiments, UI 214 and 216 may include one or more registers programmable to define at least one active region per frame stored in buffers 208 and 210. Active regions may represent those regions within an image frame that contain pixels that are to be displayed, while pixels outside of the active region of the frame are not to be displayed. In order to reduce the number of accesses that may be required to fetch pixels from frame buffers 208 and 210, when fetching frames from memory 202 (more specifically from frame buffers 208 and 210), UI 214 and 216 may fetch only those pixels of any given frame that are within the active regions of the frame, as defined by the contents of the registers within UI 214 and 216. The pixels outside the active regions of the frame may be considered to have an alpha value corresponding to a blend value of zero. In other words, pixels outside the active regions of a frame may automatically be treated as being transparent, or having an opacity of zero, thus having no effect on the resulting display frame. Consequently, the fetched pixels may be blended with pixels from other frames, and/or from processed video frame or frames provided by video pipe 220 to blend unit 218.
Turning now to
Display pipe 300 may be designed to fetch data from memory, process that data, then presents it to an external display controller through an asynchronous FIFO 320. The display controller may control the timing of the display through a Vertical Blanking Interval (VBI) signal that may be activated at the beginning of each vertical blanking interval. This signal may cause display pipe 300 to initialize (Restart) and start (Go) the processing for a frame (more specifically, for the pixels within the frame). Between initializing and starting, configuration parameters unique to that frame may be modified. Any parameters not modified may retain their value from the previous frame. As the pixels are processed and put into output FIFO 320, the display controller may issue signals (referred to as pop signals) to remove the pixels at the display controller's clock frequency (indicated as vclk in
In the embodiment shown in
The overall operation of blend unit 310 will now be described. Blend unit 310 may be situated at the backend of display pipe 300 as shown in
The sources to blend unit 310 (UI 304 and 326, and/or video pipe 328) may provide the pixel data and per-pixel Alpha values (which may be 8-bit and define the transparency for the given pixel) for an entire frame with width, display width, and height, display height, in pixels starting at a specified default pixel location, (e.g. 0,0). The Alpha values may be used to perform per-pixel blending, may be overridden with a static per-frame Alpha value (e.g. saturated Alpha), or may be combined with a static per-frame Alpha value (e.g. Dissolve Alpha). Any pixel locations outside of a source's valid region may not be used in the blending. The layer underneath it may show through as if that pixel location had an Alpha of zero. An Alpha of zero for a given pixel may indicate that the given pixel is invisible, and will not be displayed.
Blend unit 310 may functionally operate on a single layer at a time. The lowest level layer may be defined as the background color (BG, provided to blend element 314). Layer 1 may blend with layer 0 (at blend element 316). The next layer, layer 2, may blend with the output from blend element 316 (at blend element 318), and so on until all the layers are blended. For the sake of simplicity, only three blend elements 314-318 are shown, but display pipe 300 may include more or less blend elements depending on the desired number of processed layers. Each layer (starting with layer 1) may specify where its source comes from to ensure that any source may be programmatically selected to be on any layer. As mentioned above, as shown, blend unit 310 has three sources (UI 304 and 322, and video pipe 328) to be selected onto three layers (using blend elements 314-318).
A CRC (cyclic redundancy check) may also be performed on the output of blend unit 310. For example, in a first mode of operation, such as a test mode, blend unit 310 may be put into an error-checking only mode, (e.g. a CRC only mode), when an error checking operation is performed on the output pixels without the output pixels being sent to the display controller. More specifically, the error checking operation may be performed on the pixel stream output from color space converter 341 (or, in some embodiments, output from blend element 318) without the pixel stream being provided to FIFO 320. In the embodiment shown, an error check unit 319 is coupled to the output of color space converter 341, to receive the pixel stream output by the display pipe. It should be noted that for ease of illustration, certain elements are shown in
As also previously mentioned, the stream of pixels output by display pipe 300 may be presented to an external display controller through asynchronous FIFO 320, and as the pixels are processed at a first rate—e.g. corresponding to a clock rate indicated as “clk” in FIG. 3—and pushed into FIFO 320, the display controller may issue signals to remove the pixels at a second rate—e.g. the display controller's clock frequency indicated as vclk in
As indicated above, error check unit 319 may be used to perform CRC operations based on the stream of pixels received from either blend element 318 or color space converter 341 (both of which may correspond to the output of the display pipe from an error checking perspective). By performing a CRC on the output pixels, no display controller and/or display is required to be connected to the output of FIFO 320 to perform test operations or simulations of the operation of display pipe 300. Furthermore, operation of display pipe 300 may be tested and/or simulated at the pixel generation rate rather than the pixel display rate. Error check unit 319 may perform a CRC for each frame. That is, the CRC value may be calculated for a stream of pixels representing a frame, and error check unit 319 may be polled every frame, for example by processing unit 114 shown in
In one set of embodiments, valid source regions, referred to as active regions may be defined as the area within a frame that contains valid pixel data. Pixel data for an active region may be fetched from memory by UI 304 and 322, and stored within FIFOs 308 and 326, respectively. An active region may be specified by starting and ending (X,Y) offsets from an upper left corner (0,0) of the entire frame. The starting offsets may define the upper left corner of the active region, and the ending offsets may define the pixel location after the lower right corner of the active region. Any pixel at a location with coordinates greater than or equal to the starting offset and less than the ending offset may be considered to be in the valid region. Any number of active regions may be specified. For example, in one set of embodiments there may be up to four active regions defined within each frame and may be specified by region enable bits. The starting and ending offsets may be aligned to any pixel location. An entire frame containing the active regions may be sent to blend unit 310. Any pixels in the frame, but not in any active region would not be displayed, and may therefore not participate in the blending operation, as if the pixels outside of the active had an Alpha value of zero. In alternate embodiments, blend unit 310 may be designed to receive pixel data for only the active regions of the frame instead of receiving the entire frame, and automatically treat the areas within the frame for which it did not receive pixels as if it had received pixels having a blending value (Alpha value) of zero.
In one set of embodiments, one active region may be defined within UI 304 (in registers 319a-319n) and/or within UI 322 (in registers 321a-321n), and may be relocated within the display destination frame. Similar to how active regions within a frame may be defined, the frame may be defined by the pixel and addressing formats, but only one active region may be specified. This active region may be relocated within the destination frame by providing an X and Y pixel offset within that frame. The one active region and the destination position may be aligned to any pixel location. It should be noted that other embodiments may equally include a combination of multiple active regions being specified by storing information defining the multiple active regions in registers 319a-319n and in registers 321a-321n, and designating one or more of these active regions as active regions that may be relocated within the destination frame as described above.
In one set of embodiments, a parameter FIFO 352 may be used to store programming information for registers 319a-319n, 321a-321n, 317a-317n, and 323a-323n. Parameter FIFO 352 may be filled with this programming information by control logic 344, which may obtain the programming information from memory through host master interface 302. In some embodiments, parameter FIFO 352 may also be filled with the programming information through an advanced high-performance bus (AHB) via host slave interface 303.
Turning now to
Turning now to
Numerous variations and modifications will become apparent to those skilled in the art once the above disclosure is fully appreciated. It is intended that the following claims be interpreted to embrace all such variations and modifications.
| Number | Name | Date | Kind |
|---|---|---|---|
| 6768774 | MacInnis et al. | Jul 2004 | B1 |
| 20030142058 | Maghielse | Jul 2003 | A1 |
| 20080222491 | Lee et al. | Sep 2008 | A1 |
| 20090271678 | Schneider et al. | Oct 2009 | A1 |
| 20100287427 | Kim et al. | Nov 2010 | A1 |
| 20120050462 | Liu et al. | Mar 2012 | A1 |
| Number | Date | Country |
|---|---|---|
| 11110239 | Apr 1999 | JP |
| Number | Date | Country | |
|---|---|---|---|
| 20120127187 A1 | May 2012 | US |
