Patent Application
20060061517
An embodiment of the invention is directed to the digital processing of still images or video, for delivery to a display device.
Images and video that are displayed by electronic systems, such as personal computers, are composed of picture elements or pixels that define a frame or image to be displayed. A frame may consist of over a million pixels and may be formed under control of the main data processing part of the system (e.g., by a central processing unit, CPU, running a program stored in main memory). Each frame may be temporarily stored in a frame buffer of the system, prior to being transferred to the display device.
To transfer a digital image for display on a conventional analog cathode ray tube (CRT) device, its pixels may be read from the frame buffer by a display controller on a line by line basis (corresponding to the horizontal lines of the screen). Those pixels are then converted from digital to analog form. The resulting analog signal, also referred to as an analog video signal, is transmitted via cable to the CRT drivers that control the intensity and color of individual locations on the screen. The voltage of the analog video signal, which varies continuously with time, translates directly into an intensity or color of a location on the screen.
In contrast to the analog display interface, a digital monitor has what is referred to as a “digital” interface. Examples of these types of interfaces are found in digital CRTs, flat panel displays, and digital televisions. In those cases, the pixel data fetched by the display controller is kept in digital/discrete time format while being transferred to the display device (e.g., via cable). The Digital Visual Interface (DVI) 1.0 Specification, by the Digital Display Working Group (DDWG), Portland, Oreg. (“DVI”), defines the connector and other parts of an example display interface for digital displays.
In many cases, the integrated circuit containing the display controller is a graphics controller device. A graphics controller device within a computer system typically includes dedicated hardware to perform certain compute intensive tasks in digital image processing, such as geometry processing, vertex processing, texture application, and rasterization to yield a frame (the latter being one of the tasks typically performed by the display controller section). A graphics controller may interface with the display device by having its digital output pixel stream converted or translated into the format and signaling of the display interface. Examples of digital interfaces for transferring video out of a display controller are the Accelerated Graphics Port (AGP) and the Digital Video Output (DVO) port, both used by graphics controllers from Intel Corp., Santa Clara, Calif. A separate integrated circuit package, typically sold by a third party (other than the one sourcing the display controller and the display device), may be installed, as a “receiver” device for the stream, that has the needed conversion circuitry to translate between, for example, DVO and DVI
A typical scenario for displaying an image by a computer system is as follows. A pixel pipeline within the graphics controller collects a number of sub-frames or sub-images and combines them into a final image. For example, consider a computer screen showing several different windows and a cursor, where each of these items or objects may be a separate sub-image that has been separately generated. This final frame is stored in the frame buffer, waiting to be transferred to the display device. At this point, the final frame may be in the same resolution as required by the display device. For example, if a display device has 1024×768 pixel resolution, then the final frame may also be in that format of 1024×768=768,432 pixels. In other cases, the frame buffer is smaller than the screen in which case the image may be centered or located in some corner of the screen, or it may be upscaled within the pixel pipeline (prior to being applied to the screen).
Due to the nature of the materials and circuitry that make up the screen, the control signals that are applied to activate the elements of the screen need to be repeatedly applied to refresh the screen, to thereby maintain the brightness and colors of the displayed images. For example, in a typical CRT, a refresh rate of at least 80 Hz may be needed. In that case, the buffered frame is copied to the display device 80 times per second. The display screen resolution places a further requirement on data transfer rate in the interfaces between the display controller and the display device. For the example above, in the case of a 1024×768 screen with a refresh rate of 80 Hz, the rate at which pixels should be transferred for each frame is at least 1024×768×80=62,914,560 pixels per second (approximately 63 MHz). The actual rate is greater, due to the need for transferring blanking information in addition to the pixels.
Conventional graphics controllers have the ability to obtain pixel streams from the frame buffer at different rates, to accommodate the needs of different types of display devices. In the case of earlier generation Intel graphics controllers, pixel data can be sent out of the graphics controller through the DVO port at different rates, depending on how the graphics controller has been configured by software. Since the display interface has different signaling requirements than the DVO port, a protocol conversion is performed from DVO to the display interface. For example, an AGP digital display card (ADD Card) can be used to provide the needed translation from DVO/AGP to different types of display interfaces. In that case, a required pixel rate of for example 50 MHz is met by a DVO data transfer rate of 50×8=400 MHz (where in this case each pixel is represented by 8 binary bits). The ADD Card translates this 50 MHz DVO pixel input into a 50 MHz DVI pixel output. If a different display device were connected to the DVI interface, then software within the system would configure the graphics controller to change its DVO pixel rate (e.g., decrease from 50 MHz to 25 MHz). The DVO port in that case would drop the data transfer rate to 25×8=200 MHz, as would the DVI interface (via the ADD Card). Thus, the earlier generation DVO port had the capability to reduce its data transfer rate to that required by a given pixel rate.
The embodiments of the invention are illustrated by way of example and not by way of limitation in the figures of the accompanying drawings in which like references indicate similar elements. It should be noted that references to “an” embodiment of the invention in this disclosure are not necessarily to the same embodiment, and they mean at least one.
Beginning with
The received pixels are then delivered for display on a display device (not shown), over an interface that operates at a “higher” data transfer rate. The received pixels are part of a new pixel stream 108 that is being transferred at, in this example, 3 times the lower data transfer rate. Each of the pixels from the original pixel stream has been squeezed into a clock cycle that is ⅓ as long as the original pixel clock cycle. However, the effective pixel rate in the new pixel stream is still the same, namely 60 MHz. One or more fill codes 112 have been inserted between each adjacent pair of pixels. To achieve an integer multiplier effect (in this case, a 3× multiplier), two fill codes 112 are inserted between pixel1 and pixel2 where each inserted fill code is the same size as a single pixel. As a result, the time interval T for transferring both streams may be essentially equal.
The above-described method may be used between the display controller and a display device of a computer system, for sending pixel data and display timings to one or more display devices. The method is useful when one of the interfaces to the display devices requires that a higher data rate be maintained, i.e., higher in relation to a data transfer rate dictated by the display resolution and refresh requirements at which the pixel data has been received. One reason why the interface to the display device (or simply referred to here as display interface) requires a higher data rate may be that the interface is AC coupled and as a result needs the data transfer rate to be maintained above a pre-determined threshold, due to its high pass filter effect.
For example, consider the serial digital video out (SDVO) interface found on current generation graphics and memory controller hub products by Intel Corp. SDVO refers to the second generation of graphics controller digital display channels by Intel, the first being DVO. SDVO provides digital display data in a serialized format, in contrast to DVO which is a parallel interface. In many cases, a protocol converter, such as a SDVO receiver device provided by a third party vendor, accepts the serialized SDVO format and translates the pixel stream into the appropriate format for the display interface (these embodiments will be further described below). SDVO currently calls for a symbol or character transfer rate that is between 100 MHz and 200 MHz. If each character symbol were a single pixel, then the above-described pixel stuffing methodology helps maintain the higher character transfer rate on the display interface, while maintaining the lower, original pixel rate. As described further below, this may be used in another embodiment of the invention to simplify the design of a receiver device (e.g., one that can simultaneously send the pixel data over a digital display interface and an analog display interface in a cost-efficient manner).
There are several different “transfer rates” that are being used here to describe the different embodiments of the invention. First, there is “pixel rate” which gives the number of pixels per second that are to be displayed. This rate may depend upon the display resolution and the refresh rate, as well as other factors such as blanking intervals and over scan compensation. Referring to
Still referring to
Finally, the term “data transfer rate” refers to bit rate. Here, a character or symbol (e.g., a pixel; a fill code) may consist of a fixed number of bits (where in the above example, each pixel consists of 8 bits). Note that a “bit” here refers to two or more states, such that it is not limited to a binary bit. If the pixel data is being transferred serially, then the data transfer rate of an interface is a integer multiple of its character transfer rate. As another example, if each 8 bit character is encoded into a 10 bit symbol before being transferred, the data transfer rate in that case would be 10 times the character transfer rate (as in SDVO). The table below shows the relationship between pixel rates, character transfer rate (also referred to as clock rate), and data transfer rate for displaying 5 different pixel rates. The pixels are received at the pixel rate (PR), for display over an interface that operates at a “higher” data transfer rate, in the example range of 100-200 MHz.
For example, the pixel rate of 25 to 40 MP/s is translated to a character transfer rate (clock rate) of 125 to 200 MHz using a 5× multiplier. That means stuffing four fill codes following each received pixel. Although the pixel rate (PR) multipliers and clock rate (CR) multipliers are shown as integers, non-integer multipliers may alternatively be used albeit doing so may complicate the circuitry needed to implement the conversion. Upon receiving a request (such as from software running in the system) to change the pixel rate to a new rate, the number of codes that are inserted between each adjacent pair of pixels is changed in accordance with the new rate. For example, if the change is from a current pixel rate of 100 to 200 mega pixel per second to 50 mega pixel per second, then one option is to keep the character and data transfer rates unchanged while inserting a single fill code following each pixel in the original stream, so that there is enough data to fill the channel at the higher character and data transfer rates. Another option is to change to a lower CR and inserting 2, 3, or 4, fill codes (as appropriate) to fill the channel. The codes may be pixel-sized in that each code may have the same number of bits as a pixel. Each code may thus take about the same amount of time to transfer as a pixel. In addition, the codes may be inserted at regular intervals, such as between every pair of adjacent pixels of the original stream.
At the receiver end, the method for displaying pixels contemplates that the inserted codes will be detected (e.g., based on predefined patterns stored in the receiver), and used for receiver symbol alignment in the interface. The inserted fill codes may thus be designed to facilitate their detection for purposes of symbol alignment in a serial interface such as one that complies with the peripheral components interconnect (PCI) Express Base Specification 1.0a and Errata updated Jul. 14, 2004, available from PCI Special Interest Group (PCI-SIG) Portland, Oreg. 97221 (“PCI Express”). In other words, the received bit stream will be analyzed to determine the symbol boundaries, using previously stored data patterns that match the transmitted fill codes. In another embodiment, each fill code also provides error detection benefits, and may have a value such that a single bit error in any received pixel (or color value) is not likely to match a fill code, or that the fill code is not likely to be mistaken for a pixel.
The above described pixel stuffing methodology for displaying an image sequence may be combined with a further aspect where display timing data units are also included in the received pixel stream. For example, video pixel data may be sent during an “active” period, which is preceded by a “blank” period. This blank period may contain horizontal and vertical synchronization data units, conventionally used to indicate display timing for analog CRTs. These are also referred to as blank codes 210. Each blank code 210 may be the inverse of (e.g., one's complement in a binary system) a fill code. Referring now to
Turning now to
Turning now to
The pixel stream provided by the pixel pipeline 408 is sent to pixel timing replication logic 410. The logic 410 is to replicate the timing data units, as well as the pixels, in accordance with a multiplication factor. As explained above, the multiplication factor may be a positive integer greater than one. Thus, if a 3× factor were used to increase a pixel rate of 60 MHz to a character transfer rate of 180 MHz, a 3× multiplier used to duplicate the timing data unit. For example, if a blank period is two 60 MHz clocks in duration, it would be replicated to span six 180 MHz clocks.
The controller device 404 also includes fill codes insertion logic 412 that is designed to replace the replicate pixels with fill codes. As explained above, these fill codes may be selected to help synchronize the receiver of the stream, including use in, for example, symbol alignment logic and/or lane-to-lane deskew logic (not shown). In addition, these fill codes may be further designed to help with error detection at the receiver. For example, the fill code may be the 1's complement of a blank code, and selected such that no single-bit error in color (pixel value) matches a fill or blank code.
The output of the fill codes insertion logic 412 is combined with the timing signal (including timing data units) by an encoder 414 that includes encoding logic used to encode the output frame in accordance with a digital video transfer protocol. In this example, an encoding scheme referred to as 8B-10B may be used to deliver the output frame as part of a serial digital video stream over the SDVO interface.
The encoder 414 may encode each color channel of the output frame separately, such that each encoded color channel is assigned to a separate lane of a serial multi-lane point-to-point link (e.g., SDVO over PCI Express, where the SDVO protocol runs on top of physical layer interface circuitry that is in accordance with PCI Express). Other higher and lower layer protocols may alternatively be used in that interface.
The second path taken by the pixel and timing signals in
As suggested above, the controller device 404 may be capable of supporting video transfer protocols with different programmable pixel rates, where each pixel rate is met by setting a respective clock or character rate that is a multiple of the pixel rate, and by setting a data transfer rate that is a multiple of the clock rate (see Table 1 describing the five different examples for pixel rates given above).
In operation, the controller device 404 splits a single pixel stream to be displayed into two simultaneous streams, where one is in the form of a serial digital video protocol such as SDVO, while the other is analog video. Note that the replication of pixels and timing data units, in this case, helps simplify the analog video portion, because it is a simple matter of digital to analog conversion of the serial stream directly, and then applying the resulting analog signals to the analog CRT. That may be because an analog CRT need not receive a clock, and its pixel data does not undergo the same encoding and parallel to serial conversion that a digital interface needs. For example, a pixel and its replicate together (in sequence) will appear the same as a single original pixel at one half the frequency.
The embodiments of the invention described above may provide one or more of the following advantages. First, the required frequency range on a high speed AC coupled interface such as SDVO may be narrowed while supporting a relatively broad range of pixel rates, thereby avoiding issues where the data transfer rate over the AC coupled bus is too slow for its coupling capacitors. In addition, using recognizable fill codes help the receiver device reconstruct the original pixel stream, and particularly where the fill codes have been selected with a view towards error detection and synchronization of the receiver. Note that in the multi-lane serial interface embodiment, the pixel stream may be divided over the multiple lanes of an AC coupled, multi-lane serial interface (such as one that complies with PCI Express). In that case, the fill codes may have the same size and values across all lanes. In addition, the fill code may be sent simultaneously on all lanes. This helps with error detection and synchronization at the receiver. In another embodiment described above where the active pixels are first replicated and then the replicates are replaced with fill codes, this feature helps efficiently implement the simultaneous display capability of an analog CRT monitor and a digital monitor.
An additional benefit of multiplying the pixel rate is that the receiver can more easily recover a clock from the received data stream, by increasing the edge rate within the stream and enabling data to be sent without a sideband clock. Also, in an SDVO embodiment, since SDVO encoding gives identical run lengths to 8b/10b encoding used in PCI Express, keeping the SDVO character rate similar to the PCI Express character rate allows the opportunity to re-use PCI Express circuitry on the transmitter and the receiver.
Turning now to
Turning now to
A block diagram of some primary components of the protocol converter 512 is depicted in
Once the fill codes and blank codes have been detected and the pixel stream is being reliably recognized at the output of the symbol alignment logic 708, encoding logic 712 forms a new sequence containing the pixels of the stream, to be sent to a display device. The example given above was 8B-10B, where 8 bit symbols are encoded into 10 bit characters before actually being transmitted by the physical layer interface circuitry (not shown). In some cases, the encoding logic 712 need not change the size of the symbols, but merely forms a new sequence containing the pixels, without the inserted fill codes and blank codes that have been detected.
Although the different embodiments of the invention are described above in terms of logic functional blocks, most, if not all, of the functionality described may be implemented by a programmed processor. In that case, a machine accessible medium would include instructions that when executed cause a machine (e.g., the display controller) to obtain digital video at a lower data transfer rate, and insert one or more blank codes for each timing data unit as well as one or more fill codes between pixels, to meet a predetermined higher data transfer rate. In addition, the instructions may also describe how to encode the pixels, the timing data units, and the blank and fill codes, in accordance with a digital video transfer protocol.
The invention is not limited to the specific embodiments described above. For example, although
