Patent Application
20120307141
1. Technical Field
This disclosure relates generally to the use of video displays, and more specifically to the display of a set of images on multiple devices.
2. Description of the Related Art
In some situations, there is a need to display the same images concurrently on multiple displays of a computer system. For example, consider a system with a computing device having an internal display and an external interface to which an external display is coupled. This system may be used for a presentation—for example, a software demonstration to an audience in a large room. The presenter may view the demonstration on the device's internal display while the audience views the demonstration on the external display. In making such a presentation, it is typically desirable for the two displays to show the same images at the same time (or at least such that differences between the two displays are not visually apparent). Achieving such a result, however, may require significant resources of the computing device. Such an allocation of resources may not make sense from a design standpoint, particularly where real estate is at a premium on the computing device (e.g., the computing device is a tablet or smart phone device) and the presentation feature described above is not frequently used. Further complicating the situation is the multiplicity of possible external displays of differing resolutions that may be attached to the computing device.
This disclosure relates to a computing device with a scaling unit that facilitates a mode in which images are displayed concurrently on different displays, such as an internal display of the computing device and an external display coupled to the computing device via an external interface. In one embodiment, the external interface may not have sufficient bandwidth (e.g., pins) to support concurrent display of images at the same resolution being used for the internal display. Accordingly, a scaling unit may be implemented within the computing device that receives pixel values destined for the internal display and down-scales these pixel values in the horizontal direction (i.e., within a line of pixels) and provides the down-scaled pixels to the external interface. By the scaling unit operating on pixel values produced by a display pipeline for the internal display, the need for the operation of an entirely separate display pipeline for the external display is eliminated. With horizontal scaling performed on the computing device, vertical scaling may then be performed outside the computing device—for example, in a dongle or other device coupled between the computing device and the external display. Scaling horizontally but not vertically on the computing device reduces on-board memory requirements since multiple pixel lines for the external display need not be stored within the scaling unit.
In one embodiment, the down-scaling produced by the scaling unit reduces pixel bandwidth through the external interface to support concurrent display of the images. In some embodiments, the scaling unit may down-scale further to maintain the aspect ratio of the image as displayed on the primary display. In some implementations, the aspect ratio of the image may depend upon the orientation of the computing device, and the scaling unit may adjust a scaling factor accordingly. For example, in one embodiment, a computing device may have a resolution of 2048 columns by 1536 rows when the computing device is oriented in a “landscape” mode, and have a resolution of 2048 rows by 1536 columns when oriented in a “portrait” mode. The former orientation has an aspect ratio of 4:3; the latter has an aspect ratio of 3:4.
As noted above, when displaying images concurrently on two different displays, it may be desirable that the images be displayed concurrently. Because pixels are being down-scaled by the scaling unit for inclusion in an output video frame, however, these pixels are not available at the same time as in the original video frame. Similarly, the frame control signals (e.g., vertical sync, horizontal sync) for the original frame often cannot be used “as is” for the output frame without the introduction of visual artifacts.
To this end, in one embodiment, the scaling unit is configured to write down-scaled pixel values to a buffer circuit in an input clock domain (e.g., a clock domain of the internal display) and read from the buffer circuit in an output clock domain (e.g., a clock domain of the external display) during appropriate periods in the output video frame. The scaling unit may be further configured to generate control signals for the output video frames, thus creating retimed video frames that include the down-scaled pixels.
In one embodiment, the computing device, upon detecting a connection to an external display, selects a set of timing parameters from a data store that includes timing parameters for supported external displays. The selected timing parameters may then be used to program the scaling unit to generate horizontal and vertical sync signals for the retimed video frame. For example, one of the selected timing parameters may specify an initial delay needed for the scaling unit to generate a first output pixel from a group of pixels in the original video frame. This and other parameters may be used to control one or more state machines in the scaling unit that are used to generate control signals for the retimed video frame.
In various embodiments, the scaling unit may generate control signals such that a refresh rate for retimed frames being displayed on a secondary display is the same as a refresh rate for frames being displayed on a primary display. This may performed in one embodiment by ensuring that various other timing requirements are met, such as ensuring that a line time for the primary display is the same as a line time for the external display. Meeting these timing requirements may prevent image drift, thus facilitating a visually satisfying concurrent presentation on multiple displays.
This specification includes references to “one embodiment,” “an embodiment,” “one implementation,” or “an implementation.” The appearances of these phrases do not necessarily refer to the same embodiment or implementation. Particular features, structures, or characteristics may be combined in any suitable manner consistent with this disclosure.
Further, various units, circuits, or other components may be described or claimed as “configured to” perform a task or tasks. In such contexts, “configured to” is used to connote structure by indicating that the units/circuits/components include structure (e.g., circuitry) that performs those task or tasks during operation. As such, the unit/circuit/component can be said to be configured to perform the task even when the specified unit/circuit/component is not currently operational (e.g., is not on). The units/circuits/components used with the “configured to” language include hardware—for example, circuits, memory storing program instructions executable to implement the operation, etc. Reciting that a unit/circuit/component is “configured to” perform one or more tasks is expressly intended not to invoke 35 U.S.C. §112, sixth paragraph, for that unit/circuit/component.
Turning now to
As shown, device 110 is coupled to display 120. In one embodiment, display 120 is integrated or internal to computing device 110. This display may be referred to as the “primary” display of device 110. In some embodiments, primary display 120 may be connected to device 110 through an external interface. Display 120 is represented with a dotted line in
In addition to display 120, computing device 110 includes an external interface 130 that is coupled to an external or secondary display 160 via connection 150. Interface 130 may be any type of standard or proprietary interface, and may be wired or wireless. A given interface 130 can be understood to have a “data width” (e.g., a number of pins) that dictates how much data the interface can transfer at a given point in time. Connection 150 is a logical representation of the connection between device 110 and secondary display 160. In various embodiments, connection 150 may be wireless. In other embodiments, connection 150 may be wired, and may include one or more intervening hardware components, such as a vertical scaling unit discussed below. Like primary display 120, secondary display 160 may be any suitable type of device. In one embodiment, secondary display 160 is a high-definition TV (HDTV) compatible device.
Computing device 110 may include various structures (not depicted in
Processors within device 110 may implement any instruction set architecture, and may be configured to execute instructions defined in that instruction set architecture. The processors may employ any microarchitecture, including scalar, superscalar, pipelined, superpipelined, out of order, in order, speculative, non-speculative, etc., or combinations thereof. The processors may include circuitry, and optionally may implement microcoding techniques. The processors may include one or more L1 caches, as well one or more additional levels of cache between the processors and one or more memory controllers. Other embodiments may include multiple levels of caches in the processors, and still other embodiments may not include any caches between the processors and the memory controllers.
Memory controllers within device 110 may comprise any circuitry configured to interface to the various memory requestors (e.g. processors, graphics circuitry, etc.). Any sort of interconnect may be supported for such memory controllers. For example, a shared bus (or buses) may be used, or point-to-point interconnects may be used. Hierarchical connection of local interconnects to a global interconnect to the memory controller may be used. In one implementation, a memory controller may be multi-ported, with processors having a dedicated port, graphics circuitry having another dedicated port, etc.
Memory within device 110 may be any type of memory, such as dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate (DDR, DDR2, DDR3, etc.) SDRAM (including mobile versions of the SDRAMs such as mDDR3, etc., and/or low power versions of the SDRAMs such as LPDDR2, etc.), RAMBUS DRAM (RDRAM), static RAM (SRAM), etc. One or more memory devices may be coupled onto a circuit board to form memory modules such as single inline memory modules (SIMMs), dual inline memory modules (DIMMs), etc. Alternatively, the devices may be mounted with a system on a chip in a chip-on-chip configuration, a package-on-package configuration, or a multi-chip module configuration.
Graphics controllers within device 110 may be configured to render objects to be displayed into a frame buffer in the memory. The graphics controller may include one or more graphics processors that may execute graphics software to perform a part or all of the graphics operation, and/or hardware acceleration of certain graphics operations. The amount of hardware acceleration and software implementation may vary from embodiment to embodiment.
In various embodiments, different structures within computing device 110 may be located within a system on a chip (SoC). In one implementation, device 110 includes integrated display 120, an SoC, memory, and interface 130, with the SoC coupled to the display, the memory, and the interface. Other embodiments may employ any amount of integrated and/or discrete implementations.
Computing device 110 may operate to display frames of data. Generally, a frame is data describing an image to be displayed. A frame may include pixel data describing the pixels included in the frame (e.g. in terms of various color spaces, such as RGB or YCrCb), and may also include metadata such as an alpha value for blending. Static frames may be frames that are not part of a video sequence. Alternatively, video frames may be frames in a video sequence. Each frame in the video sequence may be displayed after the preceding frame, at a rate specified for the video sequence (e.g. 15-30 frames a second). Video frames may also be complete images, or may be compressed images that refer to other images in the sequence. If the frames are compressed, a video pipeline in device 110 may decompress the frames.
A display generation unit within device 110 may be configured to read frame data from memory and to process the frame data to provide a stream of pixel values for display. The display generation unit may provide a variety of operations on the frame data (e.g. scaling, video processing for frames that are part of a video sequence, etc.). The unit may be configured as a display pipeline in some embodiments. Additionally, the display generation unit may be configured to blend multiple frames to produce an output frame. For example, in one embodiment, each frame pixel may have an associated alpha value indicating its opaqueness. The display generation unit may include one or more user interface blocks configured to fetch and process static frames (that is, frames that are not part of a video sequence) and one or more video pipelines configured to fetch and process frames that are part of a video sequence. The frames output by the user interface blocks may be blended with a video frame output by the video pipeline. In one embodiment, the display generation unit may be configured to provide the output pixel stream to pixel processing units (PPUs) within device 110.
Generally, a pixel value in a stream of pixel values may be a representation of a pixel to be displayed on a display coupled to device 110. The pixel value may include a one or more color space values. For example, in an RGB color space, the pixel value includes a red value, a green value, and a blue value. Each value may range from zero to 2N−1, and describes an intensity of the color for that pixel. Similarly, in the YCrCb color space, the pixel value includes a Y value, a Cr value, and a Cb value. The location of a pixel on the display may be inferred from the position of the corresponding pixel value in the pixel stream. For example, the pixel stream may be a series of rows of pixels, each row forming a line on the display screen. In a progressive-mode display, the lines are drawn in consecutive order and thus the next line in the pixel stream is immediately adjacent to the previous line. In an interlaced-mode display, consecutive passes over the display draw either the even or the odd lines, and thus the next line in the pixel stream skips one line from the previous line in the pixel stream. For brevity, the stream of pixel values may be referred to as a pixel stream, or a stream of pixels. Pixel processing units within device 110 may be configured to perform various pixel operations on the pixel stream and may provide the processed pixel stream to the respective physical interfaces (PHYs). Generally, a pixel operation may be any operation that may be performed on a stream of pixels forming a line on a display. For example, pixel operations may include one or more of: color space conversions, backlight control, gamma correction, contrast ratio improvement, filtering, dithering, etc. The PHYs may generally include the circuitry that physically controls the corresponding displays. The PHYs may drive control signals that physically control the respective display panels in response to the pixel values. Thus, for example, a PHY for a display that is controlled by RGB signals may transmit voltages on the R, G, and B signals that correspond to the R, G, and B components of the pixel. There may also be a display clock that may be transmitted by the PHYs, or the display clock may be embedded in one of the control signals. Different PHYs for different displays may have clocks that are within different clock domains.
A “clock domain” refers to the circuitry that is controlled responsive to a given clock. Clocked storage devices such as latches, registers, flops, etc. may all be configured to launch and capture values responsive to the given clock, either directly or indirectly. That is, the clock received by a given clocked storage device may be the given clock or a clock that is derived from the given clock. On the other hand, clocked storage devices in a different clock domain launch/capture values responsive to a different clock that may not have a synchronous relationship to the given clock.
It is often desirable to use computing device 110 to make a presentation—for example, to an audience in a large room. In such a situation, the size of primary display 120 may be inadequate for audience members. To facilitate such presentations, secondary display 160 may be coupled to device 110 via interface 130 and connection 150. In this manner, the presenter may view the presentation on display 120 while the audience views the presentation on display 160.
Such dual display becomes less useful, however, if images on the displays are not synchronized (that is, someone viewing the two images can visually discern image drift or other visual discrepancies). Stated another way, it is often desirable that the two images be displayed concurrently, such that when the presenter is describing a feature of the presentation appearing on display 120, this same feature is also appearing on display 160 at the same time. (As will be described further below, there may be some inherent phase difference between images on different display. As used herein, however, references to “synchronized,” “synchronous,” or “concurrent” display of images includes display of images on different displays that do not have visually discernable image drift.)
Concurrent display of images becomes more difficult when the internal display and external display have different resolutions (i.e., different number of pixels in the horizontal and vertical directions). One possible solution is to have different display generation units for each display. Such an approach has significant drawbacks. Consider a game developer who wishes to demonstrate a new video game using internal and external displays. If the video game is pushing the processing power of device 110, it may be a waste of processing power to have a second display generation unit running for the external display, when in effect it would be rendering the same image as for the first display generation unit. Thus, such a configuration might not allow the developer the ability to showcase the video game running at peak performance.
An alternative solution is the use of a “mirror mode” in which a single display generation unit is used to provide output (e.g., pixels) to displays 120 and 160. This solution involves fetching data from memory only a single time (as opposed to twice in the solution described above). In some embodiments of computing device 110, however, the use of mirror mode may still have shortcomings. In particular, in some instances, the data width of interface 130 may not provide sufficient bandwidth to concurrently display images on both displays. For example, interface 130 may be sufficient for many data transfer applications, but may not have enough pins to display video on an HDTV secondary display concurrently with the primary display. In order to facilitate concurrent display of images through such a connector, the data sent to interface 130 may be compressed. A horizontal scaling unit for performing such compression is described next with respect to
Turning now to
As described above with reference to
As described above, in some embodiments, the data width of interface 130 is less than that of an interface to primary display 120. In these situations, in order to effectuate display of images on secondary display 160 concurrently with display of images on primary display 120, interface 130 can be redesigned or the data passing through interface 130 may be compressed. Redesign of interface 130 may be problematic, particularly in situations in which the connector has been widely adopted over time.
In one embodiment, computing device 110 achieves concurrent display on external display 160 through bandwidth-limited interface 130 by scaling the data in between display generation unit 210 and interface 130. In the embodiment shown, horizontal scaling unit 220 is configured to reduce the number of pixels (i.e., down-scale) in a line of pixels provided by display generation unit 210. As one example, horizontal scaling unit 220 may receive 2048 pixels for a given line of a frame to be displayed on display 120, and down-scale these input pixels to output 1440 pixels for a frame to be displayed on display 160. (Further scaling examples are given with respect to
The term “horizontal” scaling denotes processing within a given line of pixels of a frame (e.g., columns 1-2048 of row 1 of a frame), in contrast to processing such as vertical scaling in which pixels from multiple rows are involved. Because unit 220 performs horizontal but not vertical scaling in one embodiment, this reduces the memory needed within unit 220 to store pixels. That is, if unit 220 performed vertical scaling in lieu of or in addition to horizontal scaling, unit 220 would likely need to store multiple lines of pixels at a given time in order to facilitate the vertical scaling process. Instead, unit 220, in one embodiment, operates only on pixels within a given row of the input frame, and thus is an “inline” scaler.
The implementation of
Note that in some embodiments, device 110 may perform horizontal and vertical scaling between unit 210 and interface 130. In some designs, however, this additional complexity within device 110 is not warranted when considering the typical frequency of the use of mirror mode as compared to the additional hardware resources that would need to be allocated to perform the vertical scaling within device 110. Accordingly, in the embodiment shown in
Vertical scaling is thus performed outside of device 110. In the embodiment shown, vertical scaling unit 230 is a hardware device located within connection 150. In one embodiment, unit 230 is a dongle that couples to interface 130 and provides a connection (either wired or wireless) to external display 160. Alternate embodiments are possible. For example, unit 230 could be situated at the other end of connection 150, or even within external display 160. The configuration shown in
Turning now to
As discussed above, a problem may exist when a data width of interface 130 does not permit concurrent display of images on displays 120 and 160 (even leaving aside the differences in resolution). Horizontal scaling unit 220 may thus operate to scale in the horizontal dimension by an amount sufficient to pass data through interface 130 at a rate that supports concurrent display of images. In certain embodiments, unit 220 may apply an additional scale factor to preserve an aspect ratio of the image on display 120. The application of such a scale factor thus allows proportionately sized concurrent images to appear on displays 120 and 160.
In the example shown, an image displayed on display 120 at 2048×1536 pixels is ultimately down-scaled to fit on a 1920×1080 display. In one embodiment, the scaling factor applied by unit 220 is based on whichever dimension (horizontal or vertical) needs the greatest amount of down-scaling. In
For certain implementations of computing device 110, the aspect ratio of display 120 may change. In one embodiment, the aspect ratio of display 120 may change based on the orientation of device 110. For example, device 110 may be configured such that if it is oriented (e.g., by the user) in a “landscape” mode (as in
Turning now to
To produce a synchronized image, however, timing issues need be considered. For example, if the clock used by internal display 120 (in Clk in
The generation of vertical and horizontal control signals for display 160 need also be considered. Examples of such signals are shown with reference to
Vertical sync period 816 starts at the beginning of a frame. The vertical back porch period 818 starts at the end of vertical sync period 816 and lasts until the beginning of the first line of active pixel data (i.e., the beginning of vertical active period 812). The vertical front porch period 814 starts at the end of the last active line of pixel data and lasts until the beginning of the next frame (i.e. the beginning of the next vertical sync). Each of these periods may be defined as an integer multiple of the horizontal line time (reference numeral 854 in
Similarly, the horizontal blanking interval (HBI) 858 is the period between the last active pixel of one horizontal line and the first active pixel of the subsequent line, and is composed of a horizontal sync period 816, a horizontal back porch (HBP) period 868, and a horizontal front porch (HFP) period 864.
The horizontal sync period 816 starts at the beginning of a line. The horizontal back porch period 868 starts at the end of the horizontal sync period 816 and lasts until the first active pixel of the line (i.e., the beginning of horizontal active period 862—thus, for display 120, pixels are output on clock pulses occurring during horizontal active periods 862). The horizontal front porch period 864 starts after the last active pixel of the line, and lasts until the beginning of the next line (i.e. the beginning of the next horizontal sync). Each of these periods may be defined an integer multiple of the pixel time. Note that the HBI is typically observed for all line times, even those that occur during the VBI.
One possible solution for generating the timing for display 160 is to use the input clock (i.e., display 120's clock) as the output clock during HBI periods, which would also allow display 160 to use the input horizontal sync signal and the HBP and HFP periods associated with display 120. In this solution, during non-HBI periods, certain clocks are suppressed according to the degree of horizontal scaling that is required. Circuitry to accomplish these objectives is shown in
The configuration of
Turning now to
At a high level, the retiming mechanism of scaling unit 500 is based on ensuring that the frame time (e.g., reference numeral 804) of the input frame (i.e., a frame output to display 120) is equal to the frame time of the corresponding output frame (i.e., a frame that is output to display 160). Because scaling unit 500 does not perform any vertical scaling, however, the VBI and the height of the output frame can be unchanged from the input frame. Accordingly, an equation for keeping the input and output frame rates the same reduces to keeping input and output line times equal:
(widthin+HBIin)/clkin=(widthout+HBIout)/clkout.
Note that widthin, HBIin, clkin, and widthout for the internal display are known and may be fixed; scaling unit 500 is configured to select HBIout and clkout such that this equation and others described below are satisfied. Failing to adhere to this restriction may cause the phase offset to drift over time, resulting in visual artifacts on display 160.
Another high-level design consideration in certain embodiments relates to the size of buffer circuit 530. In one embodiment, scaling unit 500 is configured to guarantee that scaler 510 can provide scaled data at a constant rate using a clock in output clock domain 514. To do so, scaled data may be buffered before the start of active period of every line. The size of buffer circuit 530 may be related to HBIout, so that buffer circuit does not run out of pixels during this time period. In one embodiment, buffer circuit 530 includes a FIFO with 512 entries, and software within computing device 110 will program HBIout to be less than 512 pixel periods.
In one embodiment, scaler 510 is the primary data path of scaling unit 500, responsible for performing the horizontal scaling on the incoming frame. It may be composed of three sub-blocks in one embodiment: horizontal scaler (H Scale) block 524, buffer circuit 530, and CRC block 534.
In one embodiment of H Scale unit 524, horizontal scaling is accomplished via three separate 9-tap polyphase filters, one for each pixel component (R, G, B). The filters may use a shared digital differential analyzer (DDA) to control the stepping of the current center pixel and the index for the coefficients. The DDA value may start each line at a value referred to as DDAInit, and may be incremented by a value DDAStep for each subsequent output pixel. (In one implementation, DDAStep is programmed so that the input width divided by the DDAStep equals the output width.) In one embodiment, input pixels enter one end of a 9-entry shift register, making their way to the other end as subsequent pixels are received. When all the input pixels necessary for an output pixel are in the shift register (as determined from the DDA) the scaling calculation is performed, and the output pixel generated. A table describing the scaling process is described further below with reference to
The DDA (Digital Differential Analyzer) may be used to control the current position during the scaling operation. In one embodiment, it is a 36-bit fixed-point number with 16 bits in the integer portion and 20 bits in the fraction. The 16-bit integer portion determines the current pixel position, while the fraction is used to index into a coefficient table. The horizontal scaler uses 16 phase coefficient tables (one table per tap), so only the upper four fractional bits (rounded) are used for the index. The same index may be used for each table.
Each filter may multiply nine input pixel components (repeated when necessary) by nine coefficients drawn from the tables, and sums the results to form the destination pixel component. The results may be rounded as needed. For edge cases, the pixels that are used for filtering depend upon the relationship between the current DDA and either 0 (left edge) or the input width (right edge). When the DDA is close enough to the border that there are not enough pixels to perform the 9-tap filtering, then the border pixels are repeated.
Buffer circuit 530 receives scaled pixel data from H Scale unit 524. In one embodiment, buffer circuit 530 includes a 512-deep 24-bit asynchronous FIFO that is written in the input clock domain (512) and read in the output clock domain (514). Data is pushed into the FIFO whenever scaled data is available, and is read every output clock cycle during the pixel active period (the overlap of the vertical and horizontal active periods; in the embodiment shown, output data enable 568 in
Scaler 510 may use CRC block 534 to provide a mechanism for determining the correctness of the output frame. In one embodiment, the CRC is updated constantly during frame time (both when active pixels are generated and during blanking), and is optionally reset at the end of each frame. The input stream for the CRC can be thought of as the 3 8-bit outputs of the horizontal scalers organized as {Red[7:0], Blue[7:0], Green[7:0]}, with the low order bit entering the CRC first. During blanking the data input to the CRC may be 0 in one embodiment.
In the embodiment shown, control circuit 540 is responsible for retiming of the incoming frame. It is composed of three sub-blocks: skew block 550, vertical sync generator 580, and horizontal sync generator 570. As shown, the vertical and horizontal sync generators run in output clock domain 514.
Skew block 550 is responsible for setting the phase offset between the input frame and the output frame. Upon receiving input vertical sync signal 544 (in VSync), skew block 550 starts a count operation. Once the count is completed, a go signal (synchronized to output clock domain 514) is sent to other blocks in control circuit 540. In one embodiment, the programmed count is greater than the latency between receipt of the first input pixel and the generation of the first output pixel, ensuring that output data enable 538 is not driven active before pixels are available in buffer circuit 530.
Vertical sync generator 580 in
Like vertical sync generator 580, horizontal sync generator 570 in
As shown, output data enable signal 568 (outDataEn) is used to determine when to read output data 538 (outData) from buffer circuit 530. Output data enable 568 may be determined in one embodiment by ANDing horizontal active signal 562 and vertical active signal 564 produced by generators 570 and 580, respectively. Accordingly, when control circuit 540 indicates that the output frame is within a vertical active period (i.e., a non-VBI period) and within a horizontal active period (i.e., a non-HBI period), output data enable 568 is valid, and a pixel is read from buffer circuit 530.
Various units within scaling unit 500 may depend upon programmed values to operate correctly for a given set of displays 120 and 160. Accordingly, in one embodiment, once computing device 110 detects a connection to external display 160, device 110 may determine one or more characteristics of display 160 (e.g., resolution), select a predetermined set of parameters from a data store, and program scaling unit 500 with these selected parameters (e.g., into programming interface 590 via I/O bus 588).
There are numerous possible values that may be programmed via interface 590. A Skew Count Register may be used to set the latency between the start of an input frame and the start of the corresponding output frame in input clock cycles. Such a value may ensure scaled data is available when the horizontal state machine is in the horizontal active state. Equations that may restrict the programming of this register in certain embodiments are described further below with reference to
A Source Frame Size Register may specify the size (width and height) in pixels of the frame being input to scaling unit 500. Similarly, a Destination Frame Size Register may specify the size (width and height) in pixels of the frame being output from unit 500.
Values for various vertical and horizontal control signals may be programmed via interface 590. A Vertical Sync Count Register may be used for software to specify the input and output vertical sync period in line times. A Vertical Back Porch Count Register may be used to specify the input and output vertical back porch period in line times. Similarly, a Vertical Front Porch Count Register may be used to specify the input and output vertical front porch period in line times. Similar register values may be set for output horizontal sync period, output horizontal back porch period, and horizontal front porch period (all in output clock cycles). Equations restricting programming of these registers in certain embodiments are described further below. These registers may control the timing of the vertical and horizontal state machines described below with reference to
Interface 590 may also be used to set parameters pertinent to scaler 510. For example, in one embodiment, a DDA Initial Value Register may be used to set an initial value for the DDA, which is used to set the initial scale location on each line. A DDA Step Value Register may contain a step value for the DDA, which is used to control horizontal scaling. Horizontal Scaler Coefficient Registers may contain 16-bit coefficient values used to perform the 9-tap horizontal scaling function. The registers may be organized as 9 tables of 16 entries each, corresponding to the 9 taps and 16 phases of the horizontal scaler.
Thus, upon determining characteristics of external display 160, computing device 110 may program various registers within scaling unit 500 to ensure concurrent display of images on the primary and secondary display. For example, software on device 110 may, given an input clock and input HBI parameters, select an output clock and control signal parameters that permit input and output line times to be equal. These parameters may be selected from a data store (e.g., registers or other storage element) within computing device 110. In one embodiment, these parameters have been computed for some set of possible external displays so that the timing equations described with reference to
Turning now to
As described above, state machine 900 begins in idle state 902; upon skew block 550 generating a go pulse and synchronizer 560 outputting go pulse 904 in output clock domain 514, state machine 900 transitions to vertical sync state 912, which causes output vertical sync 584 to be asserted. The counter in state machine 900 then begins counting to vertical sync count 914. Upon reaching this value, output vertical sync 584 is de-asserted, and state machine transitions to vertical back porch state 922, where it remains until the counter (which has been reset in one embodiment) reaches vertical back porch count 924. At that point, state machine 900 transitions to state 932 and asserts vertical active signal 932 until vertical active count 934 is reached. State machine 900 then transitions to state 942 (vertical front porch period), de-asserts the vertical active signal, and begins counting to vertical front porch value 942. When vertical front porch 942 is reached and another input vertical sync 544 has been received by control circuit 540 for the next input frame, state machine 900 transitions back to state 912 and asserts output vertical sync 584. If another frame has not been received, state machine 900 transitions to idle state 902 and remains until another input vertical sync 544 is received.
State machine 950 operates in an analogous manner to state machine 900, and generates output horizontal sync 962 and output horizontal active 982 in response to input vertical sync 544 and the various count values programmed by computing device 110.
Turning now to
Before discussing the actual restrictions on register values, it is helpful to introduce two concepts: initial scaling latency and final scaling latency. Note that these values, though similar in concept, are not necessarily identical. Initial scaling latency, in one implementation, is the time in input clock cycles between when the first input pixel of a line is received and when the first output pixel is ready for transmission. Initial scaling latency may thus be used to determine the time range during which the first output pixel can be safely transmitted. It may be a function of the horizontal scale calculation latency (fixed at 9 in one implementation described above) and the rounded integer portion of the DDAInit (variable from 0-4).
When utilizing initial scaling latency to prevent underflow it may be necessary in some instances to include an output clock cycle in the equation to account for the possibility that synchronization into the output clock domain may take some signals one output clock cycle more than other signals. Conversely, when utilizing the initial scaling latency to prevent overflow an output clock cycle should be subtracted. These relationships are shown in
Final scaling latency may be defined as the time in input clock cycles between when the last input pixel of a line is received and when the last output pixel is ready for transmission. It may be a function of the horizontal scale calculation latency and the rounded integer portion of the DDA value corresponding to the last output pixel of a line. As with the initial scaling latency, it may be necessary to add one output clock cycle when using the final scaling latency for underflow, and subtract one output clock cycle when using it for overflow. These relationships are shown in
In addition to its use in determining the time range during which the last output pixel can be safely transmitted, the final scaling latency may also determine the minimum input horizontal blanking (the sum of HSync, HBPorch, and HFPorch). This restriction may be used when the final output pixel of any line must be generated before the first input pixel of the subsequent line is received. This restriction, however, is based on the implementation of scaling unit 500 and it is not an inherent retiming restriction. Further, this restriction is entirely within the input clock domain, so there is no asynchronous boundary crossing to consider. This restriction is shown in
In one possible design, there are three types of retiming restrictions that may be considered: line retiming, pixel retiming, and frame offsetting. Line retiming refers to the restriction that is placed on the input and output line times, while pixel retiming refers to the restrictions placed on when scaled pixels can be output. Frame offsetting refers to the restrictions placed on the time delta between the input and output frames.
As described above, retiming is performed to prevent frame drift between displays. Because scaling unit 500 does not perform vertical scaling (i.e. there are an equal number of input and output lines), scaling unit 500 thus ensures in one embodiment that input and output line times be equal. A line time is the sum of the horizontal blanking time and the horizontal active time. Thus, as long as Equation 4 in
Pixel retiming restrictions are intended to ensure that any given scaled pixel is available at the time it is supposed to be output from buffer circuit 530. In one implementation, there are four such restrictions, corresponding to the following statements: do not start outputting pixels too early, do not start outputting pixels too late, do not stop outputting pixels too early, and do not stop outputting pixels too late. The first restriction (do not start outputting pixels too early; Equation 5 in
Frame offsetting refers to the time skew between the start of the input frame and the output frame. Because, in one embodiment, scaling unit 500 will shut down if it ceases receiving input frames, it may be necessary that the skew be large enough to ensure a “next frame coming” indication is received in the output clock domain before the next frame is scheduled to start. Equation 9, indicated by reference numeral 1080 in
The equations described above can be used to determine a set of parameter values for a particular scaling unit design and a particular external display. For example, these equations may be used to determine timing parameters and output clock frequencies to be used for scaling unit 500 and a 1920×1080 display, scaling unit 500 and a 1024×768 display, and so on. Note that any values that satisfy these equations are possible. Parameters may thus be determined for some set of displays supported by computing device 110. These parameters may then be stored in device 110 in a storage element (and may be updated as needed in certain embodiments). When mirror mode is used and a particular external display 160 is detected, the appropriate set of parameters may be selected and scaling unit 500 programmed via interface 590 to prevent frame drift between input and output frames.
Turning now to
The pixel processing data path begins in step 1104, in which pixels from an input frame are received in an input clock domain (e.g., by H scale unit 524). In step 1108, the pixels are down-scaled, such as by unit 524 as described above. In step 1112, the down-scaled pixels are stored, such as in a FIFO in buffer circuit 530. Then, in step 1116, the stored pixels are output for display in an output clock domain, such as from buffer circuit 530. In one embodiment, a pixel is output when both the output horizontal and vertical active signals are asserted (e.g., during step 1128).
The control signal data path begins in step 1120, in which one or more control signals in the input clock domain are received, such as by control circuit 540. In scaling unit 500, input vertical sync 544 is used to denote the start of an input frame; input horizontal sync may not be used by the state machines in this embodiment. In step 1124, output control signals such as output horizontal sync 574, output vertical sync 584, horizontal active 562, and vertical active 564 are generated, and output in step 1128. As noted above, certain of these signals may be used in one embodiment to control the outputting of pixels in step 1116.
An example application of method 1100 can be illustrated with reference to table 1200 in
In this example timing, the input device has a line time of 2208 pixels, measured with reference to the input pixel clock of 205.2 MHz. This input line time is composed of an HBI of 160 pixel times (32 HSync; 80 HBPorch; 48 HFPorch) and a horizontal active time of 2048 pixel times. The input line time is thus 10.76×10−6 sec (2208/205.2×106). The input frame time is 1549 line times, which, as indicated above, are measured in the input clock domain. The vertical active period is 1536 lines, and a VBI of 13 line times (4 VSync; 6 VBPorch; 3 VFPorch). The frame time is thus 10.76×10−6*1549, or 0.016667 sec ( 1/60 sec, for a refresh rate of 60).
On the output device, there are 1440 pixels per line in the example shown, as opposed to 2048 pixels per line on the input device. In the example shown, the output device is supplied with a slower pixel clock (171 MHz). Additionally, the HBI is set to 400 pixel times (10 HSync; 250 HBPorch; 40 HFPorch), for a total of 1840 pixel times/line. With these settings and the output pixel clock, the output line time is 10.76×10−6 sec (1840/171×106)—the same line time for lines in the input frame.
As described above, in one embodiment, computing device 110 performs only horizontal scaling on the device (vertical scaling is performed external to device 110). Thus, the number of lines/frame in the output frame is kept identical to the number of lines/frame in the output frame: 1549 (with the same breakdown between vertical active lines and the VBI). Keeping the same number of lines/frame and the same absolute line time in both the input and output frames thus eliminates drift between images on the two displays. Note that the parameters in the table above are exemplary and other values may be used. In particular, there are other possible values which keep line times the same in both the input and output clock domains. Additionally, other sets of parameters may be used for different destinations for the same source shown above (2048×1536). Still further, other sets of parameters may be use for different source/destination combinations.
Turning now to
Turning now to
Although specific embodiments have been described above, these embodiments are not intended to limit the scope of the present disclosure, even where only a single embodiment is described with respect to a particular feature. Examples of features provided in the disclosure are intended to be illustrative rather than restrictive unless stated otherwise. The above description is intended to cover such alternatives, modifications, and equivalents as would be apparent to a person skilled in the art having the benefit of this disclosure.
The scope of the present disclosure includes any feature or combination of features disclosed herein (either explicitly or implicitly), or any generalization thereof, whether or not it mitigates any or all of the problems addressed herein. Accordingly, new claims may be formulated during prosecution of this application (or an application claiming priority thereto) to any such combination of features. In particular, with reference to the appended claims, features from dependent claims may be combined with those of the independent claims and features from respective independent claims may be combined in any appropriate manner and not merely in the specific combinations enumerated in the appended claims.
This application claims the benefit of U.S. Provisional Application No. 61/493,635, filed on Jun. 6, 2011, which is incorporated by reference herein in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| 61493635 | Jun 2011 | US |
