Patent Application
20160275916
The disclosure relates generally to providing image content to, and displaying image content on, displays, and more particularly to methods and apparatus for reducing frame delay and other problems associated with providing and displaying image content on displays with a variable refresh rate. Displays that support variable refresh rates typically operate within a range of supported refresh rates at which the displayed image must be updated periodically. Discrepancies in the viewed image may result if the display is not refreshed at a refresh rate within its supported range of refresh rates. For example, a display with a variable refresh rate may have a supported refresh rate range between a minimum display refresh rate, and a maximum display refresh rate, whereby the minimum and maximum display refresh rates define the boundaries of the range of supported refresh rates. Frames for display may be rendered and made available (e.g. presented) to the display at a rate within its supported refresh rate range. The render rate is the rate at which frames, typically new frames, are generated. For example, a graphics processing unit (GPU) may generate new frames at a certain render rate. Once rendered, new frames may be presented to the display at a presentation rate.
For example, a display may be displaying a current frame when a new frame is rendered and made available at a rate within the supported refresh rate range of the display. In this instance, the display may be refreshed with the new frame at a time dictated by this refresh rate. If, however, a new frame is rendered and made available for display at a rate less than the minimum display refresh rate, the display is refreshed with a current frame at a time corresponding to the minimum display refresh rate, such that the display is refreshed with the same current frame more than once. This can prevent, for example, image tearing (i.e. the displaying of an image whereby part of the image is from one frame, and another part of the image is from another frame). Thereafter, the display may be refreshed with the new frame at a rate corresponding to the maximum display refresh rate.
For example, a display may support a refresh rate range of 30 Hz to 120 Hz. Accordingly, it may not be possible to refresh the display at a rate faster than at a rate of 120 Hz without causing image tearing. Thus, if the display is refreshed with a current frame, and a new frame does not become available at a rate within 30 Hz to 120 Hz, the display is refreshed at a time corresponding to a 30 Hz refresh rate with the same current frame. The display, however, cannot be refreshed with the new frame, without image tearing, at a rate faster than 120 Hz. Thus, although a new frame may be rendered at a time corresponding to a render rate faster than 120 Hz (for example, 5 msec), the display may not be refreshed with the newly rendered frame sooner than until about 8.33 milliseconds (corresponding to a refresh rate of 120 Hz, i.e. 1/120 Hz seconds) have passed since the last display refresh. Thus, because the displaying of the new frame may be delayed, input lag, e.g., the amount of time between when a change to a display image is provided and when the result appears on the display, may be increased. For example, in video gaming, a key desire is to minimize input lag, so that when a user provides for an action (e.g. hits a key to move a character), the result of that action is seen as quickly as possible on the display.
Other problems, such as display flickering, may occur if a display is updated at a rate close to its maximum or minimum supported refresh rates. For example, if a display is updated at a rate near the minimum supported display refresh rate, followed by updating the display at a rate near its maximum supported display refresh rate, the display may flicker, causing unwanted discrepancies in the viewed image. Thus, for displays experiencing these issues, images ideally would be rendered and made available to them at rates well within their supported display refresh rate range. Therefore there is a need to minimize frame delays that may increase input lag times, along with other undesirable effects such as display flickering, in the displaying of images on displays that support variable refresh rates.
The embodiments will be more readily understood in view of the following description when accompanied by the below figures and wherein like reference numerals represent like elements, wherein:
Briefly, methods and apparatus to provide image content to, and display image content on, displays that support variable refresh rates that reduce frame delays and avoid display image flickering problems are disclosed. In one example, the methods and apparatus vary a display's refresh rate by, for example, varying a current frame's vertical blanking period (i.e. the time between the end of the final line of a frame and the beginning of the first line of the next frame) by re-providing the current frame for display prior to providing a new frame for display. In this fashion, the time period during which a display may be refreshed can be, in effect, advanced in time to ensure that the new frame can be provided for display as soon as it has been rendered and become available for display. The render rate of a provided video frame may be calculated, for example, with respect to the previously rendered frame or frames. For example, if a first frame is rendered at time a, and the next frame is rendered at time b, the render rate of the next frame may be calculated by: (1/(time b−time a)) frames per second (FPS). Similarly, the render rate may be calculated with respect to the times when subsequent frames are provided to a display, or to when subsequent frames are actually displayed.
In one example, by varying a display's refresh rate, new frames may be provided for display at a rate within a safe rate range such that display image flickering issues are avoided. For example, a new frame may be made available for display at a rate outside the safe rate range. To provide the new frame to the display at a rate within the safe rate range, the display's refresh rate is varied, for example, by re-providing a current frame to the display such that the vertical blanking period of the current frame is varied. In this fashion, the new frame may then be provided to the display at a rate within the safe rate range. Safe rate ranges may fall within a display's supported variable refresh rate range but exclude, for example, certain refresh rates within the display's supported variable refresh rate range. The excluded rates can be, for example, those that may lead to flickering of the display image. For example, a display may support a variable refresh rate range whereby the minimum refresh rate is 30 Hz and the maximum refresh rate 120 Hz. While the display supports this refresh range, the display may nonetheless exhibit less flicker when provided with frames at rates that are, for example, at least 5 Hz greater than the minimum or 5 Hz less than the maximum supported refresh rates (i.e. within the ranges of 35 Hz to 115 Hz). Accordingly, safe rate ranges may be determined for displaying frames.
Although the example of varying a display's refresh rate by varying a current frame's vertical blanking period is contemplated, it will be appreciated that display refresh rates may be varied by other methods as well. For example, a display's refresh rate may be varied by adjusting the pixel rate of the display, such as by adjusting the pixel clock to the display. Other methods of varying a display's refresh rate include adjusting the horizontal active pixel period, adjusting the horizontal blanking period, or any combination of these or other known methods.
In one embodiment, logic, for example, a processor such as a GPU, determines that a new frame is to be displayed at an expected new frame display rate, and varies the current frame's vertical blanking period by re-providing the current frame for display. The logic may re-provide the current frame for display, for example, at the maximum supported refresh rate of the display. For example, a display may have a display refresh rate range of 30 Hz to 120 Hz and a new frame may be rendered and made available for display at a time corresponding to a rate of 29 Hz. (This display refresh rate range is only exemplary and is not meant to be limiting. As is appreciated, a display refresh rate range may be a subset of continuous rates, any number of discrete rates, etc.) In this example, although the new frame was provided at a time corresponding to a rate of 29 Hz, that rate is too slow to be accommodated by the display with a minimum display refresh rate of 30 Hz. As a result, the display would have to be refreshed with a current frame, as the new frame was not available within a time period corresponding to the minimum supported display refresh rate of 30 Hz. In addition, this results in the new frame being further delayed from being displayed at least until the next possible refresh period after the display is refreshed with the current frame, because the display cannot be refreshed while the display is in an active refresh with the current frame if image tearing is to be prevented. For example, the new frame may be delayed for a time period corresponding to a rate of 120 Hz (i.e. 1/120 seconds) after refreshing with the current frame. To avoid these delays in displaying the new frame, the logic determines that the new frame will be made available for display at an expected new frame display rate that is too late (e.g. at 29 Hz which is below the 30 Hz minimum supported rate) and in response varies the current frame's vertical blanking period by re-providing the current frame for display. For example, rather than waiting a time corresponding to 30 Hz (i.e. 1/30 seconds) to see if the new frame becomes available, the logic re-inserts the current frame at a time corresponding to the maximum supported display rate of 120 Hz (i.e. 1/120 seconds), with the result that the new frame will become available at a rate within the display's variable refresh rate. For example, when the new frame becomes available, the new frame may be provided to the display immediately because the display is not in an active refresh with the current frame.
Thus, as one advantage, the amount of time it may take new content to be displayed is reduced. As a result, content from a new frame may be provided to the display as it becomes available. This may be advantageous, for example, in high-speed gaming applications, whereby user experience is enhanced by reducing the amount of time it takes new content to appear on one or more displays. Other advantages will be recognized by those of ordinary skill in the art.
In one embodiment, logic determines whether a new frame display rate for a new frame (the display rate determined for the new frame) falls within a frame reinsertion range that includes either the minimum supported display refresh rate or the maximum supported display refresh rate of the display. Thus, there may be more than one frame reinsertion range, such that one includes the minimum supported display refresh rate, and another includes the maximum supported display refresh rate, of the display. A frame reinsertion range includes a range of frame rates whereby the rates may include rates within, as well as outside of, a display's supported variable refresh rate range. For example, for a sample display with a variable refresh range of 30 Hz to 120 Hz, a frame reinsertion range may be determined to be 28 Hz to 32 Hz which includes the display's minimum supported display refresh rate (i.e. 30 Hz). Thus, the frame reinsertion range may go below the minimum supported refresh rate of the display, or may even have no lower bound. In one example, the frame reinsertion range includes the minimum supported display refresh rate, and the logic re-provides the current frame for display if the new frame display rate falls within the frame reinsertion range. The current frame is re-provided at a rate equal to or greater than the minimum supported display refresh rate of the display. The frame reinsertion range may be, for example, a percentage range from the minimum or maximum supported display refresh rate. In another example, the frame reinsertion rate may be determined empirically, may be based on historical operations, or any combination of these or other known methods.
In one embodiment, the determining, by the logic, of the new frame for display at the expected new frame display rate includes determining an average presentation rate for previously presented frames. The logic determines that a future frame will be provided by a frame generator for display at the expected new frame display rate when the determined average presentation rate is within the frame reinsertion range. For example, the recent average presentation time for new frames may be tracked. When the average presentation time gets close to or exceeds the maximum time the display can go without a new refresh (e.g. a frame reinsertion range that includes range the minimum supported display refresh rate), the display is refreshed again with the current frame as soon as the current refresh with the current frame completes. This may add more time for the rendering of the next new frame to complete before having to refresh the display.
In one example, the logic determines the average presentation rate for the previously presented frames by determining the average render rate for previously presented frames. For example, the logic may determine that the expected new frame rate is below the minimum supported refresh rate of a display based on an average render rate of previously rendered frames. In this example, the logic may re-provide a current frame for display such that if the new frame is rendered at a render rate below the minimum supported display refresh rate, the new frame may be provided to the display as soon as the new frame has been rendered. For example, assume that for a display with a minimum supported display refresh rate range of 30 Hz (corresponding to a maximum refresh period of approximately 33.33 milliseconds) and a maximum supported display refresh rate of 120 Hz (corresponding to a minimum refresh period of approximately 8.33 milliseconds), the expected new frame rate is determined to be approximately 29.4 Hz (e.g. indicating that it will take approximately 34 milliseconds to render the new frame). Rather than waiting a maximum refresh period of 33.33 milliseconds to determine if a new frame will be ready for display, and, assuming that a new frame was not ready for display prior to the expiration of the maximum refresh period, having to wait at least an additional minimum refresh period of 8.33 milliseconds before refreshing the display with a new frame, the current frame is re-provided to the display at a time corresponding to the minimum refresh period of 8.33 milliseconds. Thereafter, the new frame may be provided to the display when it becomes available if it becomes available at a time corresponding to a refresh rate within the display's supported refresh rate range. For example, if the new frame becomes available at a time corresponding to the previously expected new frame rate of 29.4 Hz, the new frame may be provided to the display at a time corresponding to a refresh rate of 38.9 Hz (e.g. 1/(34 milliseconds−8.33 milliseconds)). In one embodiment, the logic displays the current frame, the re-provided frame, and the new frame, where the re-provided frames are re-provided at the maximum supported display refresh rate. For example, the logic may include a display with a variable refresh rate.
As an alternate to averaging presentation or rendering rates, the expected new frame display rate may be determined in other ways. For example, the determination of the expected new frame display rate may be based on received content associated with a frame. In one example, the expected new frame display rate is based on information received in meta-data that is associated with received image content. The meta-data may include a declared predicted frame rate that indicates, for example, that video or movie content is provided at a certain rate, such as 25 or 30 frames per second. Another alternative to determining the expected new frame display rate is based on a median presentation or rendering rate. For example, the expected new frame display rate may be determined by calculating a median presentation rate for previously presented frames. Thus, the logic may, for example, determine that a future frame will be provided by a frame generator for display at an expected new frame display rate when a determined median presentation rate is within a frame reinsertion range.
In one embodiment, the frame reinsertion range includes the minimum supported display refresh rate, and the logic re-provides the current frame at a rate within a safe rate range that is within the display's supported variable refresh rate range. For example, for an example display with a variable refresh rate range of 30 Hz to 120 Hz, the frame reinsertion range may be determined to be 28 Hz to 32 Hz. If a frame is provided to the display at a time corresponding to a rate between 30 Hz and 32 Hz, the display image may flicker on this example display. The logic may determine a safe rate range that includes refresh rates above 32 Hz to 120 Hz. Thus, the logic may re-provide the current frame at a time corresponding to a rate within the safe rate range of 32 Hz to 120 Hz, for example, when a new frame is made available at a rate between 30 Hz and 32 Hz. In one example, a display may flicker when refreshed after a maximum refresh period corresponding to the display's minimum refresh rate followed by a minimum refresh period corresponding to the display's maximum refresh rate. However, by re-providing a current frame at a rate within the safe rate range, flicker of the display may be avoided, as further discussed below with respect to
In yet another embodiment, the frame reinsertion range includes the maximum supported display refresh rate, and the logic re-provides the current frame at a time corresponding to a rate within a safe rate range that is within the display's supported variable refresh rate range. For example, for an example display with a variable refresh rate range of 30 Hz to 120 Hz, the frame reinsertion range may be determined to be 118 Hz to 122 Hz. If a frame is provided at a time corresponding to a rate between 118 Hz and 120 Hz, the image may flicker on this example display. The logic may determine a safe rate range that includes refresh rates below 118 Hz down to 30 Hz. Thus, the logic may re-provide the current frame at a rate within the safe rate range of 30 Hz to 118 Hz, for example, when a new frame is made available at a rate between 118 Hz and 120 Hz. In one example, the logic may re-provide the current frame at a rate within a safe rate range when a new frame is predicted to become available at a rate within the frame reinsertion range.
In one embodiment, the logic determines a new frame presentation duration based on a time difference between when a previous frame is presented for display and when a current frame is presented for display. For example, a trigger event (e.g. an interrupt to a CPU) may indicate that a new frame has been rendered. When a frame is rendered, however, there may be a delay before the frame is ready to be presented to the display. For example there may be system delays associated with notification of a rendered frame, such as, for example, delays associated with a CPU having to be interrupted to access the rendered frame. Alternatively, there may be a delay associated with a CPU having to make system calls to gain access to the rendered frame. Once the frame is ready to be presented to the display, however, the logic may determine a presentation duration (e.g. delay from the time the previous frame was presented for display to the time the current frame is presented for display) for that new frame. For example, the logic may determine a current frame presentation duration to be 24 milliseconds for a display with a variable refresh range of 40 Hz to 144 Hz, where the frame reinsertion range is determined to be 40 Hz to 42 Hz. The logic may also determine that the expected new frame display rate is within a frame reinsertion range when the determined new frame presentation duration dictates that the new frame will be available for display at a rate that falls within the frame reinsertion range. If the frame presentation duration is such that the frame will be ready for display at a time corresponding to a rate that is within the frame reinsertion range (i.e. the frame will be ready for display at a rate of 40 to 42 Hz), the logic may then vary the vertical blanking period to the current frame by re-providing the current frame for display, as described above. In one example, the logic may determine the frame presentation duration based on a historical average of presentation durations of previous frames. In another example, the logic may determine the frame presentation duration based on the render rate of previously presented frames. In one embodiment, the logic includes a processor executing driver code that will monitor the presentation duration of frames, and may also determine and adjust the safe rate range of the display. The processor may also re-provide the current frame to the display, thereby varying the vertical blanking period of the current frame.
Turning now to the drawings, and as described in detail below, one example of the presently disclosed system is a device including logic 102 that provides display content and a display device 104 that supports a variable refresh rate. The logic is operative to generate rendered frames and to determine a frame reinsertion range and a new frame display rate, and is further able to vary the vertical blanking period of frames by re-providing current frames (e.g., the logic may insert additional current frames for display). If a new frame is made available for display, for example, at a rate that falls within the frame reinsertion range, the logic may refresh the display device with the current frame so as to vary the vertical blanking period of the current frame. The display device supports a variable refresh rate and is operative to receive provided content that may be displayed, in part or in whole, on a display.
Rendered frame generator 106 may generate rendered frames at a render rate that may be configurable or variable. For example, rendered frame generator 106 may include an accessible register that configures the render rate, among other functions. For example, a CPU that may be part of logic 102 may configure the render rate of rendered frame generator 106 via the accessible register. Rendered frame generator 106 may provide rendered frames via communication link 130, such as a bus, to frame buffer 108, whereby frame buffer 108 may store rendered frame data, such as new frames 109 and current frames 111. Alternately, rendered frame generator 106 may provide new frames directly to both frame buffer 108 and frame insertion and provider logic with varying blanking period control 112, thus alieving frame insertion and provider logic with varying blanking period control 112 from having to access frame buffer 108 for the new frame. Frame buffer 108 may be any suitable storage mechanism, including but not limited to memory, a hard drive, RAM, such as DDRAM or other suitable RAM, ROM in any suitable form, a cloud storage mechanism, or any suitable storage mechanism accessible via the web. Rendered frame generator 106 may also provide a new frame signal 134 to frame reinsertion range and new frame display rate determination logic 110 to determine whether the new frame was provided for display at a rate within a determined frame reinsertion range. For example, new frame signal 134 may be a signal indicating that a new frame has been rendered and is available for display, allowing frame reinsertion range and new frame display rate determination logic 110 to calculate the new frame render rate. Alternatively, new frame signal 134 may provide the new frame render rate, relieving frame reinsertion range and new frame display rate determination logic 110 from calculating the same. New frame signal 134 may also be provided to frame insertion and provider logic with varying blanking period control 112 to provide new frames over communication link 126, as discussed below.
Frame reinsertion range and new frame display rate determination logic 110 may receive a display's supported refresh rate range over communication link 122. Communication link 122 may be any suitable communication link that allows for the communication of display refresh rate range data which may be part of a display's extended display identification data (EDID). The frame reinsertion range and new frame display rate determination logic 110 may optionally receive, over communication link 124, display scan information such as whether the display is currently refreshing, and the current scan location that the display is refreshing at. Communication link 124 may be any suitable communication link that allows for the communication of display scan information.
Frame reinsertion range and new frame display rate determination logic 110 determines a frame reinsertion range, and also determines the rate at which a new frame may be made available for display (e.g. an expected new frame display rate). For example, the rate at which a new frame may be made available for display may be a predicted rate based on when the previous frame was made available for display. Alternately, frame reinsertion range and new frame display rate determination logic 110 may determine a rate at which a future frame may be made available for display, such as by determining an average presentation rate for previously presented frames and using the average presentation rate to predict the future frame display rate, as described above. Frame reinsertion range and new frame display rate determination logic 110 provides frame reinsertion signal 136 to frame insertion and provider logic with varying blanking period control 112, which indicates whether a new frame has been determined to be displayed at a rate within the frame reinsertion range. If frame reinsertion signal 136 indicates that a new frame is to be displayed at a rate within the determined frame reinsertion range, frame insertion and provider logic with varying blanking period control 112 may vary the vertical blanking period of a current frame for display by re-providing to the display device 104 the current frame over communication link 126. For example, frame insertion and provider logic with varying blanking period control 112 may access the current frame 111 in frame buffer 108 over communication link 132, and re-provide the current frame to display device 104. Communication link 126 allows frame insertion and provider logic with varying blanking period control 112 to provide display device 104 with frames for display, for example, such as new and current frames, over communication link 126. As noted above, frame insertion and provider logic with varying blanking period control 112 may receive new frames either directly from 106, or by accessing new frame 109 in frame buffer 108.
In another example, frame reinsertion range and new frame display rate determination logic 110 and frame insertion and provider logic with varying blanking period control 112 may both receive an indication that a new frame has been rendered, as indicated by new frame signal 134. Frame reinsertion range and new frame display rate determination logic 110 may then determine whether the new frame render rate is within a determined frame reinsertion range, and may indicate this determination over frame reinsertion signal 136 to frame insertion and provider logic with varying blanking period control 112. If the indication over frame reinsertion signal 136 is that the new frame rate is not within the determined frame reinsertion range, frame insertion and provider logic with varying blanking period control 112 may then provide the new frame to display device 104 over communication link 126 during a time period corresponding to a rate within the display device's 104 supported display refresh rate.
Communication links 122, 124, and 126 may each be any suitable communication link, including but not limited to a Display Port link, an HDMI link, I2C link, or any other suitable link, or may each be part of communication link 128. Communication link 128 may include other data communication signals or links, and may also be any suitable communication link, including Display Port, HDMI, or I2C.
Display device 104 may be a display with a variable refresh rate and may include interface 114, display driver 116, display 118, and frame buffer 120. Interface 114 may provide information on communication links 122 and 124, and may receive frame content data on communication link 126, as described above. Interface 114 also provides frame content data over communication link 140. For example, interface 114 may receive frame data (e.g. frames) over communication link 126, and provide the data to display 118. In one embodiment, frame insertion and provider logic with varying blanking period control 112 provides frames directly to driver 116 via interface 114, for example, without a need for a frame buffer such as frame buffer 120. Display driver 116 may then receive frames from interface 114 over communication link 140, and, in turn, display driver 116 may provide the frames to display 118 over communication link 142. This embodiment may minimize delays and resources associated with implementing a frame buffer 120 in display 104.
In one example, frame insertion and provider logic with varying blanking period control 112 is in control of when frames are provided by display driver 116 to display 118 (e.g. display driver 116 provides frames to display 118 as they are received from frame insertion and provider logic with varying blanking period control 112). In another example, display driver 116 may have some or full control of when frames are provided to display 118. In yet another embodiment, interface 114 provides frames to frame buffer 120 over communication link 138. Display driver 116 would then access frame content data in frame buffer 120 via interface 114, and provide the frame content data for display to display 118. For example, the display driver 116 may, every display refresh cycle, obtain frame content data from frame buffer 120 and provide it to display 118. For example, if display device 104 supports a display refresh rate range of 30 Hz to 120 Hz, the display driver 116 may update the display 118 between approximately every 8.33 milliseconds to 33.3 milliseconds (i.e. 1/120 seconds to 1/30 seconds).
The display driver 116 may also contain display refresh rate range registers 117, which may be EDID registers, indicating the supported display refresh rate range of the display. For example, the display refresh rate range registers 117 may allow logic 102 to determine the minimum and maximum supported refresh rates of display device 104, which would allow logic 102 to provide frame content for display at those rates. The display refresh rate range registers 117 may also allow logic 102 to configure display driver 116 to refresh display 118 during a time period corresponding to a rate within the range of supported refresh rates.
The example method begins at block 202 where a new frame is determined to be displayed at an expected new frame display rate, such as may be performed by frame reinsertion range and new frame display rate determination logic 110 of
In some examples, executable suitable instructions may be stored on a computer readable storage medium, where the executable instructions are executable by one or more processors to cause the one or more processors to perform the actions described herein.
First processor 604 and second processor 606 may access memory 608 over bus 614. Bus 614 can be, for example, any number of interconnects allowing communication among the various devices. Display 602 may include some or all of the functionality of the display device 104 of
The foregoing description has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the exemplary embodiments disclosed. Many modifications and variations are possible in light of the above teachings. It is intended that the scope of the invention be limited not by this detailed description of examples, but rather by the claims appended hereto. The above detailed description of the embodiments and the examples described therein have been presented for the purposes of illustration and description only and not by limitation. It is therefore contemplated that the present invention cover any and all modifications, variations, or equivalents that fall within the spirit and scope of the basic underlying principles disclosed above and claimed herein.
The present application is related to U.S. patent application Ser. No. 14/589,560 (Attorney Docket No. 100100.140308), entitled “EXTENDING THE RANGE OF VARIABLE REFRESH RATE DISPLAYS,” which was filed on Jan. 5, 2015 and the contents of which are incorporated by reference herein for all that it discloses.
