Patent Application
20190371264
A typical processing system employs a graphics processing unit (GPU) to generate images for display. In particular, based on information received from a central processing unit (CPU) or other processing unit, the GPU generates a series of frames and renders the series of frames at a display, such as a computer monitor. Two different timing factors govern the rate at which the series of frames can be displayed: the rate at which the GPU generates frames and the refresh rate of the display. Some processing systems improve the user experience by synchronizing the display refresh with the generation of frames at the GPU. For example, by adjusting the blanking interval of the display, the processing system can ensure that the display is refreshed at or near the time that a new frame is ready for display at the GPU. However, conventional synchronization techniques can cause flickering and other visual artifacts that negatively impact the user experience.
The present disclosure may be better understood, and its numerous features and advantages made apparent to those skilled in the art by referencing the accompanying drawings. The use of the same reference symbols in different drawings indicates similar or identical items.
The processing system 100 is generally configured to execute sets of instructions (e.g., computer programs) to carry out specified tasks for an electronic device. Examples of such tasks include controlling aspects of the operation of the electronic device, displaying information to a user to provide a specified user experience, communicating with other electronic devices, and the like. Accordingly, in different embodiments the processing system 100 is employed in one of a number of types of electronic device, such as a desktop computer, laptop computer, server, game console, tablet, smartphone, and the like.
To support execution of the sets of instructions, the processing system 100 includes a plurality of processor cores (not shown at
The GPU 102 is generally configured to receive the commands and data associated with graphics and other display operations from the plurality of processor cores. Based on the received commands, the GPU 102 executes operations to generate frames (e.g., frame 115) for display. Examples of operations include vector operations, drawing operations, and the like. The rate at which the GPU 102 is able to generate based on these operations is referred to as the frame generation rate, or simply the frame rate, of the GPU 102. The frame generation rate is illustrated at
To display frames, the processing system 100 includes a display control module 104 and a display 110. The display 110 is a display device generally configured to visually display images based on the frames generated by the GPU 102. Accordingly, in different embodiments the display 110 is a liquid crystal display (LCD) device, an organic light-emitting diode (OLED) device, and the like. As will be appreciated by one skilled in the art, the display 110 is generally configured to periodically render (or “draw”) the most recent frame generated by the GPU 102, thereby displaying the frame. Each frame render is associated with a portion of time, referred to as a blanking interval, during which the display 110 does not render image data. The display 110 is configured to have a blanking interval of programmable length. Accordingly, as described further herein, the refresh rate 108 of the display 110 is adjustable by programming different lengths for the blanking interval.
The display control module 104 is generally configured to control the rendering of frames at the display 110. Accordingly, in some embodiments the display control module 104 performs operations including buffering of frames generated by the GPU 102, adjustment of the refresh rate 108 of the display 110 by programming different blanking interval lengths, and the like. It will be appreciated that although the display control module 104 is illustrated as a separate module from the GPU 102 for ease of illustration, in some embodiments the display control module 104 is incorporated in the GPU 102. In other embodiments, one or more operations of the display control module 104 are performed at the display 110.
In some embodiments, the display control module 104 manages synchronization of the frame rate 105 with the refresh rate 108. By synchronizing the two rates so that they match within a specified threshold margin, the display control module 104 ensures that frames generated by the GPU 102 are displayed relatively quickly at the display 110, thereby supporting smoother display of moving objects, reduced likelihood of visual artifacts, and otherwise improving the user experience with the processing system 100.
To synchronize the generation of frames at the GPU 102 with the display of those frames, the display control module 104 monitors the frame rate 105. In some embodiments, the frame rate 105 represents a moving average of the rate at which the GPU 102 generates frames over a specified number of frames, such as over the previous N generated frames, where N is an integer. In response to a change in the frame rate 105, the display control module 104 adjusts the refresh rate 108 over time until the refresh rate 108 and the frame rate 105 match within a specified margin of error. In some embodiments, the display control module 104 adjusts the refresh rate 108 by programming different values for the blanking interval of the display 110.
As noted above, large adjustments to the refresh rate 108 can cause flickering or other visual artifacts at the display 110. To ameliorate the likelihood of such visual artifacts, the display control module 104 limits the incremental adjustment of the refresh rate 108. To illustrate, in response to identifying that the refresh rate 108 is to be adjusted based on a change in the frame rate 105, the display control module 104 determines the amount of adjustment needed to synchronize the frame rate 105 and the refresh rate 108. For purposes of description, this amount of adjustment is referred to as the full adjustment. The display control module 104 compares the full adjustment to a threshold, wherein the threshold corresponds to an adjustment that is predicted to be unlikely to result in flickering or other visual artifacts at the display 110. In response to identifying that the full adjustment is less than the threshold, the display control module 104 adjusts the refresh rate 108 by the full adjustment. In response to identifying the full adjustment exceeds the threshold, the display control module 104 adjusts the refresh rate over a number of rendered frames in specified increments corresponding, for example, to the threshold. Thus, for example, if the threshold is 1 millisecond (ms), indicating than an adjustment in the refresh rate 108 by more than 1 ms is likely to cause flicker, and the full adjustment is 3 ms, the display control module 104 adjusts the refresh rate by 1 ms each for the next three frames. The display control module 104 thereby reduces the likelihood that changes in the refresh rate 108 will result in visual artifacts, while maintaining relatively frequent synchronization between the refresh rate 108 and the frame rate 105.
In some embodiments, both positive and negative adjustments in the refresh rate 108 of greater than the threshold amount result in visual artifacts. Accordingly, in some embodiments, the display control module 104 stores the threshold as a refresh adjustment window 118 that indicates a range of adjustments, such as plus or minus 1 ms. In response to a change in the frame rate 105, the display control module 104 compares the corresponding full adjustment to boundaries of the refresh adjustment window 118. If the full adjustment would not result in an adjustment outside the boundaries of the refresh adjustment window 118. If not, the display control module 104 adjusts the refresh rate 108 by the full adjustment. If the full adjustment would result in an adjustment outside the boundaries of the refresh adjustment window 118, the display control module 104 incrementally adjust the refresh rate 108 over a plurality of frames as described above.
In some embodiments, the display control module 104 sets the size of the refresh adjustment window 118 based on a luminance profile 109 of the display 110. The luminance profile 109 reflects luminance characteristics of the display 110, and in particular reflects a sensitivity of the display 110 to changes in the refresh rate 108. Thus, for a display having a luminance profile 109 indicating relatively high sensitivity to changes in the refresh rate 108, the display control module 104 sets the size of the refresh adjustment window 118 to be relatively small. For a display having a luminance profile 109 indicating relatively low sensitivity to changes in the refresh rate 108, the display control module 104 sets the size of the refresh window to be relatively large. In some embodiments, the luminance profile 109 varies over time based on aging of the display 110, different operating conditions of the processing system 100, different types of graphics being displayed via the frames generated by the GPU 102 and the like. Accordingly, in some embodiments the display control module 104 periodically detects changes in the luminance profile 109 and based on the detected changes adjusts the size of the refresh adjustment window 118. The processing system 100 thereby continues to suppress flickering and other artifacts that could result from relatively large changes in the refresh rate 108 even as the luminance profile 109 changes.
Examples of the processing system 100 adjusting the refresh rate 108 are illustrated at
At a time 202, the display control module 104 detects an increase in the frame rate 105 of 0.5 ms, resulting in a frame rate of 20.5 ms as illustrated by interval 212. In response to the change in the frame rate 105, the display control module 104 determines that the corresponding full adjustment to the refresh rate 108 (that is, the total amount of adjustment to maintain synchronization between the frame rate 105 and the refresh rate 108) is 0.5 ms. The display control module 104 further identifies that the full adjustment is within the boundaries of the refresh adjustment window 118. Accordingly, the display control module 104 adjusts the refresh rate 108 by the full adjustment of 0.5 ms to 20.5 ms, as illustrated by interval 213. Thus, in the depicted example the GPU 102 completes rendering a frame between time 202 and a time 203, and the display 110 draws a frame between the same two time periods, thus maintaining synchronization of frame drawing and frame display.
At a time 302, the display control module 104 detects an increase in the frame rate 105 of 2 ms, resulting in a frame rate of 22 ms as illustrated by interval 312. In response to the change in the frame rate 105, the display control module 104 determines that the corresponding full adjustment to the refresh rate 108 is 2 ms. The display control module 104 further identifies that the full adjustment is greater than the upper boundary of the refresh adjustment window 118. Accordingly, the display control module 104 adjusts the refresh rate 108 by an adjustment corresponding to the upper boundary of plus 1 ms, resulting in a refresh rate of 21 ms as illustrated by interval 313. Thus, in the depicted example the display 110 draws a frame between the time 302 and a time 303, while the GPU initiates rendering of a frame at time 302 but does not complete rendering the frame until after time 303. The frame rate 105 and the refresh rate 108 are therefore at least temporarily unsynchronized. In some embodiments, such as illustrated below with respect to
At a time 402, the display control module 104 detects a decrease in the frame rate 105 of 2 ms, resulting in a frame rate of 18 ms as illustrated by interval 412. In response to the change in the frame rate 105, the display control module 104 determines that the corresponding full adjustment to the refresh rate 108 is −2 ms. The display control module 104 further identifies that the full adjustment is lower than the lower boundary of the refresh adjustment window 118. Accordingly, the display control module 104 adjusts the refresh rate 108 by an adjustment corresponding to the lower boundary of −1 ms, resulting in a refresh rate of 19 ms as illustrated by interval 413. Thus, in the depicted example the display 110 draws a frame between the time 402 and a time 403, while the GPU initiates rendering of a frame at time 302 and completes rendering the frame prior to time 403, such that the frame rate 105 and the refresh rate 108 are at least temporarily unsynchronized. In some embodiments, such as illustrated below with respect to
At a time 502, the display control module 104 detects a decrease in the frame rate 105 of 3 ms, resulting in a frame rate of 17 ms as illustrated by interval 512. In response to the change in the frame rate 105, the display control module 104 determines that the corresponding full adjustment to the refresh rate 108 is −3 ms. The display control module 104 further identifies that the full adjustment is lower than the lower boundary of the refresh adjustment window 118. Accordingly, the display control module 104 adjusts the refresh rate 108 by an adjustment corresponding to the lower boundary of −1 ms, resulting in a refresh rate of 19 ms as illustrated by interval 513.
At or around the completion of the interval 513 at time 503, the display control module 104 identifies that the frame rate 105 remains at 17 ms, and therefore again adjusts the refresh rate 108 by −1 ms, resulting in a refresh rate of 18 ms as illustrated by interval 515. Finally, at or around the completion of the interval 515 at time 501, the display control module 104 once again identifies that the frame rate 105 remains at 17 ms, and therefore once again adjusts the refresh rate 108 by −1 ms, resulting in a refresh rate of 17 ms as illustrated by interval 517. Thus, in the example of
If, at block 604, the GPU identifies that the refresh rate and the frame rate are different, the method flow moves to block 606 and the GPU identifies the full adjustment to the refresh rate that is required to synchronize the refresh rate with the frame rate of the GPU. At block 608, the GPU identifies whether the full adjustment falls outside the boundaries of a refresh adjustment window. If not, the method flow proceeds to block 610 and the GPU changes the refresh rate of the display by the full adjustment, thereby synchronizing the frame rate of the GPU and the refresh rate of the display. The method flow returns to block 602.
Returning to block 608, if the GPU identifies that the full adjustment falls outside the boundaries of the refresh adjustment window, the method flow moves to block 612 and the GPU changes the refresh rate by a maximum increment, corresponding to one of the boundaries of the refresh adjustment window. The method flow returns to block 602 and the GPU waits for the next frame to once again identify the current frame rate. If the frame rate and the refresh rate remain unsynchronized over several frames, the GPU continues to iteratively adjust the refresh rate over the several frames until the refresh rate and the frame rate are synchronized.
In some embodiments, certain aspects of the techniques described above may implemented by one or more processors of a processing system executing software. The software comprises one or more sets of executable instructions stored or otherwise tangibly embodied on a non-transitory computer readable storage medium. The software can include the instructions and certain data that, when executed by the one or more processors, manipulate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer readable storage medium can include, for example, a magnetic or optical disk storage device, solid state storage devices such as Flash memory, a cache, random access memory (RAM) or other non-volatile memory device or devices, and the like. The executable instructions stored on the non-transitory computer readable storage medium may be in source code, assembly language code, object code, or other instruction format that is interpreted or otherwise executable by one or more processors.
A computer readable storage medium may include any non-transitory storage medium, or combination of non-transitory storage media, accessible by a computer system during use to provide instructions and/or data to the computer system. Such storage media can include, but is not limited to, optical media (e.g., compact disc (CD), digital versatile disc (DVD), Blu-Ray disc), magnetic media (e.g., floppy disc, magnetic tape, or magnetic hard drive), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or Flash memory), or microelectromechanical systems (MEMS)-based storage media. The computer readable storage medium may be embedded in the computing system (e.g., system RAM or ROM), fixedly attached to the computing system (e.g., a magnetic hard drive), removably attached to the computing system (e.g., an optical disc or Universal Serial Bus (USB)-based Flash memory), or coupled to the computer system via a wired or wireless network (e.g., network accessible storage (NAS)).
Note that not all of the activities or elements described above in the general description are required, that a portion of a specific activity or device may not be required, and that one or more further activities may be performed, or elements included, in addition to those described. Still further, the order in which activities are listed are not necessarily the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, one of ordinary skill in the art appreciates that various modifications and changes can be made without departing from the scope of the present disclosure as set forth in the claims below. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present disclosure.
Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and any feature(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential feature of any or all the claims. Moreover, the particular embodiments disclosed above are illustrative only, as the disclosed subject matter may be modified and practiced in different but equivalent manners apparent to those skilled in the art having the benefit of the teachings herein. No limitations are intended to the details of construction or design herein shown, other than as described in the claims below. It is therefore evident that the particular embodiments disclosed above may be altered or modified and all such variations are considered within the scope of the disclosed subject matter. Accordingly, the protection sought herein is as set forth in the claims below.
