Patent Application
20090179909
At least one embodiment of the present invention pertains to systems that include display devices, such as televisions, video monitors, and the like, and more particularly, to a technique for triggering a screen saver for a display device.
Most televisions today are based on display technologies that suffer from a problem known as “burn-in”, which is damage to the display from long-persisting static images. The result of burn-in is an undesirable afterimage which can be seen if the set is turned off and/or while a flat color is being displayed. This problem is traditionally addressed in computers by using a “screen saver,” which either modifies the screen to use colors or intensities which are less likely to cause burn-in, or changes the display entirely to a display designed to age the screen uniformly so that no burn-in pattern is visible. Such screen savers are typically triggered by inactivity, i.e., a lack of user input from the keyboard or mouse. However, since television is typically a passive experience, it is common for the television controls not to be touched for long periods of time. As such, an inactivity-based screen saver trigger such as used in computers is generally not appropriate for televisions.
There are various known approaches to solving the burn-in problem for televisions. These are continuous approaches where no “screen saver” function is triggered to prevent the burn-in problem. One such approach is “picture rotation,” where the entire picture is shifted slightly up, down, left, or right of center over time. This technique would cause any sharp transitions on the screen, where burn-in may be more noticeable, to be blurred. However, it does not eliminate the effects of burn-in.
Another technique is to change the brightness of sidebars to match the average brightness of the screen. This technique avoids a distinctive burn-in pattern between the center of the screen and the sidebars. However, since the sidebars represent a stationary, non-changing pattern that can remain on the screen for thousands of hours during the normal course of watching television programs, burn-in still eventually occurs.
One or more embodiments of the present invention are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements and in which:
A technique for determining when to trigger a screen saver, and a display system in which such a technique can be implemented, are described. Note that references in this specification to “an embodiment”, “one embodiment”, or the like, mean that the particular feature, structure or characteristic being described is included in at least one embodiment of the present invention. Occurrences of such phrases in this specification do not necessarily all refer to the same embodiment. Further, the embodiments referred to are not necessarily mutually exclusive, unless so stated.
The technique introduced here can be implemented in any system that incorporates or cooperates with a display device that is susceptible to burn-in, such as a television or video monitor. A “television”, as the term is used herein, is any system that is capable of receiving, decoding and displaying television signals. A “video monitor”, as the term is used herein, is a display device which lacks a tuner.
As described in greater detail below, technique introduced here examines specific pixels of a display image, looking for situations where some of those pixels do not change significantly over a long period of time, even though other adjacent pixels may be changing. If such a situation is detected, a trigger event is generated to indicate the need to invoke a screen saver. If the screen saver has already been triggered, the technique introduced here also provides a method of automatically deactivating it or turning it off, thus restoring normal video display. This approach is specifically designed to handle cases such as DVD menus, which may have animation in the background and fixed text in the foreground. This approach is completely automatic and will be invoked when a fixed pattern is detected anywhere on the screen for a sufficiently long time period.
Thus, in one embodiment the method introduced here includes selecting, as a sample set, a sparse subset of the pixels that collectively form a display frame. A “sparse subset” of a frame is a subset which includes much fewer than all of the pixels that make up a complete display frame, such as, for example, fewer than 10% of all of the pixels in the frame. The term “frame”, as used herein, also is intended to refer to a field in systems where two fields make up a frame, such as in interlaced video.
During each frame, an evaluation is performed of only the pixels in the sample set, to determine whether enough pixels within the sample set have undergone enough change, over immediately preceding frames and the current frame, such that that burn-in is unlikely. If a sufficient number of pixels in the sample set are determined not to have changed enough at any given point in time, the screen saver is triggered.
In one embodiment, the method introduced here first takes several samples of each pixel in the sample set and averages them. Then it continuously tests each subsequent pixel against this initial averaged value. If the value has changed, then a state variable for that pixel resets and the process begins over. If the value stays the same for more than some predetermined number of frames, then this pixel is counted toward an “Overtime” count for the entire sample set. If more than some threshold number of pixels in the sample set are “overtime”, the screen saver is triggered.
In certain embodiments, the sample set of pixels is defined so that any horizontal or vertical single pixel thick line that completely crosses the analyzed screen area intersects at least one pixel in the sample set. Ways of achieving this objective include, for example, employing a staggered sampled pixel grid, or using a rotated, uniformly sampled pixel grid.
In addition, in certain embodiments, multiple comparison values are employed to determine the presence or absence of screen activity. There are situations in which two or more comparison values are used to provide a more accurate indication of screen activity. In these cases, if the sampled pixel matches any of the compared values within a narrow range, then the pixel is considered not to have changed. Such situations include the accommodation of flashing text, or the handling of set-top boxes which provide poorly implemented “screen savers” which cause a fixed pattern to be displayed in a limited number of screen areas. Such a screen saver approach actually causes screen burn-in those areas.
Referring now to
Pixels within such a grid can be chosen as follows. Assume the origin point (0,0) of a frame is the top left corner of the frame, with the X coordinate increasing to the right and the Y coordinate increasing downward. Assume further that the screen has width W and height H in pixels, and that the analyzed area is offset AX pixels horizontally and AY pixels vertically from the top-left corner with width AW and height AH, with the following constraints:
AX≧0,
AY≧0,
AX+AW≦W,
AY+AH≦H
A horizontal sampling interval SiX and a vertical sampling interval SiY can be chosen to create an array that comprises some number, NUM_PIXELS_TO_SAMPLE, of sampled point coordinates, where NUM_PIXELS_TO_SAMPLE=(AW/SiX)*(AH/SiY). For example, the sampled points at coordinates (SX[i], SY[i]) can be selected according to the following formula, where x varies from 0 to (AW/SiX)−1 and y varies from 0 to (AH/SiY)−1:
SX[i]=AX+(((x*SiX)+y)moduloAW);
SY[i]=AY+(((y*SiY)+x)moduloAH);
The number of sampled pixels may be as large as the total number of pixels in the analyzed area but could be much smaller and still yield nearly the same performance. For example, the total number of pixels may be as small as, for example, 1% of the total pixels in the analyzed area, perhaps smaller. In the staggered grid example above, in order to fulfill the horizontal and vertical line crossing requirements, the following criteria must be met:
SiX≦AH/SiY
SiY<AW/SiX
If SiX and SiY are made equal, then the maximum value of SiX and SiY is set to SiX=SiX≦min(SQRT(AW), SQRT(AH)), and the minimum value of NUM_PIXELS_TO_SAMPLE becomes NUM_PIXELS_TO_SAMPLE≧SQRT(AW*AH).
Note that there are other possible arrangements of sampled pixels that may be used. For example, a pattern of concentric circles or even a random sampling of pixels within the analyzed area may yield equivalent or acceptable results. The technique introduced here does not preclude using alternative arrangements of sampled pixels. However, the set of sampled pixels, and hence their pattern, should be fixed before any sampling takes place.
As an aid in improving performance, the device addresses of the pixels to be sampled can be cached. At its simplest, this address may be computed as BaseAddress+SY[i]*W+SX[i] although it could involve a much more complex hardware-specific address translation function. This cache may be made relative to the beginning of a frame buffer so that the addresses of different frame buffers with the same dimensions, which are typically used during video processing, can be easily calculated using the same cache.
In certain embodiments, a separate state machine is implemented for each pixel in the sample set, and each state machine can determine state over multiple frames for its pixel. The state machines for the entire sample set collectively determine the value of a variable, Overtime, for each frame. The variable Overtime is generally indicative of the number of pixels in the sample set that have remained relatively unchanged for a significant period of time (in terms of burn-in potential), as measured during the current frame. The variable Overtime applies to the entire sample set and is evaluated during each frame to determine whether to trigger the screen saver. In particular, during each frame, Overtime is compared to a threshold value, OvertimeThresh; if Overtime exceeds OvertimeThresh, the screen saver is triggered, otherwise, the screen saver is not triggered. OvertimeThresh is selected in advance based upon N (the sample size) and the burn-in potential for the particular type of display device being used.
At 302 the state machine determines whether the value of the pixel in the current frame differs from the average for that pixel by some predetermined difference value, NEAR_AVG_EPSILON. If it does, the process proceeds to 303, in which the state machine determines whether the difference between the current pixel value and the average has exceeded NEAR_AVG_EPSILON for a predetermined number of consecutive frames. This predetermined number of frames depend upon the burn-in potential for the particular type of display device being used; in one example, this is 12 frames. If the outcome of 303 is affirmative, this means the pixel is changing, so at 304 the state machine resets a counter variable, Count, for this pixel to one (1). The variable Count represents the number of frames for which the pixel has remained relatively unchanged. The state machine then waits until the next frame (305) for further processing. At the start of the next frame, Overtime is reset to zero at 309 (note, however, that there is one Overtime value per frame, it is not a per-pixel variable) and the state machine loops back to 302, described above. If the outcome of 303 is negative, then the state machine bypasses 304 and proceeds directly to 305, where it waits until the next frame.
Referring again to 302, if the state machine determines that the current value of the pixel does not differ from the average by at least NEAR_AVG_EPSILON, then the state machine proceeds to 306, where it increments Count for the pixel. The state machine then determines at 307 whether Count for the pixel exceeds a threshold count value, Mintime. Mintime is in units of frames and is chosen in advance based upon the burn-in potential for the particular type of display device being used. In one embodiment, Mintime is chosen to be 36,000 frames (which, at a typical frame rate of 60 Hz, corresponds to 10 minutes). If Count exceeds Mintime, then at 308 the state machine increments Overtime (note that Overtime is affected by all pixels in the sample set). The process then proceeds to 305, described above. If Count does not exceed Mintime at 307, then the process proceeds immediately from 307 to 305, described above.
Thus, in one embodiment the process introduced here first takes four samples of each pixel and averages them. Then the process continuously tests each subsequent pixel against this initial averaged value. If the value changes, then a state variable for that pixel resets and the process begins over. If the value stays the same for more than Mintime frames, then this pixel is counted toward the Overtime count. If more than OvertimeThresh pixels are “overtime”, the screen saver is triggered.
The pixel is initialized to state 0. The state of the pixel is then adjusted according to an algorithm, which is described in greater detail below. State 1 adds up the values of the pixel for some number (e.g., four) of consecutive frames. State 1 then computes the average value of the pixel for those four frames. As noted above, the use of four frames to compute the average is an example of how this process may be implemented.
State 2 compares the difference between the current value of the pixel and the average for the pixel against the difference value, NEAR_AVG_EPSILON, as described above (302), to determine whether the pixel has changed significantly from the average (where “significantly” depends on NEAR_AVG_EPSILON). If the pixel has not changed significantly (306), then Count for the pixel is incremented. Further, if Count for the pixel exceeds a value, MIN_FRAMES_NEAR AVG (307), then Overtime (which applies to the entire sample set) is also incremented.
Once a pixel is in state 2, it can remain in state 2 indefinitely for any number of frames, which will occur if the pixel does not change significantly from frame to frame. On the other hand, if the pixel does change significantly, then the pixel will proceed from state 2 to state 3.
In state 3, essentially the same comparison is made as in state 2, as just described, to determine whether the pixel has changed significantly. When in state 3, if the pixel has not changed significantly (306), then the state of the pixel goes back to state 2. However, if the pixel was determined to have changed significantly, then its state may be reset to state 0 or remain in state 3, depending on the number (NumDiff) of consecutive frames for which the pixel has deviated significantly from the average value. Specifically, if the pixel has changed significantly for only a “short” time (i.e., for less than MAX_FRAMES_NOT_NEAR_AVG frames), the pixel remain in state 3. If the pixel has changed significantly for a “long” time (i.e., more than MAX_FRAMES_NOT_NEAR_AVG frames), the state of the pixel will be reset to state 0.
At 502, Luma, the luminance value for pixel i for the current frame, is retrieved (e.g., from cache memory). A set of operations is then performed which depend on the current state of the pixel.
If the state of the pixel, State(i), is 0 (503), then at 504, Count(i) is set equal to 1, Value(i) is set equal to Luma, and the state of the pixel, State(i), is set equal to 1.
After 504, the process proceeds to 505, where the index variable i is compared to the value NUM_PIXELS_TO_SAMPLE. If the index variable i equals NUM_PIXELS_TO_SAMPLE (which means that this process has processed the entire sample set of pixels for the current frame), then the process proceeds to 506. Otherwise, the process increments I at 509 and loops back to 502.
If the variable Screen Saver is FALSE at 506 (meaning the screen saver is off) and Overtime exceeds a first threshold, ACTIVATE_OVERTIME_THRESHOLD, at 507, then the process triggers the screen saver at 508 by setting the variable Screen Saver to TRUE. If Screen Saver is found to be FALSE at 506 but Overtime is found to be less than or equal to ACTIVATE_OVERTIME_THRESHOLD at 507, the process then branches to 511.
At 511, if the screen saver is already on (i.e., Screen Saver is TRUE), then at 512 Overtime is compared to a second threshold, DEACTIVATE_OVERTIME_THRESHOLD. If Overtime is less than or equal to DEACTIVATE_OVERTIME_THRESHOLD at 512, the screensaver is deactivated by setting Screen Saver to FALSE at 513.
After activating or deactivating the screen saver at 508 or 513, respectively, or after a negative outcome of decision 511 or decision 512, the variable Overtime and the index variable i are both reset to zero at 510, and the process then branches back to 502 to process the first pixel in the sample set for the next frame.
Referring back to 502, after getting the pixel value Luma, if State(i) is equal to 1 (i.e., the pixel is in State 1) (514), then at 515 Value(i) is incremented by Luma (the current value of the current pixel) and Count(i) is incremented by one. Next, at 516 the process determines whether Count(i) is greater than or equal to the (tunable) value, NUM_FRAMES_TO_AVG, which is the number of frames over which each sample pixel is averaged. If Count(i) is greater than or equal to NUM_FRAMES_TO_AVG, then Value(i) is set equal to the average of the Value(i)'s for the last NUM_FRAMES_TO_AVG frames (including the current frame). State(i) is then set equal to 2, in the process then branches back to 505, described above. If Count(i) is less than NUM_FRAMES_TO_AVG at 516, then from 516 the process branches directly back to 505.
Referring again back to 502, after getting the pixel value Luma, if State(i) is equal to 2 (i.e., the pixel is in State 2) (519), then at 520 the process compares the magnitude of (Value(i)−Luma) to NEAR_AVG_EPSILON. If the magnitude of (Value(i)−Luma) is less than NEAR_AVG_EPSILON, then at 521 Count(i) is incremented by one. After 521, the process determines whether Count(i) is greater than the value MIN_FRAMES_NEAR_AVERAGE at 522. If Count(i) is greater than that value, then Overtime is incremented by one at 512, and the process then branches back to 505 as described above. Otherwise, at 528 the process sets State(i) equal to 3 and sets a counter variable NumDiff(i) equal to 1, and then branches back to 505. Likewise, if the magnitude of (Value(i)−Luma) is greater than or equal to NEAR_AVG_EPSILON, then from 520 the process sets State(i) equal to 3 and sets NumDiff(i) equal to 1 at 528, and then branches back to 505.
Referring again back to 502, after getting the pixel value Luma, if State(i) is not 0, 1 or 2, then State(i) must be equal to 3 (i.e., the pixel is in State 3). In that case, at 524 the process compares the magnitude of (Value(i)−Luma) to NEAR_AVG_EPSILON. If the magnitude of (Value(i)−Luma) is less than NEAR_AVG_EPSILON, then the process branches to 518, where State(i) is set equal to 2 as described above. If the magnitude of (Value(i)−Luma) is greater than or equal to NEAR_AVG_EPSILON at 524, then NumDiff(i) is incremented by one at 525, and the process next compares NumDiff(i) to a value, MAX_FRAMES_NOT_NEAR_AVG, at 526. If NumDiff(i) is greater than or equal to MAX_FRAMES_NOT_NEAR_AVG at 526, then the process resets State(i) equal to 0 at 527, and the process then branches to 505, described above. If NumDiff(i) is less than MAX_FRAMES_NOT_NEAR_AVG at 526, the process branches directly from 525 to 505.
Suggested initial values for variables mentioned above are as follows, although it should be noted that these values are only examples can change from one embodiment to another:
Numerous variations upon the above described processes are possible while still implementing the essence of the technique introduced here. For example, the analyzed area of the frame can be varied. In the above description, a simple rectangular subset of the screen is used. However, other shapes may also work. For example, a rectangle with one or more corners cut out may be appropriate for avoiding station logos (“bugs”).
Further, the spatial sample frequency of pixels can be varied. In the above description, a uniform grid of pixels is used. However, it may be reasonable to sample more pixels closer to the center of the screen than the edges, or vice versa. It is also possible to rearrange pixels to be more or less sensitive to specific patterns found in actual broadcast video, such as the logo of a specific television channel or program.
Further, chrominance can be used in addition to, or instead of, luminance. The description above focuses on only the luminance (brightness) and ignores the chrominance (color) component of all sampled pixels. Color (with or without brightness) could also be used for comparison.
The example described above also samples each pixel four times and uses the average of these first four samples as the comparison for all subsequent samples. However, the number of samples in such an average can be varied from as few as one to any arbitrarily large number of samples (and as noted above, a filter other than an average could be used instead). Note, however, that once initial samples are taken, the comparison should be fixed against these initial samples and not varied. This prevents a scene which changes very gradually over time from triggering the screen saver.
Multiple comparison values can also be employed. The description above uses only one (averaged) value with which to compare pixel values. However, there are situations where two or more comparison values may be appropriate. In such a case, if the sampled pixel matches any of the compared values, then it is considered not to have changed. Such situations include the accommodation of flashing text, or the handling of set-top boxes which include so-called “screen savers” that are implemented very poorly, because a fixed pattern is displayed in only a limited number of places on the screen, thus actually causing screen burn-in those areas.
Further, the description above treats all sampled pixels in the same way regardless of the sampled value. It may be appropriate, however, to only count pixels which are brighter than specific thresholds, assuming that darker pixels are not likely to cause burn-in even if displayed for a long time.
The above description also does not take into account the spatial relationships between pixels. Screen burn-in is most apparent when bright pixels have been displayed next to dark pixels for a long time. With this in mind, the sample set may be modified to only be sensitive to sharp transitions in color between adjacent pixels. This is typical of text over video, where light colored text is typically surrounded by a dark outline, so that it will be visible over arbitrary video displays.
Moreover, variations in various thresholds and timings can be employed. Numeric parameters in the algorithm can be tuned to be more or less sensitive to real situations that have been observed in the field.
The technique described above can be implemented in any system that incorporates or cooperates with a display device that is susceptible to burn-in, such as a television or video monitor. Note that this technique can also be used for a set-top box or other video device such as a VCR, DVD player, or video game which feeds video to a television or video monitor.
The DSP 607 provides output representing audio data to an audio encoder and digital-to-analog converter 612, which provides audio output signals to one or more audio output devices (e.g., speakers, headphones, etc.) 613. The DSP 607 also provides output representing video data to one or more display drivers 614, which provide a video output signal to the display device 601.
Thus, a technique for determining when to trigger a screen saver, and a display system in which such a technique can be implemented, have been described. The techniques introduced above can be implemented in special-purpose hardwired circuitry, in software and/or firmware in conjunction with programmable circuitry, or in a combination thereof. Special-purpose hardwired circuitry may be in the form of, for example, one or more application-specific integrated circuits (ASICs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), etc.
Software or firmware to implement the techniques introduced here may be stored on a machine-readable medium and may be executed by one or more general-purpose or special-purpose programmable microprocessors. A “machine-readable medium”, as the term is used herein, includes any mechanism that stores information in a form accessible by a machine (e.g., a microprocessor). For example, a machine-accessible medium includes recordable/non-recordable media (e.g., read-only memory (ROM); random access memory (RAM); magnetic disk storage media; optical storage media; flash memory devices; etc.), etc.
Although the present invention has been described with reference to specific exemplary embodiments, it will be recognized that the invention is not limited to the embodiments described, but can be practiced with modification and alteration within the spirit and scope of the appended claims. Accordingly, the specification and drawings are to be regarded in an illustrative sense rather than a restrictive sense.
