Patent Grant
8576336
This application claims the benefit, under 35 U.S.C. §365 of International Application PCT/US2009/006449, filed Dec. 9, 2009, which was published in accordance with PCT Article 21(2) on Jun. 16, 2011 in English.
The subject matter relates generally to video, and more particularly to systems and methods for reformatting film-based progressive video.
The motion picture industry was born when static pictures were flashed on a screen fast enough that people perceived the image to be moving. The pictures were eventually “taped” together by imprinting the static pictures on a roll of translucent film. This allowed a projector to shine light through each picture and project the image to a screen. By moving the film in front of the projector, each picture was flashed on the screen in sequence, giving the illusion of motion to the audience. Because each of the pictures is “flashed” momentarily, the speed of the flashing is important to reduce flicker. Human vision has a phenomenon known as ‘persistence of vision’—that is, the eye retains what it sees even after the picture is removed from sight. The phenomenon is momentary, and, if a motion picture projector is fast enough, the motion appears in focus and does not flicker. It has become a standard in the film industry to use a frame rate (flashes of pictures) of 24 frames (or pictures) per second. This rate works well in lighting controlled environments (persistence of vision is influenced by the amount of ambient lighting).
Modern progressive video does not have a time lag between “flashes” of pictures. The pictures (or images) are continuously shown. This does not allow the human eye to focus on the picture long enough to denote details of the picture before it changes. Essentially, the persistence of vision attempts to retain a previous picture while a new picture is being shown, causing a blurring effect. Thus, progressive video does not appear to have the same quality as when the same video is shown in a motion picture setting. Progressive video effectively reduces the quality and viewing experience of film-based motion pictures (or videos).
A film rate for film content along with a display refresh rate is utilized to determine a frame OFF period. At least one black frame that approximates the frame OFF period is then inserted into a progressive video sequence to simulate a motion picture projector's flicker. Typically, the film rate is 24 frames per second (industry standard) and, thus, the display refresh rate can be a multiple of the frame rate (e.g., 96 Hz, 120 Hz, 240 Hz, etc.). Progressive video replicates a single frame of a film several times (“a progressive video sequence”) depending on a display's refresh rate. The black frame(s) is/are substituted at the end of this sequence instead of displaying the same image throughout for a single film frame. The insertion of the black frame(s) or OFF period allows the viewer's vision time to focus on details of the motion, producing a higher subjective quality of playback for progressive video based on film.
The above presents a simplified summary of the subject matter in order to provide a basic understanding of some aspects of subject matter embodiments. This summary is not an extensive overview of the subject matter. It is not intended to identify key/critical elements of the embodiments or to delineate the scope of the subject matter. Its sole purpose is to present some concepts of the subject matter in a simplified form as a prelude to the more detailed description that is presented later.
To the accomplishment of the foregoing and related ends, certain illustrative aspects of embodiments are described herein in connection with the following description and the annexed drawings. These aspects are indicative, however, of but a few of the various ways in which the principles of the subject matter can be employed, and the subject matter is intended to include all such aspects and their equivalents. Other advantages and novel features of the subject matter can become apparent from the following detailed description when considered in conjunction with the drawings.
The subject matter is now described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the subject matter. It can be evident, however, that subject matter embodiments can be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to facilitate describing the embodiments.
As used in this application, the term “component” is intended to refer to hardware, software, or a combination of hardware and software in execution. For example, a component can be, but is not limited to being, a process running on a processor, a processor, an object, an executable, and/or a microchip and the like. By way of illustration, both an application running on a processor and the processor can be a component. One or more components can reside within a process and a component can be localized on one system and/or distributed between two or more systems. Functions of the various components shown in the figures can be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software.
When provided by a processor, the functions can be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which can be shared. Moreover, explicit use of the term “processor” or “controller” should not be construed to refer exclusively to hardware capable of executing software, and can implicitly include, without limitation, digital signal processor (“DSP”) hardware, read-only memory (“ROM”) for storing software, random access memory (“RAM”), and non-volatile storage. Moreover, all statements herein reciting instances and embodiments of the invention are intended to encompass both structural and functional equivalents. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future (i.e., any elements developed that perform the same function, regardless of structure).
The progressive display 106 can include, but is not limited to, plasma displays, liquid crystal displays (LCD), light emitting diode (LED) displays, OLED displays, and other types of displays and the like. The progressive video formatter 104 can be associated with a set top box, television, display, disc player (DVD and the like), disc recorder (DVD and the like), and/or a digital motion picture projector and the like. It 104 can reside within, on and/or near these systems. Thus, the location between a progressive video formatter 104 and a progressive display 106 can local and/or remote (fragmented implementations can permit both local and remote portions of the progressive video formatter 104). The progressive video formatter 104 can also be used with content authoring systems. These systems are typically used to generate video media content such as DVDs. However, the systems could also generate formatted content for storing on video servers, etc. for video streaming, etc.
In
By simulating motion picture projector flicker in progressive video, the subjective quality of film originated videos can be improved to a film-like experience. Motion picture projectors shine a light through a film and project the film's image onto a screen. Between each presented frame there is a moment in time where the screen goes black and afterwards the next frame of film is displayed. However, modern video displays have natively progressive output. They compose one full frame multiple times per second. Each frame leads to the next frame without a gap. This intuitively seems to be an accurate way to display the original content.
The moment of time with blackness associated with a motion picture projector is not perceived by the viewer due to a phenomenon called ‘persistence of vision.’ In a dark room, if an image is flashed ON and OFF at a speed of approximately 24 frames per second, it is perceived as dimmer than the moments the picture is flashed, but no flashing is observed. In bright daylight that flashing rate may need to be as high as 60 frames per second. Minimizing the amount of light needed to project and keeping the flicker rate to a minimum is why movie theaters are kept so dark.
Persistence of vision is motion perception. If a ball is moving across a room really fast, it is perceived as a streak of color. With a progressive display, a ball is perceived as moving as a streak of color, but the original content seen in a movie theater is perceived slightly differently. With the ball moving on a movie theater silver screen, there is a moment of blackness that allows the eye's persistence of vision to de-saturate. The next frame is then projected showing the ball in a new position. The sudden change, though not perceived as a flash or flicker, causes the eye to get a more precise visual image of the ball. The effect is that the ball is moving rapidly across the screen, but more details about the ball are perceived by the viewer. It can also be thought of in terms of a stroboscope effect. The projectors behave just like stroboscopes that have had their speed turned up to the point that the flicker is not perceived, but the motion in that type of lighting would seem very unnatural and “sharp.” However, if the film has the right amount of motion blur already in it to perfectly match the stroboscope, the viewer's eyes see things the way the creator of the content intended. Changing the light flashing presentation distorts the effect.
A film's content creator may have intended to have sharp movements in scenes with a lot of action, but progressive displays tend to make rapid movements look more streaky, soft or blurry. Thus, the original intent of the film maker is lost when it is converted to progressive video. By using a high refresh rate display and recreating the momentary flashes, progressive video can be made to look more like the original film content. For example, a 120 Hz refresh rate display has to display the same 24 fps (frames per second) image five times before moving to the next image. To simulate the flicker found in motion picture projectors, the progressive video can be altered on the 120 Hz display to show a ball displayed in one position for four frame times, then one frame time of black, followed by 4 frame times in the next position, then one frame of black, and so on.
In the example 300 illustrated in
One skilled in the art can appreciate that other refresh rates can be used as well. For example, a 96 Hz refresh rate allows for a progressive video sequence of four frames for each film frame. Substituting one black frame for the fourth frame induces flicker into the progressive video sequence (three like frames with one black frame). Thus, a ¼th OFF time provides good subjective viewing results as well. The refresh rate can also be increased. For example, a 240 Hz refresh rate has a progressive video sequence of 10 frames. Approximating the OFF time using ⅕th, two frames of black can be substituted at the end of the progressive video sequence to simulate projector flicker. If a different film frame rate is used, the OFF time can be adjusted accordingly. A faster film frame rate for the film-based content decreases the OFF time and a slow film frame rate increases the OFF time.
Some movie projectors have shutters that are open for partial amounts of the frame time with bright flashes in between. Older projectors used to flash one time per frame at 24 fps, but now flash two or three times per frame. The effect is the same, but reduces the flicker that some people might be able to see. Given a fast enough and bright enough display, this effect can also be reproduced with the techniques described herein. The fully off period while advancing the frame is still either 25% or 16.66% of the 24 fps period using a 48 Hz and/or 72 Hz example. This is typically why 20% is subjectively recommended as being the “best” technique. Although higher rate shutters in movie projectors reduce the flickering effect, it also is less efficient—requiring a more powerful light source to have the same light on the screen as a low rate shutter version.
In another instance, if the response time of a display (e.g., LCD being very slow) is known, the first of the repeated frames can be overdriven to improve the display's overall response time. If the pattern starts with OFF, then a pixel is displayed, for example, at 75%, but the display is slow, so the pixel is overdriven instead to 100% on the first frame then back to 75% for the second frame presentation of that pixel. LCD displays themselves cannot include this type of optimization because they are not aware of the brightness of the pixel during the next presented frame. Because of their difference in operation, DLP, CRT and plasma TVs would not benefit from this type of optimization.
In view of the exemplary systems shown and described above, methodologies that can be implemented in accordance with the embodiments will be better appreciated with reference to the flow charts of
In
It is to be appreciated that the systems and/or methods of the embodiments can be utilized in progressive video reformat facilitating computer components and non-computer related components alike. Some progressive video formatting can be fully and/or partially implemented in software. Further, those skilled in the art will recognize that the systems and/or methods of the embodiments are employable in a vast array of electronic related technologies, including, but not limited to, computers, set top boxes, televisions, monitors, displays, disc players/recorders, digital projectors and/or handheld electronic devices, and the like.
What has been described above includes examples of the embodiments. It is, of course, not possible to describe every conceivable combination of components or methodologies for purposes of describing the embodiments, but one of ordinary skill in the art can recognize that many further combinations and permutations of the embodiments are possible. Accordingly, the subject matter is intended to embrace all such alterations, modifications and variations that fall within the spirit and scope of the appended claims. Furthermore, to the extent that the term “includes” is used in either the detailed description or the claims, such term is intended to be inclusive in a manner similar to the term “comprising” as “comprising” is interpreted when employed as a transitional word in a claim.
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/US2009/006449 | 12/9/2009 | WO | 00 | 6/7/2012 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2011/071465 | 6/16/2011 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 3609019 | Tuber | Sep 1971 | A |
| 20060176261 | Nitta et al. | Aug 2006 | A1 |
| 20080174694 | Morad et al. | Jul 2008 | A1 |
| 20090185795 | Itani et al. | Jul 2009 | A1 |
| 20090243995 | Kimura | Oct 2009 | A1 |
| Number | Date | Country | |
|---|---|---|---|
| 20120236200 A1 | Sep 2012 | US |
