Patent Application
20090310679
1. Field of the Invention
The invention relates to video processing, and in particular to processes which involve intermediate frame interpolation.
2. Description of the Related Art
Generally, most video sources, such as films, movies, or animations, have sample rates between 24 to 30 frames per second. In addition, for normal display devices, the display frame rate ranges from 50 to 60 frames per second. Therefore, a conversion of video signals is required to convert the sample rate to the display frame rate.
Traditionally, frame repetition is commonly used to interpolate frames and perform frame rate up-conversion. However, when objects or background of the frames are moving, the conversion process using frame repetition may produce undesirable artifacts, such as movement judder artifacts, thus resulting in deterioration of video quality.
A variety of techniques have been proposed to remove the defects of frame rate conversion, caused by motion of objects in frames, such as motion compensation, wherein a technique of motion judder cancellation (MJC) has been proposed. In MJC technique, an intermediate frame is generated by spatially interpolating the position of objects and background from two successive frames to reducing judder artifacts. Referring to
The motion estimation unit 102 derives motion vectors 112 according to at least two successive frames 110. The motion vectors 112 are utilized to instruct the motion compensation unit 104 to locate and access blocks in the reference frames for generation of an intermediate frame 114. Accordingly, the motion compensation unit 104 employs the motion vectors 112 and the successive frames 110 to interpolate the intermediate frame 114, wherein the judder effects of motion are eliminated or reduced.
However, this technique may cause artifacts when one pixel of the intermediate frame is over a moving edge. More specifically, unnatural ringing artifacts, or namely creation of a “halo”, may occur at the boundary of frames with motion. The appearance of a “halo” results from incorrect boundary information derived from the motion estimation unit 102, which fails to match the real boundary of frames with motion. Consequently, ringing artifacts, or so-called “halo”, generates jagged contours or fuzzy lines in the vicinity of sharp edges of objects in motion in the intermediate frame 114, or at the motion boundary of the intermediate frame 114.
A video processing apparatus for interpolating frames between two frames is provided. The video processing apparatus comprises a motion estimation unit, a region detector and a motion compensation unit. The motion estimation unit receives the two frames and provides motion vector information of the two frames. The region detector generates boundary information associated with image boundaries of the two frames, determines a specific region in the two frames according to the boundary information and generates region information according to the determination result. The motion compensation unit generates an interpolated frame between the two frames in accordance with the region information and the motion vector information.
A video processing method for interpolating frames between two frames is further provided. The video processing method comprises: receiving the two frames; estimating motion vector information of the two frames; generating boundary information associated with image boundaries of the two frames; determining a specific region in the two frames according to the boundary information; generating region information according to the determination result; and generating an interpolated frame between the two frames in accordance with the region information and the motion vector information.
Another machine-readable storage medium storing a computer program which, when executed by a video processing apparatus, performs the video processing method is provided.
A detailed description is given in the following embodiments with reference to the accompanying drawings.
The invention can be more fully understood by reading the subsequent detailed description and examples with references made to the accompanying drawings, wherein:
The following description is of the best-contemplated mode of carrying out the invention. This description is made for the purpose of illustrating the general principles of the invention and should not be taken in a limiting sense. The scope of the invention is best determined by reference to the appended claims.
The motion estimation unit 202 receives a sequence of frames, for example, two frames 210. In an embodiment of the invention, the two frames 210 are successive frames. The motion estimation unit 202 subsequently performs motion estimation according to pertinent block data of the frames and outputs motion vector information 212 of objects moving in the two frames 210. The region detector 206 generates boundary information corresponding to image boundaries of the two frames 210, determines a specific region in the two frames 210 according to the boundary information and generates region information 224 according to the determination result. In an embodiment of the invention, the specific region is an invalid region. When all of the invalid regions in the two frames 210 have been designated with the region information 224, the region detector 206 provides the region information 224 to the motion compensation unit 204 for motion compensation. The motion compensation unit 204 interpolates the two frames 210 to generate an interpolated frame 214 according to the region information 224. In details, in accordance with the region information 224, the motion compensation unit 204 assigns a first predetermined weighting factor, such as low weighting factors, to the invalid regions and a second predetermined weighting factor to regions other than the invalid regions. For example, a weighting factor suitable for the invalid regions is zero to prevent ringing artifacts in the interpolation process. Also, movement of objects in the two frames is compensated based on the region information 224 and the motion vector information 212 from the motion estimation unit 202.
Further,
As shown in
In addition, referring to
Note that there are a variety of well known techniques available for obtaining the image boundary information of frames, for example, a predetermined threshold level may be used to determine the boundary of a frame. For this technique, the boundary detection method using the predetermined threshold level performs a scan operation for the frame from upper lines to lower lines. During the scan operation, the first line, denoted by line N, having a greater luminance value than the predetermined threshold level, is identified. Additionally, the last line, denoted by line M, having a greater luminance value than the predetermined threshold level, is further identified. As a result, an upper boundary (line N) and a lower boundary (line M) of the frame are determined. More specifically, the region which is not placed between lines N and M is defined as an outside region, or an invalid region. That is, the invalid region in the frame may comprise a plurality of regions above the upper boundary and below the lower boundary. It should be noted that other boundary detection techniques can be used as desired, in addition to the method described above.
It is noted that video processing apparatus and methods according to embodiments of the invention for interpolating frames between two frames, or certain aspects or portions thereof, may take the form of program code, i.e., instructions, embodied in tangible media, such as floppy diskettes, hard drives, nonvolatile memory devices, CD-ROMs, or any other machine-readable storage medium, wherein, when the program code is loaded into and executed by a machine, such as a video processing apparatus or similar, the machine becomes an apparatus for practicing the invention. The disclosed methods may also be embodied in the form of program codes transmitted over some transmission medium, such as electrical wiring or cabling, through fiber optics, or via any other form of transmission, wherein, when the program codes are received and loaded into and executed by a machine, such as a computer, a video processing apparatus or similar, the machine becomes an apparatus for practicing the invention. When implemented on a general-purpose processor, the program codes combine with the processor to provide a unique apparatus that operate analogously to specific logic circuits.
While the invention has been described by way of example and in terms of preferred embodiment, it is to be understood that the invention is not limited thereto. To the contrary, it is intended to cover various modifications and similar arrangements (as would be apparent to those skilled in the art). Therefore, the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements.
