This is a continuation of International Application No. PCT/JP2012/000301 filed on Jan. 19, 2012, which claims priority to Japanese Patent Application No. 2011-083308 filed on Apr. 5, 2011. The entire disclosures of these applications are incorporated by reference herein.
The present disclosure relates to a frame rate conversion method in which conversion of a frame rate to be used for, e.g., display apparatuses or video processing apparatuses and judder compensation for a pulldown signal are performed, and to a video processing apparatus using the frame rate conversion method.
With an increase in screen size and resolution, emphasis has been recently placed on display of a higher quality video picture on a television set using, e.g., a liquid crystal display device or a plasma display device. In order to smoothly display movement of an object(s) of a video picture on a screen, emphasis has been also placed on video processing for converting a frame rate. Increasing importance has been attached to the following frame rate conversion apparatus and method. A film footage or a video picture generated at 24 fps is converted into a film-originated signal by 2:3 pulldown, or an input video picture is converted into a video signal by 2:2 pulldown. Such an interlaced video signal converted by the foregoing pulldown is converted into a progressive signal to generate progressive scan frames. As a result, movement of a video picture corresponding to the video signal converted by the foregoing pulldown can be smoothly displayed, and the frame rate of the video picture can be converted into the frame rate of a display apparatus. Moreover, the frame rate conversion apparatus and method become increasingly compatible with various pulldown video signals. In the case of a pulldown signal having the high number of a plurality of frames (hereinafter referred to as “repeat frames”) containing the same image, it is expected that a difference in processing time (hereinafter referred to as a “delay time”) between two frames containing different image signals is greater than a difference in processing time between two frames containing a series of image signals, and therefore the processing time should be taken into consideration. As described in, e.g., Japanese Patent Publication No. 2005-208613 (see FIG. 1), a conventional frame rate conversion apparatus includes a plurality of frame rate converters to which different delay times are assigned and which are each configured to perform frame rate conversion in which the number of images corresponding to an input video signal is, in a unit of time, renewed to a different number and the renewed video signal is output, a first selector configured to select one of the video signals output from the frame rate converters and output the selected video signal, and an audio signal delayer configured to delay an audio signal. The audio signal delayer delays the audio signal depending on the delay time assigned to the frame rate converter outputting the video signal selected by the first selector.
However, in the conventional frame rate conversion apparatus, if, e.g., a 2:2 pulldown video picture generated at 60 fps and corresponding to a video signal changing every two frames is input, and frame rate conversion is performed with a delay by one frame, the input video signal containing image signals A, A, B, B . . . is, referring to
The case where it is detected that a 2:2 pulldown video picture is input and the processing is switched to frame rate conversion corresponding to 2:2 pulldown will be described with reference to
For a pulldown signal with the high number of repeat frames, in the case where a displayed object(s) moves in only part of a screen, the following disadvantages are caused as compared to the case where a displayed object(s) moves across the entirety of a screen: persistence of vision is increased due to frame rate conversion; and, if an error occurs in motion vector prediction, the error is noticeable due to the high number of repeat frames.
In view of the foregoing, the present disclosure is intended to provide, in the case where frame rate conversion is performed for an input pulldown video picture with the high number of repeat frames, a frame rate conversion method in which continuity of the video picture is ensured without reversing or forwarding of a moving object(s) at a switching point at which processing for a series of frames is switched to processing corresponding to pulldown. Moreover, the present disclosure is applicable to more types of pulldown video signals by employing pulldown to various numbers of repeat frames. Further, the present disclosure is intended to provide a video processing apparatus in which the frame rate conversion method is used to convert the frame rate of a display apparatus or the video processing apparatus and therefore continuity of a video picture is ensured without reversing or forwarding of a moving object(s) at a switching point at which processing for a series of frames is switched to processing for pulldown.
Further, the present disclosure is intended to reduce persistence of vision or make an error in motion vector prediction to be less noticeable in such a manner that frame rate conversion processing is switched between the case where a displayed object(s) moves across the entirety of a screen and the case where a displayed object(s) moves in only part of a screen.
In order to solve the foregoing disadvantages, a frame rate conversion method of a first aspect of the invention includes a film footage detection step of detecting a M:N pulldown video signal from an input video signal, M and N being integers; a frame rate conversion step of performing frame rate conversion for a series of frames of the input video signal; and a switching control step of, if the M:N pulldown video signal is detected based on a detection result obtained at the film footage detection step, switching, at the frame rate conversion step, the frame rate conversion for the series of frames of the video signal to frame rate conversion for the M:N pulldown video signal, and controlling the switching at a frame in which there is no difference between time required for the frame rate conversion for the series of frames of the video signal and time required for the frame rate conversion for the M:N pulldown video signal.
A second aspect of the invention is intended for the method of the first aspect of the invention, in which, at the switching control step, the switching to the frame rate conversion for the M:N pulldown is performed based on predetermined phase information indicating a frame phase of the M:N pulldown video signal.
A third aspect of the invention is intended for the method of the first aspect of the invention, in which, at the switching control step, the switching to the frame rate conversion for the M:N pulldown is performed based on the number of detections of the M:N pulldown video signal.
A fourth aspect of the invention is intended for the method of the first aspect of the invention, in which the film footage detection step includes an integration step of accumulating a frame difference value for one frame period, a comparison step of comparing an integral value obtained by accumulating the frame difference value for one frame period with a predetermined threshold value, a film footage determination step of determining, based on an output result of the comparison step, whether or not there is a frame difference value, and determining whether or not the M:N pulldown video signal is input, and a phase extraction step of outputting phase information of the M:N pulldown video signal based on the output result of the comparison step and a film footage determination result, and the M:N pulldown video signal is detected at the film footage detection step.
A fifth aspect of the invention is intended for the method of the first aspect of the invention, in which the film footage detection step includes a first integration step of accumulating a first frame difference value for one frame period, a first comparison step of comparing an integral value obtained by accumulating the first frame difference value for one frame period with a predetermined threshold value, a second integration step of accumulating a second frame difference value for one frame period, a second comparison step of comparing an integral value obtained by accumulating the second frame difference value for one frame period with a predetermined threshold value, a film footage determination step of determining, based on output results of the first and second comparison steps, whether or not there is a frame difference value, and determining whether or not the M:N pulldown video signal is input, and a phase information output step of outputting phase information of the M:N pulldown video signal based on the output results of the first and second comparison steps and a film footage determination result, and the M:N pulldown video signal is detected at the film footage detection step.
A sixth aspect of the invention is intended for a video processing apparatus including an input processing unit configured to process a signal input from broadcasting or video equipment; a frame rate conversion unit configured to convert a frame rate of a video signal output from the input processing unit; and an output processing unit configured to display the video signal having the converted frame rate on a display unit. The frame rate conversion unit includes a film footage detector configured to detect a M:N pulldown video signal from the video signal output from the input processing unit, M and N being integers, a frame rate converter configured to perform frame rate conversion for a series of frames of the video signal, and a switching controller configured to, if the M:N pulldown video signal is detected based on a detection result output from the film footage detector, switch the frame rate conversion for the series of frames of the video signal to frame rate conversion for the M:N pulldown video signal, and control the switching at a frame in which there is no difference between time required for the frame rate conversion for the series of frames of the video signal and time required for the frame rate conversion for the M:N pulldown video signal.
A seventh aspect of the invention is intended for a video processing apparatus including an input processing unit configured to process a signal decoded from a broadcasting signal or a signal reproduced from a storage unit; a frame rate conversion unit configured to convert a frame rate of a video signal output from the input processing unit; and an output processing unit configured to output the video signal having the converted frame rate to an external unit. The frame rate conversion unit includes a film footage detector configured to detect a M:N pulldown video signal from the video signal output from the input processing unit, M and N being integers, a frame rate converter configured to perform frame rate conversion for a series of frames of the video signal, and a switching controller configured to, if the M:N pulldown video signal is detected based on a detection result output from the film footage detector, switch the frame rate conversion for the series of frames of the video signal to frame rate conversion for the M:N pulldown video signal, and control the switching at a frame in which there is no difference between time required for the frame rate conversion for the series of frames of the video signal and time required for the frame rate conversion for the M:N pulldown video signal.
An eighth aspect of the invention is intended for a frame rate conversion method including a film footage detection step of detecting a M:N pulldown video signal from an input video signal, M and N being integers; a scroll detection step of detecting uniform scroll movement displayed across an entire screen from the input video signal; and a frame rate conversion step of performing frame rate conversion. At the frame rate conversion step, if a detection result obtained at the film footage detection step indicates that the M:N pulldown video signal is detected and it is detected that the uniform scroll movement is displayed across the entire screen, the frame rate conversion is switched to frame rate conversion for the M:N pulldown video signal.
A ninth aspect of the invention is intended for a frame rate conversion method including a scroll detection step of detecting uniform scroll movement displayed across an entire screen from an input video signal; a film footage detection step of detecting a M:N pulldown video signal from the input video signal, M and N being integers; and a frame rate conversion step of performing frame rate conversion. At the frame rate conversion step, if the uniform scroll movement across the entire screen is detected and a detection result obtained at the film footage detection step indicates that the M:N pulldown video signal is detected, the frame rate conversion is switched to frame rate conversion for the M:N pulldown video signal.
A tenth aspect of the invention is intended for a frame rate conversion method including a film footage detection step of detecting a M:N pulldown video signal from an input video signal, M and N being integers; a scroll detection step of detecting uniform scroll movement displayed across an entire screen from the input video signal; and a frame rate conversion step of performing frame rate conversion. At the frame rate conversion step, if a detection result obtained at the film footage detection step indicates that the M:N pulldown video signal is detected and the uniform scroll movement across the entire screen is detected, the frame rate conversion is switched to frame rate conversion for the M:N pulldown video signal. If the detection result indicates that the M:N pulldown video signal is detected and the uniform scroll movement across the entire screen is not detected, the frame rate conversion is switched to frame rate conversion for an X:Y pulldown video signal having a smaller integer ratio than M:N.
According to the present disclosure, in the case where frame rate conversion is performed when a pulldown signal having the high number of repeat frames is input, continuity of a video picture can be ensured at a switching point at which the processing is switched from processing for a series of frames to processing for pulldown, and frame rate conversion can be provided without reversing or forwarding of a moving object(s). Moreover, the present disclosure is applicable to more types of pulldown video signals by employing pulldown to various numbers of repeat frames.
The frame rate conversion method of the present disclosure is used to perform frame rate conversion for a display apparatus or a video processing apparatus. Thus, continuity of a video picture can be ensured at a switching point at which the processing is switched from processing for a series of frames to processing for pulldown, and frame rate conversion can be provided without reversing or forwarding of a moving object(s).
Further, in the case where frame rate conversion is performed when a pulldown signal having the high number of repeat frames is input, since continuity of a video picture can be ensured at a switching point at which the processing is switched from processing for a series of frames to processing for pulldown, an influence of a difference in delay of an audio signal can be reduced without switching the delay of the audio signal depending on the type of pulldown.
In the case of a pulldown signal having the high number of repeat frames, a scroll movement, i.e., uniform movement displayed on a screen, is detected, thereby performing frame rate conversion depending on the number of repeat frames during the scroll movement. Thus, persistence of vision due to frame interpolation in the case where a displayed object(s) moves in part of a screen or a small area of the screen can be reduced, and noise due to an error in motion vector prediction can be reduced.
Embodiments of the present disclosure will be described below in detail with reference to drawings.
Referring to
An operation of the first embodiment will be described with reference to
For the foregoing reasons, in the present embodiment, the film footage detector 106 detects, based on the frame data S102E, S102F output from the storage 102, the number of repeat frames and detects whether or not a M:N (“M” and “N” are integers) pulldown signal is input. Then, the film footage detector 106 outputs a detected video pattern signal S106B. If the M:N pulldown video signal is detected, the film footage detector 106 outputs a frame phase signal S106A corresponding to a frame phase per cycle of the pulldown signal. The present embodiment describes one example where a 4:4 pulldown video signal is input. Upon detection of 4:4 pulldown, a signal with a repetition of values 0, 1, 2, 3 is output as the frame phase signal S106A corresponding to the frame phase per cycle. In order to read, based on the frame phase signal S106A and the video pattern signal S106B output from the film footage detector 106, frame data required for each video processing pattern in the switching controller 107, the memory controller 103 is switched to switch frame data to be output to the motion vector detector 104 and the interpolation frame generator 105 based on an output pulldown signal S107A.
Switching timing in an operation of the switching controller 107 will be described with reference to
(First Configuration Example of Film Footage Detector)
Next, a first configuration example of the film footage detector 106 of the first embodiment will be described with reference to
Referring to
(Second Configuration Example of Film Footage Detector)
In addition to frame data S102E, S102F read with frame delay from the storage 102 using the memory, frame data S102G is input as illustrated in
In a film footage detection unit 307, the number of repeat frames is determined based on the frame correlation signals S303, 306. Moreover, it is also determined, based on the frame correlation signals S303, 306, whether or not a M:N pulldown signal is input. Meanwhile, in a phase extraction unit 308, a frame phase signal S106A is output based on a determination result S307 and the frame correlation signals S303, S306. Since the number of repeat frames is determined by the comparison with the preset correlation pattern, video patterns for various pulldown signals and various numbers of repeat frames can be detected.
In
It is determined, by film footage detection, whether or not a video signal corresponding to an input video picture is a M:N pulldown video signal. If the video signal is not the M:N pulldown signal, settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to frame rate conversion for a series of frames, and then such frame rate conversion is performed.
On the other hand, if the video signal is the M:N pulldown video signal, a frame phase is determined, and it is determined whether or not the determined frame phase is a predetermined phase. If the frame phase is not the predetermined phase, the settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to frame rate conversion for a series of frames, and then such frame rate conversion is performed. If the frame phase is the predetermined phase, the settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to M:N pulldown to switch processing for memory control, motion vector detection, and interpolation frame generation. Then, frame rate conversion for M:N pulldown is performed. During frame rate conversion for pulldown processing, film footage detection is performed for each frame. If pulldown video signals are processed in the same pulldown pattern, frame rate conversion for pulldown processing continues. If pulldown video signals are not processed in the same pulldown pattern, the settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to frame rate conversion for a series of frames, and then such frame rate conversion is performed. If frame rate conversion for a series of frames is performed, the process returns to “START” at a subsequent frame, and the process is repeated.
It is determined, by M:N pulldown determination, whether or not a video signal corresponding to an input video picture is a M:N pulldown video signal. If the video signal is not the M:N pulldown signal, settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to frame rate conversion for a series of frames, and then such frame rate conversion is performed.
On the other hand, if the video signal is the M:N pulldown video signal, it is determined, by scroll detection, whether or not the video picture is a scroll video picture in which a displayed object(s) moves in a uniform manner or in the same direction across the entirety of a screen. If the video picture is not the scroll video picture, the settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to frame rate conversion for a series of frames, and then such frame rate conversion is performed. On the other hand, if the video picture is the scroll video picture, the settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to M:N pulldown to switch processing for memory control, motion vector detection, and interpolation frame generation. Then, frame rate conversion for M:N pulldown is performed.
It is determined whether or not an input video picture is a scroll video picture in which a displayed object(s) moves in a uniform manner or in the same direction across the entirety of a screen. If the video picture is not the scroll video picture, settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to frame rate conversion for a series of frames, and then such frame rate conversion is performed. On the other hand, if the video picture is the scroll video picture, it is determined, by M:N pulldown determination, whether or not a video signal corresponding to the input video picture is a M:N pulldown video signal. If the video signal is not the M:N pulldown signal, the settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to frame rate conversion for a series of frames, and then such frame rate conversion is performed. If the video signal is the M:N pulldown signal, the settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to M:N pulldown to switch processing for memory control, motion vector detection, and interpolation frame generation. Then, frame rate conversion for M:N pulldown is performed.
In the foregoing frame rate conversion, the processes of the second and third flowcharts are different from each other in the order of performing M:N pulldown detection and scroll detection. However, M:N pulldown detection and scroll detection may be performed in parallel, and frame rate conversion for M:N pulldown may be performed based on determination results of M:N pulldown detection and scroll detection.
Referring to
In the input processing unit 401 configured to process video data generated or recorded in the storage unit 400, a video signal containing a M:N pulldown signal is generated. For example, such a pulldown video signal is converted from a 60 Hz video signal into a 120 Hz video signal by using the frame rate conversion method of the first embodiment. Since the video signal having the converted frame rate is output from the output processing unit 403 to the external unit, such a video signal is processed into such a signal format that the video signal can be output via connection through a cable or via wireless connection. For display of a video picture on the display unit 404, a synchronization signal to be reference for display and a timing signal for controlling a display apparatus are, in addition to a video signal, generated and processed to display the video picture on a screen.
The display unit 404 is required for video processing apparatuses configured to display a video picture on a screen, such as television sets or displays. However, video processing apparatuses, such as DVDs, Blu-ray discs, or hard discs, configured to record and/or reproduction a video picture can be realized with a configuration which does not include the display unit 404, and a video signal having a converted frame rate is output via an output terminal connected to the external unit.
Referring to
On the other hand, if the video signal is the M:N pulldown video signal, it is determined, by scroll detection, whether or not the video picture is a scroll video picture in which a displayed object(s) moves in a uniform manner or in the same direction across the entirety of a screen. If the video picture is not the scroll video picture, the settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to X:Y pulldown to switch processing for memory control, motion vector detection, and interpolation frame generation. Then, frame rate conversion for X:Y pulldown is performed. On the other hand, if the video picture is the scroll video picture, the settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to M:N pulldown to switch processing for memory control, motion vector detection, and interpolation frame generation. Then, frame rate conversion corresponding to M:N pulldown is performed.
Note that M:N and X:Y satisfy relationships of M>X and N>Y. Suppose that M:N pulldown is 8:7 pulldown. In such a case, X:Y pulldown is 3:2 pulldown, and frame rate conversion is switched between frame rate conversion for the lower number of repeat frames and frame rate conversion for the higher number of repeat frames depending on whether or not a displayed object(s) moves in a uniform manner or in the same direction on a screen.
As described above, the frame rate conversion apparatus and method of the present disclosure are useful as apparatus and method for performing frame rate conversion used for, e.g., display apparatuses, video processing apparatuses, or television sets and performing judder compensation for a pulldown video signal.
Number | Date | Country | Kind |
---|---|---|---|
2011-083308 | Apr 2011 | JP | national |
Number | Date | Country | |
---|---|---|---|
Parent | PCT/JP2012/000301 | Jan 2012 | US |
Child | 14042668 | US |