The present invention relates to a motion vector correction device and method and a video signal processing apparatus and method capable of reducing a feeling of visual strangeness in an interpolated image due to a false detection of motion vector.
When an image display apparatus displays a moving image on a liquid crystal panel thereof, a motion blur occurs frequently in the moving image. In order to reduce a motion blur, an interpolated frame is inserted between real frames in a video signal to increase the number of frames. For example, a frame rate of 60 Hz in a vertical frequency is converted into 120 Hz or more. Then an image display is carried out. In order to carry out a frame rate conversion, a video signal processing apparatus detects a motion vector in an image and generates each interpolated pixel based on the motion vector to obtain an interpolated frame to be inserted between real frames.
A motion vector is generated based on pixel data within a limited range in respective limited frames (e.g., successive two frames). Thus, it is difficult to completely eliminate a false detection of motion vector. In view of the above-described problem, technical improvements for reducing a false detection of motion vector are carried out. As one example, Patent document 1 discloses a frame rate conversion apparatus provided with a motion vector detection device capable of reducing a false detection of motion vector.
Patent document 1: Japanese Patent Application Laid-open Publication No. 2006-331136
In a video signal processing apparatus provided with a motion vector detection device, it is required to reduce a feeling of visual strangeness in an interpolated image due to a false detection of motion vector. In particular, in a case where a detected motion vector is relatively large, when a false detection occurs, a feeling of visual strangeness in an interpolated image is remarkably enhanced.
The present invention is made in view of the above-described problem and has an object to provide a motion vector correction device and method and a video signal processing apparatus and method capable of reducing a feeling of visual strangeness in an interpolated image even when a false detection occurs in a case where a motion vector is relatively large.
In order to resolve the above-described conventional technical problem, the present invention provides a motion vector correction device including: a motion vector detector (2) that, using pixel data in at least two real frames in an input video signal, detects a first motion vector necessary for generating interpolated pixel data forming an interpolated frame to be inserted between the two real frames; and a motion vector corrector (4) that corrects the first motion vector to decrease the magnitude of first motion vector when the magnitude of first motion vector exceeds a first threshold.
In the above-described configuration, the motion vector corrector includes: a motion vector value suppressing part (41) that decreases the magnitude of first motion vector exceeding the first threshold at a predetermined rate, and outputs the result as a second motion vector; and a motion vector value converting part (42) that decreases the magnitude of second motion vector to prevent the magnitude of second motion vector from increasing when the magnitude of second motion vector exceeds a second threshold, and outputs the result as a third motion vector.
As a preferable embodiment, the motion vector value suppressing part calculates the second motion vector according to the following calculus equation
(the magnitude of first motion vector)×m/n+(the first threshold)×(n−m)/n,
where variables m and n are natural numbers which have a relation of m<n.
The motion vector value converting part holds the magnitude of third motion vector constant within a range in the magnitude of second motion vector from the second threshold to a third threshold more than the second threshold, and decreases the magnitude of third motion vector at a predetermined rate when the magnitude of second motion vector exceeds the third threshold.
As a preferable embodiment, when the magnitude of second motion vector exceeds the third threshold, the motion vector value converting part calculates the third motion vector according to the following calculus equation
2×(the second threshold)+(the third threshold−the second threshold)−(the magnitude of second motion vector).
It is preferable to further include a motion vector state determiner (3) that determines an appearance state of the first motion vector in a screen to generate and output a state determination value, wherein the motion vector corrector varies characteristics of decrease of the magnitude of first motion vector according to the state determination value.
As a preferable embodiment, the motion vector state determiner generates a state determination value that represents an appearance amount of motion vectors indicating how many first motion vectors appear in a screen. As another preferable embodiment, the motion vector state determiner generates a state determination value that represents a distribution state of motion vectors indicating a degree of uniformity or variation of directions of first motion vectors in a screen.
In order to resolve the above-described conventional technical problem, the present invention provides a video signal processing apparatus including: an interpolated pixel data generator (5) that generates interpolated pixel data, using pixel data in at least two real frames in an input video signal and a motion vector corrected by any one of the above-described motion vector correction devices.
Also, in order to resolve the above-described conventional technical problem, the present invention provides a motion vector correction method including: a motion vector detection step for, using pixel data in at least two real frames in an input video signal, detecting a first motion vector necessary for generating interpolated pixel data forming an interpolated frame to be inserted between the two real frames; and a motion vector correction step for correcting the first motion vector to decrease the magnitude of first motion vector when the magnitude of first motion vector exceeds a first threshold.
The motion vector correction step includes: a motion vector value suppressing step for decreasing the magnitude of first motion vector exceeding the first threshold at a predetermined rate, and outputting the result as a second motion vector; and a motion vector value converting step for decreasing the magnitude of second motion vector to prevent the magnitude of second motion vector from increasing when the magnitude of second motion vector exceeds a second threshold, and outputting the result as a third motion vector.
As a preferable embodiment, the motion vector value suppressing step calculates the second motion vector according to the following calculus equation
(the magnitude of first motion vector)×m/n+(the first threshold)×(n−m)/n,
where variables m and n are natural numbers which have a relation of m<n.
The motion vector value converting step holds the magnitude of third motion vector constant within a range in the magnitude of second motion vector from the second threshold to a third threshold more than the second threshold, and decreases the magnitude of third motion vector at a predetermined rate when the magnitude of second motion vector exceeds the third threshold.
As a preferable embodiment, when the magnitude of second motion vector exceeds the third threshold, the motion vector value converting step calculates the third motion vector according to the following calculus equation
2×(the second threshold)+(the third threshold−the second threshold)−(the magnitude of second motion vector).
It is preferable to further include a motion vector state determination step for determining an appearance state of the first motion vector in a screen to generate a state determination value, wherein the motion vector correction step varies characteristics of decrease of the magnitude of first motion vector according to the state determination value.
As a preferable embodiment, the motion vector state determination step generates a state determination value that represents an appearance amount of motion vectors indicating how many first motion vectors appear in a screen. As another preferable embodiment, the motion vector state determination step generates a state determination value that represents a distribution state of motion vectors indicating a degree of uniformity or variation of directions of first motion vectors in a screen.
In order to resolve the above-described conventional technical problem, the present invention provides a video signal processing method including: an interpolated pixel data generation step for generating interpolated pixel data, using pixel data in at least two real frames in an input video signal and a motion vector corrected by any of the above-described motion vector correction methods.
According to the motion vector correction device and method and the video signal processing apparatus and method of the present invention, in a case where a motion vector is relatively large, even when a false detection occurs, it is possible to reduce a feeling of visual strangeness in an interpolated image.
[
[
[
[
[
[
[
[
[
A motion vector correction device and method and a video signal processing apparatus and method will be described below with reference to the drawings. As one example of a video signal processing apparatus, one exemplary embodiment of the present invention shown in
In
The motion vector detector 2 detects a motion vector MV1 to be used to generate interpolated pixel data in an interpolated frame to be inserted between the current frame F0 and the frame F1, using two pieces of pixel data in the current frame F0 and the frame F1 according to a matching method, for example. The motion vector MV1 may be detected in a pixel data unit or a block unit composed of plural pieces of pixel data. The motion vector detector 2 includes plural line memories, a pixel delay part, a subtraction part that calculates a difference of two pieces of pixel data and the like (not shown). The motion vector MV1 is input into a motion vector state determiner 3 and a motion vector corrector 4. The motion vector MV1 may be detected using a past frame prior to the frame F1, in addition to the current frame F0 and the frame F1.
The motion vector state determiner 3 determines a motion vector state in each frame and then outputs a determination signal D1. Concrete operation of the motion vector state determiner 3 will be described later in detail. The determination signal D1 is input into the motion vector corrector 4. The motion vector corrector 4 corrects the motion vector MV1 such that the magnitude of motion vector MV1 is decreased according to the determination signal D1, and then outputs it as a motion vector MV3. Concrete configuration and operation of the motion vector corrector 4 will be described later in detail. The motion vector MV3 is input into the interpolated pixel data generator 5.
The interpolated pixel data generator 5 includes plural line memories and a pixel delay part (not shown) used to delay pixel data in the current frame F0 in a horizontal direction and a vertical direction and generate pixel data within a certain range of the current frame F0 in the horizontal direction and the vertical direction, and plural line memories and a pixel delay part (not shown) used to delay pixel data in the frame F1 in a horizontal direction and a vertical direction and generate pixel data within a certain range of the frame F1 in the horizontal direction and the vertical direction. The interpolated pixel data generator 5 further includes an averaging part that selects pixel data in the current frame F0 and pixel data in the frame F1 according to the motion vector MV3 and averages them to generate interpolated pixel data.
In the present embodiment, although the interpolated pixel data is generated using two pieces of pixel data in two real frames, the interpolated pixel data may be generated using three pieces of pixel data or more in three real frames or more.
Pixel data in the frame F0 and interpolated pixel data from the interpolated pixel data generator 5 are sequentially input into the frame frequency converter 6. The frame frequency converter 6 may be composed of a time series conversion memory. The frame frequency converter 6 generates image data of the frame F0 which is a real frame based on the pixel data in the frame F0 sequentially input, and image data of an interpolated frame F1 which is an interpolated frame based on the interpolated pixel data sequentially input from the interpolated pixel data generator 5.
Then, the frame frequency converter 6 alternately selects the image data of the frame F0 and the image data of the interpolated frame Fi at a frequency of 120 Hz, and outputs a video signal Sout with a frame frequency of 120 Hz.
The concrete operation of the motion vector state determiner 3 will be described below. First, a first example of the operation of the motion vector state determiner 3 will be described with reference to
The motion vector state determiner 3 determines whether or not a motion vector MV1 appears every a detection unit of motion vector MV1 in the segmented region AR, and sets “1” if the motion vector MV1 appears or sets “0” if the motion vector MV1 does not appear. Then, the motion vector state determiner 3 counts the number of appearances of motion vectors MV1 by summing “1” in the segmented region AR. When the number of appearances of motion vectors MV1 (count value) exceeds a predetermined threshold, the motion vector state determiner 3 assigns “1” to the segmented region AR as a determination value for segmented region, as indicated at the right side of an arrow in
As described above, in the first example of the operation of the motion vector state determiner 3, the determination signal D1 which represents how many motion vectors MV1 appear in one frame (screen) is generated. That is, in the first example, the determination signal D1 which represents the number of appearances of motion vectors MV1 in one frame is generated without consideration of a direction of motion vector MV1. The state determination value increases as the number of appearances of motion vectors MV1 increases. A large state determination value indicates that the motion vectors MV1 stabilize.
Next, a second example of the operation of the motion vector state determiner 3 will be described with reference to
The motion vector state determiner 3 determines how motion vectors MV1 are distributed in one frame. As one example, motion vectors MV1 are classified into the following nine types based on a direction of motion vector MV1 in a horizontal direction and a vertical direction. A symbol (0, 0) indicates that there is not a motion in the horizontal direction and the vertical direction. A symbol (+, 0) indicates that there is a motion in a right direction and there is not a motion in the vertical direction. A symbol (−, 0) indicates that there is a motion in a left direction and there is not a motion in the vertical direction. A symbol (0, +) indicates that there is not a motion in the horizontal direction and there is a motion in an upward direction. A symbol (0, −) indicates that there is not a motion in the horizontal direction and there is a motion in a downward direction. A symbol (+, +) indicates that there is a motion in the right and upward directions. A symbol (+, −) indicates that there is a motion in the right and downward directions. A symbol (−, +) indicates that there is a motion in the left and upward directions. A symbol (−, −) indicates that there is a motion in the left and downward directions.
The motion vector state determiner 3 classifies motion vectors MV1 into the nine types (0, 0) to (−, −) every the detection unit, and sums the number of motion vectors for each type. Thereby, the motion vector state determiner 3 can determine whether or not motion vectors MV1 in one frame appear in a relatively uniform direction or different directions. For example, the motion vector state determiner 3 generates a state determination value that represents a degree of uniformity or variation of directions of motion vectors MV1 in one frame by a predetermined function using the summation of the number of motion vectors for each type. The state determination value has one of eight-level of “0” to “7” as well as the first example, and is defined as a determination signal D1. That is, in the second example, the determination signal D1 which represents a distribution state of motion vector MV1 in one frame is generated with consideration of a direction of motion vector MV1. The state determination value increases as the degree of uniformity of directions of motion vectors MV1 increases. A large state determination value indicates that the motion vectors MV1 stabilize.
Next, the configuration and operation of the motion vector corrector 4 will be described. As shown in
MV2=MV1×m/n+s×{(n−m)/n} (1)
In the equation (1), variables m and n are predetermined natural numbers which have the relation of m<n, and these values are arbitrarily set. Although the first threshold S, that is a position which is a start point at which the motion vector MV1 starts to decrease, may be fixed, it is preferable to vary the first threshold S according to the state determination value of the motion vector MV1. It is further preferable to increase the first threshold S as the state determination value increases.
The motion vector MV2 is input into the motion vector value converting part 42.
MV3=2×r+h−MV2 (2)
According to the equation (2), the magnitude of motion vector MV3 decreases at a predetermined rate to approach “0” when the magnitude of motion vector MV2 exceeds the third threshold t. Although the second threshold r and the range h between the second threshold r and the third threshold t may be fixed, it is preferable to vary them according to the state determination value of the determination signal D1. It is further preferable to increase the second threshold r and the range h as the state determination value increases. Alternately, while the range h is fixed, the second threshold r may be increased as the state determination value increases.
By correction operations shown in
In the present embodiment, since the range h where the motion vector MV3 has the constant value r is set as the characteristics of motion vector MV3, a state where the motion vector MV1 is not corrected by the motion vector corrector 4 is smoothly changed into a state where the motion vector MV1 is corrected. This reduces a feeling of visual strangeness in an interpolated image.
The present invention is not limited to the present embodiment described above, and various changes may be made without departing from the scope of the invention. The present invention includes programs for causing a computer to realize functions of elements described in the present embodiment. These programs may be read from a recording medium and then loaded in the computer, or be transmitted via a communication network and then loaded in the computer.
Number | Date | Country | Kind |
---|---|---|---|
2010-110962 | May 2010 | JP | national |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/JP2011/055520 | 3/9/2011 | WO | 00 | 7/10/2012 |