APPARATUS FOR CREATING INTERPOLATION FRAME

Abstract
An area detecting unit detects a first matching area in a first reference block in the first reference frame, a second matching area in a second reference block in the second reference frame. A settling unit settles a secondary motion vector to be assigned to a mismatching area in the each interpolation block, based on surrounding interpolation blocks around the mismatching area. A motion compensating unit assigns an image to the interpolation frame based on the primary motion vector and the secondary motion vector.
Description

BRIEF DESCRIPTION OF THE DRAWINGS


FIG. 1 is a functional block diagram of an interpolation-frame creating apparatus according to a first embodiment of the present invention;



FIG. 2 is a schematic for explaining an interpolation frame created by the interpolation-frame creating apparatus;



FIG. 3 is a schematic for explaining motion estimation according to the first embodiment;



FIG. 4 is a schematic for explaining the motion estimation according to the first embodiment;



FIG. 5 is a flowchart of processing of interpolation frame creation performed by the interpolation-frame creating apparatus shown in FIG. 1;



FIG. 6 depicts an example of a first reference frame and a second reference frame according to the first embodiment;



FIG. 7 is a flowchart of detailed processing at motion estimation in FIG. 5;



FIG. 8 is a schematic for explaining the motion estimation performed on the first reference frame and the second reference frame shown in FIG. 6;



FIG. 9 depicts the result of the motion estimation shown in FIG. 8;



FIG. 10 is a flowchart of detailed processing at area determination in FIG. 5;



FIG. 11 is a schematic for explaining correlation calculation between a first reference block assigned with a motion vector MV2 shown in FIG. 8 and a second reference block corresponding to the first reference block;



FIG. 12 depicts a matching area and mismatching areas obtained by the correlation calculation in FIG. 11;



FIG. 13 is a schematic for explaining a result of the area determination performed on another first reference block on the left to the first reference block shown in FIG. 8;



FIG. 14 depicts a result of the area determination performed on the whole first reference frame shown in FIG. 8;



FIG. 15 is a flowchart of detailed processing at mismatching motion-vector assignment in FIG. 5;



FIG. 16 depicts matching areas and mismatching areas in the first reference frame;



FIG. 17 is a schematic for explaining a first reference frame and a second reference frame that include three motion vectors;



FIG. 18 depicts motion vectors from the first reference frame to the second reference frame shown in FIG. 17;



FIG. 19 depicts a result of area determination in a first reference block in the first reference frame shown in FIG. 18;



FIG. 20 is a block diagram of hardware configuration of the interpolation-frame creating apparatus according to the first embodiment;



FIG. 21 is a functional block diagram of an interpolation-frame creating apparatus according to a second embodiment of the present invention;



FIG. 22 is a flowchart of detailed processing at mismatching motion-vector assignment included in interpolation frame creation performed by the interpolation-frame creating apparatus according to the second embodiment;



FIG. 23 is a functional block diagram of a motion vector detector according to a third embodiment of the present invention;



FIG. 24 a functional block diagram of an interpolation-frame creating apparatus according to a fourth embodiment of the present invention;



FIG. 25 is a schematic for explaining processing performed by a motion estimating unit in the interpolation-frame creating apparatus according to a fourth embodiment of the present invention;



FIG. 26 is a flowchart of processing of interpolation frame creation performed by the interpolation-frame creating apparatus according to the fourth embodiment;



FIG. 27 is a flowchart of detailed processing at motion estimation explained with reference to FIG. 26;



FIG. 28 is a flowchart of detailed processing at area determination explained with reference to FIG. 26;



FIG. 29 is a schematic for specifically explaining the area determination in FIG. 28;



FIG. 30 is a schematic for specifically explaining the area determination in FIG. 28;



FIG. 31 is a flowchart of detailed processing at mismatching motion-vector assignment in FIG. 26;



FIG. 32 is a schematic for specifically explaining the mismatching motion-vector assignment in FIG. 31;



FIG. 33 is a functional block diagram of an interpolation-frame creating apparatus according to a fifth embodiment of the present invention;



FIG. 34 is a schematic for explaining reference frames subjected to the interpolation-frame creating apparatus according to the fifth embodiment;



FIG. 35 is a flowchart of detailed processing at mismatching motion-vector assignment in interpolation frame creation performed by the interpolation-frame creating apparatus according to the fifth embodiment;



FIG. 36 is a schematic for explaining a moving image in which an object moves;



FIG. 37 is a schematic for explaining a result of area determination performed on the moving image shown in FIG. 36;



FIG. 38 is a functional block diagram of an interpolation-frame creating apparatus according to a sixth embodiment of the present invention;



FIG. 39 is a flowchart of detailed processing at mismatching motion-vector assignment performed by the interpolation-frame creating apparatus according to a sixth embodiment of the present invention; and



FIG. 40 is a schematic for explaining motion estimation by using a mismatching area according to the sixth embodiment.


Claims
  • 1. An apparatus that interpolates an interpolation frame between a first reference frame and a second reference frame, the apparatus comprising: a motion vector detecting unit that detects a primary motion vector of each interpolation block in the interpolation frame by referring the first reference frame and the second reference frame;an area detecting unit that detects a first matching area in a first reference block in the first reference frame, a second matching area in a second reference block in the second reference frame, a first mismatching area in the first reference block and a second mismatching area in the second reference block, wherein the both the first matching area and the second matching area are matching and corresponding to each other, the both the first mismatching area and the second mismatching area are mismatching and corresponding to each other, both the first reference block and the second reference block have equal size and identical shape to the interpolation block, and each of the first reference block and the second reference block is specified based on the primary motion vector of the interpolation block;an assigning unit that assigns the primary motion vector of the each interpolation block to an interpolation matching area in the each interpolation block, wherein the interpolation matching area corresponds to the first matching area and the second matching area;a settling unit that settles a secondary motion vector to be assigned to an interpolation mismatching area in the each interpolation block based on surrounding interpolation blocks around the interpolation mismatching area, wherein the interpolation mismatching area corresponds to the first mismatching area and the second mismatching area, the first mismatching area and the second mismatching area being determined as mismatching each other by the determining unit; anda motion compensating unit that assigns an image onto the interpolation frame based on all of at least one motion vector of the primary motion vector assigned to the interpolation matching area by the assigning unit, andthe secondary motion vector settled for the interpolation mismatching area by the settling unit.
  • 2. The apparatus according to claim 1, wherein the settling unit settles the secondary motion vector from at least one primary motion vector of the surrounding interpolation blocks around the interpolation mismatching area.
  • 3. The apparatus according to claim 2, wherein when there are a plurality of primary motion vectors of the surrounding interpolation blocks around the interpolation mismatching area, the settling unit settles the secondary motion vector from the plurality of primary motion vectors based on a correlation between an area in the first reference block and an area in the second reference block, each area being specified based on the interpolation mismatching area and each of the plurality of primary motion vectors.
  • 4. The apparatus according to claim 3, wherein when there are a plurality of primary motion vectors of the surrounding interpolation blocks around the interpolation mismatching area, the settling unit settles the secondary motion vector from the plurality of primary motion vectors based on a correlation between an area in the first reference block, an area in the second reference block, and an area in a third reference block in a third reference frame, each area being specified based on the interpolation mismatching area and each of the plurality of primary motion vectors.
  • 5. The apparatus according to claim 4, wherein when there are a plurality of primary motion vectors of the surrounding interpolation blocks around the interpolation mismatching area, the settling unit settles the secondary motion vector from the plurality of primary motion vectors based on a correlation between an area in the first mismatching area, an area in the second mismatching area, and an area in the third reference block, each area being specified based on the interpolation mismatching area and each of the plurality of primary motion vectors.
  • 6. The apparatus according to claim 1, wherein when a correlation between the first mismatching area and the second mismatching area is higher than a threshold, the settling unit settles the secondary motion vector based on the surrounding interpolation blocks around the interpolation mismatching area.
  • 7. The apparatus according to claim 6, wherein when there are a plurality of primary motion vectors of the surrounding interpolation blocks around the interpolation mismatching area, and when the first mismatching area and the second mismatching area have a lower correlation than a threshold, the settling unit settles the secondary motion vector from the plurality of primary motion vectors based on a correlation between an area in the first reference block, an area in the second reference block, and an area in a third reference block in a third reference frame, each area being specified based on the interpolation mismatching area and each of the plurality of primary motion vectors.
  • 8. The apparatus according to claim 1, wherein when number of pixels included in the interpolation mismatching area is less than a threshold, the interpolation mismatching area is converted to an interpolation matching area, and converted interpolation matching area is assigned with the primary motion vector of the interpolation block.
  • 9. An apparatus that interpolates an interpolation frame between a first reference frame and a second reference frame, the apparatus comprising: a motion vector detecting unit that detects a primary motion vector of a first reference block by referring the first reference frame and the second reference frame, the first reference block is obtained by dividing the first reference frame;an area detecting unit that detects a first matching area in the first reference block and a second matching area in a second reference block in the second reference frame, a first mismatching area in the first reference block and a second mismatching area in the second reference block, wherein the both the first matching area and the second matching area are matching and corresponding to each other, the both the first mismatching area and the second mismatching area are mismatching and corresponding to each other, the second reference block has equal size and identical shape to the first reference block, and the second matching area in the second reference block is specified based on the primary motion vector of the first reference block;an assigning unit that assigns the primary motion vector to the first matching area;a settling unit that settles a secondary motion vector to be assigned to a first mismatching area in the first reference block based on surrounding first reference blocks around the first mismatching area, wherein the first mismatching area is determined as mismatching by the determining unit; anda motion compensating unit that assigns an image onto the interpolation frame based on all of at least one motion vector of the primary motion vector assigned to the matching area by the assigning unit, andthe secondary motion vector settled for the first mismatching area by the settling unit.
  • 10. The apparatus according to claim 9, wherein the settling unit settles the secondary motion vector from at least one primary motion vector of the surrounding first reference blocks around the first mismatching area.
  • 11. The apparatus according to claim 10, wherein when there are a plurality of primary motion vectors of the surrounding first reference blocks around the first mismatching area, the settling unit settles the secondary motion vector from the plurality of primary motion vectors based on a correlation between the first mismatching area and an area in the second reference block that is specified based on the first mismatching area and each of the plurality of primary motion vectors.
  • 12. The apparatus according to claim 11, wherein the settling unit settles the secondary motion vector so as to provide a highest value of the correlation.
  • 13. The apparatus according to claim 9, wherein when a correlation between the first mismatching area and a second mismatching area in the second reference block determined as mismatching by the determining unit is higher than a threshold, the settling unit settles the secondary motion vector based on the surrounding first reference blocks around the first mismatching area.
  • 14. The apparatus according to claim 9, wherein when number of pixels included in a first mismatching area in the first reference block is less than a threshold, the first mismatching area is converted to a matching area, and converted matching area is assigned with the primary motion vector of the first reference block.
Priority Claims (1)
Number Date Country Kind
2006-095941 Mar 2006 JP national