MOTION ESTIMATION DEVICE, MOTION ESTIMATION METHOD, MOTION ESTIMATION INTEGRATED CIRCUIT, AND PICTURE CODING DEVICE

Abstract

Provided is a motion estimation device in which an amount of pixel data transferred from an external frame memory to an internal reference local memory is reduced. By the motion estimation device, it is possible to reduce a memory capacity and a size or processing of a circuit controlling the pixel transfer. In a reference memory control unit and an internal reference memory, a height of a area to be updated is set to L pixels, where L is power of 2, a logical address segments, whose size is suitable for address calculation, are allocated to picture space, and FIFO management is performs. In another application, an assistance memory is added, and another element other than the assistance memory performs the FIFO management for rectangular areas in an image of a conventional width. As a result, the address calculation is simplified, which makes it possible to reduce an embedded circuit for the reference memory control unit and the internal reference memory.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

These and other objects, advantages and features of the invention will become apparent from the following description thereof taken in conjunction with the accompanying drawings that illustrate specific embodiments of the present invention. In the Drawings:

FIG. 1 is a block diagram showing the conventional inter-picture prediction coding device;

FIG. 2 is a block diagram showing the conventional inter-picture prediction decoding device;

FIG. 3 is a block diagram showing a structure in which the inter-picture prediction coding device is connected with a frame memory;

FIG. 4 is a schematic diagram showing how pixels in one reference picture are to be transferred;

FIG. 5 is a schematic diagram showing how a memory is updated, in order to reduce a transfer amount of reference pixels;

FIG. 6 is a schematic diagram showing how stored pixels are managed, in order to reduce a capacity of the reference local memory;

FIG. 7 is a diagram showing a physical address layout around a logical boundary in the reference logical memory, when the FIFO management is used.

FIG. 8 is a block diagram showing a structure of an inter-picture prediction coding device using a motion estimation device according to the first embodiment;

FIG. 9 is a block diagram showing a structure in which the inter-picture prediction coding device is connected with a frame memory;

FIG. 10 is a schematic diagram showing memory areas in a cache memory and the reference local memory;

FIG. 11 is a schematic diagram showing a structure of rectangular areas stored in the cache memory;

FIG. 12 is a schematic diagram showing a structure of rectangular areas when next motion estimation is performed;

FIG. 13 is a schematic diagram showing positions of intermediate information of address conversion performed by the motion estimation device according to the first embodiment of the present invention;

FIGS. 14A and 14B are flowcharts of the address conversion performed by the motion estimation device according to the first embodiment of the present invention;

FIG. 15 is a schematic diagram showing a management area storing pixels of a reference picture used in a motion estimation device according to the second embodiment of the present invention;

FIG. 16 is a schematic diagram showing positions of intermediate information of address conversion performed by the motion estimation device according to the second embodiment of the present invention;

FIG. 17 is a flowchart of address conversion performed by the motion estimation device according to the second embodiment of the present invention;

FIG. 18 is a schematic diagram showing a management area storing reference picture pixels used in a motion estimation device according to the third embodiment of the present invention;

FIG. 19 is a schematic diagram showing positions of intermediate information of address conversion in the motion estimation device according to the third embodiment of the present invention;

FIG. 20 is a flowchart of address conversion performed by the motion estimation device according to the third embodiment of the present invention;

FIGS. 21A and 21B are schematic diagrams showing a structure of rectangular areas when a motion estimation range is positioned at edges of a reference picture;

FIG. 22 is a block diagram showing a structure in which an inter-picture prediction coding device is connected with a frame memory, according to the fifth embodiment of the present invention;

FIGS. 23A and 23B are schematic diagrams showing managed areas for storing pixels of a reference picture used in a motion estimation device according to the fifth embodiment of the present invention;

FIG. 24 is a block diagram showing a structure in which an inter-picture prediction coding device is connected with a frame memory, according to the sixth embodiment of the present invention;

FIGS. 25A and 25B are schematic diagrams showing managed areas for storing pixels of a reference picture used in a motion estimation device according to the sixth embodiment of the present invention;

FIGS. 26A and 26B are schematic diagrams showing variations of managed areas storing pixels of a reference picture used in a motion estimation device according to the sixth embodiment of the present invention; and

FIG. 27 is a block diagram showing a structure of an AV processing unit realizing a H.264 recorder.

Claims

1. A motion estimation device which estimates motion of each block in a picture, said motion estimation device comprising: a storage unit operable to store a reference picture;a reference memory unit operable to store pixel data included in a transferred area of the reference picture, the pixel data being transferred from said storage unit;a motion estimation unit operable to estimate motion of a target block, for which motion estimation is performed, using pixel data in a motion estimation range included in the transferred area stored in said reference memory unit; anda memory control unit operable to update a part of the transferred area for use in a next target block,wherein the transferred area includes first to third rectangular areas,the first rectangular area includes the motion estimation range,the second rectangular area has a width that is from (i) a left-most pixel in the reference picture to (ii) a pixel on immediately left of a left-most pixel in the first rectangular area, a height that is L pixels lower than a height of the first rectangular area, and a bottom that is positioned at same horizontal level as a bottom of the first rectangular area,the third rectangular area has a width that is from (i) a right-most pixel in the reference picture to (ii) a pixel on immediately right of a right-most pixel in the first rectangular area, a height that is L pixels lower than the height of the first rectangular area, and an upper limit that is positioned at same horizontal level as an upper limit of the first rectangular area,L is expressed by power of 2, andsaid memory control unit is operable to transfer pixel data from an update area in said storage unit to a memory area in said reference memory unit, the update area being positioned on immediately right of the first rectangular area and immediately under the third rectangular area and having a height of L pixels and a width of K pixels, and the memory area being positioned in the first rectangular area at top left and having a height of L pixels and a width of K pixels.

2. The motion estimation device according to claim 1, wherein coordinates of a pixel at top left of the reference picture stored in said storage unit are (0, 0), the reference picture having a height of H pixels and a width of W pixels,coordinates of a pixel at top left of the first rectangular area corresponding to the target block are (X, Y), the first rectangular area having a height of J pixels and a width of I pixels,coordinates of a pixel at top left of the second rectangular area corresponding to the target block are (0, Y+L), coordinates of a pixel at bottom right of the second rectangular area corresponding to the target block are (X−1, Y+J−1),coordinates of a pixel at top left of the third rectangular area corresponding to the target block are (X+I, Y), and coordinates of a pixel at bottom right of the third rectangular area corresponding to the target block are (W−1, Y+J−L−1).

3. The motion estimation device according to claim 2, wherein said memory control unit is operable, by transferring the pixel data from the update area in said storage unit to the memory area in said reference memory unit for the next target block, the memory area storing the pixel data, having a height of L pixels and a width of K pixels, and having coordinates of a top left pixel, which is the top left pixel in the first rectangular area, as (X, Y), and coordinates of a bottom right pixel as (X+K, Y+L), to:update, for the next target block, the coordinates of the pixel at top left of the first rectangular area to (X+K, Y) and the coordinates of the pixel at bottom right of the first rectangular area to (X+I+K−1, Y+J−1);update, for the next target block, the coordinates of the pixel at top left of the second rectangular area to (0, Y+L) and the coordinates of the pixel at bottom right in the second rectangular area to (X+K−1, Y+J−1); andupdate, for the next target block, the coordinates of the pixel at top left in the third rectangular area to (X+K+I, Y) and the coordinates of the pixel at bottom right in the third rectangular area to (W−1, Y+J−L−1).

4. The motion estimation device according to claim 1, wherein said reference memory unit includes:a first memory unit operable to store pixel data included in the first to third rectangular areas; anda second memory unit operable to store pixel data included in the motion estimation range in the first rectangular area, the pixel data being transferred from said first memory unit,said memory control unit is operable to transfer the pixel data included in the motion estimation range from said first memory unit to said second memory unit, andsaid motion estimation unit is operable to obtain the pixel data from said second memory unit and estimate the motion of the target block.

5. The motion estimation device according to claim 1, wherein said memory control unit is operable to:divide the first rectangular area into a first right rectangular area and a first left rectangular area, when a right edge of the first rectangular area is positioned beyond a right edge of the reference picture during processing of the motion estimation, (i) the first right rectangular area including the right edge of the reference picture, and (ii) the first left rectangular area including a left edge of the reference picture and being positioned L pixels lower than the first right rectangular area;combine the second and third rectangular areas into one rectangular area, when the first rectangular area is divided, (iii) the combined rectangular area being sandwiched in a horizontal direction between the first right rectangular area and the first left rectangular area, (iv) a bottom of the combined rectangular area being at same horizontal level as a bottom of the first right rectangular area, and (v) an upper limit of the combined rectangular area being at same horizontal level as an upper limit of the first right rectangular area; andset the motion estimation range to an area included in one of the first right rectangular area and the first left rectangular area, without dividing the motion estimation range.

6. The motion estimation device according to claim 1, wherein said memory control unit is operable to:convert a logical address expressed by coordinates (X, Y) of the reference picture into a physical address allocated in a raster direction; andmanage said reference memory unit by a first in first out (FIFO) method in the transferring from said storage unit to said reference memory unit.

7. The motion estimation device according to claim 6, wherein said memory control unit is operable to:divide the reference picture, by vertically dividing the reference picture into equal M columns, horizontally dividing the reference picture into rows each having a height of L pixels, and setting a height of the transferred area to N rows; anddivide said reference memory unit into memory banks whose number is equal to or more than “M×(N−1)+1” and equal to or less than “M×N−1”, and manage the divided memory banks in cycles by the FIFO method.

8. The motion estimation device according to claim 6, wherein said memory control unit is operable to:divide a virtual space, which includes the reference picture and has power of 2 addresses in a horizontal direction, by vertically dividing the virtual space into equal M columns, where M is expressed by power of 2, horizontally dividing the virtual space into rows each having a height of L pixels, and setting a height of the transferred area to N rows; anddivide said reference memory unit into memory banks whose number is equal to or more than “M×(N−1)+1” and equal to or less than “M×N−1”, and manage the divided memory banks in cycles by the FIFO method.

9. The motion estimation device according to claim 1, wherein the first rectangular area includes a fourth rectangular area and a fifth rectangular area, the fourth rectangular area having a height of L pixels from a bottom of the first rectangular area, and the fifth rectangular area being an area other than the fourth rectangular area,said reference memory unit includes a first memory unit and an assistance memory unit,said first memory unit is operable to store the fifth rectangular area, the second rectangular area, and the third rectangular area by a FIFO method,said assistance memory unit is operable to store the fourth rectangular area by the FIFO method, andsaid memory control unit is operable, for the next target block, to: (i) transfer pixel data from said assistance memory unit to said first memory unit, the pixel data being included in an area having a height of L pixels and a width of K pixels positioned in the fourth rectangular area at left; and (ii) transfer pixel data in the update area from said storage unit to said assistance memory unit by the FIFO method.

10. The motion estimation device according to claim 9, wherein said assistance memory unit is operable to store a sixth rectangular area immediately under the fourth rectangular area, a width of the sixth rectangular area being same as a width of the fourth rectangular area,said memory control unit is operable, for the next target block, to: transfer pixel data from said assistance memory unit to said first memory unit, the pixel data being included in an area having a height of L pixels and a width of K pixels positioned in the fourth rectangular area at left; and transfer both of the fourth rectangular area and the sixth rectangular area from said storage unit to said assistance memory unit by the FIFO method; and transfer pixel data in an extended update area from said storage unit to said assistance memory unit, the extended update area being obtained by extending said update area downwards, andsaid motion estimation unit is operable to perform motion estimation using the fourth rectangular area, the fifth rectangular area, and the sixth rectangular area.

11. The motion estimation device according to claim 1, wherein the first rectangular area includes a seventh rectangular area and an eighth rectangular area, the seventh rectangular area having a height of L pixels from an upper limit of the first rectangular area, and the eighth rectangular area being an area other than the seventh rectangular area,said reference memory unit includes a first memory unit and an assistance memory unit,said first memory unit is operable to store the eighth rectangular area, the second rectangular area, and the third rectangular area by a first in first out (FIFO) method,said assistance memory unit is operable to store the seventh rectangular area by the FIFO method, andsaid memory control unit is, for a next target block, operable to: (i) transfer pixel data from said first memory unit to said assistance memory unit, the pixel data being included in an area having a height of L pixels and a width of K pixels positioned in the third rectangular area at left; and (ii) transfer pixel data in the update area from said storage unit to said first memory unit by the FIFO method.

12. The motion estimation device according to claim 11, wherein said assistance memory unit is operable to store a ninth rectangular area on the seventh rectangular area, a width of the ninth rectangular area being same as a width of the seventh rectangular area,said memory control unit is, for the next target block, operable to: transfer pixel data from said first memory unit to said assistance memory unit, the pixel data being included in an area having a height of L pixels and a width of K pixels positioned in the third rectangular area at left; manage both of the seventh rectangular area and the ninth rectangular area in said assistance memory unit by the FIFO method; and transfer pixel data the update area from said storage unit to said first memory unit, andsaid motion estimation unit is operable to perform motion estimation using the seventh rectangular area, the eighth rectangular area, and the ninth rectangular area.

13. The motion estimation device according to claim 11, wherein said reference memory unit includes a second reference memory, to which (i) pixel data included in the motion estimation range and (ii) pixel data included in an update range added to the motion estimation range, the update range having pixels of a width of K pixels on immediately right to the motion estimation range, among the seventh rectangular area and the eighth rectangular area stored in said assistance memory unit and said first memory unit, are transferred,said motion estimation unit is operable to estimate motion of the target block using pixel data of the reference picture, the pixel data being obtained from said second reference memory, andsaid memory control unit is operable to manage said assistance memory unit by the FIFO method, in which pixel data of the update range is a last pixel data.

14. A motion estimation method of estimating motion of each block in a picture, said motion estimation method comprising: transferring pixel data from a storage unit to a reference memory unit, the storage unit storing the reference picture, and the pixel data being included in a transferred area of the reference picture;estimating motion of a target block, for which motion estimation is performed, using pixel data in a motion estimation range included in the transferred area stored in the reference memory unit; andupdating a part of the transferred area for use in a next target block,wherein the transferred area includes first to third rectangular areas,the first rectangular area includes the motion estimation range,the second rectangular area has a width that is from (i) a left-most pixel in the reference picture to (ii) a pixel on immediately left of a left-most pixel in the first rectangular area, a height that is L pixels lower than a height of the first rectangular area, and a bottom that is positioned at same horizontal level as a bottom of the first rectangular area,the third rectangular area has a width that is from (i) a right-most pixel in the reference picture to (ii) a pixel on immediately right of a right-most pixel in the first rectangular area, a height that is L pixels lower than the height of the first rectangular area, and an upper limit that is positioned at same horizontal level as an upper limit of the first rectangular area,L is expressed by power of 2, andin said updating, pixel data is transferred from an update area in the storage unit to a memory area in the reference memory unit, the update area being positioned on immediately right of the first rectangular area and immediately under the third rectangular area and having a height of L pixels and a width of K pixels, and the memory area being positioned in the first rectangular area at top left and having a height of L pixels and a width of K pixels.

15. An integrated circuit which estimates motion of each block in a picture, said integrated circuit comprising: a reference memory unit operable to store pixel data included in a transferred area of a reference picture, the pixel data being transferred from an external storage unit storing the reference picture;a motion estimation unit operable to estimate motion of a target block, for which motion estimation is performed, using pixel data in a motion estimation range included in the transferred area stored in said reference memory unit; anda memory control unit operable to update a part of the transferred area for use in a next target block,wherein the transferred area includes first to third rectangular areas,the first rectangular area includes the motion estimation range,the second rectangular area has a width that is from (i) a left-most pixel in the reference picture to (ii) a pixel on immediately left of a left-most pixel in the first rectangular area, a height that is L pixels lower than a height of the first rectangular area, and a bottom that is positioned at same horizontal level as a bottom of the first rectangular area,the third rectangular area has a width that is from (i) a right-most pixel in the reference picture to (ii) a pixel on immediately right of a right-most pixel in the first rectangular area, a height that is L pixels lower than the height of the first rectangular area, and an upper limit that is positioned at same horizontal level as an upper limit of the first rectangular area,L is expressed by power of 2, andsaid memory control unit is operable to transfer pixel data from an update area in the external storage unit to a memory area in said reference memory unit, the update area being positioned on immediately right of the first rectangular area and immediately under the third rectangular area and having a height of L pixels and a width of K pixels, and the memory area being positioned in the first rectangular area at top left and having a height of L pixels and a width of K pixels.

16. An image coding device which estimates motion of each block in a picture and codes an image based on the motion, said image coding device comprising: a storage unit operable to store a reference picture;a reference memory unit operable to store pixel data included in a transferred area of the reference picture, the pixel data being transferred from said storage unit;a motion estimation unit operable to estimate motion of a target block, for which motion estimation is performed, using pixel data in a motion estimation range included in the transferred area stored in said reference memory unit; anda memory control unit operable to update a part of the transferred area for use in a next target block,wherein the transferred area includes first to third rectangular areas,the first rectangular area includes the motion estimation range,the second rectangular area has a width that is from (i) a left-most pixel in the reference picture to (ii) a pixel on immediately left of a left-most pixel in the first rectangular area, a height that is L pixels lower than a height of the first rectangular area, and a bottom that is positioned at same horizontal level as a bottom of the first rectangular area,the third rectangular area has a width that is from (i) a right-most pixel in the reference picture to (ii) a pixel on immediately right of a right-most pixel in the first rectangular area, a height that is L pixels lower than the height of the first rectangular area, and an upper limit that is positioned at same horizontal level as an upper limit of the first rectangular area,L is expressed by power of 2, andsaid memory control unit is operable to transfer pixel data from an update area in said storage unit to a memory area in said reference memory unit, the update area being positioned on immediately right of the first rectangular area and immediately under the third rectangular area and having a height of L pixels and a width of K pixels, and the memory area being positioned in the first rectangular area at top left and having a height of L pixels and a width of K pixels.