The invention relates to scalable video coding (SVC), more specifically on inter-layer motion prediction used in SVC standard being defined by the Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG.
State-of-art scalable hierarchical coding methods allow to encode the information hierarchically in order that it can be decoded at different resolution and/or quality levels. A data stream generated by a scalable coding device is thus divided into several layers, a base layer and one or more enhancement layers. These devices allow to adapt a unique data stream to variable transmission conditions (bandwidth, error rate . . . ) and also to the capacities of reception devices (CPU, characteristics of reproduction device . . . ). A spatially scalable hierarchical encoding (or decoding) method encodes (or decodes) a first part of data called base layer relating to low resolution images, and from this base layer encodes (or decodes) at least another data part called enhancement layer relating to high resolution images. The coding information relating to enhancement layer are possibly inherited (i.e. derived) from coding information relating to the base layer by a method called inter-layer inheriting method. The derived coding information may possibly comprise: a partitioning pattern associated to block of pixels of the high resolution image (for splitting said block into several sub-blocks), coding modes associated to said blocks, possibly motion vectors and one or more image reference indices associated to some blocks allowing to reference the image used to predict said block. A reference image is an image of the sequence used to predict another image of the sequence. Thus, if not explicitly coded in the data stream, the coding information (more specifically, macroblock type and sub-macroblock types) relating to the enhancement layer has to be derived from the coding information relating to low resolution images. State-of-art methods for deriving coding information can be used for high resolution images whose format is not linked to the format of low resolution images by a dyadic transform. The most recent method, as described in the document from the Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG JVT-R202 entitled “Joint Scalable Video Model JSVM-5: Joint Draft 5 with proposed changes”, J. Reichel, H. Schwarz, M. Wien (denoted in the sequel JSVM5), proposes a direct inter-layer motion prediction method, based on some complex (in terms of textual description) processes.
It is an object of the invention to simplify significantly this inter-layer motion prediction process and its description. More specifically, the invention relates to a process for determining the partitioning of said macroblock in partitions and possibly the partitioning of said partitions in sub-partitions. It is another object of the invention to derive macroblock type and sub-macroblock types, for a given high layer macroblock, using inter-layer prediction, according to claim 1.
Other particularities and advantages of the invention will become clearly apparent from the following description given by way of non limiting examples and given in light of the appended figures which represent:
In the sequel, two spatial layers are considered, a low layer (called base layer) corresponding to the images of low resolution and a high layer (called enhancement layer) corresponding to the images of high resolution. As represented in
For a given high layer macroblock that uses inter-layer prediction, the partitions, possibly sub-partitions, macroblock type and sub-macroblock types derivation process is based on the use of an array of 4×4 integers, called partInfo[x][y], with x=0 to 3 and y=0 to 3.
The process works in the following steps:
These successive steps, represented in
Derivation of partInfo Values (Step 20)
Let (xM,yM) be the position of the upper left pixel of the said high layer macroblock in the high layer reference (with origin positioned at upper left pixel of the high layer picture).
According to the invention, the process comprises a step for identifying for each 4×4 block (x,y), x=0 to 3 and y=0 to 3 of the said high layer macroblock a corresponding macroblock mbBase in the base layer picture, i.e. identifying its address mbAddrBase. To this aim, let (Xin=4*x, Yin=4*y) be the input pixel position of this process, that will derive the corresponding base layer macroblock address, the base layer partition index of the said base layer macroblock, and the base layer sub-partition index of the said partition, comprising the corresponding pixel in base layer of pixel (Xin, Yin).
Let (xP,yP) be the so-called reference pixel of the 4×4 block of the said high layer macroblock defined as
The corresponding position (xB,yB) in the base layer is thus computed as follows:
The address mbAddrBase of the base layer macroblock mbBase comprising the pixel (xB,yB) is identified as follows:
If the said base layer macroblock mbBase does not exists (e.g. pixel (xB,yB) is outside the base layer picture limits), mbAddrBase is marked as not available. The interlayer prediction is forbidden for the high layer macroblock and the process is stopped.
The process comprises a step for identifying the partition (mbPartIdxBase) and subpartition (subMbPartIdxBase) of the corresponding 4×4 block mbBase. The partition (mbPartIdxBase) and subpartition (subMbPartIdxBase) are identified as follows:
The process of derivation of base macroblock, partition and sub-partition described above is applied with position (Xin=4*x,Yin=4*y) as input, to derive the base macroblock address mbAddrBase, the index mbPartIdxBase of the base layer partition and the index subMbPartIdxBase of the sub-partition, corresponding to the said 4×4 block.
The process comprises a step for deriving partInfo[x][y]. partInfo[x][y] is computed as follows:
The following process is then applied to manage configurations in which some 4×4 blocks inherit from intra base macroblocks and other from inter base macroblocks. The process cleans the array partInfo by replacing any element partInfo[x][y] equal to −1 by one of its neighboring element. This cleaning process first applies on 4×4 blocks of each 8×8 block. Then 8×8 blocks are cleaned if necessary. The process is detailed in the sequel.
For each 8×8 block (X,Y), X=0 to 1 and Y=0 to 1, the following applies:
Cleaning of 4×4 Blocks
Pseudo-Code Description of this Process
The process synoptic is illustrated in
Pseudo-Code Description of this Process
This process derives the high layer macroblock type mbtype and, if it is divided in 4 8×8 blocks, the sub-macroblock type subMbType[blkIdx] of each 8×8 block of index blkIdx, from the array partInfo and from the base layer macroblocks type (also called coding modes).
It is considered that each base layer macroblock contains an available information indicating for each of its 8×8 block which prediction list it uses. This information is stored as an bidimensional array usedPredList[mbAddr][blkIdx], the first dimension corresponding to the macroblock address mbAddr, the second dimension to the index of the 8×8 block inside the macroblock blkIdx (taking values between 0 and 3).
If 8×8 block of index blkIdx of macroblock of address mbAddr uses:
As specified in JSVM5, the macroblock type is defined among the values of Table 1. Similarly, as specified in JSVM5, the sub-macroblock type is defined among the values of Table 1.
The derivation process for macroblock type mbtype and, if applicable, sub-macroblock type subMbType[blkIdx] of each 8×8 block of index blkIdx, works as follows.
Partition Determination and mbType Derivation
The process synoptic is illustrated in
Finally, the mbtype is derived, at step 76, based on the partitioning partSize, the derived prediction list information partPredMode0 and partPredMode1 and the type of the slice slice_type comprising the high layer macroblock under consideration, using Table 2.
Pseudo-Code Description of this Process
If mbtype is equal to P—8×8 or B—8×8, sub-macroblock type of each 8×8 block is determined. as follows.
The process synoptic is illustrated in
For each 8×8 block (x0,y0) with x0,y0=0, . . . , 1, the sub-partitioning is determined depending on the analysis of partInfo values.
Finally, the subMbType of the considered 8×8 block is derived, at step 90, based on the sub-partitioning subPartSize, the derived prediction list information partPredMode and the type of the slice slice_type comprising the high layer macroblock under consideration, using Table 3.
Pseudo-Code Description of this Process
When mbType is equal to P—8×8 or B—8×8, for mbPartIdx=0 . . . 3, derivation of subMbType for each partition is achieved as follows.
Number | Date | Country | Kind |
---|---|---|---|
06290747.2 | May 2006 | EP | regional |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/EP2007/054108 | 4/26/2007 | WO | 00 | 11/5/2008 |