Patent Application
20050100098
1. Field of the Invention
The present invention relates to a video decoding unit. More specifically, a highly integrated MPEG-4 video decoding unit and a method for decoding MPEG-4 video is disclosed.
2. Description of the Prior Art
MPEG-4 is an ISO/IEC standard developed by MPEG (Moving Picture Experts Group) proven to be beneficial in digital television, interactive graphics, and interactive multimedia. Unlike its successors, MPEG-1 and MPEG-2 that basically standardized a way to sequentially present a series of pictures to the user, MPEG-4 represents a totality of possibly multiple media objects, each of which may be real or generated by a computer. The media objects are described and synchronized in such a way that they can be combined to form compound audiovisual scenes easily transmitted over a network.
Several layers of video coding hierarchy are required to implement MPEG-4. At the top is a Video Object Sequence (VS) that contains all the media objects making up the complete visual scene. Next is a Video Object (VO) that represents one of the media objects in the scene. Under the VO is a Video Object Layer (VOL) that provides scalalbe coding and a full MPEG-4 header VOL or a short header VOL. Under the VOL is a Video Object Plane (VOP), which is a sample of the VO at a moment in time, contains motion, shape, and texture information, and obviously requires proper decoding to generate a visual scene for presentation to the viewer.
There are several types of VOPs, commonly known as I (Intra), P (Predicted), and B (Bi-directional) pictures. An I picture (I-VOP) is self-contained and is encoded independently of any other VOPs. A P picture (P-VOP) is encoded using another previous VOP and motion compensation. A B picture (B-VOP) also uses motion compensation and is interpolated using other I-VOPs or P-VOPs and in either or both directions relative to the video stream. Additionally, the MPEG-4 specification provides for two additional types of VOPs. These two additional VOPs are similar to I-VOP and P-VOP frames except that they are Data-partitioned. Data-partitioned I-VOPs and P-VOPs are signaled by the VOL and separate the motion information from the texture information to localize error at the decoder and improve performance.
Decoding of VOPs is a complicated process often entailing different decoding modules for different types of VOPs. Please refer to
A decoding module 20 is provided for interpreting the signals mcbpc, or if not coded, mcsel. The mcbpc is a variable length code used with a lookup table to derive the macroblock type and the coded block pattern for chrominance. The decoded mcbpc (or mcsel) is then stored in a storage element 32 of the memory 30. A decoding module 22 is provided for the signals ac_pred_flag and cbpy. The ac_pred_flag is a 1-bit flag that indicates that either the first row or the first column of AC coefficients are differentially coded for Intra coded macroblocks. The signal cbpy is a variable length code representing a pattern of non-transparent luminance blocks with at least 1 non Intra DC transform coefficient in a macroblock and is interpreted with another lookup table. The data is then stored in a storage element 34 of the memory 30.
Similarly, decoding module 24 for decoding the signal dquant, decoding module 26 for decoding interlaced information, and decoding module 28 for decoding motion vectors also store the results of the decoding process into storage elements 36, 38, and 40 respectively. All three of these signals are 2-bit codes that specify changes in the quantizer. Again, the use of another lookup table is employed in obtaining the results that are stored in the memory 30.
Now, please refer to
The set 45 comprises a decoding module 52 for decoding the signals dct_dc_size_lum, dct_dc_size_chrominance, and dct_dc_size_differential. These three signals are variable length codes used to respectively derive differential DC coefficients of luminance, differential DC coefficients of chrominance, and differential DC coefficients in Intra macroblocks. Again, each value is obtained utilizing lookup tables and stored into a storage element 58 in the memory 60. The set 45 additionally comprises a decoding module 54 for AC coefficients according to the results obtained from the decoding module 22 of
The decoding of Data-partitioned I-VOPs and P-VOPs each require a separate pair of decoding loops. The decoding loops for Data-partitioned I-VOPs are illustrated in
In
Turning now to
A memory 134 and a set of decoding modules 132 of a second decoding loop 130 for Data-partitioned P-VOPs are illustrated in
The success of MPEG-4 is a testimonial to the effectiveness of utilizing the six decoding loops, six sets of decoding modules, and six sets of memory storage elements briefly outlined above and as laid out by the MPEP-4 specification. All said sets are required to fully comply with the MPEG-4 standard. However, implementation of MPEG-4 decoding modules as described above results in high cost, complexity, and chip size.
It is therefore a primary objective of the claimed invention to reduce the cost, complexity, and chip size of an apparatus for decoding MPEG-4 by disclosing a highly integrated MPEG-4 video decoding unit.
The claimed invention includes three primary structures, a VOP decoding switching circuit, a set of decoding modules, and a memory. The set of decoding modules includes a plurality of decoding modules, most of which are each capable of providing multiple functions during decoding. The multi-functioned decoding modules decode a predetermined signal in each of a predetermined plurality of VOP types and output a decoded result specifically corresponding to the VOP type currently being decoded. The predetermined plurality of VOP types may at least include Data-partitioned Intra VOPs and Data-partitioned Predicted VOPs as defined by the MPEG-4 specification and may also include combined I-VOPs, combined P-VOPs, and combined B-VOPs.
A VOP may be sent to the set of decoding modules via the VOP decoding switching circuit. According to the type of VOP being decoded, the VOP decoding switching circuit may then send a predetermined sequence of selection signals to a multiplexer that is connected between the decoding modules and the memory so that the appropriate decoded results are stored into the corresponding storage elements in the memory.
The claimed decoding module may allow for the storage of the lookup table needed for decoding the predetermined signal in more than one type of VOP. Here, the decoding module may include a VOP type indicating flag that is set by the VOP decoding switching circuit. During decoding of a VOP, the decoding module can select the correct lookup table according to the value of the VOP type indicating flag. If the next VOP to be decoded by the decoding module is of a different type, the VOP type indicating flag is then set to the new VOP type indicating that the decoding module is to access a different lookup table.
In another example of the claimed invention, each decoding module includes at least enough memory to store the largest lookup table that will be used by that decoding module. Upon determination by the VOP decoding switching circuit of which type of VOP is to be decoded, the necessary lookup table(s) may be electronically transferred to the corresponding decoding module. Once the decoding module has received the corresponding lookup table, decoding can be completed for that predetermined signal. If the next VOP to be decoded is of a different type, and therefore requiring a different lookup table, the necessary lookup table or tables for that VOP type are then transferred to the decoding module.
These and other objectives of the claimed invention will no doubt become obvious to those of ordinary skill in the art after reading the following detailed description of the preferred embodiment, which is illustrated in the various figures and drawings.
The present invention discloses a highly integrated MPEG-4 video decoding unit and method for decoding MPEG-4 video. The claimed method and apparatus greatly reduces the required number of decoding modules required by the prior art while retaining their functionality.
The video decoding unit 200 comprises three primary structures, the VOP decoding switching circuit 210, a set 205 of decoding modules, and a memory 215 for storing the results outputted from the set 205 of decoding modules. The memory 215 may optionally be subdivided into a plurality of regions 270 and 275 segregated by result type as shown in
The set 205 of decoding modules comprises a plurality of decoding modules, most of which are each capable of providing multiple functions during decoding. A decoding module 230 for signals performs the various functions provided by the prior art decoding modules 20, 72, and 116, with the respective results transmitted through the multiplexer 220 into an appropriate storage element 270 in the memory 215 for further processing. A decoding module 235 provides both the functions previously provided by the prior art decoding modules 24 and 74, routing the ensuing results through the multiplexer 220 into the storage element 270. A decoding module 240 supplies the functions of the prior art decoding modules 22, 96, and 136, storing the resulting into the storage element 270 via the multiplexer 220. A single decoding module 245, replaces the prior art decoding modules 26, 28, and 118, with the storage element 270 again receiving the results through the multiplexer 220, provides motion information. The decoding module 245 may also be utilized to decode the interlaced information of the prior art decoding module 26 or an additional (not shown) module may be optionally added. Luminance, chrominance, and differential signals are decoded in a decoding module 250 and provide the functions previously provided by decoding modules 52, 76, and 138. The respective results are stored into a storage element 275 via a multiplexer 225. A decoding module 255 that replaces the prior art decoding modules 54, 100, and 144 decodes AC coefficients. The results are transmitted through the multiplexer 225 into the storage element 275. A marker checking module 260 replaces the prior art decoding modules 78 and 120. In addition, a macroblock number checker module 265 performs the functions previously requiring decoding modules 98 and 140.
As can be understood from
As stated in the prior art section of this disclosure, different types of VOPs may require different lookup tables uniquely corresponding to the specific type of VOP being decoded by any given decoding module at any given time. The present invention eliminates unneeded decoding circuitry while overcoming this incompatibility issue of the prior art in at least two ways.
One embodiment of the present invention allows for the storage of all of the lookup information needed for a specific decoding function corresponding to more than one type of VOP in the decoding module responsible for that function. For example, the decoding module 230 may store the lookup table for decoding the mcbpc signal in an I-VOP and the lookup table for decoding the mcbpc signal in a P-VOP. The type of VOP currently being decoded obviously determines the decision of which lookup information is used during the decoding process. One possible implementation of this decision is for the decoding module 230 to comprise a VOP type indicating flag that is set internally or by the VOP decoding switching circuit 210. During decoding of a VOP, the decoding module 230 can select the correct lookup information according to the value of the VOP type indicating flag. If the next VOP to be decoded by the decoding module is of a different type, the VOP type indicating flag is then set to the new VOP type indicating that the decoding module is to access a different lookup table.
Another embodiment of the present invention overcomes the prior art incompatibility issue in another manner. Each decoding module comprises at least enough memory to store the largest lookup table that will be used by that decoding module. Upon determination by the VOP decoding switching circuit 210 of which type of VOP is to be decoded, the necessary lookup tables may be electronically transferred to the corresponding decoding module. Once the decoding module has received the corresponding lookup table, decoding can be completed for that predetermined signal. If the next VOP to be decoded is of a different type, and therefore requiring a different lookup table, the necessary lookup table or tables for that VOP type are then transferred to the decoding module or possibly fetched by the decoding module from the VOP decoding switching circuit 210 or alternate source.
The highly integrated MPEG-4 video decoding unit of the present invention provides all of the functionality of the prior art decoding units while greatly reducing the necessary hardware and associated costs. Individual decoding modules are designed to decode specific signals occurring in the VOPs. The specific signals are interpreted correctly according to the type of VOP being currently decoded. A switching circuit may be employed to indicate to the respective decoding modules which type of VOP is to be decoded by that decoding module. Furthermore, the switching circuit may be connected to a multiplexer to control delivery of the decoded results from each decoding circuit to a storage element in memory. Obviously, not all decoding modules need to be decoding the same VOP or the same VOP type simultaneously. As such, the switching circuit may be designed to maximize throughput by efficient selection and assignment of VOPs to the individual decoding units.
Those skilled in the art will readily observe that numerous modifications and alterations of the device and method may be made while retaining the teachings of the invention Accordingly, the above disclosure should be construed as limited only by the metes and bounds of the appended claims.
