The technical field of this invention is video data transcoding.
A transcoder changes the bitstream data format from one format to another format. A MPEG-2 to H.264 transcoder changes the bitstream data format from MPEG-2 format to H.264 format.
The H.264 standard provides macroblock-adaptive field-frame (MBAFF) coding to effectively encode interlaced video sequences. The application of MBAFF in transcoding is not straightforward because the macroblock (MB) order of the H.264 standard using MBAFF differs from the normal raster scan order followed by the MPEG-2 standards.
Conventional H.264 MBAFF transcoding uses a frame based method. A decoder decodes one frame and writes this frame to external memory. An encoder reads the decoded frame from the external memory while changing the macroblock scan order. This absorbs the difference of the scan order.
1) DMA 213 reads MPEG-2 bitstream data from external memory 220 and transmits it to MPEG-2 decoder 211.
2) MPEG-2 decoder 211 decodes one frame. DMA 213 stores this one frame of data in the ordinary scan order in external memory 220.
3) DMA 213 reads the decoded frame data from external memory 220 in MBAFF scan order and sends it to H.264 MBAFF encoder 212.
4) H.264 MBAFF encoder 212 encodes the frame data into a H.264 standard bitstream. DMA 213 writes this data to external memory 220.
The “information” in data transfers 2) and 3) in
MPEG-2 decoder 211 operates in the following sequence. MPEG-2 decoder core 314 reads bitstream data from address indicated by read pointer rd_ptr 313 in bitstream buffer 311. MPEG-2 decoder core 314 decodes the bitstream into macroblocks (MB). MPEG-2 decoder core 314 writes the macroblocks into MB data buffer 313. Read pointer rd_ptr 313 increments by the decoded bitstream size to point to the next incoming bitstream data. Thus MPEG-2 decoder core 314 decodes a macroblock and writes it to MB data buffer 315. Repeating this operation enables one frame decoding.
1) DMA 413 reads H.264 MBAFF bitstream data from external memory 420 and transmits it to H.264 MBAFF decoder 411.
2) H.264 MBAFF decoder 411 decodes one frame. DMA 413 stores this one frame of data in the MBAFF scan order in external memory 420.
3) DMA 413 reads the decoded frame data from external memory 420 in ordinary scan order and sends it to MPEG-2 encoder 412.
4) MPEG-2 encoder 412 encodes the frame data into a MPEG-2 standard bitstream. DMA 413 writes this data to external memory 420.
The “information” in data transfers 2) and 3) in
MPEG-2 decoder 411 operates in the following sequence. MPEG-2 encoder core 512 reads macroblock data from macroblock data buffer 511. MPEG-2 encoder core 512 encodes the macroblocks into bitstream data. MPEG-2 encoder core 512 writes the bitstream data into bitstream buffer 513 at the address stored in write pointer wr_ptr 514. Write pointer wr_ptr 514 increments by the decoded bitstream size to point to the next location for storage of bitstream data. Thus MPEG-2 encoder core 512 encodes a macroblock and writes the bitstream to bitstream buffer 513. DMA 413 transfers this bitstream data from bitstream buffer 513 to external memory 420 from an address specified by read pointer rd_ptr 515. Then read pointer 515 increments by the size of the macroblock. Repeating this operation enables one frame decoding.
The difference of scan order is absorbed by changing the macroblock order when encoder reads a macroblock from external memory in the two transcoding systems of
Consider a typical example. Assume external memory is 300 MHz, 32-bit DDR SDRAM. This memory has a maximum bandwidth of:
This maximum bandwidth is an ideal value. Bandwidth is usually lost in refreshing the SDRAM or in bank conflict. Assume an image format of 1920×1080 pixels at 30 fps. The external memory access bandwidth needed for changing scan order is:
The two frames are required for each frame transcoding because one is needed for read and one for write. Note that for this example:
Thus means the MBAFF macroblock scan re-ordering consumes more than 15% of the maximum SDRAM bandwidth. This may be critical for a high resolution image transcoding system.
Real-time transcoding from MPEG-2 to H.264 or from H.264 to MPEG-2 imposes heavy requirements in external memory access bandwidth. This invention reduces the external memory access bandwidth compared to the conventional method. In this invention no frame delay occurs between decoder and encoder.
This invention proposes a 2-slice MPEG-2 coding method. The MPEG-2 standard defines the slice structures such that the first and the last macroblock of a slice are in the same horizontal row of macroblocks. Two vertically adjacent macroblocks belong to two different slices. Any MPEG-2 slice can be decoded or encoded independently of other slices in the same frame because there is no dependency between slices. Due to these properties, this invention can decode or encode two consecutive macroblock rows alternately to fit the H.264 MBAFF scan order.
These and other aspects of this invention are illustrated in the drawings, in which:
1. DMA 613 reads MPEG-2 bitstream from external memory 620 and transmits it to MPEG-2 decoder 611.
2. MPEG-2 decoder 611 decodes two macroblocks in MBAFF scan order.
3. DMA 613 reads data from the two decoded macroblocks from MPEG-2 decoder 611 and transmits it H.264 MBAFF encoder 612.
4. H.264 MBAFF encoder 612 encodes the two macroblocks of data into a H.264 bitstream. DMA 613 writes this bitstream to external memory 620.
In this method, there is no external memory access between MPEG-2 decoder 611 and H.264 MBAFF encoder 612 to change macroblock scan order from ordinary scan order to MBAFF scan order. As shown in
The bitstream data is of variable length. Bitstream buffer 0711 and bitstream buffer 1718 require pointers to identify the addresses for DMA 613 write and for MEPG-2 decoder core 717 read. Bitstream buffers 711 and 718 thus include respective write pointers wr_ptr0 and wr_ptr1. These write pointers store buffer address where DMA 613 will write the next bitstream data. Bitstream buffers 711 and 718 also include respective read pointers rd_ptr0 and rd_ptr1. These read pointers store the buffer address where MPEG-2 decoder core 717 will next read.
MPEG-2 decoder 611 operates in the following sequence. DMA 613 writes upper slice bitstream data into bitstream buffer 711 at the address of write pointer wr_ptr0712. Write pointer wr_ptr0712 increments by the amount of the bitstream write. DMA 613 writes lower slice bitstream data into bitstream buffer 714 at the address of write pointer wr_ptr1715. Write pointer wr_ptr1714 increments by the amount of the bitstream write. MPEG-2 decoder core 717 initially reads upper slice bitstream data from the address rd_ptr0 of bitstream buffer 0711. MPEG-2 decoder core 717 decodes this into a macroblock and writes the macroblock into MB data buffer 718. Read pointer rd_ptr0 increments by the decoded bitstream size. MPEG-2 decoder core 717 reads lower slice bitstream data from the address rd_ptr1 of bitstream buffer 1711. MPEG-2 decoder core 717 decodes this into a macroblock and writes the macroblock into MB data buffer 718. Read pointer rd_ptr1 increments by the decoded bitstream size.
Thus MPEG-2 decoder core 717 decodes a macroblock pair, one macroblock in upper slice 820 and one macroblock in lower slice 830. MPEG-2 decoder core 717 writes both macroblocks into MB data buffer 718. The macroblock data in MB data buffer 718 is in MBAFF scan order. By repeating this operation, this system enables 2-slice alternate decoding.
1. DMA 913 reads H.264 MBAFF bitstream data from external memory 920 and transmits it to H.264 MBAFF decoder 911.
2. H.264 MBAFF decoder 911 decodes two macroblocks in MBAFF scan order (see
3. DMA 913 reads the decoded pair of macroblocks from H.264 MBAFF decoder 911 and transmits them to MPEG-2 encoder 912.
4. MPEG-2 encoder 912 encodes the data of the two macroblocks into MPEG-2 bitstream data and writes this data to external memory 920 via DMA 913.
This method does not use external memory access between H.264 MBAFF decoder 911 and MPEG-2 encoder 912 for changing the macroblock scan order. As shown in
MPEG-2 encoder 1012 operates in the following sequence. MPEG-2 encoder core 1012 receives bitstream data in MBAFF scan order. MPEG-2 encoder core 1012 encodes macroblock pairs; one from upper slice 820 and one from lower slice 830. The macroblock pairs are written to MB data buffer 1011 from H.262 MBAFF decoder 911 via DMA 913. MPEG-2 encoder core 1012 initially reads an upper macroblock from MB data buffer 1011, encodes it into a bitstream and writes the bitstream to Bitstream buffer 01013 at the location of write pointer wr_ptr01014. Write pointer wr_ptr01014 increments by the encoded bitstream size. MPEG-2 encoder core 1012 next reads lower macroblock data from MB Data Buffer 1011, encodes it into a bitstream and writes the bitstream to bitstream buffer 11016 at the location of write pointer wr_ptr11017. Write pointer wr_ptr11017 increments by the encoded bitstream size.
Thus MPEG-2 encoder core 1012 encodes a macroblock pair, one macroblock from upper slice 820 and one macroblock from lower slice 830. MPEG-2 decoder core 717 writes both macroblocks into MB data buffer 718. The macroblock data in MB data buffer 1011 is in MBAFF scan order. The upper slice bitstream data is stored in bitstream buffer 01013 and encoded lower slice bitstream data is stored in bitstream buffer 11016. This bitstream data is read by DMA 913 as steered by read pointers rd_ptr01014 and rd_ptr11017 and transferred to external memory 920. Both read pointers rd_ptr01015 and rd_ptr11018 are incremented by the sent bitstream size. Encoded upper slice bitstream and lower slice bitstream will be muxed after MPEG-2 encoder operation. Repeating this operation enables 2-slice alternate encoding.
This invention reduces external memory access bandwidth compared to the conventional method. For a video stream of 1920×1080 pixels at 30 fps images, the conventional method the external memory access bandwidth for changing macroblock scan order is 373 MBytes/sec. This invention needs no external memory access bandwidth for changing scan order. Any needed data order re-arrangement takes place at the input or output of the transcoder. This thus reduces the required memory access bandwidth by 373 MBytes/sec compared to the conventional method.
The conventional method produced a frame delay between decoder and encoder. This frame delay is needed for changing the macroblock scan order. However, this invention does not produce such a frame delay.
This invention enables real-time transcoding from MPEG-2 to H.264 MBAFF or from H.264 MBAFF to MPEG-2 with no frame delay and no external memory access for changing macroblock scan order between MBAFF scan order and ordinary scan order.
This application claims priority under 35 U.S.C. 119(e)(1) to U.S. Provisional Application No. 60/943,878 filed Jun. 14, 2007.
| Number | Date | Country | |
|---|---|---|---|
| 60943878 | Jun 2007 | US |