1. Field of the Invention
The present invention relates to an image encoding technique, and particularly relates to an image encoding technique with respect to an intra-frame predictive encoding process.
2. Description of the Related Art
MPEG-4 and H.264 are known as image encoding systems adopting an encoding method that performs intra-frame prediction. Intra-frame prediction in H.264 is an evolution of intra-frame prediction in MPEG-4 and can enhance encoding efficiency. The main differences in intra-frame predictive encoding between MPEG-4 and H.264 are that the number of data items to be predicted is increased, the number of blocks to be referred to is increased, the direction of prediction is encoded, the number of types of blocks to be predicted is increased, and so on.
Hereinafter, intra-frame predictive encoding for blocks of 4×4 pixels each in H.264 will be described using
An example of processing steps for performing 4×4 block intra-frame encoding will be described using
A case where the speed of the encoding process is increased by using a pipeline operation will be described using
As described above, when 4×4 blocks are encoded in the encoding order compliant with the recommendation, efficient parallel processing of the blocks to be encoded cannot be performed. For this reason, when high-speed processing is required, it is necessary to, for example, increase the operation frequency itself.
To address this issue, Japanese Patent Laid-Open No. 2004-140473 discloses a technique that enables parallel processing by changing the encoding order of 4×4 blocks and thus realizes a high-speed encoding process. However, since the encoding order is changed, the technique deviates from the recommendations of H.264. Furthermore, according to the above-described proposal, in order to speed up the encoding process, encoding of blocks to be encoded is performed by limiting the number of reference pixels to be referred to or using a predetermined value as a reference pixel when the reference pixel is not yet encoded. In this manner, encoding is performed using an original method, and so decoding cannot be performed with a decoder compliant with the H.264 recommendation, and a dedicated decoder is required.
As described above, there is a problem with 4×4 intra prediction in that image encoding compliant with the H.264 recommendation cannot be processed at high speed.
According to one aspect of the present invention, an image encoding apparatus that performs intra-frame predictive encoding is provided. The apparatus includes a partitioning unit configured to partition an inputted macroblock into blocks as processing units, an encoding unit configured to encode each of blocks to be processed using a prediction value for each pixel contained in the block to be processed, the prediction value being calculated by referring to pixels contained in other blocks, and a sorting unit configured to sort the encoded blocks in a predetermined encoding order. The encoding unit starts encoding in an order in which the first block for which all the pixels to be referred to are available for calculation of the prediction value is the first to be encoded, and the encoding is performed by pipeline processing.
Further features of the present invention will become apparent from the following description of exemplary embodiments (with reference to the attached drawings).
The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention, and together with the description, serve to explain the principles of the invention.
Hereinafter, embodiments of the present invention will be described with reference to the attached drawings.
An image data control unit 102 partitions each macroblock into blocks of 4×4 pixels each. A “block” as simply expressed hereinbelow means a block of 4×4 pixels. The image data control unit 102 then reads out a block to be processed from the image buffer 101. A prediction direction computing unit 103 calculates a difference value between a reference pixel and a pixel of the block to be encoded for each of a plurality of prediction directions. A prediction mode determination unit 104 selects the difference value for the optimum prediction direction from the results of computation by the prediction direction computing unit 103. An integer transformation unit 105 performs an integer transformation on the difference value selected by the prediction mode determination unit 104. A quantization unit 106 quantizes the resultant value of the integer transformation performed by the integer transformation unit 105. An entropy encoding unit 107 performs variable-length encoding on the value quantized by the quantization unit 106. An encoded data buffer 108 accumulates at least two blocks of data encoded by the entropy encoding unit 107.
An inverse quantization unit 109 performs inverse quantization on the value quantized by the quantization unit 106. An inverse integer transformation unit 110 performs an inverse integer transformation on the inverse-quantized value. A reference pixel buffer 111 accumulates pixels to be used for prediction. A reference pixel control unit 112 instructs the reference pixel buffer 111 what reference pixel is to be outputted according to control by the image data control unit 102. An output data control unit 113 reads out and outputs the encoded data from the encoded data buffer 108 in the order compliant with the recommendation, according to control by the image data control unit 102.
A CPU 201, which is a microprocessor, controls the image encoding apparatus 100 based on programs, data, or the like stored in a ROM 203, in a hard disk (HD) 212, or on a storage medium set in an external memory drive 211.
A RAM 202 functions as a work area for the CPU 201 and holds a program stored in the ROM 203, the HD 212, or the like. Moreover, the RAM 202 functions also as the above-described image buffer 101, encoded data buffer 108, or reference pixel buffer 111.
The ROM 203, the storage medium set in the external memory drive 211, or the HD 212 stores a program or the like, such as that shown by a later-described flowchart, the program or the like being executed by the CPU 201.
A keyboard controller (KBC) 205 controls input from a keyboard (KB) 209 or a pointing device, such as a mouse, which is not shown. A display controller (DPC) 206 controls display of a monitor 210. A disk controller (DKC) 207 controls access to the HD 212 and the external memory drive 211 and reads and writes various types of programs and various types of data, such as font data, a user file, and an edit file, from and to those storage media. A printer controller (PRTC) 208 is connected to a printer 222 via a predetermined bidirectional interface 221 and controls communication with the printer 222.
It should be noted that the CPU 201 executes a process of expanding (rasterizing) an outline font into, for example, a display information area allocated on the RAM 202 or a dedicated video memory (VRAM) to enable the outline font to be displayed on the monitor 210. Moreover, the CPU 201 opens various types of registered windows and executes various types of data processing based on commands given via a mouse cursor or the like on the monitor 210.
An encoding process with the image encoding apparatus 100 will be described using
In step S301, the image data control unit 102 partitions a macroblock into blocks of 4×4 pixels each, which are the minimum units for intra-frame predictive encoding.
The blocks will be described using
In step S302, the image data control unit 102 reads out the blocks 401 from the image buffer 101 to the prediction direction computing unit 103 one by one according to an order and timings that will be described later.
In step S303, the prediction direction computing unit 103 generates, from a block input by the image buffer 101 and reference pixels according to each prediction mode output from the reference pixel buffer 111, a prediction value for each pixel and calculates the difference. The prediction direction computing unit 103 outputs all the difference pixel values obtained by computation in the nine prediction modes to the prediction mode determination unit 104. The reference pixel control unit 112 performs control according to the location of a block designated by the image data control unit 102 so that reference pixel values outputted by the reference pixel buffer 111 are appropriate for the current block to be encoded.
In step S304, the prediction mode determination unit 104 selects the optimum mode from the nine prediction modes output from the prediction direction computing unit 103 and outputs only the difference pixel values of the selected prediction mode to the integer transformation unit 105.
In step S305, the integer transformation unit 105 performs an integer transformation on the difference pixel values and outputs the resultant data to the quantization unit 106.
In step S306, the quantization unit 106 quantizes the integer-transformed data and outputs the resultant data to the entropy encoding unit 107 and the inverse quantization unit 109. The data output to the inverse quantization unit 109 is used to obtain reference pixels to be referred to in processing of blocks after the current block.
In step S307, the inverse quantization unit 109 performs inverse quantization on the data quantized by the quantization unit 106 and outputs the resultant data to the inverse integer transformation unit 110.
In step S308, the inverse integer transformation unit 110 writes only pixels to be used for prediction out of the data obtained by an inverse integer transformation to the reference pixel buffer 111.
In step S309, the entropy encoding unit 107 performs variable-length encoding on the data output from the quantization unit 106 and writes the resultant data to the encoded data buffer 108.
In step S310, the output data control unit 113 reads out the entropy encoded data from the encoded data buffer 108 after sorting the data in the order of blocks compliant with the recommendation and then outputs the data for subsequent processing. As described above, the description of the subsequent processing, for example, inter-frame predictive encoding, will be omitted.
Next, pipeline processing of the blocks will be described using
To comply with the recommendation, during processing of a target block, encoded data for four blocks, that is, blocks to the upper left, above, to the upper right, and to the left of the target block, is required. For this reason, in an initial stage of processing of a macroblock 500, only B0 can be processed. After the completion of processing of B0 up to and including inverse integer transformation, which is step S308 shown in
When processing of B1 up to and including inverse integer transformation is completed, both B2 and B4 can be processed. In other words, the encoding process of B4 can be started at this stage. Thus, in the present embodiment, B4, which would be processed after B3 according to the recommendation, is processed in parallel with B2.
Next, the details of pipeline processing will be described using
At a stage corresponding to the period 600 shown in
When inverse integer transformation 706 for B2 is completed, 4×4 prediction 701 for B3 can be started as can be seen from the relationship shown by the arrow 710. Accordingly, the image data control unit 102 reads out B3 from the image buffer 101 to the prediction direction computing unit 103. Similarly, when inverse integer transformation 706 for B4 is completed, the prediction direction computing unit 103 can start 4×4 prediction 701 for B5 as can be seen from the relationship shown by the arrow 720. As in the foregoing description, the image data control unit 102 reads out the blocks to the prediction direction computing unit 103 in an order in which the first block for which all the prerequisite reference pixels are available is the first to be read out, without being constrained by the encoding order specified by the recommendation. Moreover, when there is a plurality of blocks for which all the prerequisite reference pixels are available, any one of the blocks may be read out first; however, for example, the blocks are read out according to the encoding order specified by the recommendation.
Now, referring again to
After encoding by the entropy encoding unit 107, the output data control unit 113 reads out and outputs the blocks in the order compliant with the encoding order specified by the recommendation. Since the encoding processes of B0, B1, and B2 are completed in that order, these blocks are output without changing the order. The block whose encoding is completed after B2 is B4, but B4 is held in the encoded data buffer 108. After outputting B3, whose encoding is completed after B4, the output data control unit 113 reads out and outputs B4 from the encoded data buffer 108. Blocks after B4 are output in the same manner. Although processing of data read out from the encoded data buffer 108 is not defined in the present embodiment, any processing is possible as long as it is in compliance with the recommendation because the order of the read-out data is in compliance with the encoding order specified by the recommendation.
From the foregoing, according to the present embodiment, H.264 recommendation-compliant and high-speed image encoding can be performed in 4×4 intra prediction.
In the first embodiment, the efficiency of pipeline processing was improved by performing encoding processes in the order in which the first block for which all of the prerequisite reference pixels are available is the first to be processed. However, when entropy encoding and the like 807 takes a long processing time, in some cases, a sufficient effect cannot be obtained with the configuration of the first embodiment. An example of such a case will be described using
Thus, the speeding up of encoding is realized by providing two entropy encoding units as shown in
The output data control unit 113 switches the input switch 900 so that output from the quantization unit 106 is outputted to the entropy encoding unit that is not performing processing. As previously described using
As described above, even when entropy encoding processing and the like takes a longer period of time than the other processing steps, H.264 recommendation-compliant and high-speed image encoding can be performed in 4×4 intra prediction.
The above-described exemplary embodiments of the present invention can also be achieved by providing a computer-readable storage medium that stores program code of software (computer program) which realizes the operations of the above-described exemplary embodiments, to a system or an apparatus. Further, the above-described exemplary embodiments can be achieved by program code (computer program) stored in a storage medium read and executed by a computer (CPU or micro-processing unit (MPU)) of a system or an apparatus.
The computer program realizes each step included in the flowcharts of the above-mentioned exemplary embodiments. Namely, the computer program is a program that corresponds to each processing unit of each step included in the flowcharts for causing a computer to function. In this case, the computer program itself read from a computer-readable storage medium realizes the operations of the above-described exemplary embodiments, and the storage medium storing the computer program constitutes the present invention.
Further, the storage medium which provides the computer program can be, for example, a floppy disk, a hard disk, a magnetic storage medium such as a magnetic tape, an optical/magneto-optical storage medium such as a magneto-optical disk (MO), a compact disc (CD), a digital versatile disc (DVD), a CD read-only memory (CD-ROM), a CD recordable (CD-R), a nonvolatile semiconductor memory, a ROM and so on.
Further, an OS or the like working on a computer can also perform a part or the whole of processes according to instructions of the computer program and realize functions of the above-described exemplary embodiments.
In the above-described exemplary embodiments, the CPU jointly executes each step in the flowchart with a memory, hard disk, a display device and so on. However, the present invention is not limited to the above configuration, and a dedicated electronic circuit can perform a part or the whole of processes in each step described in each flowchart in place of the CPU.
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2008-153394, filed Jun. 11, 2008, which is hereby incorporated by reference herein in its entirety.
Number | Date | Country | Kind |
---|---|---|---|
2008-153394 | Jun 2008 | JP | national |