This application is based upon and claims the benefit of priority from prior Japanese Patent Application No. 2005-084775, filed Mar. 23, 2005, the entire contents of which are incorporated herein by reference.
1. Field of the Invention
The present invention relates to a moving picture encoding apparatus for encoding a moving picture, and more particularly, to a moving picture encoding apparatus capable of reducing an amount of processing pertaining to motion estimation in moving picture encoding.
2. Description of the Related Art
In recent years, a technique of encoding and decoding a moving picture has been advanced more remarkably. This is because an amount of information is increased with a higher picture quality of a moving picture and also because there has been a growing demand for developing a wired or wireless network, thereby transmitting picture information through these networks.
In the technique of encoding and decoding a moving picture, there is a demand for high compression efficiency, high picture quality during decoding, and good transmission efficiency. The technique of encoding and decoding a moving picture which meets these demands includes a technique referred to as H. 264/AVC (Advanced video coding) recognized as an international standard (hereinafter, simply referred to “H264”). This technique of encoding and decoding a moving picture is disclosed in, for example, IEEE TRANSACTION ON CIRCUIT AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 13, No. 7, 2003, “Overview of the H. 264/AVC Video Coding Standard”, Thomas Wiegand.
In general, in a moving picture encoding apparatus for encoding a video signal, an inter-prediction is carried out based on a reference frame or reference frames to compress and encode picture data. In the H. 264 standard which is one of moving picture encoding standards, inter-prediction can be carried out from a plurality of reference frames. In such a moving picture encoding apparatus configured in accordance with the H. 264 standard, an enormous amount of computation is required to carry out motion estimation from all the reference frames.
In a conventional moving picture encoding apparatus that is compliant with the H. 264, assuming that a frame in which a 1-schene change has occurred is defined as an instantaneous decoding refresh (IDR), a reference frame is initialized, so that an amount of calculation can be reduced. However, there is a problem that a frame earlier than the IDR picture cannot be referenced in a sequence including such a flash that a dark scene and a bright scene are repeated or a sequence which includes a video picture in which the same scene is repeated. Thus, all the scenes corresponding to the scene changes are defined as the IDR pictures so that an encoding efficiency is degraded.
In addition, in the H. 264 standard, although it is necessary to periodically insert IDR pictures because an intra-macro block (Intra MB) is not inserted unlike MPEG-4, it may not always necessary to set a scene corresponding to the scene change to IDR.
According to the above described situation, in any case, there is a demand for a moving picture encoding apparatus capable of reducing an amount of processing pertaining motion estimation, in particular, motion estimation in a frame produced after a scene change.
An object of the invention is to provide a moving picture encoding apparatus capable of reducing an amount of computation in a moving picture encoding.
According to an aspect of the present invention, there is provided a moving picture encoding apparatus for encoding a moving picture comprising:
a scene change detecting section configured to compare successive frames of the moving picture to detect a scene change, inputted in time series, and assigns a scene identifier for identifying an identical or similar scene to each frame in response to detection of the scene change;
a storage section configured to store the frame which belongs to the each scene as a reference frame specified by the scene identifier;
a setting section configured to set any one of an intra-prediction mode and an inter-prediction mode; and
an encoding section configured to encode the frame inputted between the scene changes in the inter-prediction mode and to search a reference frame by a scene identifier inputted after the scene change, the encoding section encoding the frame inputted after the scene change in the intra-prediction mode if there is no reference picture specified by the scene identifier, and encoding section encoding the frame inputted after the scene change in one of the inter-prediction mode and intra-prediction mode inputted so as to minimize an encoding cost if there is a reference picture specified by the scene identifier.
According to another aspect of the present invention, there is provided a moving picture encoding apparatus for encoding a moving picture comprising:
a scene change detecting section configured to compare successive frames of the moving picture to detect a scene change, which have frame numbers and are inputted in time series, and assign a scene number for identifying an identical or similar scene to each frame in response to detection of the scene change, wherein, the scene change detecting section assigns a scene number identical to the scene if an identical or similar scene exists in a predetermined range before the scene change, and the scene change detecting section assigns a new scene number if no identical or similar scene exists in a predetermined range before the scene change;
a storage section configured to store a frame which belongs to the each scene as a reference frame specified by the scene number and frame number;
a setting section configured to set any one of an intra-prediction mode and an inter-prediction mode; and
an encoding section configured to encode frames to which the identical scene numbers are assigned, the frames being continuously inputted, in the inter-prediction mode, and search the reference picture in a scene number assigned to a frame inputted after the scene change, the encoding section encoding the frame inputted after the scene change in the intra-prediction mode if there is no reference picture specified by the scene number, and the encoding section encoding the frame inputted after the scene change in one of the intra-prediction mode and inter-prediction mode so as to minimize an encoding cost if there is a reference picture specified by the scene number.
Hereinafter, a moving picture encoding apparatus according to an embodiment of the present invention will be described with reference to the accompanying drawings as required.
First, with reference to
In a moving picture decoding apparatus conforming to the H. 264 standard, inter-prediction can be carried out from a plurality of reference frames. Specifically, 16 preceding and succeeding frames can be referenced when a frame is encoded. In general, an enormous amount of computation is required for motion estimation in which a target block to be referenced is determined from among all the reference frames when respective blocks constituting the frame is encoded. As shown in
When a scene change has occurred, and the motion of frames produced after scene change is estimated, a frame of the scene to which the reference frame immediately before the scene change belongs is excluded from reference candidates. Therefore, an amount of computation required for motion estimation can be reduced. That is, a scene number is assigned for each inputted frame, and the same scene number is assigned to the frame of the same scene, and a different scene number is assigned to the frame of a different scene. This scene number is referenced, and motion estimation is carried out in the scene or a search is made for a scene similar to the scene, whereby motion estimation is determined from the frame to which that scene belongs. The term “scene” used here denotes each scene in a moving picture. The term “scene change” used here denotes that a scene is changed. For example, in a sequence including a picture in which a flash or an identical scene is repeated such that bright and dark matters are repeated, the current scene is often returned to its original scene even if a scene change occurs.
As described above, in motion estimation (ME) utilizing a detection of a scene change, an amount of processing can be reduced by narrowing the reference frames. In this manner, in a frame in which an amount of processing can be reduced, high level motion estimation, mode selection or the like can be used.
In the moving picture encoding apparatus shown in
The scene change detecting section 101 also repeats the evaluation and determination and assigns frame number and scene number to following frames FR4, FR5, . . . , and the relation between the scene number and frame number is notified to an encoding control section 106.
As shown in
As described above, in the scene change detecting section 101, a frame FR of frame number n is compared with a preceding frame FR of frame number (n-1) which belongs to a predetermined scene number “m”, and an occurrence of a scene change is detected based on the correlation between the frames FR of frame numbers n and (n-1). If a scene change occurs, frame FR of frame number “n” is compared with frame FR belonging to an another scene of another scene number (m-1, m-2, . . . , m-k (k is integer)) without defining the frame FR of the scene number “m” as a reference candidate. If there is a predetermined correlation, the same scene is determined, and that scene number is assigned. As described later, in the case where the same scene is determined, a frame which belongs to that scene is restored as a reference frame candidate, and motion estimation is carried out.
From the scene change detector 101, frames FR0 to FR5 are supplied one after another to a frequency transforming and quantizing section 104 via a subtracting section 102. The frequency transforming and quantizing section 104 carries out a frequency transforming process and a quantizing process. That is, in the frames FR inputted to the frequency transforming and quantizing section 104, respective macro-blocks (MC) constituting each of the frames, which is a minimum unit of the frame, is quadrature-transformed to computed frequency transforming coefficient and the compute frequency transforming coefficient is quantized, under the control of the encoding control section 106. Then, the quantized frequency transforming coefficient is supplied to an entropy encoding section (variable length encoding section) 105. In the entropy encoding section (variable length encoding section) 105, the quantized frequency transforming coefficient is encoded on a variable length basis under the control of the encoding control section 106, and the encoded information is outputted as an encoded bit stream.
The quantized frequency transforming coefficient outputted from the frequency transforming and quantizing section 104 is also inputted to a reverse quantizing and frequency transforming section 107, the inputted frequency transforming coefficient is reverse quantized and reverse frequency transformed, and the resulting coefficient is supplied to an adder 108 in a macro-block unit. In the adder 108, a local decode picture signal supplied in a unit of macro-blocks is added to a prediction picture signal from a switch 103, and a local decoded signal is outputted to a de-blocking filter 109.
In the de-blocking filter 109, a distortion between blocks which occurs in the local decoded signal is filtered, and this local decoded signal is stored as a reference frame in a frame memory 111 in a unit of frames.
A frame FR outputted from the scene change detecting section 102 is supplied to a motion vector detecting section 112, and a motion vector is detected. This detected motion vector is supplied to a motion compensating section 110 and the entropy encoding section (variable length encoding section) 105. The motion compensating section 110 generates a prediction frame on the basis of a motion vector with reference to the reference frame stored in the frame memory 111, and the generated prediction frame is supplied to the subtracting section 102 via the switch 103.
In the subtracting section 102, every blocks of the newly inputted frame FR are compared with the blocks of the prediction frame from the motion compensating section 110, and differential picture data between the blocks of the frame FR and the prediction frame is calculated and is supplied to the frequency transforming and quantizing section 104. In the frequency transforming and quantizing section 104, quantizing frequency transforming coefficient is obtained from the differential picture data as described above.
The frequency transforming coefficient is outputted to the entropy encoding section (variable length encoding section) 105 from the frequency. transforming and quantizing section 104 and is encoded as a payload in the entropy encoding section (variable length encoding section) 105. Then, the motion vector assigned to the entropy encoding section (variable length encoding section) 105 is also encoded, and these encoded frequency transforming coefficient and motion vector are outputted as additional information on a payload to be referenced on the decoder side.
After a scene change is detected in the scene detecting section 101 and the encoding control section 106 determines that there is no highly correlative frame FR with respect to a new scene, a new scene number is assigned to an input new frame FR pertaining to the new scene, and the switch 103 is connected to the intra-prediction section 113 to execute an intra-prediction process on the new frame FR.
In the intra-prediction process, every macro blocks in the new frame FR are compared with an intra-frame prediction macro block predicted in the picture, and difference data between the macro blocks is calculated and is inputted to the frequency transforming and quantizing section 104. In the frequency transforming and quantizing section 104, the difference data for each macro blocks is quadrature transformed (frequency transforming-processed) into a frequency transforming coefficient. In the frequency transforming and quantizing section 104, the frequency transforming coefficient is quantized, and the quantized frequency transforming coefficient is outputted from the frequency transforming and quantizing section 104. The quantized frequency transforming coefficient is supplied to the reverse quantizing and frequency transforming section 107. The quantized frequency transforming coefficient is reverse quantized and reverse frequency transformed in the reverse quantizing and frequency transforming section 107, and the thus quantized and transformed coefficient is supplied to the adder 108 in a macro block unit.
In the adder 108, the supplied coefficient is added to a prediction picture signal for respective blocks from the switch 103, the added signal is supplied as a local decoded signal to an intra-predicting section 113 via the adder 108, and a prediction picture in a block unit is generated. The generated block picture is compared with a next block picture at the subtracting section 102, the difference is supplied to the frequency transforming and quantizing section 104, and the difference data is frequency transformed and quantized. A correlation with the macro block which exists at the periphery of a macro block which is a repetition of this process is obtained, and intra-frame or intra-slice encoding is carried out.
The quantized frequency transforming coefficient is outputted to the entropy encoding section 105 from the frequency transforming and quantizing section 104, and is so encoded as to have a variable encoded length in the entropy encoding section 105, and the thus encoded frequency transforming coefficient is outputted as a payload of a bit stream.
Information on the intra-prediction process is supplied as additional information to the entropy encoding section 105 from the encoding control section 106, and the additional information is so encoded as to have a variable length. Then, the thus encoded information is outputted together with the payload.
The scene change detecting section 101 shown in
After frame FR2 that follows the frame FR1 is supplied to the buffer section 201, the sum of absolute difference (SAD) is greater than reference value Ref1 as a result of macro block comparison. In this case, it is determined that a scene change is occurred between the frame FR1 and the frame FR2, and a scene change signal is assigned to the scene comparing section 203 as a comparison signal. Therefore, in the scene comparing section 203, the frame FR2 is compared one after another with the frames in the frame memory 204 in which typical frames of other scenes are stored. In this comparison, after the sum of absolute difference (SAD) from the frame compared with the frame FR2 has been obtained, if the thus obtained sum is smaller than reference value Rf2, it is determined that both of the frames belong to a similar scene. If there is no frame similar to the scene of the frame FR2 at the scene comparing section 203, a new scene number 1 is assigned to the frame FR2. These scene number 1 and frame number 2 are supplied to the encoding control section 106, and the frame FR2 of such a new scene number 1 is stored in the frame memory 204. Then, the frame FR2 is supplied from the scene comparing section 203 to the subtracting section 102.
The frame FR3, FR4 is processed as belonging to the same scene as in the frame FR2. The frame number FR3, FR4 and the scene number 1 are supplied from the scene comparing section 203 to the encoding control section 106, and the frames FR3 and FR4 are supplied to the subtracting section 102 one after another. When frame FR5 is supplied to the buffer section 201, the sum of absolute difference (SAD) becomes greater than the reference value Ref1, and the scene change signal is supplied to the scene comparing section 203. In the scene comparing section 203, in the case where it is determined that the frame FR5 is similar to the frame FR1 stored in the frame memory 204, the frame FR5 is supplied to the subtracting section 102, and concurrently, the similar scene number 0 and frame number 5 are supplied to the encoding control section 106.
With reference to a flow chart shown in
When frames FR1 to FR5 shown in
In step S16, the encoding control section 106 sets a new scene number for the frame to be encoded. In addition, as shown in step S18, a prediction mode in the intra-prediction is determined for respective blocks divided from the frame so as to minimize cost for each blocks with referring to a relation between the prediction errors in the intra-prediction section 113 and generated encoding amount in the entropy encoding section 105.
The term “cost” used here is defined by a cost function. In general, the following formula is used by a parameter for determining an encoding mode:
Cost function=D+λ′·R
wherein D denotes a distortion; a sum of absolute difference (SAD) or a sum of square difference (SSD) is used. SSD is obtained by squaring a respective one of prediction errors and computing a sum thereof. x denotes a constant (Lagrangian multiple), and R denotes a rate defined by a generated encoding amount of bits for encoding a target block with utilizing a candidate mode, which corresponds to generated bits of the intra-prediction mode encoding in the intra-prediction and generated encoding amount of encoding the motion vector and the reference frame in the inter-prediction. The above cost is computed with respect to each prediction modes, and a combination of the prediction modes which minimizes a cost is handled as an optimal parameter value.
Next, as shown in step S20, the inter-prediction mode is not selected because the encoding control section 106 determines that there is no reference frame in the step S14, and the intra-prediction mode determined in the step S18 is selected as the prediction mode for minimizing the cost.
In step S22, a block is encoded in the prediction mode selected in step S20. That is, frequency trans-forming and quantizing are executed, and a frequency transforming coefficient of the macro block is obtained in the frequency transforming and quantizing section 104. Thereafter, in step S24, it is verified whether or not encoding process has terminated with respect to all the blocks of a frame FR to be encoded in the encoding control section 106. If the encoding process does not terminate, the current step is returned to step S18. When the encoding process has terminated with respect to all the blocks of the frame FR, the current step is advanced to step S26. Then, the current step is returned to step S12 for processing of a next frame FR.
In step S14, if no scene change occurs and the frame FR has a same scene as that of a previous frame, the encoding control section 106 determines that a reference scene exists, and the current step is moved to step S28. Alternatively, if a scene change occurs, but there is a reference frame in a frame memory 204 of the scene change detecting section 101, which belongs to a same scene as that of the frame FR inputted after the scene change, the encoding control section 106 also determines that a reference scene exists, and the current step is moved to step S28. As shown in step S28, a scene number to be referenced to the frame FR is set. Next, in step S30, it is verified whether or not a scene number of a frame FR stored in the frame memory 11 and having a frame number “n” assigned thereto, is identical to that set in the frame FR. If they are not identical in scene, the current frame is changed to a new frame, as shown in step S34. If this selected frame FR is stored in the frame memory 111, and does not exceed N, the current step is returned to step S30 again in which it is verified whether or not the scene of the selected frame FR is identical to a frame of a current frame FR.
If the scene of the selected frame FR is identical to the frame of the current frame FR in the encoding control section 106 in step S30, a motion vector (MV) for minimizing a cost is determined in the motion vector detecting section 106 in step S32. This cost is calculated from the prediction error in the motion compensating section 110 and a generated encoding amount of the reference frame and motion vector (MV) in the entropy encoding section. The frame number of this reference frame and the determined motion vector (MV) are temporarily stored for later comparison in the encoding control section 106. In step S34, the current frame is changed to a new frame. If this selected frame FR is stored in the frame memory 111, and does not exceed N, the current step is returned to step S30 again in which it is verified whether or not the scene of the selected frame FR is identical to the frame of the current frame FR. If a frame FR of the same scene exists, that frame is defined as a reference frame, and then, a motion vector (MV) for minimizing a cost is decided in the motion vector detecting section 112. In the case where there exist a plurality of frames whose scene numbers are identical to each other, the frame numbers of a plurality of reference frames and a plurality of decided motion vectors (MV) are temporarily stored together with the cost in step S32.
In step S36, when a search of the same scene has been terminated with respect to N reference frames, the plurality of temporarily stored costs are compared with each other. Then, as shown in step S38, a reference frame whose cost is minimum and a motion vector (MV) are decided. Therefore, the reference frame and motion vector (MV) for inter-prediction are decided.
Next, in step S40, intra-prediction for minimizing a cost is decided for the sake of comparison. In step S42, the inter-prediction cost decided in step S40 is compared with the intra-prediction cost decided in step S42, and a prediction mode for minimizing the cost is selected. In the case where intra-prediction has been decided, the switch 103 shown in
In step S44, the block is encoded in the prediction mode selected in step S42. That is, frequency transforming transform and quantizing are executed in the frequency transforming and quantizing section 104, and a frequency transforming coefficient of the macro-block is obtained. Thereafter, in step S46, it is verified an encoding process is terminated with respect to all the macro-blocks of frames FR to be encoded. If the verification result is negative, the current step is returned to step S30. If the encoding process is terminated with respect to all the macro-blocks of the frames FR, the current step is advanced to step S26. Then, the current step is returned to step S12 for the sake of processing of a next frame FR.
As has been described above, in motion estimation (ME) after scene change detection, an amount of processing can be reduced by narrowing reference frames. In this manner, in a frame in which the amount of processing is reduced, high level motion estimation or mode selection can be used.
In a moving picture encoding apparatus (encoder) for carrying out motion estimation with respect to N reference frames having a search range R×R, the maximum number of reference pixels is obtained as R×R×N, and the search range in encoding of one frame is obtained as “r=sqrt” (R×R×N/n), wherein “n” is the number of references (n<=N) to be actually subjected to motion estimation.
In a two-pass encoding process, in the case where a periodicity between frames exists, a frame before scene change is stored as Non-IDR (Non Instantaneous Decoding Refresh) picture in a frame memory. If no periodicity exists, it is preferable that an IDR frame is stored. In accordance with this process, the efficiency of encoding can be improved.
As has been described above, according to the present invention, there is provided a moving picture encoding apparatus capable of reducing an amount of processing relating to motion estimation in a frame after scene change.
Number | Date | Country | Kind |
---|---|---|---|
2005-084775 | Mar 2005 | JP | national |