None.
The field of the disclosure is that of data compression (audio and/or video).
More precisely, the disclosure relates to a real time encoding technique, with bit rate and time constraints. In one example of the disclosure, a real time encoder/decoder (codec) is proposed that has bit rate and time constraint.
In the present description, a frame is defined as set of successive items of data and a scene as a set of successive frames. In the specific case of video, a frame is an image.
The quantity of information (entropy) contained in a signal can vary hugely in time. For example, in the case of a video signal, it is possible to switch from a static scene containing smooth textures to a scene featuring many moving objects and complex textures. In this case, a significant increase of the complexity of the scene may be observed, and therefore of the quantity of information.
When compression techniques are used, this natural variability has two consequences:
Solutions have been developed to overcome these two problems.
Bit rate control techniques permit the output bit rate of the encoder to be regulated. It is possible to ensure either a constant bit rate or a variable bit rate. In both cases, an external constraint is respected, such as for example the physical capacity of a communication channel. If this external constraint varies in time, then this is the case of the variable bit rate (VBR). The control of the bit rate is based on the use of modelling of the emission buffer memory, which is also called “emission buffer” or even EBV (Emission Buffer Verifier). The emission buffer memory is also called VBV (for Video Buffer Verifier) in the case of the compression of a video signal or “reservoir-bit” in the case of the compression of an audio signal.
Further details on bit rate control may be found in: “Efficient algorithms for MPEG video compression”, Dzung Tien Hoang, Jeffrey Scott Vitter, WILEY, 2002.
To ensure timely constant encoding per frame, the most simple solution consists of over sizing the equipment (processor, FPGA, etc.). However this solution is costly and not optimal, as the equipment is only rarely used to its full capacity. The opposite logic consists of limiting the complexity of the encoder.
In this case, the efficiency of the compression is limited and consequently the quality of the data compressed. Similarly, the equipment is only rarely used to its full capacity. Furthermore, the quality is not optimal in scenes of low complexity.
Ideally, it would be possible to limit the complexity of the encoder simply when the scene is too complex so that it could be processed in real time with the equipment available. Such a dynamic complexity control technique is presented in the patent document WO 03/061298 A1, dated 24 Jul. 2003.
Before resuming this dynamic complexity control technique, one example of a classic real time encoder will be described, based on the
The example of the real time encoder of
The various elements 1 to 5 in the encoder are classically made using computer technology, with one or more elements of equipment (especially memory and processor components) and/or software (programs).
The data transfers, references A to E in
Transfer A (synchronised): at each start, a frame leaves the capture module 1 and is stored in the capture buffer memory 2.
Transfer B (unsynchronised): at the instant t, the encoding module 3 takes in input Ve(t) frames per clock period.
Transfer C (non-synchronised): at the instant t, the encoding module 3 also sends to the output Ve(t) compressed frames per clock period.
Transfer D (synchronised): at each start, the EBV 5 takes in input Vo(t) data packets (which generally correspond to one compressed frame).
Transfer E (synchronised): the VBV sends to the output a data packet at each start. It thus guarantees a controlled bit rate on the communication channel. In the case of a constant bit rate (CBR), the size of the data packets is constant and thus the output bit rate is constant.
Classically, the process formed by the set of transfers A to E is controlled by a program run by one or several processors in the encoder.
The function of the two capture buffer 2 and intermediate 4 memories is to compensate the variation in processing time per frame of the encoder (Ve(t)). These two buffer memories are synchronised. When one is emptied, the other is filled. Wherein Mc is the maximum size of each of these two buffer memories. Where: No(t)=Mc−Nc(t).
If the processing time by the encoding module remains greater than P (which is to say if Ve(t)<1) for too long a period, the capture buffer memory is filled. The only way to respect the real time constraint is then to eliminate frames. In the case of video decoding, when the compressed video is decoded and displayed, this is translated by visually unacceptable jerky images.
In order to avoid the suppression of frames, the technique presented in the patent document WO 03/061298 A1 consists of using a complexity control mechanism. This mechanism will now be described in relation to
Unfortunately, the solution presented above is efficient but not optimal for two reasons:
By increasing the size of the capture buffer memory 2, it is possible to process longer complexity peaks, or to maintain the quality of the compressed data (for example video) until the complexity peak has passed. However this introduces an extra time to the overall output of the encoder. In fact, the encoder as it is presented introduces a time whose size is at least Mc. Which is to say that if a frame enters the encoder at the instant t, it will leave at the earliest at the instant t+P·Mc. Similarly, each element of the encoder may introduce a time. If De is the time introduced by the encoding module 3 of the encoder and DEBV is the time introduced by the emission buffer memory 5 (EBV), then the time total of the encoder is: D=Mc+De+Debv.
In one specific example, a real time encoder with bit rate constraint and time constraint is proposed, of the type comprising:
said encoder comprising a control module comprising dynamic adjustment means for the maximum size of the capture buffer memory, permitting the following rules to be applied:
The general principle of the disclosure includes therefore of temporarily increasing the size of the capture buffer memory without increasing the overall time of the encoder. The idea consists of using the filling level of the emission buffer memory (EBV), which is to say the number of packets contained in the emission buffer memory. In fact, for a given filling level, the emptying time of the EBV is a priori known, thanks to the bit rate constraint. The capacity of the capture buffer memory may temporarily be increased, as long as the EBV is not empty. The delay thus incurred must be compensated before the EBV is emptied. Consequently, it is possible to support longer complexity peaks without reducing the quality of the compressed frames, or increasing the overall time of the encoder.
Advantageously, the maximum temporary size Mc′ of the capture buffer memory is defined by the following equation: Mc≦Mc′≦Mc+Dv(t), where Mc is the maximum initial size of the capture buffer memory, and Dv(t) is the filling level of the emission buffer memory expressed in number of packets contained in the emission buffer memory.
Advantageously, the control module comprises means of managing the speed of emptying the intermediate buffer memory to suit the filling level of the capture buffer memory expressed in number of frames contained in the capture buffer memory.
In one advantageous characteristic, said means de managing the speed of emptying the intermediate buffer memory apply the following rules:
In one advantageous example, the control module comprises means of dynamically adjusting the complexity of the encoding module by dynamic selection of a level of complexity from at least two distinct levels of complexity, according to the filling level Nc(t) of the capture buffer memory expressed in number of frames contained in the capture buffer memory and the filling level Dv(t) of the emission buffer memory expressed in number of packets contained in the emission buffer memory.
Advantageously, said means of dynamically adjusting the complexity of the encoding module apply the following rules:
In one advantageous variant, said means of dynamically adjusting the complexity of the encoding module apply the following rules:
Advantageously, k is between 0.7 and 0.9, r is between 0.2 and 0.4, and K=Mc/2.
In another example, the disclosure relates to a method for a real time encoder with bit rate constraint and time constraint, of the type comprising:
said method comprising a dynamic adjustment step of the maximum size of the capture buffer memory, permitting the following rules to be applied:
More generally, different examples of this control method may be envisaged, of which the characteristics result from the transposition in the form of steps of the means included in the control module mentioned above.
In another example, the disclosure relates to a computer program product that may be downloaded from a communication network and/or saved to a support that may be read by a computer and/or run by a processor, wherein said computer program product comprises program code instructions to run the steps of the control process mentioned above, when said program is run on a computer.
In another example, the disclosure relates to storage means, that may be totally or partially removable, that may be read by a computer, storing a set of instructions that may be run by said computer to use the control process mentioned above.
Other characteristics and advantages of will become clearer after reading the following description, provided purely by way of example and in no way restrictively (none of the embodiments of the disclosure are limited to the characteristics and advantages of the embodiments described hereunder), and appended drawings.
In all of the figures of the present document, the identical elements and steps are designated with a same numerical reference.
In the following description, the specific case of the compression of a video signal will be considered, wherein a frame is an image.
Now will be presented, in relation to
As the classic real time encoder of
It further comprises a control module 10 (also called RTCC, Real-Time Capture Controller) which, in one specific example, carries out the following actions:
This monitoring is shown by the arrow reference 11;
The control module 10 (RTCC) is for example made by a program run by a processor.
Now will be presented, in relation to
This second example is different from the first (described above in relation to
This second example is used with an encoder H.264/MPEG4-10 AVC, for example. Such an encoder divides the images into macro-blocks. Each macro-block is encoded according to a mode selected in a predefined list. In particular, the 16×16, 16×8, 8×16 and 8×8 modes are defined for the “P” macro-blocks. The selection of the best encoding mode for a macro-block is a costly process. In order to reduce the complexity of the encoder, the list of modes tested for each macro-block is reduced. In this way, two levels of complexity are defined for the encoder (obviously however in other examples, more than two levels of complexity may be used). A standard level where all modes are authorised for the P blocks, and a reduced level of complexity where only the 16×16 mode is authorised. For more information on the operation of the H.264/MPEG4-10 AVC standard, refer to “H.264 and MPEG-4 Video Compression”, lain E G Richardson, John Wiley & Sons, September 2003, ISBN 0-470-84837-5.
Different configurations are possible. For example, the thresholds may be adjusted either according to Mc, or according to Mc′.
In a first configuration, the reduced level of complexity is chosen when the filling level Nc(t) of the capture buffer memory 2 is greater than 0.8*Mc″, where Mc″=MAX (Mc′)=Mc+Dv(t).
In a second, more interesting configuration, there is:
AND Dv(t)<0.3*Mv, where Mv is the maximum size of the emission buffer memory 5′ (VBV), which is to say the maximum number of packets that the VBV can contain.
This second configuration permits better use of the knowledge of the filling level of the emission buffer memory (VBV). The value of Mc′ is limited in order to avoid reaching excessive filling levels of the capture buffer memory 2. The purpose is to ensure the success of passing Nc(t) below Mc.
The control module 20 (RTCCC) is for example made by a program run by a processor.
It may be noted that the disclosure is not limited to a purely software installation (sequence of instructions for a computer program), but that it may also be used in the form of equipment or any mixed form using one part of equipment and one part software. In the case of the disclosure being partially or totally installed in software form, the corresponding sequence of instructions may be stored on removable storage means (such as for example a floppy disk, a CD-ROM or a DVD-ROM) or non removable storage means, wherein this storage means may be read partially or totally by a computer or a microprocessor.
The present disclosure provides a real time encoding technique with bit rate constraints, that also has a time constraint.
The disclosure provides such a technique permitting the tolerance of an encoder to the variation in complexity of a scene to be increased without necessarily requiring dynamic control of the complexity of said encoder.
The disclosure provides such a technique permitting, with respect to the prior art, to reduce the number of frames eliminated due to the capture buffer memory being exceeded without increasing the overall time induced by the encoder, nor reducing the quality of the compressed frames.
The disclosure provides such a technique that may be combined with the known principle of dynamic control of the complexity of the encoding module 3.
The disclosure provides such a technique that is simple and cheap to use.
Although the present disclosure has been described with reference to one or more illustrative examples, workers skilled in the art will recognize that changes may be made in form and detail without departing from the spirit and scope of the disclosure and/or the appended claims.
Number | Date | Country | Kind |
---|---|---|---|
06 09473 | Oct 2006 | FR | national |
Number | Name | Date | Kind |
---|---|---|---|
20030123540 | Zhong et al. | Jul 2003 | A1 |
20030185222 | Goldstein | Oct 2003 | A1 |
20040047367 | Mammen | Mar 2004 | A1 |
20070271358 | Gaddy | Nov 2007 | A1 |
Number | Date | Country |
---|---|---|
0 562 221 | Sep 1993 | EP |
1 349 334 | Oct 2003 | EP |
WO 02063461 | Aug 2002 | WO |
WO 03061298 | Jul 2003 | WO |
Number | Date | Country | |
---|---|---|---|
20080101461 A1 | May 2008 | US |