Packetization of variable bit rate encoded data based on rate control

Abstract

A coding apparatus 100 that generates coded data by coding inputted data in a predetermined unit basis, maps one or more units of the coded data onto a packet of fixed length and output the packet including: a storage unit 103 that stores the coded data and outputs the data size of the stored coded data; a packetization unit 104 that maps the coded data stored in the storage unit 103 onto the packet and outputs the header size assigned to the one or more units of the coded data and the payload size of the packet; a control unit 107 that generates the control information based on the data size of the outputted coded data, the header size and the payload size and outputs the control information; and a coding unit 102 that generates the coded data by calculating an outer candidate value that makes the data size of the coded data to be mapped onto the packet an integral multiple of the payload size of the packet based on the outputted control information and coding the predetermined units of inputted data up to the outer candidate value.

Description

TECHNICAL FIELD

This present invention relates to the coding method and the coding apparatus of audio data and video data in digital broadcasting and communication field, especially to the technique for improving a transmission efficiency of the coded data.

BACKGROUND ART

Some of conventional coding apparatuses (encoders) that improve a transmission efficiency of coded data determine a quantization width based on buffer amount (for example, refer to document 1).

FIG. 1 is a block diagram showing the structure of a conventional encoder (coding apparatus) described in the above-mentioned document.

As shown in FIG. 1, a conventional audio encoder 200 includes a coding unit 202, a packetization unit 204, a quantization width calculation unit 206 and the like.

The coding unit 202 receives an input of audio data 201 and calculates coded data size inside. The coding unit 202 generates coded data in a way that the coded data falls within the calculated coded data size, and the generated coded data is stored in the sequential storage unit 203. The data stored in the storage unit 203 is divided into the packet 205 by the packetization unit 204. The storage unit 203 transfers the stored data size to the quantization width calculation unit 206, and the quantization width calculation unit 206 transfers the quantization width to the coding unit 202.

In the case where the size of the coded data outputted by the coding unit 202 is variable length, the total size of the data stored in the storage unit 203 is also variable length.

The procedure at the time of dividing the data stored by the storage unit 203 will be shown in FIG. 2.

The coded data stored by the storage unit 203 is called as elementary stream (abridged as ES from here). As to audio data, ESs are made in a plurality of audio frames 301, 302 and 308. Next, these ESs are converted into a packetized elementary stream (abridged as PES). ESs are converted into PESs by adding a PES header 303 to ESs, and the whole is called as PES packet. In the PES packet, the data part stored by the storage unit 203 is called as PES payload 304. This PES packet is converted into a transport stream (called TS from here). This PES packet is divided into fixed lengths, and the TS header 305 is added to each of the packets so as to make each of the corresponding TS packet. A TS payload 306 follows the TS header. In mapping from an ES to a PES or a TS, there may be a case of mapping an integer number of audio frames onto a single PES packet and mapping in a way that a single audio frame is not divided into plural PES packets, and also, mapping in a way that a single TS packet is not divided into plural PES packets with a view to making it easier to access on an audio frame basis as shown in FIG. 2.

Japanese Laid-Open Patent application No. 2001-292448 publication.

However, in a conventional audio encoder, in the above-mentioned case, there is a case where a padding 307 is generated on a PES packet basis when dividing the PES packet into TS packets. Especially, percentage of the padding in the size of the PES packet becomes bigger when the size of the PES packet is small, and there emerges a problem that the transmission efficiency of the TS packets decreases.

Also, it is possible to improve the transmission rate of the TS packets by changing the coded data size in order to restrain padding, there is a problem that sound quality deteriorates when changing coded data size, to be more specific, reducing the coded data size or always changing the coded data size during the encode processing with no restriction.

The first object of the present invention is to provide a coding apparatus that solves the above-mentioned first problem, that is, to reduce the padding data and improve the transmission efficiency.

Also, the second object of the present invention is to provide a coding apparatus that solves the above-mentioned second problem and minimize sound deterioration improving the transmission efficiency.

DISCLOSURE OF INVENTION

In order to solve the above-mentioned problem, the coding apparatus concerning the present invention generates coded data by coding inputted data in a predetermined unit basis, maps one or more predetermined units of the coded data onto a packet of fixed length and outputs the packet, comprising: a storage unit operable to store the coded data and output a data size of the stored coded data; a packetization unit operable to map the coded data stored in the storage unit onto the packet and output a header size and a payload size of the packet assigned to one or more units of the coded data; a control unit operable to generate control information based on the data size of the outputted coded data, the header size and the payload size, and output the control information; and a coding unit operable to calculate an outer candidate value that makes the data size of the coded data to be mapped onto the packet become an integral multiple of the payload size of the packet based on the outputted control information, and code the predetermined units of inputted data up to the outer candidate value so as to generate the coded data.

In this way, it is possible to restrain padding of packets and improve the transmission efficiency by feeding back the information such as the size of the data stored by the storage unit, the packet size of the packetization unit and the like and adjusting the size of the coded data.

Also, in the coding apparatus concerning the present invention, further, the packetization unit may output a flag for judging timing for calculating the outer candidate value, the control unit may generate the control information including the outputted flag, and the coding unit may generate the coded data up to the outer candidate value in the case where the flag is ON.

Also, in the coding apparatus concerning the present invention, the packetization unit turns the flag ON before the data size of the coded data to be mapped onto the packet becomes the integral multiple of the payload size of the packet.

Also, in the coding apparatus concerning the present invention, the coding unit previously calculates an inner candidate value that is a unique coded data size at the time of coding, judges which of the inner candidate value and the outer candidate value should be used and generates the coded data up to the candidate value determined by the judgment.

Also, in the coding apparatus concerning the present invention, the coding unit generates the coded data up to the inner candidate value in the case where a ratio of difference between an inner candidate value and an outer candidate value to the inner candidate value exceeds a threshold.

Also, in the coding apparatus concerning the present invention, the inputted data is audio data, the coded data is an elementary stream of MPEG audio, and the packet of fixed length composes a MPEG transport stream.

Also, in the coding apparatus concerning the present invention, the inputted data is video data, the coded data is an elementary stream of MPEG video, and the packet of fixed length composes a MPEG transport stream.

As shown from the above-mentioned explanation, according to the coding apparatus of the present invention, it becomes possible to restrain the number of cases where padding occupies the most part at the time of dividing packets, and improve the transmission efficiency.

Also, in the case where the change of the coded data size, to be more specific, the reduce of the data size exceeds the threshold value, not changing the coded data size can avoid sound deterioration.

Also, it is possible to minimize sound deterioration because coded data size is changed only at a specific timing.

Therefore, this present invention can improve the transmission efficiency and minimize the deterioration of sound, and the present invention is highly practical today when digital broadcasting or the content distribution by the Internet and the like have been become popular.

Note that the present invention can not be only realized as this coding apparatus but also realized as an integrated circuit including characteristic units of this coding apparatus, as a coding method making these characteristic units of this coding apparatus become steps, and realized as a program that causes a computer to execute these steps. In addition, the program can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet.

Further Information about Technical Background to this Application

The disclosure of Japanese Patent Application No. 2003-383526 filed on Nov. 13, 2003 including specification, drawings and claims is incorporated herein by reference in its entirety.

BRIEF DESCRIPTION OF DRAWINGS

These and other objects, advantages and features of the invention will become apparent from the following description thereof taken in conjunction with the accompanying drawings that illustrate a specific embodiment of the invention. In the Drawings:

FIG. 1 is a block diagram showing the structure of a conventional encoder;

FIG. 2 is a key map showing the deterioration of the transmission efficiency in a conventional audio encoder;

FIG. 3 is a block diagram showing the functional structure of the audio encoder concerning the embodiment of the present invention;

FIG. 4 is a diagram showing the structural example of the data structure of the packet information shown in FIG. 3;

FIG. 5 is a diagram showing the structural example of the data structure of the storage data information shown in FIG. 3;

FIG. 6 is a diagram showing the structural example of the data structure of the control information shown in FIG. 3;

FIG. 7 is a flow chart showing an operation of the audio encoder;

FIG. 8 is a flow chart showing a subroutine of the step S702 shown in FIG. 7; and

FIG. 9 is a diagram for explaining the difference between the conventional coding processing and the decoding processing in the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION

The embodiment of the present invention will be explained in detail with reference to figures below.

FIG. 3 is a block diagram showing the functional structure of an audio encoder (coding apparatus) concerning the embodiment of the present invention.

As shown in FIG. 3, the coding apparatus (audio encoder) 100 includes a coding unit 102, a storage unit 103, a packetization unit 104, a control unit 107 and the like. Note that the audio data 101 and the packet 105 are the same as the conventional ones and the explanations on them will be omitted.

The coding unit 102 generates an audio frame data from the audio data to be inputted and a coded data by coding the audio frame data based on the control information 106 from the control unit 107.

The storage unit 103 includes a buffer for storing audio frame data and coded data or a unit for obtaining the size of the stored audio frame. Also, the size of the audio frame stored in the storage unit 103 is transferred from the storage unit 103 to the control unit 107 as storage data information 108. The structural example of the storage data information 108 will be shown in FIG. 5. The storage data information 108 is composed of a storage data size 501 and the like as shown in FIG. 5.

The packetization unit 104 transmits, to the control unit 107, a flag indicating the timing for dividing coded data stored by the storage unit 103 into packets and the packetization information 109 including the size of the packet 105 and the like. The structural example of the packetization information 109 will be shown in FIG. 4.

The packetization information 109, as shown in FIG. 4, includes a packetization flag 401 that shows the timing for dividing the coded data into packets, a PES header size 402, a TS payload size 403 and the like.

The control unit 107 generates the control information 106 based on the storage data information 108 that is sent and the packetization information 109 and communicates the generated control information 106 to the coding unit 102. The structural example of the control information will be shown in FIG. 6. The control information 106 includes a combination of a packetization information 109 and a storage data information 108. In other words, the control information 106, as shown in FIG. 6, includes a packetization flag 601, a PES header size 602, a TS payload size 603 and a storage data size 604.

Next, the procedure of packetization processing of the audio frame in the coding apparatus 100 will be explained with reference to FIG. 7.

The coding unit 102 converts the audio data 101 (S701) that is inputted first into an audio frame (S702) and stores the audio frame in the storage unit 103 (5703). At this time, the storage unit 103 updates the storage data information 108 inside (S704). After that the storage unit 103 transmits the storage data information 108 to the control unit 107 (S704). At this time, the data size of the audio frame to be stored in a PES packet and a TS packet that are generated next is stored in the storage data information 108.

After that, when the coded data is stored in the storage unit 103, the packetization unit 104 packetizes the stored data. Also, the packetization unit 104 controls the cycle of the packetization. In this embodiment, it is assumed that packetization is performed every six audio frames. In this case, packetization is performed after these six audio frames are stored in the storage unit 103.

The packetization unit 104 transfers the packetization information 109 where the packetization flag 401 is turned ON to the control unit. 107 after the phase before the packetization is finished, in other words, the 5th audio frame is stored (S705, S706 and S707). In this embodiment, in the case of “turn ON”, all the 32 bit data is made to be “1”. Actual packetization is performed in the next phase, that is, after the sixth audio frame is stored (S709, S710). At this time, the packetization flag 401 is turned OFF and the packetization information 109 is transmitted to the control unit 107 (S711). In this embodiment, in the case of “turn OFF”, all the 32 bit data is made to be “0”. After the other phases finish (“N” in S709), the packetization flag 401 is kept OFF and the packetization information 109 is transmitted to the control unit 107.

Note that, in the embodiment, 32 bit data structure is employed as a packetization flag, all the 32 bit data is made to be “1” in the case of “turn ON”, and all the 32 bit data is made to be “0” in the case of “turn OFF”. However, 1 bit data structure may be used as a packetization flag as long as two statuses are used as an indication of a packetized flag.

The control unit 107 receives the packetization information 109 from the packetization unit 104, combines the packetization information 109 with the storage data information 108 from the storage unit 103 so as to generate the control information 106 and transmits the generated control information 106 to the coding unit 102 (S708).

After receiving the control information 106, the coding unit 102 generates the coded data by executing the coding processing of the audio frame data based on the control information 106 (S702).

The procedure of the coding processing using the control information in the audio encoder (detailed processing in S702) will be explained in FIG. 8.

The control unit 107 generates control information 106 from the sent packetization information 109 and the storage data information 108 and transmits the control information 106 to the coding unit 102 (S802).

The coding unit 102 inputs PCM data (S801), calculates the candid value of the coded data size at the time of outputting the audio frame and performs coding up to this size. The coding unit 102 calculates the candid value of the coded data size internally inside (S803). This candid value is called as inner candidate value (nInnerCandidateValue). Also, the coding unit 102 uses the inner candidate value as the coded data size candidate value in the case where the packetization flag 401 of the control information 106 sent from the control unit 107 is OFF (S808). In other words, inner candidate value is used for 1 to 5 frames.

On the other hand, in the case where the packetization flag is ON, the coding unit 102 calculates the candidate valule of the coded data size based on the control information 106 (S805). This candidate value is called as outer candidate value (nOuterCandidateValue). In other words, outer candidate value is basically used for six frames.

As shown in formula (1), an outer candidate value is calculated by subtracting, from an inner candidate value, the remainder of dividing the total of the inner candidate value, a PES header size 602 and a storage data size 604 by a TS payload size 603. Note that, “%” in the formula (1) is an operator for calculating the surplus. In the case of calculating the outer candidate value, the coding unit 102 performs processing for determining one of the inner candidate value and the outer candidate value as the coded data size candidate value next. $\begin{array}{cc}\mathrm{nOuterCandidateValue}=\mathrm{nInnerCandidateValue}-\left(\mathrm{nInnerCandidateValue}+\mathrm{nStockedDateSize}+\mathrm{nPESHeaderSize}\right)\%\mathrm{nTSPayloadSize}& \left(1\right)\end{array}$ nStockedDateSize:Size of coded data presently stored in storage unit 103 nPESHeaderSize:PES header size nTSPayloadSize:TS payload size

The processing for determining the candidate value of the coded data size will be explained as follows. Initially, the coding unit 102 calculates the evaluation values. dEvaluateValue=(nInnerCandidateValue−nOuterCandidateValue)/nOuterCandidateValue  (2) dEvaluateValue:Evaluatioin value

The coding unit 102 judges whether the evaluation value is small enough to the inner candidate value or not. This judgment is made using the threshold processing (S807). In the case where the differential value is small enough (“Y” in S807), the coding unit 102 judges that the sound deterioration is small in the case where coding is performed up to the outer candidate value, substitutes the outer candidate value into the coded data size candidate value (S809), and performs coding up to this value (S810). Note that threshold processing is performed on condition that the value of the threshold is constant, in the case where the differential slightly exceeds the threshold, it is possible to change the threshold into a larger threshold, and perform coding up to the outer candidate value.

On the other hand, in the case where the differential value is judged as big (“N” in S807), the coding unit 102 judges that the sound deterioration is remarkable in the case where coding is performed up to the outer candidate value, substitutes the inner candidate value into the coded data size candidate value (S808) and performs coding up to this value (S810).

Determining the target of the coded data size in this way, it becomes possible to perform the restriction of sound deterioration and the restriction of padding efficiently. Also, as the coded data size is changed only when the packetization flag is ON, not all the time during the encoding processing, and it is possible to minimize the number of audio frames whose data size is changed in controlling the encoding processing, and the deterioration in sound by the change of the coded data size.

Note that the form of mapping six audio frames into a single PES packet has already been explained in the embodiment, but the method for mapping the audio frames onto the PES packet is not limited to the embodiment, but any mapping may be used.

Also, in this embodiment, packetization flag is turned ON at the timing when the fifth audio frame is stored, and the coded data size is changed, but the present invention is not limited to this, and a similar effect can be obtained even in the case of changing the coded data size at an arbitrary timing. In this case, changing the coded data size of only the data to be coded immediately before performing packetization makes it possible to reduce the deterioration in sound by the change of the coded data size.

Also, it is possible to change the coded data size of the fifth and the sixth audio frames in the case of an encoder whose inner candidate value of the sixth audio frame becomes equal to its inner candidate value of the fifth audio frame. In this encoder, the sizes of the fifth and the sixth audio frames are always equal to each other as shown in FIG. 9A, the packetization flag is turned ON at the timing when the fourth audio frame is stored. At this time point, frames up to the audio frame of 133 size in FIG. 9A are stored. The coding unit 102 confirms that the packetization flag is turned ON at the time of encoding next fifth audio frame, and calculates the inner candidate value of the fifth and the sixth audio frames.

nInnerCandidateValue=179

After that, as two audio frames are handled before executing the formula (1), the processing that doubles nInnerCandidateValue is added like below.

nInnerCandidateValue=179×2=358

Next, the formula (1) is executed like the conventional method, and the outer candidate value is calculated. Note that the PES header size is made to be 10, and the PES payload size is made to be 80. $\begin{array}{c}\mathrm{nOuterCandidateValue}=358-\left(358+10+131+169+182+133\right)\mathrm{\%80}\\ =358-983\mathrm{\%80}\\ =358-23\\ =335\end{array}$

At last, two audio frames are handled, the processing for dividing nOuterCandidateValue into two is added like below, this value is used for an outer candidate value of the fifth and the sixth audio frames. $\begin{array}{c}\mathrm{nOuterCandidateValue}=335/2\\ =167\end{array}$

In the case where this outer candidate value is employed, the storage unit 103 becomes like. FIG. 9B.

The padding becomes 57 in the case of FIG. 9A, and becomes 1 in the case of FIG. 9B.

Unlike the case where outer candidate value is used only in a single audio frame, in the case where the outer candidate value is calculated and employed in a plurality of audio frames like this, it is possible to disperse the sound deterioration in a single audio frame into plural audio frames and restrain the sound deterioration of the whole music.

Also, mapping onto the TS packet is performed on a basis of PES packet in this embodiment, padding is generated by preventing plural PES packets from being mapped onto a single TS packet, but the present invention is not limited to this. With an audio encoder that performs mapping where padding is frequently generated at the time of generating the TS packet, the same effect is obtained by using the present invention.

Also, the audio encoder has already been shown in the embodiment, the present invention is not limited to this. The present invention is applicable in a video encoder or an encoder where padding is generated at the time of mapping onto the coded data.

Although only an exemplary embodiment of this invention has been described in detail above, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiment without materially departing from the novel teachings and advantages of this invention. Accordingly, all such modifications are intended to be included within the scope of this invention.

INDUSTRIAL APPLICABILITY

The present invention may be used not only for an audio encoder and a video data but also for improving the transmission efficiency in the case of transmitting the variable length frame data in a fixed length packet in a communication field.

Claims

1. A coding apparatus for generating coded data by coding inputted data in a predetermined unit basis, mapping one or more predetermined units of the coded data onto a packet of fixed length and outputting the packet, comprising: a storage unit operable to store the coded data and output a data size of the stored coded data; a packetization unit operable to map the coded data stored in the storage unit onto the packet and output a header size and a payload size of the packet assigned to one or more units of the coded data; a control unit operable to generate control information based on the data size of the outputted coded data, the header size and the payload size, and output the control information; and a coding unit operable to calculate an outer candidate value that makes the data size of the coded data to be mapped onto the packet become an integral multiple of the payload size of the packet based on the outputted control information, and code the predetermined units of inputted data up to the outer candidate value so as to generate the coded data.

2. The coding apparatus according to claim 1, wherein the packetization unit further outputs a flag for judging timing for calculating the outer candidate value, the control unit generates the control information including the outputted flag, and the coding unit generates the coded data up to the outer candidate value in the case where the flag is ON.

3. The coding apparatus according to claim 2, wherein the packetization unit turns the flag ON before the data size of the coded data to be mapped onto the packet becomes the integral multiple of the payload size of the packet.

4. The coding apparatus according to claim 1, wherein the coding unit previously calculates an inner candidate value that is a unique coded data size at the time of coding, judges which of the inner candidate value and the outer candidate value should be used and generates the coded data up to the candidate value determined by the judgment.

5. The coding apparatus according to claim 4, wherein the coding unit generates the coded data up to the inner candidate value in the case where a ratio of difference between an inner candidate value and an outer candidate value to the inner candidate value exceeds a threshold.

6. The coding apparatus according to claim 1, wherein the inputted data is audio data, the coded data is an elementary stream of MPEG audio, and the packet of fixed length composes a MPEG transport stream.

7. The coding apparatus according to claim 1, wherein the inputted data is video data, the coded data is an elementary stream of MPEG video, and the packet of fixed length composes a MPEG transport stream.

8. A coding method used by a coding apparatus for generating coded data by coding inputted data in a predetermined unit basis, mapping one or more units of the coded data onto a packet of fixed length, and outputting the packet, comprising: a storage step of storing the coded data in a storage unit and outputting a data size of the stored coded data; a packetization step of mapping the coded data stored in the storage unit onto the packet and outputting a header size and a payload size of the packet assigned to one or more units of the coded data; a control step of generating control information based on the data size of the outputted coded data, the header size and the payload size, and outputting the control information; and a coding step by calculating an outer candidate value that makes the data size of the coded data to be mapped onto the packet become an integral multiple of the payload size of the packet based on the outputted control information, and coding the predetermined units of inputted data up to the outer candidate value so as to generate the coded data.

9. The coding method according to claim 8, wherein in the packetization step, a flag for judging timing for calculating the outer candidate value is further outputted, in the control step, the control information including the outputted flag is generated, and in the coding step, the coded data up to the outer candidate value is generated in the case where the flag is ON.

10. The coding method according to claim 9, wherein in the packetization step, the flag is turned ON before the data size of the coded data to be mapped onto the packet becomes the integral multiple of the payload size of the packet.

11. The coding method according to claim 8, wherein in the coding step, an inner candidate value that is a unique coded data size is previously calculated at the time of coding, which of the inner candidate value and the outer candidate value should be used is judged and the coded data up to the candidate value determined by the judgment is generated.

12. The coding method according to claim 11, wherein in the coding step, the coded data up to the inner candidate value is generated in the case where a ratio of difference between an inner candidate value and an outer candidate value to the inner candidate value exceeds a threshold.

13. The coding method according to claim 8, wherein the inputted data is audio data, the coded data is an elementary stream of MPEG audio, and the packet of fixed length composes a MPEG transport stream.

14. The coding method according to claim 8, wherein the inputted data is video data, the coded data is an elementary stream of MPEG video, and the packet of fixed length composes a MPEG transport stream.

15. A program used by a coding apparatus for generating coded data by coding inputted data in a predetermined unit basis, mapping one or more units of the coded data onto a packet of fixed length, and outputting the packet, the program causing a computer to execute: a storage step of storing the coded data in a storage unit and outputting a data size of the stored coded data; a packetization step of mapping the coded data stored in the storage unit onto the packet and outputting a header size and a payload size of the packet assigned to one or more units of the coded data; a control step of generating control information based on the data size of the outputted coded data, the header size and the payload size, and outputting the control information; and a coding step of calculating an outer candidate value that makes the data size of the coded data to be mapped onto the packet become an integral multiple of the payload size of the packet based on the outputted control information, and coding the predetermined units of inputted data up to the outer candidate value so as to generate the coded data.

16. A computer-readable recording medium for recording a program used by a coding apparatus for generating coded data by coding inputted data in a predetermined unit basis, mapping one or more units of the coded data onto a packet of fixed length, and outputting the packet, the program causing a computer to execute: a storage step of storing the coded data in a storage unit and outputting a data size of the stored coded data; a packetization step of mapping the coded data stored in the storage unit onto the packet and outputting a header size and a payload size of the packet assigned to one or more units of the coded data; a control step of generating control information based on the data size of the outputted coded data, the header size and the payload size, and outputting the control information; and a coding step of calculating an outer candidate value that makes the data size of the coded data to be mapped onto the packet become an integral multiple of the payload size of the packet based on the outputted control information, and coding the predetermined units of inputted data up to the outer candidate value so as to generate the coded data.

17. An integrated circuit functioning as a coding apparatus for generating coded data by coding inputted data in a predetermined unit basis, mapping one or more units of the coded data onto a packet of fixed length, and outputting the packet, comprising: a storage unit operable to store the coded data and output a data size of the stored coded data; a packetization unit operable to map the coded data stored in the storage unit onto the packet and output a header size and a payload size of the packet assigned to one or more units of the coded data; a control unit operable to generate control information based on the data size of the outputted coded data, the header size and the payload size, and output the control information; and a coding unit operable to calculate an outer candidate value that makes the data size of the coded data to be mapped onto the packet become an integral multiple of the payload size of the packet based on the outputted control information, and code the predetermined units of inputted data up to the outer candidate value so as to generate the coded data.