Patent Application
20090326931
The present invention relates to a hierarchical audio coding system. It also relates to a hierarchical audio coder and a hierarchical audio decoder.
The invention finds a particularly advantageous application in the field of transmission of speech and/or audio signals over packet networks, of the voice over IP type. More specifically, in this context, the invention provides a quality that can be modulated, running from a telephone band to a wideband, as a function of the bitrate capacity of the transmission and guaranteeing interworking with an existing telephone band core.
Many techniques exist at present for converting an audio-frequency (speech and/or audio) signal into the form of a digital signal and processing the signals digitized in this way. The standard high-quality audio coding methods are generally classified as “waveform coding”, “parametric coding by analysis by synthesis”, and “perceptual coding in sub-bands or by transforms”.
The first category includes quantizing techniques with or without memory such as PCM or ADPCM coding.
The second category includes techniques that represent the signal by means of a model, generally a linear predictive model, having parameters that are determined using methods derived from waveform coding. For this reason, this category is often referred to as hybrid coding. For example, CELP (code excited linear prediction) coding belongs to this second category. In CELP coding, the input signal is coded by means of a “source-filter” model inspired by the speech production process. The parameters transmitted represent separately the source (or “excitation”) and the filter. The filter is generally an all-pole filter. The basic concepts of coding audio-frequency signals and more particularly of CELP coding and quantization are explained in the following works in particular: W. B. Kleijn and K. K. Paliwal, editors, Speech Coding and Synthesis, Elsevier, 1995, and Nicolas Moreau, Techniques de compression des signaux [Signal compression techniques], Collection Technique et Scientifique des Télécommunications, Masson, 1995.
The third category includes coding techniques such as MPEG 1 and 2 Layer III, better known as MP3, or MPEG 4 AAC.
The ITU-T G.729 system is one example of CELP coding designed for speech signals in the telephone band (300 hertz (Hz)-3400 Hz) sampled at 8 kilohertz (kHz). It operates at a fixed bitrate of 8 kilobits per second (kbps) with 10 milliseconds (ms) frames. Its operation is specified in detail in ITU-T Recommendation G.729, Coding of Speech at 8 kbps using Conjugate Structure Algebraic Code Excited Linear Prediction (CS-ACELP), March 1996.
a), 1(b) and 1(c) together constitute a simplified diagram of the associated coder and decoder.
The excitation decoded in this way is shaped by a 10th order LPC (linear predictive coding) synthesis filter 1/A(z) (120), having coefficients that are decoded (119) in the LSF (line spectrum frequency) domain from pairs of spectrum lines and interpolated at 5 ms sub-frame level. To improve quality and to mask certain coding artefacts, the reconstructed signal is then processed by an adaptive post-filter (121) and a post-processing high-pass filter (122). The
a) represents a very high level diagram of the G.729 coder. It therefore shows the pre-processing high-pass filtering (101), the LPC analysis and quantization (102), the coding of the excitation (103) and the multiplexing of the coding parameters (104). The pre-processing and LPC analysis and quantizing blocks of the G.729 coder are not discussed here; for more details see the ITU-T recommendation referred above.
The excitation parameters are determined by minimizing the quadratic error (111) between the CELP target (105) and the excitation filtered by W(z)/Â(z) (110). This process of analysis by synthesis is described in detail in the ITU-T recommendation referred to above.
In practice, the complexity of the G.729 coder/decoder (codec) is relatively high (around 18 WMOPS (weighted million operations per second)). To meet the requirements of applications such as simultaneous transmission of voice and data via DSVD (digital simultaneous voice and data) modems, an interworking system of lesser complexity (around 9 WMOPS) is also recommended by the ITU-T: the G.729A codec. This is described and compared to the G.729 codec in R. Salami et al., Description of ITU-T Recommendation G.729 Annex A: Reduced complexity 8 kbps CS-ACELP codec, ICASSP 1997.
Of the significant differences between G.729 and G.729A, that which reduces the G.729 complexity the most relates to searching in the ACELP dictionary: in the G.729A coder an in-depth search firstly of the four signed pulses replaces the interleaved loop search used in the G.729 coder. By virtue of its low complexity, the G.729A codec is now very widely used in voice over IP or ATM applications in the telephone band (300-3400 Hz).
With the growth of optical fiber and broadband networks such as ADSL, deploying new services can now be envisaged, such as bidirectional communication of much higher quality than standard systems using the telephone band. One step in this direction is to provide “wideband” quality, i.e. to use audio-frequency signals sampled at 16 kHz and limited to a usable band of 50 Hz-7000 Hz. The quality obtained is then similar to that of AM radio.
The choice of a codec for deploying “wideband” quality instead of “narrowband” quality must take a number of important factors into account.
The approach known as “hierarchical” coding is the technical solution best suited to taking account of all these constraints.
Unlike conventional coding, such as G.729 or G.729A coding, generating a bit stream at fixed bitrate, hierarchical coding generates a bit stream that can be decoded in whole or in part. As a general rule, hierarchical coding comprises a core layer and one or more enhancement layers. The core layer is generated by a low fixed bitrate core codec, guaranteeing the minimum coding quality. This layer must be received by the decoder to maintain an acceptable quality level. The enhancement layers serve to improve quality. However, it can happen that they are not all received by the decoder, because of transmission errors, for example in the event of congestion of an IP network.
This technique therefore offers great flexibility in terms of the choice of the bitrate and the quality of reconstruction. The coder always assumes that the bitrate is the maximum bitrate. However, anywhere in the communication chain the bitrate can be adapted simply by truncating the bit stream. Hierarchical coding can moreover progressively deploy wideband quality, relying on a standard of the CELP coding in the telephone band type (such as the ITU-T G.729 and G.729A standards).
Of the various approaches to hierarchical coding based on a CELP core coder, the following four techniques may be mentioned:
The difference between the concept of hierarchical CELP coding by excitation enrichment and the coding shown in
The band extension system proposed in the above paper by J.-M. Valin is shown in the
a baseband regenerated by the block (32);
Note more particularly in this diagram the extension of the highband, which is founded on the “source-filter” model. This begins with a narrowband LPC analysis (34) that determines the coefficients of the prediction filter ANB(z) (36). The result of this LPC analysis is also used by the LPC envelope extension unit (35) to determine the coefficients of a full-band LPC synthesis filter 1/BWB(z) (38). Envelope extension can be effected using codebook mapping techniques, for example, with no transmission of auxiliary information, or with explicit information requiring transmission by quantization at a low additional bitrate. In parallel, the narrowband LPC residual (or excitation) signal is calculated by the unit (36). The resulting excitation sampled at 8 kHz is extended to the sampling frequency of 16 kHz by the unit (37). This operation can be carried out in the excitation domain by employing non-linearity, oversampling and filtering, in order to extend the harmonic structure and to whiten the full-band excitation. The extended excitation is then shaped by the full-band synthesis filter 1/BWB (38) and the result is limited by the high-pass filter (39) to the 3400 Hz-8000 Hz band.
All known techniques of the prior art give rise to the following problems, however:
Moreover, certain fundamental problems are rarely touched on in the prior art: the phase non-linearity of pre-processing and post-processing is only rarely taken into account. The enhancement layers rely on coding a difference signal between original (pre-processed or not) and synthesis of the lower layer have badly degraded performance if the phase non-linearity (or group delay) of the pre-processing and post-processing filters is not compensated or eliminated.
The invention therefore has the object of remedying the various problems set out above by proposing a system for coding a hierarchical audio signal, comprising, at least, a core layer using parametric coding by analysis by synthesis in a first frequency band, a band extension layer for widening said first frequency band into a second frequency band, or wideband, noteworthy in that said system also comprises a wideband audio coding quality enhancement layer based on transform coding using a spectral parameter obtained from said band extension layer.
It should be emphasized here that the term “wideband” used in this description corresponds to a particular instance of the general concept of “extended band”. Here “wideband” means a frequency band resulting from the extension of a first band, the telephone band of 300 Hz to 3400 Hz, to a second band, the wideband, of 50 Hz to 7000 Hz.
An advantageous embodiment of said system also comprises a first frequency band audio coding quality enhancement layer.
In a first embodiment of the coding system of the invention, said spectral parameter is a spectral envelope obtained from the band extension layer. Two embodiments can be envisaged: said spectral envelope is specified by a wideband linear prediction filter, or said spectral envelope is given by the energy per sub-band of the signal.
In a second embodiment of the coding system of the invention, said spectral parameter is at least a portion of the transform of the signal synthesized by the band extension layer. Said system then advantageously comprises a module for progressive adjustment of the energy in the sub-bands of the transform of the signal synthesized by the band extension layer.
The invention also provides for said parametric coding by analysis by synthesis to be CELP coding. In particular, said CELP coding is G.729 coding or G.729A coding.
Accordingly, as seen in detail below, the coding system proposed by the invention constitutes a hierarchical coding system able to operate at bitrates of 8 kbps to 12 kbps, for example, and at all bitrates of 14 kbps to 32 kbps.
In response to the problems raised by the prior art, the coding/decoding system according to the invention is such that:
The invention also provides a method of implementing the coding system according to the first embodiment, comprising the following steps:
The invention further provides a method of implementing the coding system according to the second embodiment, comprising the following steps:
Said method advantageously comprises a step of progressively adjusting the energy in the sub-bands of the transform of the signal synthesized by the band extension layer.
The invention further provides a computer program comprising program instructions for executing the steps of the method according to the invention when said program is executed by a computer.
The invention further provides a first hierarchical audio coder comprising:
Similarly, the invention provides a second hierarchical audio coder comprising:
The invention further provides a first hierarchical audio decoder comprising:
Finally, the invention provides a second hierarchical audio decoder comprising:
The following description with reference to the appended drawings, provided by way of non-limiting example, explains in what the invention consists and how it can be reduced to practice.
a) is a diagram of the first three stages of a coder according to the present invention.
b) is a diagram of the fourth stage of the coder from
a) is a general diagram of the four-layer decoder according to the present invention.
b) is a detailed diagram of the transform predictive decoding stage of the decoder from
a) to 10(b) show a hierarchical coding/decoding system consisting of a coder and a decoder that are described in succession next.
In the remainder of this description it should be recalled that the term “wideband” refers to the particular circumstance of a telephone band 300 Hz-3400 Hz extended to 50 Hz-7000 Hz domain.
a) is a block diagram of the coder. An original audio signal with a usable band between 50 and 7000 Hz and sampled at 16 kHz is divided into frames of 320 samples, or 20 ms. High-pass filtering 601 with a cut-off frequency of 50 Hz is applied to the input signal. The signal SWB obtained is used in multiple branches of the coder and corresponds to the signal really coded.
Firstly, in a first branch, low-pass filtering (having coefficients as set out in the
A first enhancement layer then introduces a second stage 603 of CELP coding. This second stage consists in an innovator code consisting of four additional ±1 pulses for a 5 ms subframes (dictionary equivalent to that of G.729A), these pulses are scaled by a gain genh. The principle of this enhancement stage has already been described above with reference to the paper by R. D. De lacovo. This dictionary enriches the CELP excitation and offers a quality improvement, particularly for non-voiced sounds. The bitrate of this second coding stage is 4 kbps and the associated parameters are the positions and the signs of the pulses and the associated gain for each sub-frame of 40 samples (5 ms at 8 kHz). In a variant of this embodiment, this coding stage uses other enhancement modes, for example those described in the De lacovo paper referred to above.
The core coder and the first enhancement layer are decoded to obtain the 12 kbps telephone band synthesis signal. It is important to note that the adaptive post-filtering and post-processing (high-pass filtering) of the core coder are deactivated in order to take account of the non-linear phase-shift of these operations; the difference between the original pre-process signal and the synthesis at 8 and 12 kbps is therefore minimized. Oversampling and low-pass filtering 604 produce the version sampled at 16 kHz of the first two stages of the coder.
The wideband signal is produced by the second enhancement layer, also called the band extension layer. The input signal SWB can be filtered by a pre-emphasis filter 605 with μ=0.68. This filter provides a better representation of the higher frequencies from the wideband linear prediction filter. To compensate the effect of the pre-emphasis filter, a dual de-emphasis filter 606 is then used in the synthesis process. In a preferred embodiment, no pre-emphasis and de-emphasis filters are used in the coding and decoding structure. The next step calculates and quantizes the wideband linear prediction filter 607. The linear prediction filter is an 18th order filter, but in a variant of this embodiment another prediction order is chosen, for example a lower order (16th order). The linear prediction filter can be calculated by the autocorrelation method using the Levinson-Durbin algorithm.
This wideband linear prediction filter ÂWB(z) is quantized using a prediction of these coefficients, where applicable from the filter ÂWB(z) from the telephone band core coder 603. The coefficients can then be quantized using multistage vector quantization, for example, and the dequantized LSF parameters of the telephone band core coder, as described in the paper by H. Ehara, T. Morii, M. Oshikiri and K. Yoshida, Predictive VQ for bandwidth scalable LSP quantization, ICASSP 2005.
The wideband excitation 608 is obtained from telephone band excitation parameters of the core coder: the pitch delay, the associated gain, and the algebraic excitations of the core coder and the first CELP excitation enrichment layer and the associated gains. This excitation is generated using an oversampled version of the parameters of the telephone band stage excitation. In a variant of this embodiment, the excitation is calculated from the pitch delay and the associated gain, these parameters being used to generate harmonic excitation from white noise. In this variant, the excitation from the algebraic dictionary is replaced by white noise.
This wideband excitation is then filtered by the synthesis filter 609 previously calculated. If pre-emphasis has been applied to the input signal, the de-emphasis filter 606 is applied to the output signal of the synthesis filter. The signal obtained is a wideband signal that has not had its energy adjusted. To calculate the gain for leveling the energy of the high band (3400-7000 Hz), high-pass filtering 611 (having coefficients as set out in the
The remainder of coding is effected in the frequency domain using a transform predictive coding scheme using the linear prediction filter from the band extension layer.
This coding stage constitutes the wideband coding quality enhancement layer.
b) shows this portion of the coder. The delayed input signal 614 and synthesis signal at 14 kbps 615 are filtered by respective perceptual weighting 616 and 617 of AWB(z/γ)*(1−μz), typically with γ=0.92 and μ=0.68. These signals are then encoded by the transform coding scheme.
A modified discrete cosine transform (MDCT) is applied: both to blocks of 640 samples of the weighted input signal 618 with an overlap of 50% (refreshing of the MDCT analysis every 20 ms), and also to the weighted synthesis signal 619 from the preceding band extension stage at 14 kbps (same block length and same overlap). The MDCT spectrum 620 to be encoded corresponds to the difference between the weighted input signal and the synthesis signal at 14 kbps for the 0 to 3400 Hz band and to the weighted input signal from 3400 Hz to 7000 Hz. The spectrum is limited to 7000 Hz by setting to zero the last 40 coefficients (only the first 280 coefficients are coded). The spectrum is divided into 18 bands: one band of eight coefficients and 17 bands of 16 coefficients as set out in the
The scale factors of the high band (3400 Hz-7000 Hz) are transmitted before those of the low band (0-3400 Hz), as the bit stream format shown in
Dynamic bit allocation is based on the energy of the bands of the spectrum from the de-quantized version of the spectral envelope. This achieves compatibility between the binary allocation of the coder and the decoder. The allocation of bits in the TDAC (time domain aliasing cancellation) module 620 is effected in two phases. Firstly, a first calculation of the number of bits to allocate to each band is effected; each of the values obtained is rounded to the closest available dictionary bitrate. If the total bitrate allocated is not exactly equal to that available, a second phase is used to make the adjustment. This step is effected by an iterative procedure based on an energy criterion that adds bits to the bands or removes bits from the bands as described in the paper by Y. Mahieux and J. P. Petit, Transform coding of audio signals at 64 kbps, IEEE GLOBECOM 1990. Thus if the total number of bits distributed is less than that available, bits are added to the bands in which the perceptual enhancement is the greatest (greatest energy). In the contrary situation where the total number of bits distributed is greater than that available, the extraction of bits from the bands is effected in a dual manner.
The normalized (fine structure) MDCT coefficients in each band are then quantized by vectorial quantizers using dictionaries interleaved in size and in resolution, the dictionaries consisting of a union of permutation codes as described in international application WO/0400219. Finally, the information on the core coder, the telephone band CELP enrichment stage, the wideband CELP stage, and, finally, the spectral envelope and decoded normalized coefficients, is multiplexed and transmitted in frames.
The number of bits allocated to each of the parameters of the coder and decoder is set out in the
The frame structure of the bit stream is shown in
The structure of the decoder is described next with reference to
The module 701 demultiplexes the parameters contained in the bit stream. There are multiple decoding situations as a function of the number of bits received for a frame, of which the first three are described with reference to
1. The first concerns the reception of the minimum number of bits by the decoder. In this situation, only the first stage is decoded. Thus only the bit stream relating to the CELP (G.729+) type core decoder 702 is received and decoded. This synthesis can be processed by the adaptive post-filter and the post-processing of the G.729 decoder. This signal is oversampled and filtered to produce a signal sampled at 16 kHz (703).
2. The second situation concerns the reception of the number of bits relating to the first and second decoding stages. In this situation, the core decoder and the first CELP excitation enrichment stage are decoded. This synthesis can be processed by the adaptive post-filter and the post-processing of the G.729 decoder. This signal is oversampled and filtered to produce a signal sampled at 16 kHz (703).
3. The third situation corresponds to the reception of the number of bits relating to the first three decoding stages. In this situation, the first two decoding stages are first effected as in situation 2, after which the band extension module generates a signal sampled at 16 kHz after decoding the parameters of the wideband pairs of spectral lines (WB-LSF) (704) and the gains associated with the excitation. The wideband excitation is generated from the parameters of the core coder and the first CELP enrichment stage 705. This excitation is then filtered by the synthesis filter 706 and where appropriate by the de-emphasis filter 707 if a pre-emphasis filter was used in the coder. A high-pass filter 708 is applied to the signal obtained and the energy of the band extension signal is adapted by means of the associated gains (709) every 5 ms. This signal is then added to the telephone band signal sampled at 16 kHz obtained from the first two decoder stages. With the aim of obtaining a signal limited to 7000 Hz, this signal is filtered in the transform domain by setting to 0 the last 40 MDCT coefficients before passing through the inverse MDCT transform 713 and the weighted synthesis filter 714.
4. This last situation corresponds to the decoding of the last stage of the decoder (
An inverse MDCT transform is then applied to the decoded MDCT coefficients (713) and filtering by the weighted synthesis filter (714) produces the output signal.
In a variant of the embodiment described above, the predictive transform coding/decoding stage operates entirely on the difference signal between the original signal and the synthesis signal of the band extension stage in the range 0 to 7000 Hz.
In another variant of this embodiment, band extension is effected on coding and on decoding in the transform domain from a spectral envelope given by the energy of each sub-band of the signal and coding of the fine structure. This spectral envelope can be quantized by factor quantization. In this variant, the wideband enhancement stage uses TDAC type transform coding as described above (with no weighting filtering). Thus the spectral envelope that is given by the energy in each sub-band of the signal and that constitutes a spectral parameter is transmitted in band extension stage and re-used by the wideband enhancement layer.
Moreover, in an alternative embodiment, the first coded frequency band could correspond to the 50 Hz-7000 Hz wideband and the second coded frequency band could be an FM band (50 Hz-15000 Hz) or a HiFi band (20 Hz-2400 Hz).
| Number | Date | Country | Kind |
|---|---|---|---|
| 0552199 | Jul 2005 | FR | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/FR2006/050690 | 7/7/2006 | WO | 00 | 6/23/2009 |
