Patent Application
20130030819
The present invention is related to audio processing and, particularly, to multi-channel audio processing of a multi-channel signal having two or more channel signals.
It is known in the field of multi-channel or stereo processing to apply the so-called mid/side stereo coding. In this concept, a combination of the left or first audio channel signal and the right or second audio channel signal is formed to obtain a mid or mono signal M. Additionally, a difference between the left or first channel signal and the right or second channel signal is formed to obtain the side signal S. This mid/side coding method results in a significant coding gain, when the left signal and the right signal are quite similar to each other, since the side signal will become quite small. Typically, a coding gain of a quantizer/entropy encoder stage will become higher, when the range of values to be quantized/entropy-encoded becomes smaller. Hence, for a PCM or a Huffman-based or arithmetic entropy-encoder, the coding gain increases, when the side signal becomes smaller. There exist, however, certain situations in which the mid/side coding will not result in a coding gain. The situation can occur when the signals in both channels are phase-shifted to each other, for example, by 90°. Then, the mid signal and the side signal can be in a quite similar range and, therefore, coding of the mid signal and the side signal using the entropy-encoder will not result in a coding gain and can even result in an increased bit rate. Therefore, a frequency-selective mid/side coding can be applied in order to deactivate the mid/side coding in bands, where the side signal does not become smaller to a certain degree with respect to the original left signal, for example.
Although the side signal will become zero, when the left and right signals are identical, resulting in a maximum coding gain due to the elimination of the side signal, the situation once again becomes different when the mid signal and the side signal are identical with respect to the shape of the waveform, but the only difference between both signals is their overall amplitudes. In this case, when it is additionally assumed that the side signal has no phase-shift to the mid signal, the side signal significantly increases, although, on the other hand, the mid signal does not decrease so much with respect to its value range. When such a situation occurs in a certain frequency band, then one would again deactivate mid/side coding due to the lack of coding gain. Mid/side coding can be applied frequency-selectively or can alternatively be applied in the time domain.
There exist alternative multi-channel coding techniques which do not rely on a kind of a waveform approach as mid/side coding, but which rely on the parametric processing based on certain binaural cues. Such techniques are known under the term “binaural cue coding”, “parametric stereo coding” or “MPEG Surround coding”. Here, certain cues are calculated for a plurality of frequency bands. These cues include inter-channel level differences, inter-channel coherence measures, inter-channel time differences and/or inter-channel phase differences. These approaches start from the assumption that a multi-channel impression felt by the listener does not necessarily rely on the detailed waveforms of the two channels, but relies on the accurate frequency-selectively provided cues or inter-channel information. This means that, in a rendering machine, care has to be taken to render multi-channel signals which accurately reflect the cues, but the waveforms are not of decisive importance.
This approach can be complex particularly in the case, when the decoder has to apply a decorrelation processing in order to artificially create stereo signals which are decorrelated from each other, although all these channels are derived from one and the same downmix channel. Decorrelators for this purpose are, depending on their implementation, complex and may introduce artifacts particularly in the case of transient signal portions. Additionally, in contrast to waveform coding, the parametric coding approach is a lossy coding approach which inevitably results in a loss of information not only introduced by the typical quantization but also introduced by looking on the binaural cues rather than the particular waveforms. This approach results in very low bit rates but may include quality compromises.
There exist recent developments for unified speech and audio coding (USAC) illustrated in
b illustrates the situation when the MPEG Surround decoder 706 would perform a mid/side decoding. Alternatively, the MPEG Surround decoder block 706 could perform a binaural cue based parametric decoding for generating stereo signals from a single mono core decoder signal. Naturally, the MPEG Surround decoder 706 could also generate a plurality of low band output signals to be input into the SBR decoder block 709 using parametric information such as inter-channel level differences, inter-channel coherence measures or other such inter-channel information parameters.
When the MPEG Surround decoder block 706 performs the mid/side decoding illustrated in
Using a combination of a block 706 and a block 709 causes only a small increase in computational complexity compared to a stereo decoder used as a basis, because the complex QMF representation of the signal is already available as part of the SBR decoder. In a non-SBR configuration, however, QMF-based stereo coding, as proposed in the context of USAC, would result in a significant increase in computational complexity because of the necessitated QMF banks which would necessitate in this example 64-band analysis banks and 64-band synthesis banks. These filter banks would have to be added only for the purpose of stereo coding.
In the MPEG USAC system under development, however, there also exist coding modes at high bit rates where SBR typically is not used.
According to an embodiment, an audio decoder for decoding an encoded multi-channel audio signal, the encoded multi-channel audio signal including an encoded first combination signal generated based on a combination rule for combining a first channel audio signal and a second channel audio signal of a multi-channel audio signal, an encoded prediction residual signal and prediction information, may have: a signal decoder for decoding the encoded first combination signal to obtain a decoded first combination signal, and for decoding the encoded residual signal to obtain a decoded residual signal; and a decoder calculator for calculating a decoded multi-channel signal having a decoded first channel signal, and a decoded second channel signal using the decoded residual signal, the prediction information and the decoded first combination signal, so that the decoded first channel signal and the decoded second channel signal are at least approximations of the first channel signal and the second channel signal of the multi-channel signal, wherein the prediction information includes a real-valued portion different from zero and/or an imaginary portion different from zero, wherein the prediction information includes an imaginary factor different from zero, wherein the decoder calculator includes a predictor configured for estimating an imaginary part of the decoded first combination signal using a real part of the decoded first combination signal, wherein the predictor is configured for multiplying the imaginary part of the decoded first combination signal by the imaginary factor of the prediction information when obtaining a prediction signal; wherein the decoder calculator further includes a combination signal calculator configured for linearly combining the prediction signal and the decoded residual signal to obtain a second combination signal; and wherein the decoder calculator further includes a combiner for combining the second combination signal and the decoded first combination signal to obtain the decoded first channel signal, and the decoded second channel signal.
According to another embodiment, an audio encoder for encoding a multi-channel audio signal having two or more channel signals may have: an encoder calculator for calculating a first combination signal and a prediction residual signal using a first channel signal and a second channel signal and prediction information, so that a prediction residual signal, when combined with a prediction signal derived from the first combination signal or a signal derived from the first combination signal and the prediction information results in a second combination signal, the first combination signal and the second combination signal being derivable from the first channel signal and the second channel signal using a combination rule; an optimizer for calculating the prediction information so that the prediction residual signal fulfills an optimization target; a signal encoder for encoding the first combination signal and the prediction residual signal to obtain an encoded first combination signal and an encoded residual signal; and an output interface for combining the encoded first combination signal, the encoded prediction residual signal and the prediction information to obtain an encoded multi-channel audio signal, wherein the first channel signal is a spectral representation of a block of samples; wherein the second channel signal is a spectral representation of a block of samples, wherein the spectral representations are either pure real spectral representations or pure imaginary spectral representations, wherein the optimizer is configured for calculating the prediction information as a real-valued factor different from zero and/or as an imaginary factor different from zero, wherein the encoder calculator includes a real-to-imaginary transformer or an imaginary-to-real transformer for deriving a transform spectral representation from the first combination signal, and wherein the encoder calculator is configured to calculate the first combined signal and the first residual signal so that the prediction signal is derived from the transformed spectrum using the imaginary factor.
According to another embodiment, a method of decoding an encoded multi-channel audio signal, the encoded multi-channel audio signal including an encoded first combination signal generated based on a combination rule for combining a first channel audio signal and a second channel audio signal of a multi-channel audio signal, an encoded prediction residual signal and prediction information, may have the steps of: decoding the encoded first combination signal to obtain a decoded first combination signal, and decoding the encoded residual signal to obtain a decoded residual signal; and calculating a decoded multi-channel signal having a decoded first channel signal, and a decoded second channel signal using the decoded residual signal, the prediction information and the decoded first combination signal, so that the decoded first channel signal and the decoded second channel signal are at least approximations of the first channel signal and the second channel signal of the multi-channel signal, wherein the prediction information includes a real-valued portion different from zero and/or an imaginary portion different from zero, wherein the prediction information includes an imaginary factor different from zero, wherein an imaginary part of the decoded first combination signal is estimated using a real part of the decoded first combination signal, wherein the imaginary part of the decoded first combination signal is multiplied by the imaginary factor of the prediction information when obtaining a prediction signal; wherein the prediction signal and the decoded residual signal are linearly combined to obtain a second combination signal; and wherein the second combination signal and the decoded first combination signal are combined to obtain the decoded first channel signal, and the decoded second channel signal.
According to another embodiment, a method of encoding a multi-channel audio signal having two or more channel signals may have the steps of: calculating a first combination signal and a prediction residual signal using a first channel signal and a second channel signal and prediction information, so that a prediction residual signal, when combined with a prediction signal derived from the first combination signal or a signal derived from the first combination signal and the prediction information results in a second combination signal, the first combination signal and the second combination signal being derivable from the first channel signal and the second channel signal using a combination rule; calculating the prediction information so that the prediction residual signal fulfills an optimization target; encoding the first combination signal and the prediction residual signal to obtain an encoded first combination signal and an encoded residual signal; and combining the encoded first combination signal, the encoded prediction residual signal and the prediction information to obtain an encoded multi-channel audio signal, wherein the first channel signal is a spectral representation of a block of samples; wherein the second channel signal is a spectral representation of a block of samples, wherein the spectral representations are either pure real spectral representations or pure imaginary spectral representations, wherein the prediction information is calculated as a real-valued factor different from zero and/or as an imaginary factor different from zero, a real-to-imaginary transform or an imaginary-to-real transform is performed for deriving a transform spectral representation from the first combination signal, and wherein the first combined signal and the first residual signal are calculated so that the prediction signal is derived from the transformed spectrum using the imaginary factor.
Another embodiment may have a computer program for performing, when running on a computer or a processor, the inventive methods.
The present invention relies on the finding that a coding gain of the high quality waveform coding approach can be significantly enhanced by a prediction of a second combination signal using a first combination signal, where both combination signals are derived from the original channel signals using a combination rule such as the mid/side combination rule. It has been found that this prediction information is calculated by a predictor in an audio encoder so that an optimization target is fulfilled, incurs only a small overhead, but results in a significant decrease of bit rate necessitated for the side signal without losing any audio quality, since the inventive prediction is nevertheless a waveform-based coding and not a parameter-based stereo or multi-channel coding approach. In order to reduce computational complexity, it is advantageous to perform frequency-domain encoding, where the prediction information is derived from frequency domain input data in a band-selective way. The conversion algorithm for converting the time domain representation into a spectral representation is a critically sampled process such as a modified discrete cosine transform (MDCT) or a modified discrete sine transform (MDST), which is different from a complex transform in that only real values or only imaginary values are calculated, while, in a complex transform, real and complex values of a spectrum are calculated resulting in 2-times oversampling.
A transform based on aliasing introduction and cancellation is used. The MDCT, in particular, is such a transform and allows a cross-fading between subsequent blocks without any overhead due to the well-known time domain aliasing cancellation (TDAC) property which is obtained by overlap-add-processing on the decoder side.
The prediction information calculated in the encoder, transmitted to the decoder and used in the decoder comprises an imaginary part which can advantageously reflect phase differences between the two audio channels in arbitrarily selected amounts between 0° and 360°. Computational complexity is significantly reduced when only a real-valued transform or, in general, a transform is applied which either provides a real spectrum only or provides an imaginary spectrum only. In order to make use of this imaginary prediction information which indicates a phase shift between a certain band of the left signal and a corresponding band of the right signal, a real-to-imaginary converter or, depending on the implementation of the transform, an imaginary-to-real converter is provided in the decoder in order to calculate a prediction residual signal from the first combination signal, which is phase-rotated with respect to the original combination signal. This phase-rotated prediction residual signal can then be combined with the prediction residual signal transmitted in the bit stream to re-generate a side signal which, finally, can be combined with the mid signal to obtain the decoded left channel in a certain band and the decoded right channel in this band.
To increase audio quality, the same real-to-imaginary or imaginary-to-real converter which is applied on the decoder side is implemented on the encoder side as well, when the prediction residual signal is calculated in the encoder.
The present invention is advantageous in that it provides an improved audio quality and a reduced bit rate compared to systems having the same bit rate or having the same audio quality.
Additionally, advantages with respect to computational efficiency of unified stereo coding useful in the MPEG USAC system at high bit rates are obtained, where SBR is typically not used. Instead of processing the signal in the complex hybrid QMF domain, these approaches implement residual-based predictive stereo coding in the native MDCT domain of the underlying stereo transform coder.
In accordance with an aspect of the present invention, the present invention comprises an apparatus or method for generating a stereo signal by complex prediction in the MDCT domain, wherein the complex prediction is done in the MDCT domain using a real-to-complex transform, where this stereo signal can either be an encoded stereo signal on the encoder-side or can alternatively be a decoded/transmitted stereo signal, when the apparatus or method for generating the stereo signal is applied on the decoder-side.
Embodiments of the present invention will be detailed subsequently referring to the appended drawings, in which:
a illustrates an implementation of the encoder calculator of
b illustrates an alternative implementation of the encoder calculator of
c illustrates a mid/side combination rule to be applied on the encoder side;
a illustrates an implementation of the decoder calculator of
b illustrates an alternative implementation of the decoder calculator in form of a matrix calculator;
c illustrates a mid/side inverse combination rule corresponding to the combination rule illustrated in
a illustrates an embodiment of an audio encoder operating in the frequency domain which is a real-valued frequency domain;
b illustrates an implementation of an audio decoder operating in the frequency domain;
a illustrates an alternative implementation of an audio encoder operating in the MDCT domain and using a real-to-imaginary transform;
b illustrates an audio decoder operating in the MDCT domain and using a real-to-imaginary transform;
a illustrates an audio postprocessor using a stereo decoder and a subsequently connected SBR decoder;
b illustrates a mid/side upmix matrix;
a illustrates a detailed view on the MDCT block in
b illustrates a detailed view on the MDCT−1 block of
a illustrates an implementation of an optimizer operating on reduced resolution with respect to the MDCT output;
b illustrates a representation of an MDCT spectrum and the corresponding lower resolution bands in which the prediction information is calculated;
a illustrates an implementation of the real-to-imaginary transformer in
b illustrates a possible implementation of the imaginary spectrum calculator of
The decoder calculator 116 is configured for calculating a decoded multi-channel signal having the decoded first channel signal 117 and the decoded second channel signal 118 using the decoded residual signal 114, the prediction information 108 and the decoded first combination signal 112. Particularly, the decoder calculator 116 is configured to operate in such a way that the decoded first channel signal and the decoded second channel signal are at least an approximation of a first channel signal and a second channel signal of the multi-channel signal input into a corresponding encoder, which are combined by the combination rule when generating the first combination signal and the prediction residual signal. Specifically, the prediction information on line 108 comprises a real-valued part different from zero and/or an imaginary part different from zero.
The decoder calculator 116 can be implemented in different manners. A first implementation is illustrated in
c illustrates an example for an inverse combination rule applied by the combiner 1162 in
The prediction information is generated by an optimizer 207 for calculating the prediction information 206 so that the prediction residual signal fulfills an optimization target 208. The first combination signal 204 and the residual signal 205 are input into a signal encoder 209 for encoding the first combination signal 204 to obtain an encoded first combination signal 210 and for encoding the residual signal 205 to obtain an encoded residual signal 211. Both encoded signals 210, 211 are input into an output interface 212 for combining the encoded first combination signal 210 with the encoded prediction residual signal 211 and the prediction information 206 to obtain an encoded multi-channel signal 213, which is similar to the encoded multi-channel signal 100 input into the input interface 102 of the audio decoder illustrated in
Depending on the implementation, the optimizer 207 receives either the first channel signal 201 and the second channel signal 202, or as illustrated by lines 214 and 215, the first combination signal 214 and the second combination signal 215 derived from a combiner 2031 of
An optimization target is illustrated in
Other optimization targets may relate to the perceptual quality. An optimization target can be that a maximum perceptual quality is obtained. Then, the optimizer would necessitate additional information from a perceptual model. Other implementations of the optimization target may relate to obtaining a minimum or a fixed bit rate. Then, the optimizer 207 would be implemented to perform a quantization/entropy-encoding operation in order to determine the necessitated bit rate for certain a values so that the a can be set to fulfill the requirements such as a minimum bit rate, or alternatively, a fixed bit rate. Other implementations of the optimization target can relate to a minimum usage of encoder or decoder resources. In case of an implementation of such an optimization target, information on the necessitated resources for a certain optimization would be available in the optimizer 207. Additionally, a combination of these optimization targets or other optimization targets can be applied for controlling the optimizer 207 which calculates the prediction information 206.
The encoder calculator 203 in
The combiner 2031 outputs the first combination signal 204 and a second combination signal 2032. The first combination signal is input into a predictor 2033, and the second combination signal 2032 is input into the residual calculator 2034. The predictor 2033 calculates a prediction signal 2035, which is combined with the second combination signal 2032 to finally obtain the residual signal 205. Particularly, the combiner 2031 is configured for combining the two channel signals 201 and 202 of the multi-channel audio signal in two different ways to obtain the first combination signal 204 and the second combination signal 2032, where the two different ways are illustrated in an exemplary embodiment in
The residual calculator 2034 in
a illustrates an implementation of an audio encoder. Compared to the audio encoder illustrated in
a additionally illustrates the residual calculator 2034 as an adder which receives the side signal at its “plus” input and which receives the prediction signal output by the predictor 2033 at its “minus” input. Additionally,
The predictor control information 206 is a factor as illustrated to the right in
When, however, the prediction control information only comprises a second portion which can be the imaginary part of a complex-valued factor or the phase information of the complex-valued factor, where the imaginary part or the phase information is different from zero, the present invention achieves a significant coding gain for signals which are phase shifted to each other by a value different from 0° or 180°, and which have, apart from the phase shift, similar waveform characteristics and similar amplitude relations.
A prediction control information is complex-valued. Then, a significant coding gain can be obtained for signals being different in amplitude and being phase shifted. In a situation in which the time/frequency transforms provide complex spectra, the operation 2034 would be a complex operation in which the real part of the predictor control information is applied to the real part of the complex spectrum M and the imaginary part of the complex prediction information is applied to the imaginary part of the complex spectrum. Then, in adder 2034, the result of this prediction operation is a predicted real spectrum and a predicted imaginary spectrum, and the predicted real spectrum would be subtracted from the real spectrum of the side signal S (band-wise), and the predicted imaginary spectrum would be subtracted from the imaginary part of the spectrum of S to obtain a complex residual spectrum D.
The time-domain signals L and R are real-valued signals, but the frequency-domain signals can be real- or complex-valued. When the frequency-domain signals are real-valued, then the transform is a real-valued transform. When the frequency domain signals are complex, then the transform is a complex-valued transform. This means that the input to the time-to-frequency and the output of the frequency-to-time transforms are real-valued, while the frequency domain signals could e.g. be complex-valued QMF-domain signals.
b illustrates an audio decoder corresponding to the audio encoder illustrated in
The bitstream output by bitstream multiplexer 212 in
Depending on the implementation of the system, the frequency/time converters 52, 53 are real-valued frequency/time converters when the frequency-domain representation is a real-valued representation, or complex-valued frequency/time converters when the frequency-domain representation is a complex-valued representation.
For increasing efficiency, however, performing a real-valued transform is advantageous as illustrated in another implementation in
Concerning the position of the quantization/coding (Q/C) module 2072 for a, it is noted that the multipliers 2073 and 2074 use exactly the same (quantized) α that will be used in the decoder as well. Hence, one could move 2072 directly to the output of 2071, or one could consider that the quantization of a is already taken into account in the optimization process in 2071.
Although one could calculate a complex spectrum on the encoder-side, since all information is available, it is advantageous to perform the real-to-complex transform in block 2070 in the encoder so that similar conditions with respect to a decoder illustrated in
The real-to-imaginary transformer 1160a or the corresponding block 2070 of
Specifically, as illustrated in
It has to be noted that the control information 1003 can additionally indicate to use more frames than the two surrounding frames or to, for example, only use the current frame and exactly one or more preceding frames but not using “future” frames in order to reduce the systematic delay.
Additionally, it is to be noted that the stage-wise weighted combination illustrated in
As indicated in
a illustrates a more detailed implementation of block 50 or 51. Particularly, a sequence of time domain audio samples is input into an analysis windower 500 which performs a windowing operation using an analysis window and, particularly, performs this operation in a frame by frame manner, but using a stride or overlap of 50%. The result of the analysis windower, i.e., a sequence of frames of windowed samples is input into an MDCT transform block 501, which outputs the sequence of real-valued MDCT frames, where these frames are aliasing-affected. Exemplarily, the analysis windower applies analysis windows having a length of 2048 samples. Then, the MDCT transform block 501 outputs MDCT spectra having 1024 real spectral lines or MDCT values. The analysis windower 500 and/or the MDCT transformer 501 are controllable by a window length or transform length control 502 so that, for example, for transient portions in the signal, the window length/transform length is reduced in order to obtain better coding results.
b illustrates the inverse MDCT operation performed in blocks 52 and 53. Exemplarily, block 52 comprises a block 520 for performing a frame-by-frame inverse MDCT transform. When, for example, a frame of MDCT values has 1024 values, then the output of this MDCT inverse transform has 2048 aliasing-affected time samples. Such a frame is supplied to a synthesis windower 521, which applies a synthesis window to this frame of 2048 samples. The windowed frame is then forwarded to an overlap/add processor 522 which, exemplarily, applies a 50% overlap between two subsequent frames and, then, performs a sample by sample addition so that a 2048 samples block finally results in 1024 new samples of the aliasing free output signal. Again, it is advantageous to apply a window/transform length control using information which is, for example, transmitted in the side information of the encoded multi-channel signal as indicated at 523.
The α prediction values could be calculated for each individual spectral line of an MDCT spectrum. However, it has been found that this is not necessitated and a significant amount of side information can be saved by performing a band-wise calculation of the prediction information. Stated differently, a spectral converter 50 illustrated in
The bands are shaped in a psychoacoustic way so that the bandwidth of the bands increases from lower frequencies to higher frequencies as illustrated in
For calculating the α values the high resolution MDCT spectrum is not necessitated.
Alternatively, a filter bank having a frequency resolution similar to the resolution necessitated for calculating the α values can be used as well. When bands increasing in frequency are to be implemented, then this filterbank should have varying bandwidth. When, however, a constant bandwidth from low to high frequencies is sufficient, then a traditional filter bank with equi-width sub-bands can be used.
Depending on the implementation, the sign of the a value indicated in
The modules 2070 in the encoder and 1160a in the decoder should exactly match in order to ensure correct waveform coding. This applies to the case, in which these modules use some form of approximation such as truncated filters or when it is only made use of one or two instead of the three MDCT frames, i.e. the current MDCT frame on line 60, the preceding MDCT frame on line 61 and the next MDCT frame on line 62.
Additionally, it is advantageous that the module 2070 in the encoder in
Subsequently, several aspects of embodiments of the present invention are discussed in more detail.
Standard parametric stereo coding relies on the capability of the oversampled complex (hybrid) QMF domain to allow for time- and frequency-varying perceptually motivated signal processing without introducing aliasing artifacts. However, in case of downmix/residual coding (as used for the high bit rates considered here), the resulting unified stereo coder acts as a waveform coder. This allows operation in a critically sampled domain, like the MDCT domain, since the waveform coding paradigm ensures that the aliasing cancellation property of the MDCT-IMDCT processing chain is sufficiently well preserved.
However, to be able to exploit the improved coding efficiency that can be achieved in case of stereo signals with inter-channel time- or phase-differences by means of a complex-valued prediction coefficient α, a complex-valued frequency-domain representation of the downmix signal DMX is necessitated as input to the complex-valued upmix matrix. This can be obtained by using an MDST transform in addition to the MDCT transform for the DMX signal. The MDST spectrum can be computed (exactly or as an approximation) from the MDCT spectrum.
Furthermore, the parameterization of the upmix matrix can be simplified by transmitting the complex prediction coefficient α instead of MPS parameters. Hence, only two parameters (real and imaginary part of α) are transmitted instead of three (ICC, CLD, and IPD). This is possible because of redundancy in the MPS parameterization in case of downmix/residual coding. The MPS parameterization includes information about the relative amount of decorrelation to be added in the decoder (i.e., the energy ratio between the RES and the DMX signals), and this information is redundant when the actual DMX and RES signals are transmitted.
Because of the same reason, the gain factor g, shown in the upmix matrix above, is obsolete in case of downmix/residual coding. Hence, the upmix matrix for downmix/residual coding with complex prediction is now:
Compared to Equation 1169 in
Two options are available for calculating the prediction residual signal in the encoder. One option is to use the quantized MDCT spectral values of the downmix. This would result in the same quantization error distribution as in M/S coding since encoder and decoder use the same values to generate the prediction. The other option is to use the non-quantized MDCT spectral values. This implies that encoder and decoder will not use the same data for generating the prediction, which allows for spatial redistribution of the coding error according to the instantaneous masking properties of the signal at the cost of a somewhat reduced coding gain.
It is advantageous to compute the MDST spectrum directly in the frequency domain by means of two-dimensional FIR filtering of three adjacent MDCT frames as discussed. The latter can be considered as a “real-to-imaginary” (R2I) transform. The complexity of the frequency-domain computation of the MDST can be reduced in different ways, which means that only an approximation of the MDST spectrum is calculated:
As long as the same approximation is used in the encoder and decoder, the waveform coding properties are not affected. Such approximations of the MDST spectrum, however, can lead to a reduction in the coding gain achieved by complex prediction.
If the underlying MDCT coder supports window-shape switching, the coefficients of the two-dimensional FIR filter used to compute the MDST spectrum have to be adapted to the actual window shapes. The filter coefficients applied to the current frame's MDCT spectrum depend on the complete window, i.e. a set of coefficients is necessitated for every window type and for every window transition. The filter coefficients applied to the previous/next frame's MDCT spectrum depend only on the window half overlapping with the current frame, i.e. for these a set of coefficients is necessitated only for each window type (no additional coefficients for transitions).
If the underlying MDCT coder uses transform-length switching, including the previous and/or next MDCT frame in the approximation becomes more complicated around transitions between the different transforms lengths. Due to the different number of MDCT coefficients in the current and previous/next frame, the two-dimensional filtering is more complicated in this case. To avoid increasing computational and structural complexity, the previous/next frame can be excluded from the filtering at transform-length transitions, at the price of reduced accuracy of the approximation for the respective frames.
Furthermore, special care needs to be taken for the lowest and highest parts of the MDST spectrum (close to DC and fs/2), where less surrounding MDCT coefficients are available for FIR filtering than necessitated. Here the filtering process needs to be adapted to compute the MDST spectrum correctly. This can either be done by using a symmetric extension of the MDCT spectrum for the missing coefficients (according to the periodicity of spectra of time discrete signals), or by adapting filter coefficients accordingly. The handling of these special cases can of course be simplified at the price of a reduced accuracy in vicinity of the borders of the MDST spectrum.
Computing the exact MDST spectrum from the transmitted MDCT spectra in the decoder increases the decoder delay by one frame (here assumed to be 1024 samples).
The additional delay can be avoided by using an approximation of the MDST spectrum that does not necessitate the MDCT spectrum of the next frame as an input.
The following bullet list summarizes the advantages of the MDCT-based unified stereo coding over QMF-based unified stereo coding:
Important properties of an implementation can be summarized as follows:
Additional or alternative implementation details comprise:
Embodiments relate to an inventive system for unified stereo coding in the MDCT-domain. It enables to utilize the advantages of unified stereo coding in the MPEG USAC system even at higher bit rates (where SBR is not used) without the significant increase in computational complexity that would come with a QMF-based approach.
The following two lists summarize configuration aspects described before, which can be used alternatively to each other or in addition to other aspects:
In an embodiment, the encoder additionally comprises: a spectral converter (50, 51) for converting a time-domain representation of the two channel signals to a spectral representation of the two channel signals having subband signals for the two channel signals, wherein the combiner (2031), the predictor (2033) and the residual signal calculator (2034) are configured to process each subband signal separately so that the first combined signal and the residual signal are obtained for a plurality of subbands, wherein the output interface (212) is configured for combining the encoded first combined signal and the encoded residual signal for the plurality of subbands.
Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.
In an embodiment of the present invention, a proper handling of window shape switching is applied. When
The subsequent table gives different MDST filter coefficients for a current window sequence for different window shapes and different implementations of the left half and the right half of the window.
Additionally, the window shape information 109 provides window shape information for the previous window, when the previous window is used for calculating the MDST spectrum from the MDCT spectrum. Corresponding MDST filter coefficients for the previous window are given in the subsequent table.
Hence, depending on the window shape information 109, the imaginary spectrum calculator 1001 in
The window shape information which is used on the decoder side is calculated on the encoder side and transmitted as side information together with the encoder output signal. On the decoder side, the window shape information 109 is extracted from the bitstream by the bitstream demultiplexer (for example 102 in
When the window shape information 109 signals that the previous frame had a different transform size, then it is advantageous that the previous frame is not used for calculating the imaginary spectrum from the real-valued spectrum. The same is true when it is found by interpreting the window shape information 109 that the next frame has a different transform size. Then, the next frame is not used for calculating the imaginary spectrum from the real-valued spectrum. In such a case when, for example, the previous frame had a different transform size from the current frame and when the next frame again has a different transform size compared to the current frame, then only the current frame, i.e. the spectral values of the current window, are used for estimating the imaginary spectrum.
The prediction in the encoder is based on non-quantized or quantized frequency coefficients such as MDCT coefficients. When the prediction illustrated by element 2033 in
In an embodiment, the real part and the imaginary part of the complex prediction coefficient per prediction band are quantized and encoded directly, i.e. without for example MPEG Surround parameterization. The quantization can be performed using a uniform quantizer with a step size, for example, of 0.1. This means that any logarithmic quantization step sizes or the like are not applied, but any linear step sizes are applied. In an implementation, the value range for the real part and the imaginary part of the complex prediction coefficient ranges from −3 to 3, which means that 60 or, depending on implementational details, 61 quantization steps are used for the real part and the imaginary part of the complex prediction coefficient.
The real part applied in multiplier 2073 in
In a further embodiment of the present invention, a cheap signaling in case all prediction coefficients are real is applied. It can be the situation that all prediction coefficients for a certain frame, i.e. for the same time portion of the audio signal are calculated to be real. Such a situation may occur when the full mid signal and the full side signal are not or only little phase-shifted to each other. In order to save bits, this is indicated by a single real indicator. Then, the imaginary part of the prediction coefficient does not need to be signaled in the bitstream with a codeword representing a zero value. On the decoder side, the bitstream decoder interface, such as a bitstream demultiplexer, will interpret this real indicator and will then not search for codewords for an imaginary part but will assume all bits being in the corresponding section of the bitstream as bits for real-valued prediction coefficients. Furthermore, the predictor 2033, when receiving an indication that all imaginary parts of the prediction coefficients in the frame are zero, will not need to calculate an MDST spectrum, or generally an imaginary spectrum from the real-valued MDCT spectrum. Hence, element 1160a in the
The complex stereo prediction in accordance with embodiments of the present invention is a tool for efficient coding of channel pairs with level and/or phase differences between the channels. Using a complex-valued parameter a, the left and right channels are reconstructed via the following matrix. dmxIm denotes the MDST corresponding to the MDCT of the downmix channels dmxRe.
The above equation is another representation, which is split with respect to the real part and the imaginary part of a and represents the equation for a combined prediction/combination operation, in which the predicted signal S is not necessarily calculated.
The following data elements are used for this tool:
These data elements are calculated in an encoder and are put into the side information of a stereo or multi-channel audio signal. The elements are extracted from the side information on the decoder side by a side information extractor and are used for controlling the decoder calculator to perform a corresponding action.
Complex stereo prediction necessitates the downmix MDCT spectrum of the current channel pair and, in case of complex_coef=1, an estimate of the downmix MDST spectrum of the current channel pair, i.e. the imaginary counterpart of the MDCT spectrum. The downmix MDST estimate is computed from the current frame's MDCT downmix and, in case of use_prev_frame=1, the previous frame's MDCT downmix. The previous frame's MDCT downmix of window group g and group window b is obtained from that frame's reconstructed left and right spectra.
In the computation of the downmix MDST estimate, the even-valued MDCT transform length is used, which depends on window_sequence, as well as filter_coefs and filter_coefs_prev, which are arrays containing the filter kernels and which are derived according to the previous tables.
For all prediction coefficients the difference to a preceding (in time or frequency) value is coded using a Huffman code book. Prediction coefficients are not transmitted for prediction bands for which cplx_pred_used=0.
The inverse quantized prediction coefficients alpha_re and alpha_im are given by
alpha—re=alpha—q—re*0.1
alpha—im=alpha—q—im*0.1
It is to be emphasized that the invention is not only applicable to stereo signals, i.e. multi-channel signals having only two channels, but is also applicable to two channels of a multi-channel signal having three or more channels such as a 5.1 or 7.1 signal.
The inventive encoded audio signal can be stored on a digital storage medium or can be transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.
Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed.
Some embodiments according to the invention comprise a non-transitory or tangible data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.
Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.
Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.
In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.
A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein.
A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may for example be configured to be transferred via a data communication connection, for example via the Internet.
A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.
A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
In some embodiments, a programmable logic device (for example a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are performed by any hardware apparatus.
While this invention has been described in terms of several advantageous embodiments, there are alterations, permutations, and equivalents which fall within the scope of this invention. It should also be noted that there are many alternative ways of implementing the methods and compositions of the present invention. It is therefore intended that the following appended claims be interpreted as including all such alterations, permutations, and equivalents as fall within the true spirit and scope of the present invention.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10169432.1 | Jul 2010 | EP | regional |
This application is a continuation of copending International Application No. PCT/EP2011/054485, filed Mar. 23, 2011, which is incorporated herein by reference in its entirety, and additionally claims priority from U.S. Applications Nos. 61/322,688, filed Apr. 9, 2010, 61/363,906, filed Jul. 13, 2010 and European Application 10169432.1-2225, filed Jul. 13, 2010, which are all incorporated herein by reference in their entirety.
| Number | Date | Country | |
|---|---|---|---|
| 61322688 | Apr 2010 | US | |
| 61363906 | Jul 2010 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/EP2011/054485 | Mar 2011 | US |
| Child | 13645700 | US |
