The invention relates to the field of audio signal coding. Especially, the invention relates to efficient audio coding adapted for low bit rates. More specifically, the invention relates to scalable audio coding. The invention relates to an encoder, a decoder, methods for encoding and decoding, an encoded audio signal, storage and transmission media with data representing such encoded signal, and devices with an encoder and/or decoder.
Within low bit rate audio coding often the available bit rate is too low to model an entire spectrum of an audio signal with a deterministic type of encoder, such as a sinusoidal or a waveform encoder. Two approaches have been used to overcome this problem.
According to one approach bandwidth of the signal to be modeled is limited such that the available bit rate is sufficient to model the limited bandwidth with the deterministic encoder. A disadvantage of this approach is that the necessary bandwidth limitation is effectively a reduction in audio quality.
According to a second approach the entire bandwidth is modeled. Part of the signal is modeled with the deterministic encoder using a large portion of the available bit rate and the remaining parts of the audio signal are modeled with noise. This often leads to reasonable results because the perceived bandwidth and timbre of the original audio signal is nearly maintained. However, regarding the second mentioned approach a problem is to determine how the noise signal should be generated.
When a sinusoidal encoder is used as a deterministic encoder, often a residual signal, i.e. a signal that is left after subtracting the sinusoidal components in each audio segment, is used as a basis for estimating noise parameters. Many advanced encoders prepare the residual signal before noise parameter estimation to overcome some artefacts such as an overly noisy sound quality of the decoded signal or low frequency artefacts due to poor spectral resolution of the noise encoder. An example on such approach is seen in WO 2004049311.
When a waveform encoder is used, e.g. a transform encoder, the encoder decides which audio bands should not or can not be modeled by the transform encoder. Information about these omitted bands is then transmitted so as to allow the decoder to generate noise accordingly.
The above described methods suffer from the disadvantage that already at the encoder side final decisions have to be made about the noise signal that is going to be generated at the decoder side. As a consequence, it is not permitted that parameters or data for the deterministic part of the decoder are changed once the signal has been encoded. This may happen for example during transmission of the encoded signal or during fast rescaling of a compressed audio file where certain layers of information are dropped. If this is done, the consequence will be that, at the decoder side, the generated noise signal will not match the resulting signal from the deterministic decoder part and considerable audible artefacts can be the result. In other words, noise coding according to the described principles is not scalable because it does not allow modifications to the deterministic signal after noise parameters have been estimated.
It may be seen as an object of the present invention to provide a method and an audio encoder and decoder providing a scalable encoding, i.e. allowing modifications of the encoded signal prior to decoding, without considerable audible artefacts of the resulting decoded signal.
According to a first aspect of the invention, this object is complied with by providing an audio encoder adapted to encode an audio signal, the audio encoder comprising:
encoder means adapted to encode the audio signal into a first encoded signal part,
computation means adapted to compute a representation of an excitation pattern of the audio signal and provide it as a second encoded signal part, the computation means further being adapted to compute a representation of a masking curve based on the representation of the excitation pattern, and provide the representation of the masking curve to the encoder means so as to optimize encoding efficiency.
By the term ‘excitation pattern’ is understood spectral energy distribution across auditory filters in the human auditory system, see also [1] (referring to the list of references at the end of the section “Description of preferred embodiments”). An excitation pattern is a representation of the human basilar membrane or human auditory nerve response to an audio signal. This response can be modeled by a filter bank of e.g. 40 parallel auditory filters. Thus, a representation of the excitation pattern comprising 40 values each of which relate to a signal level of a frequency band of an auditory filter, is considered an appropriate model of the human auditory system. Thus, the excitation pattern of an audio signal is a parametric spectral description of the audio signal. By a representation of e.g. 40 values, that are correlated due to the spectral overlap of the auditory filter shapes, the inclusion of the excitation pattern is quite inexpensive in terms of amount of data to be included in the encoded audio signal if for example differential encoding is used. Depending on e.g. target frequency range, the excitation pattern may be represented by fewer than 40 values, such as 30 values, such as 20 values, or even fewer.
By ‘masking curve’ related to an audio signal is understood a spectral representation of the human hearing threshold given the audio signal as input to the human auditory system. With respect to encoding precision this is important since it provides the encoder means with information that possible distortion or noise products added to the original signal are not perceivable as long as these products do not exceed the masking curve. Thus, encoding of e.g. sinusoidal amplitudes or transform coefficients can be performed avoiding unnecessary bit allocation for details of the original signal that can not be perceived e.g. by encoding signal components relative to the masking curve. Hereby, the masking curve representation helps to improve encoding efficiency of the encoder means.
The audio encoder according to the first aspect provides a scalable encoded signal due to the inclusion of the second encoded signal part, i.e. the inclusion of the excitation pattern of the original audio signal in an output bit stream of the encoder. Thus, since a decoder receiving the encoded signal is provided with information regarding the excitation pattern of the original signal, it is possible to add an appropriate signal, for instance noise, to a first decoded signal part so as to generate a resulting signal exhibiting an excitation pattern nearly identical to that of the original signal. As a result the perceived timbre of the reproduced signal will resemble the original signal, and thus a crucial parameter relating to overall sound quality is ensured.
Perceptually, recreating the original excitation pattern is an appropriate perceptual target because the excitation pattern describes an energy distribution across different auditory filters and as such comprises no more and no less spectral envelope information than necessary for reconstruction of he original spectrum envelope appropriately. It should be noted, though, that the excitation pattern does not include all perceptually relevant information. Temporal structure of an audio signal is generally not captured within the excitation pattern. As far as this temporal information is perceptually relevant it is assumed that in part this is modeled with the encoder means, and as such included in the first encoded signal part. However, the excitation pattern encoder can also encode temporal information in two ways. First, by regular update of the excitation parameters. Second, by using a temporal envelope including required temporal information to modulate the signal to be added to the first decoded signal part.
Another advantage of including the excitation pattern of the original audio signal in the encoded bit stream is that it provides convenient information for easy computation of a representation of a corresponding masking curve of the original signal—both at the encoder and the decoder side. Knowledge of the masking curve is important with respect to coding efficiency of the first encoded signal part since the masking curve comprises information that enables the encoder to decide whether certain parts of parameter values can be omitted since they will not be perceived by a listener in the final signal due to masking by the human auditory system. Preferably, the representation of the masking curve is computed based on a quantized representation of the excitation pattern at the encoder side. Hereby, it is ensured that identically the same masking curve is available at the encoder and the decoder side.
Preferably the audio encoder means comprises a deterministic signal type of encoder selected from the group consisting of: parametric encoders (e.g. a sinusoidal encoder), transform encoders, waveform encoders, Regular Pulse Excitation encoders, and Codebook Excited Linear Predictive encoders.
A second aspect of the invention provides an audio decoder adapted to regenerate an audio signal from an encoded audio signal, the audio decoder comprising:
means adapted to generate, from a second encoded audio signal part, a representation of an excitation pattern of the audio signal,
decoder means adapted to generate a first decoded signal part from a first encoded signal part,
signal generator means adapted to generate a second decoded signal part, so that a sum of the first and second decoded signal parts exhibits an excitation pattern being substantially equal to the excitation pattern of the audio signal.
For the purpose of creating a decoded audio signal with perceivably spectral properties similar to the original signal, the excitation pattern of the original signal is compared to an excitation pattern of a decoded first encoded signal part. A possible deviation will be compensated by the decoder by adding an appropriate signal so that at least the resulting signal will be similar to the original audio signal with respect to excitation pattern. Thus, the decoder does not need to comprise decoding means being exactly inverse to the encoder means.
Preferably, the decoder comprises means for providing a sum of the first and second decoded signal parts as a representation of the original audio signal.
Preferably, the decoder means comprises a deterministic signal type of decoder selected from the group consisting of: parametric decoders (e.g. a sinusoidal encoder), transform decoders, waveform decoder, Regular Pulse Excitation encoders, and Codebook Excited Linear Predictive encoders.
The decoder means may utilize a representation of the masking curve based on the original audio signal that was used in the encoder. This masking curve is conveniently based on the representation of the excitation pattern extracted from the second decoded signal part.
The signal generator means may comprise a noise generator or spectral band replication means or a combination thereof. Preferably, the signal generator comprises means to generate the second decoded signal part based on the representation of the excitation pattern by using an iterative method.
In a third aspect the invention provides a method of encoding an audio signal, comprising the steps of:
computing a representation of an excitation pattern of the audio signal,
computing a representation of a masking curve based on the representation of the excitation pattern,
encoding the audio signal according to an encoding scheme into a first encoded signal part by utilizing the masking curve, and
providing a second encoded signal part comprising the representation of the excitation pattern of the audio signal.
The same explanation applies as for the first aspect.
In a fourth aspect the invention provides a method of regenerating an audio signal from an encoded audio signal, the method comprising the steps of:
generating from a second encoded signal part, a representation of an excitation pattern of the audio signal,
generating from the representation of the excitation pattern, a representation of a masking curve,
decoding a first encoded signal part, according to a decoding scheme, into a first decoded signal part,
generating a second decoded signal part, based on the representation of the excitation pattern, so that a sum of the first and second decoded signal parts exhibits an excitation pattern substantially equal to the excitation pattern of the audio signal.
The same explanation applies as for the second aspect.
In a fifth aspect the invention provides an encoded audio signal representing an original audio signal, the encoded signal comprising a first part comprising a first encoded signal part, and a second part comprising a representation of an excitation pattern of the audio signal.
The encoded signal may be a digital electrical signal with a format according to standard digital audio formats. The signal may be transmitted using an electrical connecting cable between two audio devices. However, the encoded signal could be a wireless signal, such as an air-borne signal using a radio frequency carrier, or it may be an optical signal adapted for transmission using an optical fiber.
In a sixth aspect the invention provides a storage medium comprising data representing an encoded audio signal according to the fifth aspect. The storage medium is preferably a standard audio data storage medium such as DVD, DVD+r, DVD+rw, DVD-r, DVD-rw, CD, CD-r, CD-rw, read-writable CD, compact flash, memory stick etc. However, it may also be a computer data storage medium such as a computer hard disk, a computer memory, a solid-state device, a floppy disk etc.
In a seventh aspect the invention provides a device comprising an audio encoder according to the first aspect.
In an eighth aspect the invention provides a device comprising an audio decoder according to the second aspect.
Preferred devices according to the seventh and eighth aspects are all different types of tape, disk, or memory based audio recorders and players. For example: Portable audio devices, car CD players, DVD players, audio processors for computers etc. In addition, it may be advantageous for mobile phones.
In the following the invention is described in more details with reference to the accompanying figures of which:
While the invention is susceptible to various modifications and alternative forms, specific embodiments have been shown by way of example in the drawings and will be described in detail herein. It should be understood, however, that the invention is not intended to be limited to the particular forms disclosed. Rather, the invention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention as defined by the appended claims.
In the embodiment of
Preferably, the excitation pattern is calculated for short segments of the input signal in such a way that changes over time in the excitation pattern can be tracked. The excitation pattern is applied to the bit stream encoder BSE and is thus included in the output bit stream OUT.
The audio encoder comprises a masking curve computation unit MCC adapted to receive the excitation pattern computed by the excitation pattern computation means EPC. A masking curve computed by the masking curve computation unit MCC based on the excitation pattern is applied to the encoder means ENC. The encoder means ENC is adapted to improve its encoding efficiency based on the masking curve since the masking curve informs the encoder means about parts of the audio input signal IN that need not be encoded since they will be masked by the human auditory system and thus are not perceivable in the final signal. Additionally, encoding of the parameters of the first encoded signal part can be performed e.g. relative to the masking curve, thus avoiding unnecessary bit allocation. Preferably the masking curve is computed in accordance with [2]. Further details regarding masking curve computation are given below.
The first encoded signal part is applied to decoder means DEC that preferably comprises a deterministic type of decoder, such as a sinusoidal or a transform decoder. The decoder means DEC is necessarily of the same type as the encoder that produced the first encoded signal part. However, it may be the case that in the decoder a downscaled version of the bit stream/parameters is received than originally transmitted or available at the encoder. The decoder means DEC generates a first decoded signal part in response to the first encoded signal part.
The second encoded signal part, i.e. the excitation pattern of the original audio signal, is applied to a signal generator, in this preferred embodiment illustrated as a noise modeler NM. The first decoded signal part is also applied to the noise modeler NM that generates a second decoded signal part in response. The noise modeler NM is adapted to generate the second decoded signal part, i.e. a noise signal, so that a sum of the first and second decoded signal parts forms a representation of the original audio signal and exhibits an excitation pattern deviating only insignificantly from the excitation pattern of the original audio signal. Further details in this regards are given below.
The first and second decoded signal parts are applied to summation means SUM adapted to add the first and second decoded signal parts so as to generate an output signal OUT being a decoded representation of the encoded audio signal received in the input bit stream IN and thus being a representation of the original audio signal.
The audio decoder further comprises a masking curve computation unit MCC adapted to receive the second encoded signal part, i.e. the original signal excitation pattern. In response the masking curve computation unit MCC applies to the decoder means DEC a masking curve representation based on the original excitation pattern. This masking curve representation is used by the decoder DEC to decode the first encoded signal part, if encoding of the parameters of the first encoded signal part was performed e.g. using the masking curve, thus avoiding unnecessary bit allocation.
In the following an audio encoder embodiment scheme as shown in
A first step by encoding the audio input signal IN is to estimate the excitation pattern. This estimation is preferably based on a perceptual model described in [2]. In [2] it is found that a masking function ν(ƒm) is given by:
where θm is a frequency for which a masking curve is calculated, ƒ is a frequency of a component in a masker spectrum, {circumflex over (L)} is an effective duration of an audio segment under evaluation, Hom is an assumed filtering in the human outer and middle ear, γi is a transfer function of the i-th gamma tone filter modeling the human auditory filter function, m is a spectrum of the original audio input signal, while Ca and Cs are calibration constants.
The excitation pattern is defined by the following quantity:
This excitation pattern has an index i specifying an auditory filter number. In general, the number of auditory filters can be limited to about 40 values, and therefore a relatively inexpensive representation is obtained of the spectrum of the original input audio signal. Each of the excitation parameters, Ei, needs to be quantized before encoding is possible. A logarithmic quantization is preferred. Preferably, a step size between 0.5 dB and 5 dB is used, more preferably the step size is about 2 dB. Resulting quantized parameters are denoted Eqi.
Once the excitation pattern is known, the masking curve is also known, as can be seen from Eq. (1), where the denominator comprises an expression equal to the i-th excitation pattern parameter and the numerator does not depend on the input signal. Thus, Eq. (1) can be rewritten to:
Preferably the quantized excitation parameters are used for generating the masking curve. This ensures that the masking curve used by the encoder will be identical to the one used by the decoder, since the masking curve computed at the decoder side necessarily is based on the quantized excitation parameters received in the second encoded signal part.
The encoding of the excitation pattern parameters Eqi by the bit stream encoder BSE can be done efficiently by using intra-frame differential encoding. By defining EΔqi=Eq(i+1)−Eqi a suitable set of differential parameters can be obtained that do not vary much and in this case additional time-differential encoding may be used for some of the frames.
In the encoder embodiment with a sinusoidal encoder, part of the input audio signal IN is modeled with sinusoids. The sinusoidal parameters can be encoded more effectively by use of the masking curve. There are several ways to benefit from the information contained in the masking curve. One method is to divide all sinusoidal amplitude values by the masking curve. By performing this transformation, entropy of the amplitude parameters will decrease because the distribution of amplitude values is compacted considerably by the masking curve division.
An alternative method of gaining benefit from it is to utilize the masking curve in a high rate quantization scheme such as proposed in [4]. Note that alternatively, when a transform encoder is used for encoding a deterministic signal part, some techniques (see e.g. [5]) weight the transform coefficients by the masking function before encoding the transform coefficients. At the decoder side an inverse transformation is performed. The weighting curve effectively removes the need for encoding side information specifying scaling of transform coefficients.
The decoding process starts with decoding the excitation pattern parameters. Using Eq. (3) the masking curve can be derived which is made available to the decoder means DEC in its decoding of the first encoded signal part.
The noise modeler NM generates a noise signal in response to the excitation pattern and the first decoded signal part. Various algorithms exist that can be used for synthesizing a noise signal such that this noise signal together with the first decoded signal part has an excitation pattern similar to the original audio signal. In the following one method will be described that yields good results with a relatively low computational complexity.
Assuming that the length of the analysis and the synthesis segment is M, where M is an even number, then in the spectral representation of synthesis segment the first ½M complex numbers define the complete signal because it is known that the time-domain signal is real. The ½M numbers are partitioned in L noise bands with a bandwidth proportional to Equivalent Rectangular Bandwidth (ERB) such as proposed in [6]. The L start positions of each noise band are denoted kj. In addition, kj+1 is the end position plus one of the last noise band.
A spreading matrix G is defined as:
The spreading matrix defines how the energy within each noise band j is distributed across auditory filters i. Based on the spreading matrix a backward spreading matrix is defined as:
The algorithm will now try to find energy values Xj for each noise band such that
is as close as possible to the excitation pattern Eqi of the original signal for each i. Note that Edi is the excitation pattern of the first encoded signal part, and bi, bi≧1, is a factor adapted to compensate for the effects of quantization in the first and second encoded signal parts which could lead to an excess of noise that is generated by the decoder. A good value for bi has been found to be 1.3, however, a dependence on the chosen quantization scheme and on i, with larger values for small i's (i.e. low frequencies) may lead to improved results. For bi=1 no compensation is made.
The following 6 steps define a preferred iterative method of finding a suitable solution for Xj:
Preferably a stop criterion for this iterative method is chosen so that the iteration stops after all cj values are close enough to unity or alternatively after a fixed number of iterations. It the latter is chosen as stop criterion a total of 20 iterations has been found to be enough to yield a good quality noise signal.
The energy values Xj are now applied to the spectral representation of a noise signal W such that for each energy band j:
An inverse discrete Fourier Transform is used to convert this signal to the time domain. This is followed by a scaling, windowing, and overlap-add to allow for the final construction of the noise signal that is ready to be added to the first decoded signal part.
The above described embodiment using a sinusoidal encoder to generate the first encoded signal part has been tested at a sampling frequency of 44.1 kHz using a segment length M=2048 and a 50% overlap between segments. When only intra-frame differential encoding of the excitation pattern parameters is used, a bit rate of 9-10 kbps is required to represent the excitation pattern, i.e. the second encoded signal part.
In combination with the sinusoidal encoder/decoder a good audio quality can be obtained where generally the noise is integrated well with the deterministic signal part from the sinusoidal decoder. The noise model has been proven to be scalable. Independent of the number of sinusoids that were used in the sinusoidal decoder the same excitation pattern could be transmitted and a suitable noise signal could be generated at the decoder side to complement the sinusoidal signal part.
Encoders and decoders according to the invention may be implemented on a single chip with a digital signal processor. The chip may then be built into devices such as audio devices. The encoders and decoders may alternatively be implemented purely by algorithms running on a main signal processor of the application device.
In addition to coding efficiency in terms of bit rate, the described coding methods provide a high efficiency also with respect to computational load to be carried out by the encoder.
Number | Date | Country | Kind |
---|---|---|---|
04103940.5 | Aug 2004 | EP | regional |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/IB05/52483 | 7/25/2005 | WO | 2/12/2007 |