The present invention relates to predictive encoding and decoding of a signal, more particularly it relates to predictive encoding and decoding of a signal representing sound, such as speech, audio, or video.
Real-time transmissions over packet switched networks, such as speech, audio, or video over Internet Protocol based networks (mainly the Internet or Intranet networks), has become increasingly attractive due to a number of features. These features include such things as relatively low operating costs, easy integration of new services, and one network for both non-real-time and real-time data. Real-time data, typically a speech, an audio, or a video signal, in packet switched systems is converted into a digital signal, i.e. into a bitstream, which is divided in portions of suitable size in order to be transmitted in data packets over the packet switched network from a transmitter end to a receiver end.
As packet switched networks originally were designed for transmission of non-real-time data, transmissions of real-time data over such networks causes some problems. Data packets can be lost during transmission, as they can be deliberately discarded by the network due to congestion problems or transmission errors. In non-real-time applications this is not a problem since a lost packet can be retransmitted. However, retransmission is not a possible solution for real-time applications that are delay sensitive. A packet that arrives too late to a real-time application cannot be used to reconstruct the corresponding signal since this signal already has been, or should have been, delivered to the receiving end, e.g. for playback by a speaker or for visualization on a display screen. Therefore, a packet that arrives too late is equivalent to a lost packet.
When transferring a real-time signal as packets, the main problem with lost or delayed data packets is the introduction of distortion in the reconstructed signal. The distortion results from the fact that signal segments conveyed by lost or delayed data packets cannot be reconstructed.
When transferring a signal it is most often desired to use as little bandwidth as possible. As is well known, many signals have patterns containing redundancies. Appropriate coding methods can avoid the transmission of the redundant information thereby enabling a more bandwidth effective transmission of the signal. Typical coding methods taking advantage of such redundancies are predictive coding methods. A predictive coding method encodes a signal pattern based on dependencies between the pattern representations. It encodes the signal for transmission with a fixed bit rate and with a tradeoff between the signal quality and the transmitted bit rate. Examples of predictive coding methods used for speech are Linear Predictive Coding (LPC) and Code Excited Linear Prediction (CELP), which both coding methods are well known to a person skilled in the art.
In a predictive coding scheme a coder state is dependent on previously encoded parts of the signal. When using predictive coding in combination with packetization of the encoded signal, a lost packet will lead to error propagation since information on which the predictive coder state at the receiving end is dependent upon will be lost together with the lost packet. This means that decoding of a subsequent packet will start with an incorrect coder state. Thus, the error due to the lost packet will propagate during decoding and reconstruction of the signal.
One way to solve this problem of error propagation is to reset the coder state at the beginning of the encoded signal part included by a packet. However, such a reset of the coder state will lead to a degradation of the quality of the reconstructed signal. Another way of reducing the effect of a lost packet is to use different schemes for including redundancy information when encoding the signal. In this way the coder state after a lost packet can be approximated. However, not only does such a scheme require more bandwidth for transferring the encoded signal, it furthermore only reduces the effect of the lost packet. Since the effect of a lost packet will not be completely eliminated, error propagation will still be present and result in a perceptually lower quality of the reconstructed signal.
Another problem with state of the art predictive coders is the encoding, and following reconstruction, of sudden signal transitions from a relatively very low to a much higher signal level, e.g. during a voicing onset of a speech signal. When coding such transitions it is difficult to make the coder states reflect the sudden transition, and more important, the beginning of the voiced period following the transition. This in turn will lead to a degraded quality of the reconstructed signal at a decoding end.
An object of the present invention is to overcome at least some of the above-mentioned problems in connection with predictive encoding/decoding of a signal which is transmitted in packets.
Another object is to enable an improved performance at a decoding end in connection with predictive encoding/decoding when a packet with an encoded signal portion transmitted from an encoding end is lost before being received at the decoding end.
Yet another object is to improve the predictive encoding and decoding of a signal which undergoes a sudden increase of its signal power.
According to the present invention, these objects are achieved by methods, apparatuses and computer-readable mediums having the features as defined in the appended claims and representing different aspects of the invention.
According to the invention, a signal is divided into blocks and then encoded, and eventually decoded, on a block by block basis. The idea is to provide predictive encoding/decoding of a block so that the encoding/decoding is independent on any preceding blocks, while still being able to provide predictive encoding/decoding of a beginning end of the block in such way that a corresponding part of the signal can be reproduced with the same level of quality as other parts of the signal. This is achieved by basing the encoding and the decoding of a block on a coded start state located somewhere between the end boundaries of the block. The start state is encoded/decoded using any applicable coding method. A second block part and a third block part, if such a third part is determined to exist, on respective sides of the start state and between the block boundaries are then encoded/decoded using any predictive coding method. To facilitate predictive encoding/decoding of both block parts surrounding the start state, and since encoding/decoding of both of these parts will be based on the same start state, the two block parts are encoded/decoded in opposite directions with respect to each other. For example, the block part located at the end part of the block is encoded/decoded along the signal pattern as it occurs in time, while the other part located at the beginning of the block is encoded/decoded along the signal pattern backwards in time, from later occurring signal pattern to earlier occurring signal pattern.
By encoding the block in three stages in accordance with the invention, coding independency between blocks is achieved and proper predictive encoding/decoding of the beginning end of the block always facilitated. The three encoding stages are:
Correspondingly, decoding of an encoded block is performed in three stages when reproducing a corresponding decoded signal block.
The signal subject to encoding in accordance with the present invention either corresponds to a digital signal or to a residual signal of an analysis filtered digital signal. The signal comprises a sequential pattern which represents sound, such as speech or audio, or any other phenomena that can be represented as a sequential pattern, e.g. a video or an ElectroCardioGram (ECG) signal. Thus, the present invention is applicable to any sequential pattern that can be coded so as to be described by consecutive states that are correlated with each other.
Preferably, the encoding/decoding of the start state uses a coding method which is independent of previous parts of the signal, thus making the block self-contained with respect to information defining the start state. However, when the invention is applied in the LPC residual domain, predictive encoding/decoding is preferably used also for the start state. By the assumption that the quantization noise in the decoded signal prior to the beginning of the start state can be neglected, the error weighting or error feedback filter of a predictive encoder can be started from a zero state. Hereby the self-contained coding of the start state is achieved.
Preferably, the signal block is divided into a set of consecutive intervals and the start state chosen to correspond to one or more consecutive intervals of those intervals that have the highest signal energy. This means that encoding/decoding of the start state can be optimized towards a signal part with relatively high signal energy. In this way an encoding/decoding of the rest of the block is accomplished which is efficient from a perceptual point of view since it can be based on a start state which is encoded/decoded with a high accuracy.
An advantage of the present invention is that it enables the predictive coding to be performed in such way that the coded block will be self-contained with respect to information in the excitation domain, i.e. the coded information will not be correlated with information in any previously encoded block. Consequently, at decoding, the decoding of the encoded block is based on information self-contained in the encoded block. This means that if a packet carrying an encoded block is lost during transmission, the predictive decoding of subsequent encoded blocks in subsequent received packets will not be affected by lost state information in the lost packet.
Thus, the present invention avoids the problem of error propagation that conventional predictive coding/decoding encounter during decoding when a packet carrying an encoded block is lost before reception at the decoding end. Accordingly, a codec applying the features of the present invention will become more robust to packet loss.
Preferably, the start state is chosen so as to be located in the part of the block which is associated with the highest signal power. For example, in a speech signal composed of voiced and unvoiced parts, this implies that the start state will be located well within the voiced part in a block including an unvoiced and a voiced part.
In a speech signal, high correlation exists between signal samples within a voiced part and low correlation between signal samples within an unvoiced part. The correlation in the transition region between an unvoiced part and a voiced part, and vice versa, is minor and difficult to exploit. From a perceptual point of view it is more important to achieve a good waveform matching when reproducing a voiced part of the signal, whereas the waveform matching for an unvoiced part is less important.
Conventional predictive coders operate on the signal representations in the same order as that with which the corresponding signal is produced by the signal source. Thus, any coder state representing the signal at a certain time will be correlated with previous coder states representing earlier parts of the signal. Due to the difficulties of exploiting any correlation during a transition from an unvoiced period to a voiced period, the coder states for conventional predictive coders will during the beginning of a voiced period following such a transition include information which gives a quite poor approximation of the original signal. Consequently, the regeneration of the speech signal at the decoding end will provide a perceptually degraded signal for the beginning of the voiced region.
By placing the start state well within a voiced region of a block, and then encoding/decoding the block from the start state towards the end boundaries, the present invention is able to more fully exploit the high correlation in the voiced region to the benefit for the perception. The transition from unvoiced to highly periodic voiced sound takes a few pitch periods. When placing the start state well within a voiced region of a block, the high bit rate of the start state encoding will be applied in a pitch cycle where high periodicity has been established, rather than in one of the very first pitch cycles of the voiced region.
The above mentioned and further features of, and advantages with, the present invention, will be more fully described from the following description.
The encoding and decoding functionality according to the invention is typically included in a codec having an encoder part and a decoder part. With reference to
In
In
The essence of the codec is linear predictive coding (LPC) as is well known from adaptive predictive coding (APC) and code excited linear prediction (CELP). A codec according to the present invention, however, uses a start state, i.e., a sequence of samples localized within the signal block to initialize the coding of the remaining parts of the signal block. The principle of the invention complies with an open-loop analysis-synthesis approach for the LPC as well as the closed-loop analysis-by-synthesis approach, which is well known from CELP. An open-loop coding in a perceptually weighted domain, provides an alternative to analysis-by-synthesis to obtain a perceptual weighting of the coding noise. When compared with analysis-by-synthesis this method provides an advantageous compromise between voice quality and computational complexity of the proposed scheme. The open-loop coding in a perceptually weighted domain is described later in this description.
Encoder
In the embodiment of
In principle any method can be used to extract a spectral envelope from the signal block without diverging from the spirit of the invention. One method is outlined as follows: For each input block, the encoder does a number, e.g. two, linear-predictive coding (LPC) analysis, each with an order of e.g. 10. The resulting LPC coefficients are encoded, preferably in the form of line spectral frequencies (LSF). The encoding of LSF's is well known to a person skilled in the art. This encoding may exploit correlations between sets of coefficients, e.g., by use of predictive coding for some of the sets. The LPC analysis may exploit different, and possibly non-symmetric window functions in order to obtain a good compromise between smoothness and centering of the windows and lookahead delay introduced in the coding. The quantized LPC representations can advantageously be interpolated to result in a larger number of smoothly time varying sets of LSF coefficients. Subsequently the LPC residual is obtained using the quantized and smoothly interpolated LSF coefficients converted into coefficients for an analysis filter.
An example of a residual signal block 315 and its partition into sub-blocks 316, 317, 318, 319, 320 and 321 is illustrated in
Encoding of Start State
Without diverging from the spirit of the invention, the start state can be encoded with basically any encoding method.
According to an embodiment of the invention scalar quantization with predictive noise shaping is used, as illustrated in
Any noise shaping weighting filter 540 and 560 can be applied in this embodiment. Advantageously the same noise shaping is applied in the encoding of the start state as in the subsequent encoding of the remaining signal block, described later. As an example, the noise shaping can be implemented by minimizing the quantization error after weighting it with a weighting filter equal to A(z/L1)/(Aq(z)*A(z/L2)), where A(z) is the unquantized LPC analysis filter after a possible initial bandwidth expansion, Aq(z) is the quantized LPC analysis filter, and L1 and L2 are bandwidth expansion coefficients, which can advantageously be set to L1=0.8 and L2=0.6, respectively. All LPC and weighting coefficients needed in this filtering is in
Below follows a c-code example implementation of a start state encoder
Decoding of Start State
The Decoding of the start state follows naturally from the method applied in the encoding of the start state. A decoding method corresponding to the encoding method of
Encoding from the Start State Towards the Block Boundaries
Within the scope of the invention the remaining samples of the block can be encoded in a multitude of ways that all exploit the start state as an initialization for the state of the encoding algorithm. Advantageously, a linear predictive algorithm can be used for the encoding of the remaining samples. In particular, the application of an adaptive codebook enables an efficient exploitation of the start state during voiced speech segments. In this case, the encoded start state is used to populate the adaptive codebook. Also an initialization of the state for error weighting filters is advantageously done using the start state. The specifics of such initializations can be done in a multitude of ways well known by a person skilled in the art.
The encoding from the start state towards the block boundaries is exemplified by the signals in
In an embodiment based on sub-blocks for which the start state is identified as an interval of a predefined length towards one end of an interval defined by a number of sub-blocks, it is advantageous to first apply the adaptive codebook algorithm on the remaining interval to reach encoding of the entire interval defined by a number of sub-blocks. As example, the start state 715, which is an example of the signal 645 and which is a decoded representation of the start state target 325, is extended to an integer sub-block length start state 725. Thereafter, these sub-blocks are used as start state for the encoding of the remaining sub-blocks within the block A-B (the number of sub-blocks being merely illustrative).
This encoding can start by either encoding the sub-blocks later in time, or by encoding the sub-blocks earlier in time. While both choices are readily possible under the scope of the invention, we describe in detail only embodiments which start with the encoding of sub-blocks later in time.
Encoding of Sub-Blocks Later in Time
If the block contains sub-blocks later in time of the ones encoded for start state, then an adaptive codebook and weighting filter are initialized from the start state for encoding of sub-blocks later in time. Each of these sub-blocks are subsequently encoded. As an example, this can result in the signal 735 in
If more than one sub-block is later in time than the integer sub-block start state within the block, then the adaptive codebook memory is updated with the encoded LPC excitation in preparation for the encoding of the next sub-block. This is done by methods which are well known by a person skilled in the art.
Encoding of Sub-Blocks Earlier in Time
If the block contains sub-blocks earlier in time than the ones encoded for the start state, then a procedure equal to the one applied for sub-blocks later in time is applied on the time-reversed block to encode these sub-blocks. The difference is, when compared to the encoding of the sub-blocks later in time, that now not only the start state, but also the LPC excitation later in time than the start state, is applied in the initialization of the adaptive codebook and the perceptual weighting filter. As an example, this will extend the signal 735 into a full decoded representation 745, which is the resulting decoded representation of the LPC residual 315. The signal 745 constitute the LPC excitation for the decoder.
The encoding steps of the present invention have been exemplified on a block of speech LPC residual signal in
Example c-Code for the Encoding from the Start State Towards Block Boundaries
Weighted Adaptive Codebook Search
In the described forward and backward encoding procedures. The adaptive codebook search can be done in an un-weighted residual domain, or a traditional analysis-by-synthesis weighting can be applied. We here describe in detail a third method applicable to adaptive codebooks. This method supplies an alternative to analysis-by-synthesis, and gives a good compromise between performance and computational complexity. The method consist of a pre-weighting of the adaptive codebook memory and the target signal prior to construction of the adaptive codebook and subsequent search for the best codebook index.
The advantage of this method, compared to analysis-by-synthesis, is that the weighting filtering on the codebook memory leads to less computations than what is needed in the zero state filter recursion of an analysis-by-synthesis encoding for adaptive codebooks. The drawback of this method is that the weighted codebook vectors will have a zero-input component which results from past samples in the codebook memory not from past samples of the decoded signal as in analysis-by-synthesis. This negative effect can be kept low by designing the weighting filter to have low energy in the zero input component relative to the zero state component over the length of a codebook vector. Advantageous parameters for a weighting filter of the form A(z/L1)/(Aq(z)*A(z/L2)), is to set L1=1.0 and L2=0.4.
An implementation of this third method is schematized in
Below follows a c-code example implementation of this third method for weighted codebook search.
Decoder
The decoder covered by the present invention is any decoder that interoperates with an encoder according to the above description. Such a decoder will extract from the encoded data a location for the start state. It will decode the start state and use it as an initialization of a memory for the decoding of the remaining signal frame. In case a data packet is not received a packet loss concealment could be advantageous.
Below follows a c-code example implementation of a decoder.
Number | Date | Country | Kind |
---|---|---|---|
0104059 | Dec 2001 | SE | national |
This application is a Continuation of application Ser. No. 10/497,530 filed on Nov. 30, 2004 now U.S. Pat. No. 7,895,046, and for which priority is claimed under 35 U.S.C. §120. Application Ser. No. 10/497,530 is a National Phase of International Application No. PCT/SE02/002226 filed on Dec. 3, 2002, which claims priority to Application No. 0104059-1, filed in Sweden on Dec. 4, 2001. The entire contents of all are hereby incorporated by reference.
Number | Name | Date | Kind |
---|---|---|---|
6389388 | Lin | May 2002 | B1 |
6760478 | Adiletta et al. | Jul 2004 | B1 |
6970479 | Abrahamsson et al. | Nov 2005 | B2 |
6973132 | Sato et al. | Dec 2005 | B2 |
7209878 | Chen | Apr 2007 | B2 |
20010048680 | Yoshimura et al. | Dec 2001 | A1 |
20020037049 | Hayashita et al. | Mar 2002 | A1 |
20030063745 | Boykin et al. | Apr 2003 | A1 |
20060153286 | Andersen et al. | Jul 2006 | A1 |
Number | Date | Country |
---|---|---|
20020037049 | Mar 2002 | GB |
Entry |
---|
Andersen, S.V. et al., “Multiplexed Predictive Coding of Speech,” In: 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2001. roceedings. Salt Lake City, UT, USA, Mar. 7-11, vol. 2, pp. 741-744, ISBN: 0-7803-7041-4. |
Boyce, J.M., “Packet loss resilient transmission of MPEG video over the Internet,” Signal Processing:Image Communications, Sep. 1999, vol. 15, No. 1-2, pp. 7-24. |
Leslie, B. et al., “Packet Loss Resilient, Scalable Audio compression and Streaming for IP Networks,” In: Second International Conference on 3G Mobile Communication Technologies, 2001. (Conf. Publ: No. 477), London, UK, Mar. 26-28, 2001, pp. 119-123, ISBN: 0-85296-731-4. |
Number | Date | Country | |
---|---|---|---|
20110142126 A1 | Jun 2011 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 10497530 | US | |
Child | 13030929 | US |