Patent Grant
7529673
The present invention relates to speech decoders, and more particularly to methods used to handle bad frames received by speech decoders.
In digital cellular systems, a bit stream is said to be transmitted through a communication channel connecting a mobile station to a base station over the air interface. The bit stream is organized into frames, including speech frames. Whether or not an error occurs during transmission depends on prevailing channel conditions. A speech frame that is detected to contain errors is called simply a bad frame. According to the prior art, in case of a bad frame, speech parameters derived from past correct parameters (of non-erroneous speech frames) are substituted for the speech parameters of the bad frame. The aim of bad frame handling by making such a substitution is to conceal the corrupted speech parameters of the erroneous speech frame without causing a noticeable degrading of the speech quality.
Modern speech codecs operate by processing a speech signal in short segments, the above-mentioned frames. A typical frame length of a speech codec is 20 ms, which corresponds to 160 speech samples, assuming an 8 kHz sampling frequency. In so-called wideband codecs, frame length can again be 20 ms, but can correspond to 320 speech samples, assuming a 16 kHz sampling frequency. A frame may be further divided into a number of subframes.
For every frame, an encoder determines a parametric representation of the input signal. The parameters are quantized and then transmitted through a communication channel, in digital form. A decoder produces a synthesized speech signal based on the received parameters (see
A typical set of extracted coding parameters includes spectral parameters (so called linear predictive coding parameters, or LPC parameters) used in short-term prediction, parameters used for long-term prediction of the signal (so called long-term prediction parameters or LTP parameters), various gain parameters, and finally, excitation parameters.
What is called linear predictive coding is a widely used and successful method for coding speech for transmission over a communication channel; it represents the frequency shaping attributes of the vocal tract. LPC parameterization characterizes the shape of the spectrum of a short segment of speech. The LPC parameters can be represented as either LSFs (Line Spectral Frequencies) or, equivalently, as ISPs (Immittance Spectral Pairs). ISPs are obtained by decomposing the inverse filter transfer function A(z) to a set of two transfer functions, one having even symmetry and the other having odd symmetry. The ISPs, also called Immittance Spectral Frequencies (ISFs), are the roots of these polynomials on the z-unit circle. Line Spectral Pairs (also called Line Spectral Frequencies) can be defined in the same way as Immittance Spectral Pairs; the difference between these representations is the conversion algorithm, which transforms the LP filter coefficients into another LPC parameter representation (LSP or ISP).
Sometimes the condition of the communication channel through which the encoded speech parameters are transmitted is poor, causing errors in the bit stream, i.e. causing frame errors (and so causing bad frames). There are two kinds of frame errors: lost frames and corrupted frames. In a corrupted frame, only some of the parameters describing a particular speech segment (typically of 20 ms duration) are corrupted. In a lost frame type of frame error, a frame is either totally corrupted or is not received at all.
In a packet-based transmission system for communicating speech (a system in which a frame is usually conveyed as a single packet), such as is sometimes provided by an ordinary Internet connection, it is possible that a data packet (or frame) will never reach the intended receiver or that a data packet (or frame) will arrive so late that it cannot be used because of the real-time nature of spoken speech. Such a frame is called a lost frame. A corrupted frame in such a situation is a frame that does arrive (usually within a single packet) at the receiver but that contains some parameters that are in error, as indicated for example by a cyclic redundancy check (CRC). This is usually the situation in a circuit-switched connection, such as a connection in a system of the global system for mobile communication (GSM) connection, where the bit error rate (BER) in a corrupted frame is typically below 5%.
Thus, it can be seen that the optimal corrective response to an incidence of a bad frame is different for the two cases of bad frames (the corrupted frame and the lost frame). There are different responses because in case of corrupted frames, there is unreliable information about the parameters, and in case of lost frames, no information is available.
According to the prior art, when an error is detected in a received speech frame, a substitution and muting procedure is begun; the speech parameters of the bad frame are replaced by attenuated or modified values from the previous good frame, although some of the least important parameters from the erroneous frame are used, e.g. the code excited linear prediction parameters (CELPs), or more simply the excitation parameters.
In some methods according to the prior art, a buffer is used (in the receiver) called the parameter history, where the last speech parameters received without error are stored. When a frame is received without error, the parameter history is updated and the speech parameters conveyed by the frame are used for decoding. When a bad frame is detected, via a CRC check or some other error detection method, a bad frame indicator (BFI) is set to true and parameter concealment (substitution for and muting of the corresponding bad frames) is then begun; the prior-art methods for parameter concealment use parameter history for concealing corrupted frames. As mentioned above, when a received frame is classified as a bad frame (BFI set to true), some speech parameters may be used from the bad frame; for example, in the example solution for corrupted frame substitution of a GSM AMR (adaptive multi-rate) speech codec given in ETSI (European Telecommunications Standards Institute) specification 06.91, the excitation vector from the channel is always used. When a speech frame is lost (including the situation where a frame arrives too late to be used, such as for example in some IP-based transmission systems), obviously no parameters are available from the lost frame to be used.
In some prior-art systems, the last good spectral parameters received are substituted for the spectral parameters of a bad frame, after being slightly shifted towards a constant predetermined mean. According to the GSM 06.91 ETSI specification, the concealment is done in LSF format, and is given by the following algorithm,
Such prior-art systems always shift the spectrum coefficients towards constant quantities, here indicated as mean_LSF(i). The constant quantities are constructed by averaging over a long time period and over several successive talkers. Such systems therefore offer only a compromise solution, not a solution that is optimal for any particular speaker or situation; the tradeoff of the compromise is between leaving annoying artifacts in the synthesized speech, and making the speech more natural in how it sounds (i.e. the quality of the synthesized speech).
What is needed is an improved spectral parameter substitution in case of a corrupted speech frame, possibly a substitution based on both an analysis of the speech parameter history and the erroneous frame. Suitable substitution for erroneous speech frames has a significant effect on the quality of the synthesized speech produced from the bit stream.
Accordingly, the present invention provides a method and corresponding apparatus for concealing the effects of frame errors in frames to be decoded by a decoder in providing synthesized speech, the frames being provided over a communication channel to the decoder, each frame providing parameters used by the decoder in synthesizing speech, the method including the steps of: determining whether a frame is a bad frame; and providing a substitution for the parameters of the bad frame based on an at least partly adaptive mean of the spectral parameters of a predetermined number of the most recently received good frames.
In a further aspect of the invention, the method also includes the step of determining whether the bad frame conveys stationary or non-stationary speech, and, in addition, the step of providing a substitution for the bad frame is performed in a way that depends on whether the bad frame conveys stationary or non-stationary speech. In a still further aspect of the invention, in case of a bad frame conveying stationary speech, the step of providing a substitution for the bad frame is performed using a mean of parameters of a predetermined number of the most recently received good frames. In another still further aspect of the invention, in case of a bad frame conveying non-stationary speech, the step of providing a substitution for the bad frame is performed using at most a predetermined portion of a mean of parameters of a predetermined number of the most recently received good frames.
In another further aspect of the invention, the method also includes the step of determining whether the bad frame meets a predetermined criterion, and if so, using the bad frame instead of substituting for the bad frame. In a still further aspect of the invention with such a step, the predetermined criterion involves making one or more of four comparisons: an inter-frame comparison, an intra-frame comparison, a two-point comparison, and a single-point comparison.
From another perspective, the invention is a method for concealing the effects of frame errors in frames to be decoded by a decoder in providing synthesized speech, the frames being provided over a communication channel to the decoder, each frame providing parameters used by the decoder in synthesizing speech the method including the steps of: determining whether a frame is a bad frame; and providing a substitution for the parameters of the bad frame, a substitution in which past immittance spectral frequencies (ISFs) are shifted towards a partly adaptive mean given by:
ISFq(i)=α*past—ISFq(i)+(1−α)* ISFmean(i), for i=0 . . . 16,
where
α=0.9,
ISFq(i) is the ith component of the ISF vector for a current frame,
past_ISFq(i) is the ith component of the ISF vector from the previous frame,
ISFmean(i) is the ith component of the vector that is a combination of the adaptive mean and the constant predetermined mean ISF vectors, and is calculated using the formula:
ISFmean(i)=β*ISFconst
where β=0.75, where
and is updated whenever BFI=0 where BFI is a bad frame indicator, and where ISFconst
The above and other objects, features and advantages of the invention will become apparent from a consideration of the subsequent detailed description presented in connection with accompanying drawings, in which:
According to the invention, when a bad frame is detected by a decoder after transmission of a speech signal through a communication channel (
An analysis according to the invention also makes use of the localized nature of the spectral impact of the spectral parameters, such as line spectral frequencies (LSFs). The spectral impact of LSFs is said to be localized in that if one LSF parameter is adversely altered by a quantization and coding process, the LP spectrum will change only near the frequency represented by the LSF parameter, leaving the rest of the spectrum unchanged.
The Invention in General, for Either a Lost Frame or a Corrupt Frame
According to the invention, an analyzer determines the spectral parameter concealment in case of a bad frame based on the history of previously received speech parameters. The analyzer determines the type of the decoded speech signal (i.e. whether it is stationary or non-stationary). The history of the speech parameters is used to classify the decoded speech signal (as stationary or not, and more specifically, as voiced or not); the history that is used can be derived mainly from the most recent values of LTP and spectral parameters.
The terms stationary speech signal and voiced speech signal are practically synonymous; a voiced speech sequence is usually a relatively stationary signal, while an unvoiced speech sequence is usually not. We use the terminology stationary and non-stationary speech signals here because that terminology is more precise.
A frame can be classified as voiced or unvoiced (and also stationary or non-stationary) according to the ratio of the power of the adaptive excitation to that of the total excitation, as indicated in the frame for the speech corresponding to the frame. (A frame contains parameters according to which both adaptive and total excitation are constructed; after doing so, the total power can be calculated.)
If a speech sequence is stationary, the methods of the prior art by which corrupted spectral parameters are concealed, as indicated above, are not particularly effective. This is because stationary adjacent spectral parameters are changing slowly, so the previous good spectral values (not corrupted or lost spectral values) are usually good estimates for the next spectral coefficients, and more specifically, are better than the spectral parameters from the previous frame driven towards the constant mean, which the prior art would use in place of the bad spectral parameters (to conceal them).
During stationary speech segments, concealment is performed according to the invention (for either lost or corrupted frames) using the following algorithm:
For i=0 to N−1 (elements within a frame):
adaptive_mean—LSF(i)=(past—LSF_good(i)(0)+past—LSF_good(i)(1)+ . . . +past—LSF_good(i)(K−1))/K;
LSF—q1(i)=α*past_LSF_qood(i)(0)+(1−α)*adaptive_mean_LSF(i);
LSF—q2(i)=LSF—q1(i). (2.1)
where α can be approximately 0.95, N is the order of LP filter, and K is the adaptation length. LSF_q1(i) is the quantized LSF vector of the second subframe and LSF_q2(i) is the quantized LSF vector of the fourth subframe. The LSF vectors of the first and third subframes are interpolated from these two vectors. The quantity past_LSF_qood(i)(0) is equal to the value of the quantity LSF_q2(i−1) from the previous good frame. The quantity past_LSF_good(i)(n) is a component of the vector of LSF parameters from the n+1th previous good frame (i.e. the good frame that precedes the present bad frame by n+1 frames). Finally, the quantity adaptive_mean_LSF(i) is the mean (arithmetic average) of the previous good LSF vectors (i.e. it is a component of a vector quantity, each component being a mean of the corresponding components of the previous good LSF vectors).
It has been demonstrated that the adaptive mean method of the invention improves the subjective quality of synthesized speech compared to the method of the prior art. The demonstration used simulations where speech is transmitted through an error-inducing communication channel. Each time a bad frame was detected, the spectral error was calculated. The spectral error was obtained by subtracting, from the original spectrum, the spectrum that was used for concealing during the bad frame. The absolute error is calculated by taking the absolute value from the spectral error.
As mentioned above, the spectral coefficients of non-stationary signals (or, less precisely, unvoiced signals) fluctuate between adjacent frames, as indicated in
in which energypitch is the energy of pitch excitation and energyinnovation is the energy of the innovation code excitation. When most of the energy is in long-term prediction excitation, the speech being decoded is mostly stationary. When most of the energy is in the fixed codebook excitation, the speech is mostly non-stationary.)
For β=1.0, equation (2.3) reduces to equation (1.0), which is the prior art. For β=0.0, equation (2.3) reduces to the equation (2.1), which is used by the present invention for stationary segments. For complexity sensitive implementations (in applications where it is important to keep complexity to a reasonable level), β can be fixed to some compromise value, e.g. 0.75, for both stationary and non-stationary segments.
Spectral Parameter Concealment Specifically for Lost Frames
In case of a lost frame, only the information of past spectral parameters is available. The substituted spectral parameters are calculated according to a criterion based on parameter histories of for example spectral and LTP (long-term prediction) values; LTP parameters include LTP gain and LTP lag value. LTP represents the correlation of a current frame to a previous frame. For example, the criterion used to calculate the substituted spectral parameters can distinguish situations where the last good LSFs should be modified by an adaptive LSF mean or, as in the prior art, by a constant mean.
Alternative Spectral Parameter Concealment Specifically for Corrupted Frames
When a speech frame is corrupted (as opposed to lost), the concealment procedure of the invention can be further optimized. In such a case, the spectral parameters can be completely or partially correct when received in the speech decoder. For example, in a packet-based connection (as in an ordinary TCP/IP Internet connection), the corrupted frames concealment method is usually not possible because with TCP/IP type connections usually all bad frames are lost frames, but for other kinds of connections, such as in the circuit switched GSM or EDGE connections, the corrupted frames concealment method of the invention can be used. Thus, for packet-switched connections, the following alternative method cannot be used, but for circuit-switched connections, it can be used, since in such connections bad frames are at least sometimes (and in fact usually) only corrupted frames.
According to the specifications for GSM, a bad frame is detected when a BFI flag is set following a CRC check or other error detection mechanism used in the channel decoding process. Error detection mechanisms are used to detect errors in the subjectively most significant bits, i.e. those bits having the greatest effect on the quality of the synthesized speech. In some prior art methods, these most significant bits are not used when a frame is indicated to be a bad frame. However, a frame may have only a few bit errors (even one being enough to set the BFI flag), so the whole frame could be discarded even though most of the bits are correct. A CRC check detects simply whether or not a frame has erroneous frames, but makes no estimate of the BER (bit error rate).
As can be seen from
Table 1 demonstrates the idea behind the corrupted frame concealment according to the invention in the example of an adaptive multi-rate (AMR) wideband (WB) decoder.
In case of an AMR WB decoder, mode 12.65 kbit/s is a good choice to use when the channel carrier to interference ratio (C/I) is in the range from approximately 9 dB to 10 dB. From Table 1, it can be seen that in case of GSM channel conditions with a C/I in the range 9 to 10 dB using a GMSK (Gaussian Minimum-Shift Keying) modulation scheme, approximately 35-50% of received bad frames have a totally correct spectrum. Also, approximately 75-85% of all bad frame spectral parameter coefficients are correct. Because of the localized nature of the spectral impact, as mentioned earlier, spectral parameter information can be used in the bad frames. Channel conditions with a C/I in the range 6-8 dB or less are so poor that the 12.65 kbit/s mode should not be used; instead, some other, lower mode should be used.
The basic idea of the present invention in the case of corrupted frames is that according to a criterion (described below), channel bits from a corrupt frame are used for decoding the corrupt frame. The criterion for spectral coefficients is based on the past values of the speech parameters of the signal being decoded. When a bad frame is detected, the received LSFs or other spectral parameters communicated over the channel are used if the criterion is met; in other words, if the received LSFs meet the criterion, they are used in decoding just as they would be if the frame were not a bad frame. Otherwise, i.e. if the LSFs from the channel do not meet the criterion, the spectrum for a bad frame is calculated according to the concealment method described above, using equations (2.1) or (2.2). The criterion for accepting the spectral parameters can be implemented by using for example a spectral distance calculation such as a calculation of the so-called Itakura-Saito spectral distance. (See, for example, page 329 of Discrete-Time Processing of Speech Signals by John R Deller Jr, John H. L. Hansen, and John G. Proakis, published by IEEE Press, 2000.)
The criterion for accepting the spectral parameters from the channel should be very strict in the case of a stationary speech signal. As shown in
Thus, although the invention includes a method for concealing corrupted frames, it also comprehends as an alternative using a criterion in case of a corrupted frame conveying non-stationary speech, which, if met, will cause the decoder to use the corrupted frame as is; in other words, even though the BFI is set, the frame will be used. The criterion is in essence a threshold used to distinguish between a corrupted frame that is useable and one that is not; the threshold is based on how much the spectral parameters of the corrupted frame differ from the spectral parameters of the most recently received good frames.
The use of possible corrupted spectral parameters is probably more sensitive to audible artifacts than use of other corrupted parameters, such as corrupted LTP lag values. For this reason, the criterion used to determine whether or not to use a possibly corrupt spectral parameter should be especially reliable. In some embodiments, it is advantageous to use as the criterion a maximum spectral distance (from a corresponding spectral parameter in a previous frame, beyond which the suspect spectral parameter is not to be used); in such an embodiment, the well-known Itakura-Saito distance calculation could be used to quantify the spectral distance to be compared with the threshold. Alternatively, fixed or adaptive statistics of spectral parameters could be used for determining whether or not to use possibly corrupted spectral parameters. Also other speech parameters, such as gain parameters, could be used for generating the criterion. (If the other speech parameters are not drastically different in the current frame, compared to the values in the most recent good frame, then the spectral parameters are probably okay to use, provided the received spectral parameters also meet the criteria. In other words, other parameters, such as LTP gain, can be used as an additional component to set proper criteria to determine whether or not to use the received spectral parameters. The history of the other speech parameters can be used for improved recognition of speech characteristic. For example, the history can be used to decide whether the decoded speech sequence has a stationary or non-stationary characteristic. When the properties of the decoded speech sequence are known, it is easier to detect possibly correct spectral parameters from the corrupted frame and it is easier to estimate what kind of spectral parameter values are expected to have been conveyed in a received corrupted frame.)
According to the invention in the preferred embodiment, and now referring to
The criterion according to the preferred embodiment involves making one or more of four comparisons: an inter-frame comparison, an intra-frame comparison, a two-point comparison, and a single-point comparison.
In the first comparison, the inter-frame comparison, the differences between LSF vector elements in adjacent frames of the corrupted frame are compared to the corresponding differences of previous frames. The differences are determined as follows:
dn(i)=|Ln-1(i)−Ln(i)|, 1≦i≦P−1,
where P is the number of spectral coefficients for a frame, Ln(i) is the ith LSF element of corrupted frame, and Ln-1(i) is the ith LSF element of the frame before corrupted frame. The LSF element, Ln(i), of the corrupted frame is discarded if the difference, dn(i), is too high compared to dn-1(i), dn-2(i), . . . , dn-k(i), where k is the length of the LSF buffer.
The second comparison, the intra-frame comparison, is a comparison of difference between adjacent LSF vector elements in the same frame. The distance between the candidate ith LSF element, Ln(i), of the nth frame and the (i−1)th LSF element, Ln-1(i), of the nth frame is determined as follows:
en(i)=Ln(i−1)−Ln(i), 2≦i≦P−1,
where P is the number of spectral coefficients and en(i) is the distance between LSF elements. Distances are calculated between all LSF vector elements of the frame. One or another or both of the LSF elements Ln(i) and Ln(i−1) will be discarded if the difference, en(i), is too large or too small compared to en-1(i), en-2(i), . . . , en-k(i).
The third comparison, the two-point comparison, determines whether a crossover has occurred involving the candidate LSF element Ln(i), i.e. whether an element Ln(i−1) that is lower in order than the candidate element has a larger value than the candidate LSF element Ln(i). A crossover indicates one or more highly corrupted LSF values. All crossing LSF elements are usually discarded.
The fourth comparison, the single-point comparison, compares the value of the candidate LSF vector element, Ln(i) to a minimum LSF element, Lmin(i), and to a maximum LSF element, Lmax(i), both calculated from the LSF buffer, and discards the candidate LSF element if it lies outside the range bracketed by the minimum and maximum LSF elements.
If an LSF element of a corrupted frame is discarded (based on the above criterion or otherwise), then a new value for the LSF element is calculated according to the algorithm using equation (2.2).
Referring now to
The invention can be applied in a speech decoder in either a mobile station or a mobile network element. It can also be applied to any speech decoder used in a system having an erroneous transmission channel.
It is to be understood that the above-described arrangements are only illustrative of the application of the principles of the present invention. In particular, it should be understood that although the invention has been shown and described using line spectrum pairs for a concrete illustration, the invention also comprehends using other, equivalent parameters, such as immittance spectral pairs. Numerous modifications and alternative arrangements may be devised by those skilled in the art without departing from the spirit and scope of the present invention, and the appended claims are intended to cover such modifications and arrangements.
This application claims priority under 35 USC §119(e)(1) to provisional application Ser. No. 60/242,498 filed Oct. 23, 2000. This application is also a continuation of U.S. patent application Ser. No. 09/918,300 filed 30 Jul. 2001 now U.S. Pat. No. 7,031,926, from which priority is claimed under all applicable sections of Title 35 of the United States Code including, but not limited to, Sections 120, 121, and 365(c).
| Number | Name | Date | Kind |
|---|---|---|---|
| 5406632 | Sogo et al. | Apr 1995 | A |
| 5502713 | Lagerqvist et al. | Mar 1996 | A |
| 5530750 | Akagiri | Jun 1996 | A |
| 5598506 | Wigren et al. | Jan 1997 | A |
| 5634082 | Shimoyoshi et al. | May 1997 | A |
| 5717822 | Chen | Feb 1998 | A |
| 5862518 | Nomura et al. | Jan 1999 | A |
| 5873065 | Akagiri et al. | Feb 1999 | A |
| 6122607 | Ekudden et al. | Sep 2000 | A |
| 6292774 | Taori et al. | Sep 2001 | B1 |
| 6373842 | Coverdale et al. | Apr 2002 | B1 |
| 6377915 | Sasaki | Apr 2002 | B1 |
| 6418408 | Udaya Bhaskar et al. | Jul 2002 | B1 |
| Number | Date | Country |
|---|---|---|
| 7271391 | Oct 1995 | JP |
| 325594 | Dec 1995 | JP |
| 7325594 | Dec 1995 | JP |
| 8305398 | Nov 1996 | JP |
| 10 22936 | Jan 1998 | JP |
| WO 9429851 | Dec 1994 | WO |
| WO 9966494 | Dec 1999 | WO |
| WO 0060579 | Oct 2000 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20070239462 A1 | Oct 2007 | US |
| Number | Date | Country | |
|---|---|---|---|
| 60242498 | Oct 2000 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 09918300 | Jul 2001 | US |
| Child | 11402220 | US |
