Patent Application
20060173687
1. Field
The present disclosure relates generally to voice communications, and more particularly, to frame erasure concealment techniques for voice communications.
2. Background
Traditionally, digital voice communications have been performed over circuit-switched networks. A circuit-switched network is a network in which a physical path is established between two terminals for the duration of a call. In circuit-switched applications, a transmitting terminal sends a sequence of packets containing voice information over the physical path to the receiving terminal. The receiving terminal uses the voice information contained in the packets to synthesize speech. If a packet is lost in transit, the receiving terminal may attempt to conceal the lost information. This may be achieved by reconstructing the voice information contained in the lost packet from the information in the previously received packets.
Recent advances in technology have paved the way for digital voice communications over packet-switched networks. A packet-switch network is a network in which the packets are routed through the network based on a destination address. With packet-switched communications, routers determine a path for each packet individually, sending it down any available path to reach its destination. As a result, the packets do not arrive at the receiving terminal at the same time or in the same order. A jitter buffer may be used in the receiving terminal to put the packets back in order and play them out in a continuous sequential fashion.
The existence of the jitter buffer presents a unique opportunity to improve the quality of reconstructed voice information for lost packets. Since the jitter buffer stores the packets received by the receiving terminal before they are played out, voice information may be reconstructed for a lost packet from the information in packets that precede and follow the lost packet in the play out sequence.
A voice decoder is disclosed. The voice decoder includes a speech generator configured to receive a sequence of frames, each of the frames having voice parameters, and generate speech from the voice parameters. The voice decoder also includes a frame erasure concealment module configured to reconstruct the voice parameters for a frame erasure in the sequence of frames from the voice parameters in one of the previous frames and the voice parameters in one of the subsequent frames.
A method of decoding voice is disclosed. The method includes receiving a sequence of frames, each of the frames having voice parameters, reconstructing the voice parameters for a frame erasure in the sequence of frames from the voice parameters in one of the previous frames and the voice parameters from one of the subsequent frames, and generating speech from the voice parameters in the sequence of frames.
A voice decoder configured to receive a sequence of frames is disclosed. Each of the frames includes voice parameters. The voice decoder includes means for generating speech from the voice parameters, and means for reconstructing the voice parameters for a frame erasure in the sequence of frames from the voice parameters in one of the previous frames and the voice parameters in one of the subsequent frames.
A communications terminal is also disclosed. The communications terminal includes a receiver and a voice decoder configured to receive a sequence of frames from the receiver, each of the frames having voice parameters. The voice decoder includes a speech generator configured to generate speech from the voice parameters, and a frame erasure concealment module configured to reconstruct the voice parameters for a frame erasure in the sequence of frames from the voice parameters in one of the previous frames and the voice parameters in one of the subsequent frames.
It is understood that other embodiments of the present invention will become readily apparent to those skilled in the art from the following detailed description, wherein various embodiments of the invention are shown and described by way of illustration. As will be realized, the invention is capable of other and different embodiments and its several details are capable of modification in various other respects, all without departing from the spirit and scope of the present invention. Accordingly, the drawings and detailed description are to be regarded as illustrative in nature and not as restrictive.
Aspects of the present invention are illustrated by way of example, and not by way of limitation, in the accompanying drawings, wherein:
The detailed description set forth below in connection with the appended drawings is intended as a description of various embodiments of the present invention and is not intended to represent the only embodiments in which the present invention may be practiced. The detailed description includes specific details for the purpose of providing a thorough understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced without these specific details. In some instances, well known structures and components are shown in block diagram form in order to avoid obscuring the concepts of the present invention.
The transmitting terminal 102 is shown with a voice encoder 106 and the receiving terminal 104 is shown with a voice decoder 108. The voice encoder 106 may be used to compress speech from a user interface 110 by extracting parameters based on a model of human speech generation. A transmitter 112 may be used to transmit packets containing these parameters across the transmission medium 114. The transmission medium 114 may be a packet-based network, such as the Internet or a corporate intranet, or any other transmission medium. A receiver 116 at the other end of the transmission medium 112 may be used to receive the packets. The voice decoder 108 synthesizes the speech using the parameters in the packets. The synthesized speech may then be provided to the user interface 118 on the receiving terminal 104. Although not shown, various signal processing functions may be performed in both the transmitter and receiver 112, 116 such as convolutional encoding including Cyclic Redundancy Check (CRC) functions, interleaving, digital modulation, and spread spectrum processing.
In most applications, each party to a communication transmits as well as receives. Each terminal would therefore require a voice encoder and decoder. The voice encoder and decoder may be separate devices or integrated into a single device known as a “vocoder.” In the detailed description to follow, the terminals 102, 104 will be described with a voice encoder 106 at one end of the transmission medium 114 and a voice decoder 108 at the other. Those skilled in the art will readily recognize how to extend the concepts described herein to two-way communications.
In at least one embodiment of the transmitting terminal 102, speech may be input from the user interface 110 to the voice encoder 106 in frames, with each frame further partitioned into sub-frames. These arbitrary frame boundaries are commonly used where some block processing is performed, as is the case here. However, the speech samples need not be partitioned into frames (and sub-frames) if continuous processing rather than block processing is implemented. Those skilled in the art will readily recognize how block techniques described below may be extended to continuous processing. In the described embodiments, each packet transmitted across the transmission medium 114 may contain one or more frames depending on the specific application and the overall design constraints.
The voice encoder 106 may be a variable rate or fixed rate encoder. A variable rate encoder dynamically switches between multiple encoder modes from frame to frame, depending on the speech content. The voice decoder 108 also dynamically switches between corresponding decoder modes from frame to frame. A particular mode is chosen for each frame to achieve the lowest bit rate available while maintaining acceptable signal reproduction at the receiving terminal 104. By way of example, active speech may be encoded at full rate or half rate. Background noise is typically encoded at one-eighth rate. Both variable rate and fixed rate encoders are well known in the art.
The voice encoder 106 and decoder 108 may use Linear Predictive Coding (LPC). The basic idea behind LPC encoding is that speech may be modeled by a speech source (the vocal chords), which is characterized by its intensity and pitch. The speech from the vocal cords travels through the vocal tract (the throat and mouth), which is characterized by its resonances, which are called “formants.” The LPC voice encoder 106 analyzes the speech by estimating the formants, removing their effects from the speech, and estimating the intensity and pitch of the residual speech. The LPC voice decoder 108 at the receiving end synthesizes the speech by reversing the process. In particular, the LPC voice decoder 108 uses the residual speech to create the speech source, uses the formants to create a filter (which represents the vocal tract), and runs the speech source through the filter to synthesize the speech.
Further compression techniques may be used to dramatically decrease the information required to represent speech by eliminating redundant material. This may be achieved by exploiting the fact that there are certain fundamental frequencies caused by periodic vibration of the human vocal chords. These fundamental frequencies are often referred to as the “pitch.” The pitch can be quantified by “adaptive codebook parameters” which include (1) the “delay” in the number of speech samples that maximizes the autocorrelation function of the speech segment, and (2) the “adaptive codebook gain.” The adaptive codebook gain measures how strong the long-term periodicities of the speech are on a sub-frame basis. These long term periodicities may be subtracted 210 from the residual speech before transmission to the receiving terminal.
The residual speech from the subtractor 210 may be further encoded in any number of ways. One of the more common methods uses a codebook 212, which is created by the system designer. The codebook 212 is a table that assigns parameters to the most typical speech residual signals. In operation, the residual speech from the subtractor 210 is compared to all entries in the codebook 212. The parameters for the entry with the closest match are selected. The fixed codebook parameters include the “fixed codebook coefficients” and the “fixed codebook gain.” The fixed codebook coefficients contain the new information (energy) for a frame. It basically is an encoded representation of the differences between frames. The fixed codebook gain represents the gain that the voice decoder 108 in the receiving terminal 104 should use for applying the new information (fixed codebook coefficients) to the current sub-frame of speech.
The pitch estimator 208 may also be used to generate an additional adaptive codebook parameter called “Delta Delay” or “DDelay.” The DDelay is the difference in the measured delay between the current and previous frame. It has a limited range however, and may be set to zero if the difference in delay between the two frames overflows. This parameter is not used by the voice decoder 108 in the receiving terminal 104 to synthesize speech. Instead, it is used to compute the pitch of speech samples for lost or corrupted frames.
The jitter buffer 302 may be positioned at the front end of the voice decoder 108. The jitter buffer 302 is a hardware device or software process that eliminates jitter caused by variations in packet arrival time due to network congestion, timing drift, and route changes. The jitter buffer 302 delays the arriving packets so that all the packets can be continuously provided to the speech generator 308, in the correct order, resulting in a clear connection with very little audio distortion. The jitter buffer 302 may be fixed or adaptive. A fixed jitter buffer introduces a fixed delay to the packets. An adaptive jitter buffer, on the other hand, adapts to changes in the network's delay. Both fixed and adaptive jitter buffers are well known in the art.
As discussed earlier in connection with
The voice parameters, whether released from the jitter buffer 302 or reconstructed by the frame erasure concealment module 306, are provided to the speech generator 308. Specifically, an inverse codebook 312 is used to convert the fixed codebook coefficients to residual speech and apply the fixed codebook gain to that residual speech. Next, the pitch information is added 318 back into the residual speech. The pitch information is computed by a pitch decoder 314 from the “delay.” The pitch decoder 314 is essentially a memory of the information that produced the previous frame of speech samples. The adaptive codebook gain is applied to the memory information in each sub-frame by the pitch decoder 314 before being added 318 to the residual speech. The residual speech is then run through a filter 320 using the LPC coefficient from the inverse transform 322 to add the formants to the speech. The raw synthesized speech may then be provided from the speech generator 308 to a post-filter 324. The post-filter 324 is a digital filter in the audio band that tends to smooth the speech and reduce out-of-band components.
The quality of the frame erasure concealment process improves with the accuracy in reconstructing the voice parameters. Greater accuracy in the reconstructed speech parameters may be achieved when the speech content of the frames is higher. This means that most voice quality gains through frame erasure concealment techniques are obtained when the voice encoder and decoder are operated at full rate (maximum speech content). Using half rate frames to reconstruct the voice parameters of a frame erasure provides some voice quality gains, but the gains are limited. Generally speaking, one-eight rate frames do not contain any speech content, and therefore, may not provide any voice quality gains. Accordingly, in at least one embodiment of the voice decoder 108, the voice parameters in a future frame may be used only when the frame rate is sufficiently high to achieve voice quality gains. By way of example, the voice decoder 108 may use the voice parameters in both the previous and future frame to reconstruct the voice parameters in an erased frame if both the previous and future frames are encoded at full or half rate. Otherwise, the voice parameters in the erased frame are reconstructed solely from the previous frame. This approach reduces the complexity of the frame erasure concealment process when there is a low likelihood of voice quality gains. A “rate decision” from the frame error detector 304 may be used to indicate the encoding mode for the previous and future frames of a frame erasure.
The frame erasure concealment module 306 reconstructs the speech parameters for the frame by first determining whether information from future frames is available in the jitter buffer 302. In step 410, the frame erasure concealment module 306 makes this determination by monitoring a “future frame available flag” generated by the frame error detector 304. If the “future frame available flag” is cleared, then the frame erasure concealment module 306 must reconstruct the speech parameters from the previous frames in step 412, without the benefit of the information in future frames. On the other hand, if the “future frame available flag” is set, the frame erasure concealment module 306 may provide enhanced concealment by using information from both the previous and future frames. This process is performed however, only if the frame rate is high enough to achieve voice quality gains. The frame erasure concealment module 306 makes this determination in step 413. Either way, once the frame erasure concealment module 306 reconstructs the speech parameters for the current frame, it waits for the next frame in step 408, and then repeats the process.
In step 412, the frame erasure concealment module 306 reconstructs the speech parameters for the erased frame using the information from the previous frame. For the first frame erasure in a sequence of lost frames, the frame erasure concealment module 306 copies the LSPs and the “delay” from the last received frame, sets the adaptive codebook gain to the average gain over the sub-frames of the last received frame, and sets the fixed codebook gain to zero. The adaptive codebook gain is also faded and element of randomness is the LSPs and the “delay” if power (adaptive codebook gain) is low.
As indicated above, improved error concealment may be achieved when information from future frames is available and the frame rate is high. In step 414, the LSPs for a sequence of frame erasures may be linearly interpolated from the previous and future frames. In step 416, the delay may be computed using the DDelay from the future frame, and if the DDelay is zero, then the delay may be linearly interpolated from the previous and future frames. In step 418, the adaptive codebook gain may be computed. At least two different approaches may be used. The first approach computes the adaptive codebook gain in a similar manner to the LSPs and the “delay.” That is, the adaptive codebook gain is linearly interpolated from the previous and future frames. The second approach sets the adaptive codebook gain to a high value if the “delay” is known, i.e., the DDelay for the future frame is not zero and the delay of the current frame is exact and not estimated. A very aggressive approach may be used by setting the adaptive codebook gain to one. Alternatively, the adaptive codebook gain may be set somewhere between one and the interpolation value between the previous and future frames. Either way, there is no fading of the adaptive codebook gain as might be experienced if information from future frames is not available. This is only possible because having information from the future tells the frame erasure concealment module 306 whether the erased frames have any speech content (the user may have stopped speaking just prior to the transmission of the erased frames). Finally, in step 420, the fixed codebook gain is set to zero.
The various illustrative logical blocks, modules, circuits, elements, and/or components described in connection with the embodiments disclosed herein may be implemented or performed with a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic component, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing components, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.
The methods or algorithms described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM) flash memory, Read Only Memory (ROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. A storage medium may be coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
