Orthogonal LMS algorithms for fast line echo canceler training

Abstract

An echo canceler for calculating an echo impulse response during a training sequence, wherein a sequence of data samples is selected such that a full rank data correlation matrix comprised of such data samples is diagonal at the Nth iteration. The echo canceler estimates the echo impulse response based on such data samples and an echo generated during the training sequence using a normalized least means square algorithm, wherein the length of the data sequence to sound the echo channel is N. This normalized LMS algorithm, wherein the data sequence is specifically selected, converges toward an echo impulse response much faster than standard LMS algorithms, while having a computational complexity comparable to that of the standard LMS algorithm.

Description

FIELD OF THE INVENTION

The present invention relates to an algorithm for training an echo canceler used in voice band modems, and in particular an echo canceler employing a variation of the Least Mean Squares (LMS) algorithm during a training sequence.

BACKGROUND OF THE INVENTION

Standard telephone lines are used for digital transmissions. A modem (modulator/demodulator) is used to transmit digital data by converting bit-serial digital quantities into analog signals which are transmitted over the telephone lines. The transmitting portion of the modem modulates the digital data into a sine wave carrier signal having a frequency within the voice transmission band. The receiving portion of the modem receives the analog signals and demodulates them back into digital data. U.S. Pat. No. 4,977,591, entitled "Dual Mode LMS Nonlinear Data Echo Canceler," to Walter Y. Chen and Richard A. Haddad and Richard D. Gitlin, Jeremiah F. Hayes, and Stephen B. Weinstein, Data Communication Principles 607 et seq. (1992) provide a detailed description of modem communication via the telephone channel. Both these references are incorporated herein by reference in their entirety.

A full duplex modem transmits and receives using the same bandwidth in a two-wire (bilateral) telephone channel used for both voice and data transmission. When signals are transmitted over the telephone channel an echo is generated which is added to the transmitted signal. Thus, the signal received at the receiving end includes not only the originally transmitted signal, but echo components.

The echo occurs as a result of a hybrid coupler ("hybrid") converting a four-wire loop to a two-wire loop within the telephone network. These conversion from four-wire to two wire creates an echo in the transmitted data signal which arises as a result of an impedance mismatch. This impedance mismatch causes a portion of the data signal generated by the transmitter of the near end modem to return to the receiver of the near end modem. The near end echo is comprised of two components. The first component of the near end echo arises when a hybrid converts the two-wire transmitter and two-wire receiver, a total of four wires, to the two-wire local loop of the telephone network. The second component of the near end echo results when a hybrid converts the two-wire local loop to the four-wire loop of the telephone network.

The far end echo is also comprised of two components. The first component of the near end echo arises when a hybrid converts the four-wire network to a two-wire local loop. The second component arises when a hybrid converts the two-wire local loop to the four-wire connection within the receiving modem, the two-wire receiver and two-wire transmitter. This far end echo is less significant than the near end echo.

It is the role of an echo canceler to remove or minimize the echo component from the signal received at the near end modem receiver. The signal received by the modem receiver can be modeled as follows:

y.sub.n =r.sub.n +z.sub.n

Where y.sub.n is the data signal received by the receiver. It is comprised of r.sub.n, which is the actual signal transmitted from a remote modem or source, and z.sub.n, which is the actual echo component of the received signal. The echo canceler estimates z.sub.n to obtain z'.sub.n and then subtracts this value from the received signal y.sub.n to get the an estimate of the received signal, r'.sub.n, which has the estimated echo removed. Below is an equation showing the estimated received signal with the estimated echo subtracted:

r'.sub.n =y.sub.n -z'.sub.n

In a conventional echo canceler, the modem initiates a training sequence when the carrier signal is received, prior to receiving data or going into a data mode. During this training sequence, the echo canceler estimates the impulse response of the echo path by modeling the transmission channel as an adaptive digital filter, such as an FIR filter. Such adaptive FIR filters are known in the art to cancel echos, and described in Data Communications, supra, at 619-624. This calculated echo impulse response is then used to calculate an estimated echo. During the data mode when the modem is receiving data signals, the echo canceler within the modem receives data from the data transmitter of the modem. The echo canceler performs an operation on the transmitted data signal with the estimated echo channel impulse response signal to obtain an estimated echo signal, which is then subtracted from the data received from a remote modem as discussed above.

Two commonly used methods to estimate the echo path impulse response are the LMS and Recursive Least Squares (RLS) algorithms. The LMS algorithm adjusts the calculated values to minimize the error between the actual echo signal (z.sub.n) at time n and the estimated echo signal (z'.sub.n). The RLS algorithm converges about 10 times faster than the LMS algorithm. However, the LMS algorithm is preferred over the RLS algorithm because the LMS algorithm has significantly less computational complexity. For instance, setting N as the number of data taps or length of the data sequence to sound the echo channel, the RLS algorithm requires N.sup.2 computational steps, whereas the LMS method only requires N computations. One attempt to reduce computational complexity has been the use of Fast Kalman algorithms that have computational complexities of 7-8 N. Although the Fast Kalman method may converge quickly, it is problematic because it exhibits high sensitivity to roundoff and other errors.

SUMMARY OF THE DISCLOSURE

It is an object of the present invention to estimate a discrete echo channel impulse response using an algorithm having the low computational complexity of an LMS algorithm and the superior convergence properties of RLS algorithms.

This object can be achieved in a modem having a transmitter, receiver, and echo canceler. During a training sequence prior to dual mode transmission, the transmitter generates a data sequence of length N including data samples d.sub.(n). This data sequence is selected such that a data matrix formed from data vectors comprised of the generated data samples d.sub.(n) produces a data matrix diagonal to the Nth iteration. Also during this training sequence, the transmitter generates a signal to sound the echo channel which returns to the modem as echo z.sub.(n).

Based on these values, the selected data sequence generated by the transmitter d.sub.(n) and the sounded echos z.sub.(n), the echo canceler implements a normalized LMS algorithm based on the under-determined approach to estimate the discrete echo channel impulse response x.sub.(n). In the normalized LMS, the algorithm runs for N iterations. This echo impulse response has convergence properties similar to RLS, yet realizes this convergence with significantly less computational complexity than that used by the RLS algorithm.

In a further embodiment, iterations can be performed beyond N. In such case, error correction terms can be added to the equation for estimating the echo impulse response. By going beyond N iterations to estimate the echo impulse response, the normalized algorithm can match exactly the superior convergence rate of RLS for calculating the echo impulse response.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 schematically illustrates the channel model for line echos in a telephone network.

FIG. 2 schematically illustrates an echo canceler within a modem.

FIG. 3 provides a convergence performance comparison for the OLMS, SOLMS, RLMS, and simplified LMS algorithms.

FIG. 4 provides a convergence performance comparison of the RLS, OLMS, and SOLMS algorithms.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

FIG. 1 shows a model of how modems are linked with a telephone network channel 12 and used for the transmission of digital data. Two modems, a near end modem 16 and a far end modem 68, are connected to the telephone network. The near end modem 16 includes a transmitter 20 and receiver 24 and the far end modem 68 also includes a transmitter 72 and receiver 76. The transmitter 72 of the far end modem 68 communicates digital data over the channel 12 to the receiver 24 of the near end modem 16. Similarly, the transmitter 20 of the near end modem 16 transmits data over the channel 12 to the receiver 76 of the far end modem 68.

In the near end modem 16, a transmitting two-wire pair 28 is connected to transmitter 20 and a receiving two-wire pair 32 is connected to the receiver 24. The transmitting two-wire pair 28 and receiving two-wire pair 32 (comprising a total of four wires) are converted by a hybrid 24 into a single analog two-wire pair 48, which forms a local loop of the telephone network and transmits in both directions at the same time.

At hybrid 44, an impedance mismatch is formed due to design and hardware limitations. Due to this impedance mismatch, a portion of the signal passing through the transmitting two-wire pair 28 travels back through the receiving two-wire pair 32 as an echo. This echo is part of what is known as the near end echo. Path 36 illustrates how a fraction of the transmitted signal from transmitter 20 returns to the receiver 24.

With reference to FIG. 1, a hybrid 52 converts the two-wire near end local loop 48 to a four-wire network loop 56. This conversion further adds to the near end echo received by receiver 24.

A far end echo arises as a result of two conversions. The first component of the far end echo arises as a result of hybrid 60 converting the four-wire network 56 of the telephone network to the two-wire local loop 64. The second component arises as a result of hybrid 96 converting the two-wire local loop 64 to the four-wires made up of the two-wire transmitting pair 84 and two-wire receiving pair 80.

In the far end modem 42, the same echo situation occurs. The hybrid conversions at hybrids 96 and 60 contribute to the near end echo for the far end modem 42, and the conversions at hybrids 44, 52 contribute to the far end echo for the far end modem 42. However, the echos received at the far end modem 42 are different than those echos received at the near end modem 16.

FIG. 2 shows preferred embodiments of how an echo canceler 26 may be employed in a modem. The echo canceler 26 may be comprised of a known adaptive FIR filter, such as those described in Data Communications, supra, at 619-624, for estimating the discrete time echo channel impulse response, referred to herein as x'.sub.n,. The echo canceler 26 may be implemented as a microprocessor operating under stored program control or in dedicated logic circuits or in some combination thereof. FIG. 2 shows the echo canceler 26 within the near end modem 16. The echo canceler 26 is coupled to receive data transmitted from the transmitter 20 and the data received by the modem via the receiving two wire pair 32.

After the modem connects to the telephone line, it initiates a training sequence before it receives data from a remote location. The echo canceler 26 estimates the echo channel impulse response (x'.sub.n) using two known variables, the input or data transmitted from the transmitter 20, referred to as d.sub.n, and the echo z.sub.n generated when the echo channel is sounded during the training sequence.

An equation for estimating the echo z.sub.n at time n based on the data d.sub.n from the transmitter and the echo impulse response x.sub.n is modeled as follows: ##EQU1##

As discussed, during the training sequence, the echo canceler 26 knows z.sub.n and d.sub.n. v.sub.n represents noise. During the approximation of the echo, the noise v.sub.n is assumed to have zero means, i.e., averages out to zero. Based on these known variables, the echo canceler estimates the echo channel impulse response, x'.sub.n. To estimate x'.sub.n, the discrete time equation is rewritten in matrix form to solve the linear equation, such that:

H.sub.N .multidot.x+v.sub.n =z.sub.n where ##EQU2##

The variable N is the length of the data sequence used to sound the echo channel and the number of iterations performed by the algorithm. Each data vector h.sub.(n) is comprised of a set of data samples generated by the transmitter 20. In preferred embodiments each h.sub.(n) data vector would include N data samples. For instance, if N=5, then the h.sub.(n) data vectors may have the following format:

h.sub.(1) =�d.sub.(5) d.sub.(4) d.sub.(3) d.sub.(2) d.sub.(1) ! h.sub.(2) =�d.sub.(6) d.sub.(5) d.sub.(4) d.sub.(3) d.sub.(2) !, etc.

It is a feature of the present invention to carefully select the data d.sub.n, used in the linear equation by particularly selecting the data matrix H.sub.n and then employing an LMS algorithm to estimate the data echo impulse responses x'.sub.n. In preferred embodiments, the data vector h.sub.(n) is selected such that it would produce a data matrix H.sub.N H.sub.N.sup.T that is full rank and generates a data correlation matrix that is diagonal at the Nth iteration.

To obtain H.sub.N H.sub.N.sup.T that is full rank and generates a data correlation matrix that is diagonal to the Nth iteration, the data sequence h.sub.(n) can be selected using a Constant Amplitude Zero Autocorrelation (CAZAC) sequence. Other methods for selecting the data sequences such that H.sub.N H.sub.N.sup.T is diagonal may also be used.

In a preferred embodiment the data values can be selected using polyphase codes with a periodic autocorrelation function that is zero everywhere except at a single maximum per period. If the data vector has length N, then the data values d.sub.k can be selected using the following equations.

First select a value M, which is a prime number relative to N, i.e., not divisible by N. If N is even, then the data samples d.sub.k in the data vector h.sub.(n) used during the training sequence can be calculated with the following equation involving complex numbers: ##EQU3##

If N is odd, then the data samples d.sub.k in the data vector h.sub.(n) used during the training sequence can be calculated with the following equation involving complex numbers: ##EQU4##

These data samples d.sub.k may be calculated by a processor and inputted to the transmitter 20 for further transmittal to the echo canceler 26. Alternatively, these calculated data samples d.sub.k may be stored in a look-up table in a memory.

When this carefully selected data sequence is applied to the under-determined system algorithm, the update equation for estimating the echo impulse response (x'.sub.n) reduces down to: ##EQU5##

The component

(z.sub.(n) -h.sub.n-1 .multidot.x'.sub.n-1) represents the adaptation or error component used by the echo canceler.

The echo impulse channel estimates generated using this method and data selection technique will minimize mean square errors between the calculated echo and estimated echo, wherein each subsequent iteration converges toward a better approximation. Moreover, using the normalized LMS algorithm with the carefully selected data matrix results in the same rapid convergence properties of the RLS method up to the first N iterations. Still further, this normalized LMS algorithm accomplishes such results with the computational complexity of an LMS algorithm as opposed to the far more computationally complex RLS method. The above algorithm is also known as the simplified orthogonal least means squares algorithm ("SOLMS")

After the training sequence, the modem enters data mode and begins receiving data. As discussed, the data received is referred to as y.sub.n. The data signal can be represented as:

y.sub.n =r.sub.n +z.sub.n

The goal is to estimate r.sub.n or the received signal. The estimated echo, z'.sub.n, is determined in accordance with: ##EQU6##

The echo canceler 26 uses the transmitted data from the transmitter 20 (d.sub.n), selected in the manner discussed above, and the echo channel impulse response, x'.sub.n, estimated during the training sequence, to estimate the echo z'.sub.n. The estimated echo is then subtracted from the data signal received from the far end modem 68 (y.sub.n) to estimate what the far end modem 68 transmitted (r'.sub.n), or

r'.sub.n =y.sub.n -z'.sub.n

Assuming the echo channel is not time varying, in full duplex data transmission mode, there should be almost no change in the value of the echo impulse response calculated during the training sequence. In preferred embodiments, the echo canceler 26 will continue approximating the echo impulse response x'.sub.n during the data transmission mode using a standard LMS algorithm. Although the standard LMS algorithm is slower to converge, this is not problematic because time varying changes in the echo channel during data transmission mode are relatively gradual. Accordingly, any changes in the echo channel during data transmission mode may be tracked using the standard LMS algorithm.

In alternative embodiments, the echo impulse response x'.sub.n may be calculated out to a number of iterations greater than N. In preferred embodiments, an additional 5 times N iterations have been performed, when N=60. However, other values for N and multiples thereof for performing iterations may also be selected. For those iterations beyond N, an error value is used to calculate the echo impulse response x'.sub.n. This new algorithm incorporating the error term is referred to as the orthogonal LMS algorithm ("OLMS"), which calculates the echo impulse response as follows: ##EQU7## and k=ceiling (n/N).

Note that the calculation for (err.sub.1 -err.sub.2) reduces to: ##EQU8##

This equation including the error correction terms matches exactly the computations performed by the RLS algorithm. Moreover, the use of the error terms is averaging the received data and, thus, averages out the noise v.sub.n. When there is no noise, the SOLMS algorithm closely approximates the OLMS algorithm. However, when there is noise, the OLMS provides a better approximation of the echo impulse response because the OLMS system averages out the noise. The reason for this is that the SOLMS and OLMS algorithms produce the same result for the first N iterations. The OLMS algorithm, however, goes beyond N iterations, adding the error terms to better approximate the echo impulse response and average out the noise.

This OLMS algorithm using the errors within the equation matches exactly the computation performed by the RLS algorithm and, thus, possesses the same superior convergence rate of the RLS algorithm. Moreover, the OLMS algorithm has only a slightly higher computational complexity than the normalized LMS algorithm, and far less complexity than the RLS algorithm. After obtaining the estimated echo impulse path response using the OLMS algorithm, in data mode the echo canceler 26 would remove the estimated echo z'.sub.n and calculate the transmitted signal r'.sub.n as discussed above.

When implemented, the OLMS and SOLMS algorithms can run with N=60. If the OLMS algorithm is used, an additional 5 N iterations can be performed, adding up to a total of 300 (6.times.50) iterations. This is more effective than the typical implementation using the standard LMS algorithm which often uses 2000 iterations (N=2000) during the training sequence. Setting N=50 for the OLMS and SOLMS algorithms is usually sufficient to converge the echo impulse response such that the MMSE is reached. In alternative embodiments N can be set at any other sufficient value to minimize errors in calculating the echo impulse response. For example, in the embodiment illustrated in FIGS. 3 and 4, N is set at 16.

In alternative embodiments, instead of performing a predetermined number of iterations, the training sequence will continue to perform iterations until the echo impulse response x'.sub.n appropriately converges. This means that at a particular iteration the value x'.sub.n -x'.sub.n-1 reaches a low enough threshold such that the estimation of the echo impulse response has sufficiently converged.

FIG. 3 illustrates the convergence results for the OLMS, SOLMS, RLS, and standard LMS algorithms, wherein the length of the data sequence used to sound the echo channel was set to N=16. The number of iterations performed was plotted versus the error.sup.2 =(x-x'.sub.n).sup.T (x-x'.sub.n). As illustrated, both the OLMS and SOLMS algorithms feature rapid convergence speed, matching that of the RLS algorithm, and exhibited superior convergence to the traditional LMS method. In fact, as shown in FIG. 3, in 400 iterations, the LMS algorithm approached the convergence achieved by the SOLMS and OLMS in approximately 100 iterations.

FIG. 4 provides further detail of the results of FIG. 3 for the first 24 iterations. As can be seen, the OLMS and SOLMS converged almost as rapidly as the RLMS. In fact, the OLMS and SOLMS approximated the convergence of RLMS after only 17 or so iterations.

Claims

1. A method for estimating the echo impulse response, comprising:

transmitting with a transmitter a data sequence of length N to sound an echo channel, wherein said echo z.sub.n returns as a result of such sounding;

selecting data samples d.sub.n such that vector h.sub.n which is comprised of the data samples d.sub.n produces a data matrix which is diagonal to the Nth iteration;

receiving with an echo canceler the data samples d.sub.n and the echo z.sub.n ;

calculating with the echo canceler the echo impulse response x'.sub.n using a least means square algorithm and the data vector h.sub.n of data samples d.sub.n and the sounded echo z.sub.n, wherein the echo canceler estimates the echo impulse response x'.sub.n with the following equation: ##EQU9## wherein the echo impulse response x.sub.n is calculated out to N iterations.

2. The method of claim 1, wherein N=50.

3. A method for estimating the echo impulse response, comprising:

transmitting with a transmitter a data sequence of length N to sound an echo channel, wherein said echo z.sub.n returns as a result of such sounding;

selecting data samples d.sub.n such that vector h.sub.n which is comprised of the data samples d.sub.n produces a data matrix which is diagonal to the Nth iteration;

receiving with an echo canceler the data samples d.sub.n and the echo z.sub.n ;

calculating with the echo canceler the echo impulse response x'.sub.n using a least means square algorithm and the data vector h.sub.n of data samples d.sub.n and the sounded echo z.sub.n, wherein the echo canceler estimates the echo impulse response x'.sub.n with the following equation: ##EQU10## wherein k=ceiling (n/N); and calculating the echo impulse response x'.sub.n in more than N Iterations.

4. The method of claim 3, wherein the echo impulse response is calculated out to a multiple of N iterations.

5. The method of claim 4, wherein the estimated echo z'.sub.n is calculated by the echo canceler as follows: ##EQU11##

6. The method of claim 3, wherein the echo impulse response is calculated for 6 times N iterations.

7. A method for estimating the echo impulse response, comprising: transmitting with a transmitter a data sequence of length N to sound an echo channel, wherein said echo z.sub.n returns as a result of such sounding;

selecting data samples d.sub.n such that vector h.sub.n which is comprised of the data samples d.sub.n produces a data matrix which is diagonal to the Nth iteration;

receiving with an echo canceler the data samples d.sub.n and the echo z.sub.n ;

calculating with the echo canceler the echo impulse response x'.sub.n using a least means square algorithm and the data vector h.sub.n of data samples d.sub.n and the sounded echo z.sub.n wherein the echo canceler estimates the echo impulse response x'.sub.n with the following equation: ##EQU12## for N iterations; and wherein the echo canceler estimates the echo impulse response x'.sub.n with the following equation: ##EQU13## wherein k=ceiling (n/N); and for iterations in excess of N.