Patent Application
20190130929
This invention relates to the field of acoustics, and in particular to a method and apparatus for determining the inherent acoustic delay or audio latency in an audio system, and to an echo cancellation circuit.
It is often desirable to measure the time delay of a signal through an audio system. For example, it may be necessary to measure the difference in time delay as an audio signal passes through each of a plurality of speakers compared to a reference audio signal when trying to ensure synchronization between the speakers. Traditionally, time delay measurement techniques in an audio system involve using a known impulse signal. One method involves performing a cross-correlation between a transmitted impulse signal and the recorded audio signal. This method involves a training period while the adaptive algorithm adapts to the audio characteristics of the room, and requires calibration tones or known reference tones. Other methods include use of time-domain reflectometry where a pulse or a short sine wave burst is transmitted from the audio system. Measurements are then made of the timing of the return echo. The afore mentioned methods are susceptible to ambient noise and multi-modal reverberation and/or echo in a room. As a result, the recorded audio signal or return echo signal is never an exact replica of the original transmitted signal.
Also, adaptive filters are used for echo cancellation. In certain applications, such as in the case of TV set top boxes, the echo is delayed by an amount that exceeds the capacity of the adaptive filter. Increasing the filter size is not practical for digital signal processing reasons.
Embodiments of the invention address these problems by employing a converged adaptive filter coupled to a programmable delay buffer to determine the inherent delay of an audio system.
According to as aspect of the present invention there is provided an acoustic delay measurement apparatus for measuring an audio delay introduced by an audio system, comprising a programmable delay buffer operative to receive an audio stream from an audio source and output a reference signal representing the audio stream, said programmable delay buffer being operative to introduce a programmable delay into the audio stream; an adaptive filter responsive to said reference signal to generate an estimate signal to match on convergence of the adaptive filter an audio signal output by the audio system, said audio signal comprising said audio stream delayed by an amount representative of the audio delay introduced by the audio system; and a processor including a coefficient analysis block operative to read coefficients in the adaptive filter after convergence, compute a delay introduced into said estimate signal by the adaptive filter, and add the computed delay to the programmable delay buffer by to provide a measurement of said audio delay.
Initially the programmable delay may be set to zero. Alternatively, it could be set to a reasonable estimate of the anticipated delay in the audio system. If the adaptive filter converges, which is determined by comparing the error signal output by the filter with the audio signal, the delay introduced by the adaptive filter is computed.
If the adaptive filter fails to converge on reaching its maximum delay, a preset delay is programmed into the programmable delay buffer and the process repeated. This process can be done iteratively until the adaptive filter converges.
The adaptive filter may be a Least Mean Squares filter, but other types of filter may be employed. The filter delays the reference signal output by the programmable delay buffer and generates an estimate signal, which attempts to match the input audio signal. The difference between the two represents the error signal.
Once convergence has been found, the delay introduced by the adaptive filter into the estimated signal may be computed by finding the coefficient where a predetermined threshold value is exceeded.
If this coefficient represents the last coefficient in the adaptive filter, the programmable delay can again be set to a certain value, say 75% of the size of the adaptive filter, and the adaptive filter again allowed to converge. In this case, it is likely that the threshold will be exceeded before the coefficient index reaches the last coefficient in the filter. The effect of the programmable delay buffer is thus effectively to extend the size of the adaptive filter.
The delay measured by the apparatus in accordance with the invention can be used for various purposes. For example, it can be used to align the audio from being played out from different sources.
Also, it can be used to extend the effective size of an adaptive filter used for echo cancellation by incorporating a delay buffer in the reference path. The delay buffer is programmed with the delay measured by the measuring apparatus.
According to another aspect of the present invention, there is provided a computer-implemented method of determining audio latency in an audio system, comprising applying an audio stream obtained from an audio source to the audio system possessing latency; applying a first audio signal obtained from the audio system to a first input of an adaptive filter; applying a reference signal to a second input of said adaptive filter to permit said adaptive filter to generate an estimate signal of said first audio signal upon convergence, said reference signal being obtained from said audio stream after passing through a programmable delay buffer; if said adaptive filter fails to converge, incrementing said programmable delay buffer by a determined amount and restarting said adaptive filter; and upon convergence of said adaptive filter computing the delay introduced by said adaptive filter into said estimate signal and adding the computed delay to the programmable delay buffer to provide a measurement of the audio delay.
According to yet another aspect of the invention there is provided an echo cancellation circuit, comprising a first input for receiving an input signal potentially containing an echo signal; an output for outputting an output signal with said echo at least partially attenuated; a second input for receiving an audio signal subject to delay that produces said echo signal; an adaptive filter responsive to a reference signal and said output signal to generate an estimate of said echo signal; a programmed delay buffer introducing a pre-measured delay into said audio signal to provide said reference signal for said adaptive filter; and a subtractor for subtracting said estimate signal from said input signal to provide said output signal.
An audio stream 12 is output from an audio source 13. The audio stream 12 does not need to be a pre-known signal or tone, and can be, for example, any wideband signal. The audio stream 12 is input into the audio system 14 and to the acoustic delay measurement apparatus 10.
The audio system 14 has an inherent time delay or latency, which is commonly due to contributors such as analog-to-digital conversion, buffering, digital signal processing and digital-to-analog conversion. Due to the inherent latency of the audio system 14, a delay is created in the audio stream 12 resulting in a delayed audio stream 16, which is input to the speaker 18.
The purpose of the apparatus 10 is to measure this inherent delay. In order to make a measurement of this delay, the audio stream 12 is digitized in analog-to-digital converter 11 and input to the programmable delay buffer 20, which generates a programmable delay that may adjusted during the measurement process depending on the magnitude of the delay.
The delay introduced by the programmable delay buffer 20 is incremented when delay introduced by the audio system is greater than the maximum delay that can be offset by the adaptive filter. In this case part of the delay is offset in the programmable delay buffer 20, so the delay between the input signal In and the reference signal ref will be less than the total delay in the system by an amount equal to the delay programmed into the programmable delay buffer 20.
In order to make a measurement, the audio stream 12 is digitized in analog-to-digital converter 11 and input to the programmable delay buffer 20. The initial value of the adjustable delay is normally set to zero seconds, but is increased to a non-zero value if the delay is greater than the maximum delay available in the adaptive filter 24. Alternatively, the initial delay could be set to an approximation of the anticipated delay in the audio system.
The programmable delay buffer 20 outputs a reference signal ref representing the audio stream 12, which may be delayed by the amount programmed into the delay buffer 20, although as noted the reference signal may initially be output with zero delay. The reference signal ref is applied to one input of the adaptive filter 24.
Microphone 26 picks up the output of speaker 18, which corresponds to the delayed audio stream 16, and outputs a signal 28, which after passing through ADC 27 is applied as input signal In to the other input of the adaptive filter 24.
The adaptive filter 24 adjusts its filter coefficients automatically to generate an estimate signal est (
After the prescribed convergence time, which as noted is typically 1-2 seconds, adaptive filtering is stopped. Provided the average error signal err has fallen to an acceptable value, the current coefficients are analyzed in the coefficient analysis block 30 to compute the delay determined by the adaptive filter 24 that should be applied to the reference signal to match the input signal. If the error signal err is to above a predetermined threshold, the delay is incremented in the programmable delay buffer 20 and the adaptive filtering restarted based on the delayed reference signal.
The resultant coefficient values of the adaptive filter 24 follow a predictive pattern. The speaker 18 and mic 26 are acoustically coupled. The coefficients represent the profile of the acoustic coupling. The values of the first few coefficients during the delay period will be close to zero. Acoustically coupled coefficients will have larger values. One of the coefficients will show an abrupt increase in value as the first audio is received from the speaker 18.
The coefficient analysis block 30 determines the number of coefficients between the first coefficient and the coefficient having the abrupt change in value. Using this number, as explained below, coefficient analysis block 30 determines a delay value A by dividing the number of coefficients between the first coefficient and the coefficient having the abrupt change in value, by the sampling rate of the adaptive filter 24. In the present embodiment, the sampling rate of the adaptive filter 24 is assumed to be the same as the sampling rate of the programmable delay buffer 20. In instances where the sampling rate differs, translation is required. The delay value A is added to the delay in the programmable delay buffer 20, which gives the current delay for the audio system.
The signal In is compared with the estimated signal est in adder 40 to generate error signal err, which represents the error between the input signal In and the estimated signal est. It will be understood in this context that the term “adder” covers a subtractor since a minus value is added to a positive value to determine the error.
As noted, if the ratio of error signal err is above a threshold, for example, 10 dB of the input signal In, the delay in programmable delay buffer 20 is incremented to reduce the delay between the signals ref and In, and the process restarted. The programmable delay buffer 20 thus has the effect of increasing the apparent size of the adaptive filter 24.
Of interest is the number of coefficients between the first coefficient and the first coefficient showing the abrupt change in coefficient value that exceeds the threshold value. In
Initially at step 51, the delay in the delay buffer 20 is set to zero.
At step 52, the audio stream 12 is input to the audio system 14, which could, for example, be a TV sound bar or speaker, and the programmable delay buffer 20 of the measurement apparatus 10. As noted, the audio system 14 generates the inherent delay, which is to be measured and which is included in the audio stream that is output from speaker 18. The programmable delay buffer 20 is normally set initially to zero delay, or as noted above, it could be set to a known finite value.
At step 54, the coefficients of the adaptive filter are reset to zero. The signal In, representing the inherent delay of the audio system 14, is input to the signal input of the adaptive filter 24.
Adaptive filtering is performed at step 56. The adaptive filter 24 is allowed to converge on a solution. At step 58, after a prescribed convergence time, typically 1 to 2 seconds, a determination is made as to whether the average error signal err is less than a threshold amount of the microphone input signal In, in this non-limiting example, less than 10 dB of the microphone input signal In; if yes, the adaptive filter 24 is deemed converged and the coefficients are read by the coefficient analysis block 30 at step 60.
If not, a determination is made at step 59 whether the delay in the programmable delay buffer 20 has reached its maximum delay setting. If the answer is negative, control passes to step 70, where delay in the programmable delay buffer 24 is incremented, for example, by 75% of the maximum delay available in the adaptive filter 24.
Control then passes back to step 54, and the adaptive filtering process is restarted. Adaptive filtering is then recommenced with some of the delay between signal In and the audio stream already accounted for by the programmable delay buffer 24. For example, if the total delay in the audio system is Td, and the delay in the delay buffer is D, the new reference delay refnew presented to the adaptive filter 24 will be refnew=Td, −D. If not the adaptive filter 24 still does not converge, the sub-routine is repeated iteratively with a further delay added to the programmable delay buffer 24 until a programmable delay is found where the adaptive filter 24 will converge.
At step 62, the first coefficient xi is selected and a compared with a predetermined threshold value Th. A determination 66 is made as to whether the selected coefficient lies above the threshold value Th. As the initial coefficients will be at a value close to zero, the initial coefficients will have a value less than the coefficient threshold Th. If the coefficient value of the coefficient at index i is less than the coefficient threshold Th, the process moves to step 64, where the index i is incremented and the sub-routine repeated for the next selected coefficient until the first coefficient is found that is above the predetermined threshold, whereupon at step 72 the inter-block delay is computed as equal to the coefficient index/the sampling rate. The inter-block delay is the delay between the input signal In and the current reference signal ref which if the programmable delay buffer 20 has been incremented will include the delay introduced by the programmable delay buffer 20. The total delay will therefore be the inter-block delay plus the delay in the programmable delay buffer 20.
If at step 68 it is determined that the coefficient index is equal to the maximum filter size, meaning that the filter has no more capacity to increase the delay, control is passed to step 70, where the programmable delay buffer 20 is set to a percentage, in this non-limiting example, 75% of the maximum delay of the adaptive filter 24, and the sub-routine repeated to find a convergence value coefficient having a value that will likely be less than the maximum filter size because the delay between the input signal In and the reference signal ref will be less by an amount equal to 75% of the maximum delay of the adaptive filter.
At step 73 a determination is made as to whether there is room in the programmable delay buffer 20 to add inter-block delay; if not, an error is generated. If, yes control is passed to step 74.
At step 74, the inter-block delay is added to the current delay setting in the delay buffer and at step 76 the process is terminated. The measured delay, which is the total delay introduced by the audio system 14, can then be read out of the programmable delay buffer 20.
The measured delay can be used to align audio coming from a single source but being played out different systems. For example, a home audio system may use both Bluetooth and WiFi to send audio to different speakers. The inherent delay introduced by these devices is different and would be played out at slightly different times. With Amazon AVS being use to link multiple endpoints and designed by different end customers this becoming an important issue that needs to be solved.
Another example of where delay measurement is important in addressing the inherent delay in set top boxes. A tapped off version of the audio is passed to the TV or sound bar for further processing. This adds a significant amount of delay and pushes the echo portion of the return signal outside the filter. It seems like a common issue that needs to be solved for this application.
In this case a programmed delay buffer can be used to extend the effective size of the adaptive filter so that the adaptive filter can filter out echoes with a delay greater than its actual size.
This embodiment will cope with a delay greater than that which could be removed by the adaptive filter alone. As noted above there is a practical limit to the size of the adaptive filter 94. By pre-measuring and introducing part of the delay, which has been pre-measured by the circuit shown in
It should be appreciated by those skilled in the art that any block diagrams herein represent conceptual views of illustrative circuitry embodying the principles of the invention. For example, a processor may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared. Moreover, explicit use of the term “processor” should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (DSP) hardware, network processor, application specific integrated circuit (ASIC), field programmable gate array (FPGA), read only memory (ROM) for storing software, random access memory (RAM), and non volatile storage. Other hardware, conventional and/or custom, may also be included. The functional blocks or modules illustrated herein may in practice be implemented in hardware or software running on a suitable processor. The term circuit includes an assembly of coupled functional components.
| Number | Date | Country | |
|---|---|---|---|
| 62580603 | Nov 2017 | US |
