This invention relates to estimating features of a signal, particularly for the purpose of reducing noise in the signal. The features could be noise power and gain. The signal could be an audio signal.
There are many types of devices that detect and process speech signals. Examples include headsets and mobile phones. In those devices it is often desired to reduce the noise in the detected signal in order to more accurately represent the speech component of the signal. For instance, in a mobile phone or a headset any audio that is detected by a microphone may include a component representing a user's speech and a component arising from ambient noise. If that noise can be removed from the detected signal then the signal can sound better when it is played out, and it might also be possible to compress the signal more accurately or more efficiently. To achieve this, the noise component of the detected audio signal must be separated from the voice component.
If a speech signal s(n) is corrupted by additive background noise v(n), the resulting noisy speech signal d(n) can be expressed in the time domain as:
d(n)=s(n)+v(n) (1)
The objective of noise reduction in such a situation is normally to estimate v(n) and subtract it from d(n) to find s(n).
One algorithm for noise reduction operates in the frequency-domain. It tackles the noise reduction problem by employing a DFT (discrete Fourier transform) filter bank and tracking the average power of quasi-stationary background noise in each sub-band from the DFT. A gain value is derived for each sub-band based on the noise estimates, and those gain values are applied to each sub-band to generate an enhanced time domain signal in which the noise is expected to be reduced.
In general, it can be assumed that the speech signal and the background noise are independent, and thus the power of the noisy speech signal is equal to the power of the speech signal plus the power of background noise in each sub-band k
|Dk|2=Sk|2+|Vk|2. (2)
If the noise power is known then an estimate of the speech power can be got from:
|Sk|2=|Dk2−|Vk|2, (3)
It is necessary to estimate the gain in order to generate the signals Goms,k. One of the most widely used methods of estimating gain is by means of the optimal Wiener filter gain, which is computed as
The estimated clean speech signal in each sub-band, Ŝk, is then simply derived as
Ŝk=Gwiener,k·Dk. (5)
It can be identified that the estimation of noise power (|Vk|2) and gain (Goms) is crucial to the success of the algorithm. Unfortunately, obtaining reliable estimates of these has shown to be extremely difficult in the past due to the high complexity of various noisy environments. Many algorithms perform well in one situation but fail in other situations. Since the nature of the environment is not normally known in advance, and may change as a user moves from place to place, many algorithms provide inconsistent and unsatisfactory results.
It would therefore be valuable to have an improved mechanism for estimating noise power in a signal.
According to aspects of the present invention there are provided signal processing apparatus and methods as set out in the accompanying claims.
The present invention will now be described by way of example with reference to the accompanying drawings, in which:
The system described below estimates noise in an audio signal by means of an adaptive system having cascaded controller blocks.
This example will be described in the context of a device for estimating noise in a source audio signal.
Let a noise power estimate be denoted by Pk(l), where k is the sub-band index and l is the frame index of the data frame under consideration after processing by the analysis filter bank 10 with downsampling rate L. As shown by
For better illustration, in the following the operation of the modules 10 to 13 will be described in reverse order.
Adaptive Noise Estimation Module
Noise power Pk(l) is commonly estimated by applying a first-order IIR filter to the noisy signal power:
Pk(l)=Pk(l−1)+α(|Dk(l)|2−Pk(l−1)), (6)
where the parameter α is a constant between 0 and 1 that sets the weight applied to each frame, and hence the effective average time.
Adaptive noise estimation is achieved by weighting α in equation (6) dynamically with a speech absence probability (SAP) model. That model is described below.
Let H0 be the hypothesis of speech absence; then the speech absence probability (SAP) given an input signal in the frequency domain (D) is p(H0|D). For simplicity, time and frequency indices will be ignored in the description below. Applying Bayes' rule one obtains:
where λ is a constant between 0 and 1, inclusive, then for a complex Gaussian distribution of DFT coefficients (D), we have
where σD2 is the variance of D. (See Vary, P.; Martin, R. Digital Speech Transmission. Enhancement, Coding and Error Concealment, John Wiley-Verlag, 2006; Y. Ephraim and D. Malah, “Speech enhancement using a minimum mean-square error log-spectral amplitude estimator,” IEEE Trans. Acoustics, Speech and Signal Processing, vol. ASSP-33, pp. 443-445, 1985; and I. Cohen, “Noise Spectrum Estimation in Adverse Environments: Improved Minima Controlled Recursive Averaging,” IEEE Trans. Speech and Audio Processing, vol. 11, pp. 466-475, September 2003).
Combining equations 7 to 10 gives the conditional speech absence probability as being:
By substituting σD2 with instantaneous signal power |D|2, and also adding additional constraints to differentiate between different conditions, equation 11 can be re-written as
and the noise power estimation becomes
Pk(l)=Pk(l−1)+αqk(l)(|Dk(l)|2−Pk(l−1)). (13)
It can be observed that qk(l) reaches λ only when |Dk(l)|2 is equal to Pk(l), and approaches 0 when their difference increases. This feature allows smooth transitions to be tracked but prevents any dramatic variation from affecting the noise estimate. Note that setting qk(l)to λ when |Dk(l)|2 is smaller than Pk(l) enables full speed noise adaptation which can preserve weak speech segments better as it reduces the weight of previous noise estimates. The drawback of this is the noise estimates are biased toward lower values that results in less noise reduction. This can be mitigated in a manner described below.
The SAP model in equations 12 is derived from the energy ratio between a noisy speech signal and estimated noise within each individual frequency band. It does not take advantage of the following known facts:
By supposing that noise under consideration does not have those structures characteristic of speech, a more effective SAP model can be derived to detect speech or noise. One option is to modify equations 12 to incorporate cross-band averaging, in the following way:
where b(k) is a predefined bandwidth value for sub-band k.
Such cross-band averaging results in greater variance reduction on noise than on speech, and makes the SAP model more robust. However, excessive averaging (i.e. a value of b(k) that is too large) will reduce both frequency and time resolution, which can cause significant speech distortion. To avoid this bandwidth values should be selected to be in-keeping with the formants present in speech, for example:
Speech absence probability can alternatively be estimated by other voice activity detection algorithms, conveniently those that output SAP based on input signal power information.
Statistical Analysis Module
Adaptive noise estimation performed as described above may need a long time to converge when there is a sudden change of noise floor. One possible solution is to use minimum statistics to correct noise estimation. (See Rainer Martin, “Noise power spectral density estimation based on optimal smoothing and minimum statistics,” IEEE Transactions on speech and audio processing, vol. 9, no. 5, pp. 504-512, July 2001; Myron J. Ross, Harry L. Shaffer, Andrew Cohen, Richard Freudberg).
The approach employed in the present system essentially involves searching for a minimum value either:
In the present system, minimum statistics are used to control the adaptive noise estimator, whereby the requirement for high frequency resolution can be greatly relaxed. Specifically, instead of performing minimum tracking in each sub-band, we group frequency bins into several subsets and obtain one minimum value for each subset. The benefit of grouping is two-fold: (1) it reduces system complexity and resource cost; and (2) it smoothes out unwanted fluctuation. Without loss of generality, we split the spectrum into two groups in our implementation, which span low frequency and high frequency regions, respectively. More groups could be used, and non-adjacent portions of the frequency spectrum could be combined in a single group. For each group, a fixed length FIFO (first-in first-out) queue is formed by taking the summation of noisy signal power (|Dk(l)|2) for each frame. Finally one minimal value is identified for each queue.
Minimum statistics are used in the following way to aid adaptive noise estimation. Let Pmin,g(l) be the minimum power value for group g at frame index l determined in the manner described above, and let Psum,g(l) represent the total estimated noise power for group g at frame l. Then a correction factor C is derived as
The control of noise estimation using minimum statistics is realized through applying this correction factor to the noise estimates Pk(l).
To take further advantage of minimum statistics information, a more complex scheme can be used. The range of C {C≧0} can be divided into four zones by defining two threshold values T1 and T2, where T1<1<T2. Then a state machine is implemented as shown in
When the minimum Pmin,g(l) is only slightly lower than estimated noise power Psum,g(l) as in state 2 (T1≦C≦1), nothing needs to be done because this is fully expected. However, if the minimum value is significantly smaller than noise estimate as in state 1 (C<T1) then a correction is triggered. State 1 corresponds to a condition where noise becomes mistakenly adapted to speech level or there is a sudden drop of noise floor. To avoid over-adjustment, the correction factor C is normalized by T1 so that the corrected noise estimates are still higher than the minimum value. When Pmin,g(l) is greater than Psum,g(l) as in state 3 (1<C≦T2), simple correction is applied as there might be a sudden jump of noise floor and our noise estimate is lagging behind. Special treatment is needed when the minimum value (Pmin,g(l)) is significantly higher than the noise estimate (Psum,g (l)) as in state 4 (C>T2). A plain correction of multiplying by the correction factor may run into problems when there is a substantial spectrum mismatch between the old noise floor and the new noise floor. It may take very long time to converge to the new noise spectrum. Or, even more problematically, narrow band noise could be produced which might well create annoying audio artefacts. This is addressed in the state machine of
Harmonicity Module
The minimum-search window duration has a crucial impact on noise estimation. A short window allows faster response to noise variation but may also misclassify speech as noise when continuous phonation is longer than the window length. A long window on the other hand will slow down noise adaptation. One approach is to define an advantageous window length empirically, but this may not suit a wide range of situations. Instead, the present system employs a dynamic window length which can vary during operation. In this example the window length is controlled by speech harmonicity (periodicity).
There are many ways to determine harmonicity of speech. AMDF (Average Magnitude Difference Function) is one method, and is described in Harold J. Manley; Average magnitude difference function pitch extractor, IEEE Trans. Acoust., Speech, Signal Processing, vol. 22, pp. 353-362, October 1974. A variant of AMDF is CAMDF (Cross Average Magnitude Difference Function). CAMDF has been found to be relatively efficient and to provide relatively good performance.
For a short-term signal x(n) {n:0 . . . N−1} CAMDF can be defined as below:
where τ is the lag value that is subject to the constraint 0<τ≦N−U.
One representation of harmonicity based on CAMDF can simply be the ratio between its minimum and maximum:
A harmonicity value is conventionally used directly to determine voicing status. However, its reliability degrades significantly in a high noise environment. On the other hand, under medium to high SNR conditions, harmonicity offers some unique yet important information previously unavailable to adaptive noise estimation and minimum statistics which exploit mostly energy variation patterns. The present system uses harmonicity to control the manner of operation of the statistical analysis module. Specifically, when a frame is classified as voiced by the harmonicity function, it is skipped by the minimum statistics calculation. This is equivalent to lengthening the minimum search window duration when speech is present. As a result, the default search duration can be set relatively short for fast noise adaptation.
The harmonicity detector/module can be alternatively implemented through other pitch detectors described in the literature, for example by auto-correlation. However, it is preferable to use a simpler method than fully-fledged pitch detection since pitch detection is computationally intensive. Alternatives include determining any one or more of harmonicity, periodicity and voicing and/or by analysing over a partial pitch range. If voicing is used then the detector need not perform any pitch detection.
Instant Noise Estimation Using Fourier Transform of AMDF and Variable Start Minima Search [Zhong Lin; Goubran, R.; Acoustics, Speech, and Signal Processing, 2005. Proceedings. (ICASSP apos;05). Volume 1, Issue, Mar. 18-23, 2005 Page(s): 161-164 discloses a speech processor that employs a speech detector based on Fourier Transform of AMDF that running in parallel with Variable Start Minima Search. Such a parallel approach—unlike the cascading approach described herein—increases the system's sensitivity to speech detector failures and can be computationally less efficient.
Hybrid Gain from Wiener Filter with Over-Subtraction and MMSE-LSA
Gain calculated based on the Wiener filter in equation 4 often results in musical noise. One of the commonly used solutions is to use over-subtraction during gain calculation as shown below.
where β is the over-subtraction factor.
As mentioned earlier, the noise estimate Pk(l) in the present system can be found to be biased toward lower values. Thus, using over-subtraction also compensates noise estimation to achieve greater noise reduction.
In the present system, an adaptive over-subtraction scheme is used, which is based on the SAP obtained as described above. First, let βmin and βmax be the minimum and maximum over-subtraction values, respectively. Then in a similar manner to the analysis performed in the statistical analysis module described above, and ignoring time and frequency subscripts for simplicity, we divide the range of speech absence probability q into three zones by defining two threshold values QS and QN such that 0<QS<QN<1. This represents a crude categorization of SAP into speech only, speech mixed with noise, and noise only states, respectively. Finally we use a state machine to determine the value of over-subtraction factor β. The state machine is illustrated in
In state 1 (speech only) or state 3 (noise only), β is simply set to the pre-determined minimum or the maximum over-subtraction values respectively. In state 2 which corresponds to a mixed speech and noise condition, β is calculated by linear interpolation between βmin and βmax based on SAP q. With properly selected threshold values, over-subtraction can effectively suppress musical noise and achieve significant noise reduction overall.
To further suppress musical noise, additional processing is applied to the instantaneous gain Gwiener,k(l).
Because noise is a random process, the true noise power at any instance varies around the noise estimate Pk(l). When Gwiener,k(l) is much larger than Pk(l), the fluctuation of noise power is minor compared to |Dk(l)|2, and hence Gwiener,k(l) is very reliable and its normalized variance is small. On the other hand, when |Dk(l)|2 approximates Pk(l), the fluctuation of noise power becomes significant, and hence Gwiener,k(l) is unreliable and its normalized variance is large. If Gwiener,k(l) is left without further smoothing, the large normalized variance in low SNR periods would cause musical or watering artefacts. However, if a constant average rate is used to suppress these artefacts, it would cause over smoothing in high SNR periods and thus results in tonal or ambient artefacts. To achieve the same normalized variation for the gain factor, the average rate needs to be proportional to the square of the gain. Therefore the final gain factor Gk(l) is computed by smoothing Gwiener,k(l) with the following algorithm:
Gk(l)=Gk(l−1)+(αG·G0,k2(l))(Gweiner,k(l)−Gk(l−1)), (23)
G0,k(l)=Gk(l−1)+0.25(Gwiener,k(l)−Gk(l−1)), (24)
where αG is a time constant between 0 and 1, and G0,i(k) is a pre-estimate of Gk(l) based on the latest gain estimate Gk(l−1) and the instantaneous Wiener gain G0,k(l). Using a variable average rate G0,k2(l), and specifically one based on a pre-estimate of the moderated Wiener gain value, to smooth the Wiener gain can help regulate the normalized variance in the gain factor Gk(l)
It can be observed that Gk(l) is averaged over a long time when it is close to 0, but is with very little average when it approximates 1. This creates a smooth noise floor while avoiding generating ambient-sounding (i.e. thin, watery-sounding) speech.
While over-subtraction and gain smoothing create a smooth noise floor and achieve significant noise reduction, they could also cause speech distortion, particularly on weak speech components. To improve voice quality, we choose MMSE-LSA gain function described in Ephraim and D. Malah to replace equation 21 for certain conditions which will be specified later.
The formulation of MMSE-LSA is described below.
First, define:
where γ is the a posteriori SNR, and ξ is the a priori SNR.
Then the MMSE-LSA gain function is:
In MMSE-LSA, a priori SNR ξ is the dominant factor, which enables filter to produce less musical noise and better voice quality. However, because of the diminishing role of a posteriori SNR γ, on which the over-subtraction can be applied, the noise reduction level of MMSE-LSA is limited. For this reason the present system only uses MMSE-LSA for speech dominant frequency bands of voiced frames. This is because on those frames: (1) speech quality matters most, and (2) less noise reduction may be tolerable as some noise components might be masked by stronger speech components.
Results
Tests using the system described above have indicated that the system can achieve over 20 dB noise reduction while preserving high voice quality. The system has been found to perform well from quiet to high noise conditions. It has also been found to have a fast convergence time of less than 0.5 seconds in some typical environments. These results place it among the best currently available algorithms for single microphone noise reduction performance.
The system described above can be used to estimate noise power and/or gain for use in a noise reduction system of the type shown in
The system described above can be implemented in any device that processes audio data. Examples include headsets, phones, radio receivers that play back speech signals and stand-alone microphone units.
The system described above could be implemented in dedicated hardware or by means of software running on a microprocessor. The system is preferably implemented on a single integrated circuit.
The inventors hereby disclose in isolation each individual feature described herein and any combination of two or more such features, to the extent that such features or combinations are capable of being carried out based on the present specification as a whole in the light of the common general knowledge of a person skilled in the art, irrespective of whether such features or combinations of features solve any problems disclosed herein, and without limitation to the scope of the claims. The inventors indicate that aspects of the present invention may consist of any such individual feature or combination of features. In view of the foregoing description it will be evident to a person skilled in the art that various modifications may be made within the scope of the invention.
Number | Name | Date | Kind |
---|---|---|---|
6023674 | Mekuria | Feb 2000 | A |
6122610 | Isabelle | Sep 2000 | A |
6459914 | Gustafsson et al. | Oct 2002 | B1 |
6529868 | Chandran et al. | Mar 2003 | B1 |
RE38269 | Liu | Oct 2003 | E |
6810273 | Mattila et al. | Oct 2004 | B1 |
6862567 | Gao | Mar 2005 | B1 |
6980950 | Gong et al. | Dec 2005 | B1 |
7031916 | Li et al. | Apr 2006 | B2 |
7043428 | Li | May 2006 | B2 |
7117148 | Droppo et al. | Oct 2006 | B2 |
7181390 | Droppo et al. | Feb 2007 | B2 |
7447630 | Liu et al. | Nov 2008 | B2 |
7590530 | Zhao et al. | Sep 2009 | B2 |
7680653 | Yeldener | Mar 2010 | B2 |
7873114 | Lin | Jan 2011 | B2 |
7912231 | Yang et al. | Mar 2011 | B2 |
8015002 | Li et al. | Sep 2011 | B2 |
8364479 | Schmidt et al. | Jan 2013 | B2 |
8412520 | Furuta et al. | Apr 2013 | B2 |
8571231 | Ramakrishnan et al. | Oct 2013 | B2 |
8577675 | Jelinek | Nov 2013 | B2 |
20050027520 | Mattila et al. | Feb 2005 | A1 |
20070055508 | Zhao et al. | Mar 2007 | A1 |
20080140395 | Yeldener | Jun 2008 | A1 |
20080243496 | Wang | Oct 2008 | A1 |
20080281589 | Wang et al. | Nov 2008 | A1 |
20090254340 | Sun et al. | Oct 2009 | A1 |
Number | Date | Country |
---|---|---|
1635331 | Mar 2006 | EP |
2006114101 | Nov 2006 | WO |
Entry |
---|
Cohen, I., “Noise spectrum estimation in adverse environments: improved minima controlled recursive averaging,” Speech and Audio Processing, IEEE Transactions on , vol. 11, No. 5, pp. 466,475, Sep. 2003. |
Z. Lin , R. A. Goubran and R. M. Dansereau “Noise estimation using speech/non-speech frame decision and subband spectral tracking”, Speech Commun., vol. 49, pp. 542-557 2007. |
Rangachari, Sundarrajan, and Philipos C. Loizou. “A noise-estimation algorithm for highly non-stationary environments.” Speech communication 48.2 (2006): 220-231. |
Rangachari, Sundarrajan, Philipos C. Loizou, and Yi Hu. “A noise estimation algorithm with rapid adaptation for highly nonstationary environments.” Acoustics, Speech, and Signal Processing, 2004. Proceedings.(ICASSP'04). IEEE International Conference on. vol. 1. IEEE, 2004. |
Lin, L.; Holmes, W.H.; Ambikairajah, E.; , “Adaptive noise estimation algorithm for speech enhancement,” Electronics Letters , vol. 39, No. 9, pp. 754-755, May 1, 2003 doi: 10.1049/el:20030480. |
Martin, R.; , “Noise power spectral density estimation based on optimal smoothing and minimum statistics,” Speech and Audio Processing, IEEE Transactions on , vol. 9, No. 5, pp. 504-512, Jul. 2001 doi: 10.1109/89.928915 URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=928915&isnumber=20081. |
Li Hui; Bei-qian Dai; Lu Wei; , “A Pitch Detection Algorithm Based on AMDF and ACF,” Acoustics, Speech and Signal Processing, 2006. ICASSP 2006 Proceedings. 2006 IEEE International Conference on , vol. 1, no., pp. I, May 14-19, 2006 doi: 10.1109/ICASSP.2006.1660036 URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1660036&isnumber=34757. |
Zhong Lin; Goubran, R.; , “Instant Noise Estimation Using Fourier Transform of AMDF and Variable Start Minima Search,” Acoustics, Speech, and Signal Processing, 2005. Proceedings. (ICASSP '05). IEEE International Conference on , vol. 1, no., pp. 161-164, Mar. 18-23, 2005 doi: 10.1109/ICASSP.2005.1415075. |
Zhong et al., “Instant Noise Estimation Using Fourier Transform of AMDF and Variable Start Minima Search,” 2005 IEEE International Conference on Acoustics, Speech, and Signal Processing, Mar. 18, 2005, pp. 161-164, vol. 1, IEEE, Piscataway, NJ. |
Cohen, “Noise Spectrum Estimation in Adverse Environments: Improved Minima Controlled Recursive Averaging,” IEEE Transactions on Speech and Audio Processing, Sep. 1, 2003, pp. 466-475, vol. 11, No. 5, IEEE Service Center, New York, NY. |
Tilp, “Verfahren zur Verbesserung gestoerter Sprachsignale unter Beruecksichtigung der Gundfrequenz stimmhafter Sprachlaute,” VDI Verlag, 2002, sections 2.3.2, 2.3.4, 0178-9627, Duesseldorf, Germany. |
David Malah, et al., “Tracking Speech-Presence Uncertainty to Improve Speech Enhancement in Non-Stationary Noise Environments,” AT&T Labs-Research, Florham Park, NJ 07932. |
Number | Date | Country | |
---|---|---|---|
20090254340 A1 | Oct 2009 | US |