Patent Grant
8712074
The invention relates to identification of noise in acoustic signals, e.g. speech signals, using fast noise power spectral density tracking. The invention relates specifically to a method of estimating noise power spectral density PSD in an input sound signal comprising a noise signal part and a target signal part.
The invention furthermore relates to a system for estimating noise power spectral density PSD in an input sound signal comprising a noise signal part and a target signal part.
The invention furthermore relates to use of a system according to the invention, to a data processing system and to a computer readable medium.
The invention may e.g. be useful in listening devices, e.g. hearing aids, mobile telephones, headsets, active earplugs, etc.
In order to increase quality and decrease listener fatigue of noisy speech signals that are processed by digital speech processors (e.g. hearing aids or mobile telephones) it is often desirable to apply noise reduction as a pre-processor. Noise reduction methods can be grouped in methods that work in a single-microphone setup and methods that work in a multi-microphone setup.
The focus of the current invention is on single-microphone noise reduction methods. An example where we can find these methods is in the so-called completely in the canal (CIC) hearing aids. However, the use of this invention is not restricted to these single-microphone noise reduction methods. It can easily be combined with multi-microphone noise reduction techniques as well, e.g., in combination with a beam former as a post-processor.
With these noise reduction methods it is possible to remove the noise from the noisy speech signal, i.e., estimate the underlying clean speech signal. However, to do so it is required to have some knowledge of the noise. Usually it is necessary to know the noise power spectral density (PSD). In general the noise PSD is unknown and time-varying as well (dependent on the specific environment), which makes noise PSD estimation a challenging problem.
When the noise PSD is estimated wrongly, too much or too little noise suppression will be applied. For example, when the actual noise level suddenly decreases and the estimated noise PSD is overestimated too much suppression will be applied with a resulting loss of speech quality. When, on the other hand, the noise level suddenly increases, an underestimated noise level will lead to too little noise suppression leading to the generation of excess residual noise, which again decreases the signal quality and increases listeners' fatigue.
Several methods have been proposed in the literature to estimate the noise PSD from the noisy speech signal. Under rather stationary noise conditions the use of a voice activity detector (VAD) [KIM 99] can be sufficient for estimation of the noise PSD. With a VAD the noise PSD is estimated during speech pauses. However, VAD based noise PSD estimation is likely to fail when the noise is non-stationary and will lead to a large estimation error when the noise level or spectrum changes. An alternative for noise PSD estimation are methods based on minimum statistics (MS) [Martin 2001].
These methods do not rely on the use of a VAD, but make use of the fact that the power level in a noisy speech signal at a particular frequency bin seen across a sufficiently long time interval will reach the noise-power level. The length of the time interval provides a trade off between how fast MS can track a time-varying noise PSD on one hand and the risk to overestimate the noise PSD on the other hand.
Recently in [Hendriks 2008] a method was proposed for noise tracking which allows estimation of the noise PSD when speech is continuously present. Although the method proposed in [Hendriks 2008] has been shown to be very effective for noise PSD estimation under non-stationary noise conditions and can be implemented in MATLAB in real-time on a modern PC, the necessary eigenvalue decompositions might be too complex for applications with very low-complexity constraints, e.g. due to power consumption limitations, e.g. in battery driven devices, such as e.g. hearing aids.
As do the methods described in [Martin 2001] and [Hendriks 2008], the present invention aims at noise PSD estimation. The advantage of the proposed method over methods proposed in the aforementioned references is that with the proposed method it is possible to accurately estimate the noise PSD, i.e., also when speech is present, at relatively low computational complexity.
An object of the present invention is to provide a scheme for estimating the noise PSD in an acoustic signal consisting of a target signal contaminated by acoustic noise.
Objects of the invention are achieved by the invention described in the accompanying claims and as described in the following.
A Method:
An object of the invention is achieved by a method of estimating noise power spectral density PSD in an input sound signal comprising a noise signal part and a target signal part. The method comprises
d) providing a digitized electrical input signal to a control path and performing;
d1) storing a number of time frames of the input signal each comprising a predefined number N2 of digital time samples xn (n=1, 2, . . . , N2), corresponding to a frame length in time of L2=N2/fs;
d2) performing a time to frequency transformation of the stored time frames on a frame by frame basis to provide corresponding spectra Y of frequency samples;
d3) deriving a periodogram comprising the energy content |Y|2 for each frequency sample in a spectrum, the energy content being the energy of the sum of the noise and target signal;
d4) applying a gain function G to each frequency sample of a spectrum, thereby estimating the noise energy level |Ŵ|2 in each frequency sample, |Ŵ|2=G·|Y|2;
d5) dividing the spectra into a number Nsb2 of sub-bands, each sub-band comprising a predetermined number nsb2 of frequency samples, and assuming that the noise PSD level is constant across a sub-band;
d6) providing a first estimate |{circumflex over (N)}|2 of the noise PSD level in a sub-band based on the non-zero estimated noise energy levels of the frequency samples in the sub-band;
d7) providing a second, improved estimate |Ñ|2 of the noise PSD level in a sub-band by applying a bias compensation factor B to the first estimate, |Ñ|2=B·|{circumflex over (N)}|2.
This has the advantage of providing an algorithm for estimating noise spectral density in an input sound signal.
In the spectra of frequency samples resulting from the time to frequency domain transformation, the frequency samples (e.g. X) are generally complex numbers, which can be described by a magnitude |X| and a phase angle arg(X).
In the present context the ‘descriptors’ ^ and {tilde over ( )} on top of a parameter, number or value e.g. G or I (i.e. Ĝ and Ĩ, respectively) are intended to indicate estimates of the parameters G and I. When e.g. an estimate of the absolute value of the parameter, ABS(G), here written as |G|, an estimate of the absolute value should ideally have the descriptor outside the ABS or |.|-signs, but this is, due to typographical limitations not always the case in the following description. It is however intended that e.g. |Ĝ| and |Ĩ|2 should indicate an estimate of the absolute value (or magnitude) |G| of the parameter G and an estimate of the magnitude squared |I|2 (i.e. neither the absolute value of the estimate Ĝ of G nor the magnitude squared of the estimate Ĩ of I). Typically the parameters or numbers referred to are complex.
In a preferred embodiment, the method further comprises a step d8) of providing a further improved estimate of the noise PSD level in a sub-band by computing a weighted average of the second improved estimate of the noise energy levels in the sub-band of a current spectrum and the corresponding sub-band of a number of previous spectra. This has the advantage of reducing the variance of the estimated noise PSD.
In a preferred embodiment, the step d1) of storing time frames of the input signal further comprises a step d1.1) of providing that successive frames having a predefined overlap of common digital time samples.
In a preferred embodiment, the step d1) of storing time frames of the input signal further comprises a step d1.2) of performing a windowing function on each time frame. This allows the control of the trade-off between the height of the side-lobes and the width of the main-lobes in the spectra.
In a preferred embodiment, the step d1) of storing time frames of the input signal further comprises a step d1.3) of appending a number of zeros at the end of each time frame to provide a modified time frame comprising a number K of time samples, which is suitable for Fast Fourier Transform-methods, the modified time frame being stored instead of the un-modified time frame.
In a preferred embodiment, the number of time samples K is equal to 2p, where p is a positive integer. This has the advantage of providing the possibility to use a very efficient implementation of the FFT algorithm.
In a preferred embodiment, a first estimate |{circumflex over (N)}|2 of the noise PSD level in a sub-band is obtained by averaging the non-zero estimated noise energy levels of the frequency samples in the sub-band, where averaging represent a weighted average or a geometric average or a median of the non-zero estimated noise energy levels of the frequency samples in the sub-band.
In a preferred embodiment, one or more of the steps d6), d7) and d8) are performed for several sub-bands, such as for a majority of sub-bands, such as for all sub-bands of a given spectrum. This adds the flexibility that the proposed algorithm steps can be applied to a sub-set of the sub-bands, in the case that it is known beforehand that only a sub-set of the sub-bands will gain from this improved noise PSD estimation.
In a preferred embodiment, the steps of the method are performed (repeated) for a number of consecutive time frames, such as continually.
In a preferred embodiment, the method comprises the steps
a1) converting the input sound signal to an electrical input signal;
a2) sampling the electrical input signal with a predefined sampling frequency fs to provide a digitized input signal comprising digital time samples xn;
b) processing the digitized input signal in a, preferably relatively low latency, signal path and in a control path, respectively.
In a preferred embodiment, the method comprises providing a digitized electrical input signal to the signal path and performing
c1) storing a number of time frames of the input signal each comprising a predefined number N1 of digital time samples xn (n=1, 2, . . . , N1), corresponding to a frame length in time of L1=N1/fs;
c2) performing a time to frequency transformation of the stored time frames on a frame by frame basis to provide corresponding spectra X of frequency samples;
c5) dividing the spectra into a number Nsb1 of sub-bands, each sub-band comprising a predetermined number nsb1 of frequency samples.
In a preferred embodiment, the frame length L2 of the control path is larger than the frame length L1 of the signal path, e.g. twice as large, such as 4 times as large, such as eight times as large. This has the advantage of providing a higher frequency resolution in the spectra used for noise PSD estimation.
In a preferred embodiment, the number of sub-bands of the signal path Nsb1 and control path Nsb2 are equal, Nsb1=Nsb2. This has the effect that for each of the sub-bands in the control path there is a corresponding sub-band in the signal path.
In a preferred embodiment, the number of frequency samples nsb1 per sub-band of the signal path is one.
In a preferred embodiment, step c1) relating to the signal path of storing time frames of the input signal further comprises a step c1.1) of providing that successive frames having a predefined overlap of common digital time samples.
In a preferred embodiment, step c1) relating to the signal path of storing time frames of the input signal further comprises a step c1.2) of performing a windowing function on each time frame. This has the effect of allowing a tradeoff between the height of the side-lobes and the width of the main-lobes in the spectra
In a preferred embodiment, step c1) relating to the signal path of storing time frames of the input signal further comprises a step c1.3) of appending a number of zeros at the end of each time frame to provide a modified time frame comprising a number J of time samples, which is suitable for Fast Fourier Transform-methods, the modified time frame being stored instead of the un-modified time frame.
In a preferred embodiment, the number of samples J is equal to 2q, where q is a positive integer. This has the advantage of enabling a very efficient implementation of the FFT algorithm.
In a preferred embodiment, the number K of samples in a time frame or spectrum of a signal of the control path is larger than or equal to the number J of samples in a time frame or spectrum of a signal of the signal path.
In a preferred embodiment, the second, improved estimate |Ñ|2 of the noise PSD level in a sub-band is used to modify characteristics of the signal in the signal path.
In a preferred embodiment, the second, improved estimate |Ñ|2 of the noise PSD level in a sub-band is used to compensate for a persons' hearing loss and/or for noise reduction by adapting a frequency dependent gain in the signal path.
In a preferred embodiment, the second, improved estimate |Ñ|2 of the noise PSD level in a sub-band is used to influence the settings of a processing algorithm of the signal path.
A System:
A system for estimating noise power spectral density PSD in an input sound signal comprising a noise signal part and a target signal part is furthermore provided by the present invention.
It is intended that the process features of the method described above, in the detailed description of ‘mode(s) for carrying out the invention’ and in the claims can be combined with the system, when appropriately substituted by corresponding structural features.
The system comprises
Embodiments of the system have the same advantages as the corresponding methods.
In a particular embodiment, the system further comprises a second estimating unit for providing a further improved estimate of the noise PSD level in a sub-band by computing a weighted average of the second improved estimate of the noise energy levels in the sub-band of a current spectrum and the corresponding sub-band of a number of previous spectra.
In a particular embodiment, the system is adapted to provide that the memory for storing a number of time frames of the input signal comprises successive frames having a predefined overlap of common digital time samples.
In a particular embodiment, the system further comprises a windowing unit for performing a windowing function on each time frame.
In a particular embodiment, the system further comprises an appending unit for appending a number of zeros at the end of each time frame to provide a modified time frame comprising a number K of time samples, which is suitable for Fast Fourier Transform-methods, and wherein the system is adapted to provide that a modified time frame is stored in the memory instead of the un-modified time frame.
In a particular embodiment, the system further comprises one or more microphones of the hearing instrument picking up a noisy speech or sound signal and converting it to an electric input signal and a digitizing unit, e.g. an analogue to digital converter to provide a digitized electrical input signal. In a particular embodiment, the system further comprises an output transducer (e.g. a receiver) for providing an enhanced signal representative of the input speech or sound signal picked up by the microphone. In a particular embodiment, the system comprises an additional processing block adapted to provide a further processing of the input signal, e.g. to provide a frequency dependent gain and possibly other signal processing features.
In a particular embodiment, the system form part of a voice controlled devices, a communications device, e.g. a mobile telephone or a listening device, e.g. a hearing instrument.
Use:
Use of a system as described above, in the section describing mode(s) for carrying out the invention and in the claims is moreover provided by the present invention.
In a preferred embodiment, use in a hearing aid is provided. In an embodiment, use in communication devices, e.g. mobile communication devices, such as mobile telephones, is provided. Use in a portable communications device in acoustically noisy environments is provided. Use in an offline noise reduction application is furthermore provided.
In a preferred embodiment, use in voice controlled devices is provided (a voice controlled device being e.g. a device that can perform actions or influence decisions on the basis of a voice or sound input.
A Data Processing System:
In a further aspect, a data processing system is provided, the data processing system comprising a processor and program code means for causing the processor to perform at least some of the steps of the method described above, in the detailed description of ‘mode(s) for carrying out the invention’ and in the claims. In an embodiment, the program code means at least comprise the steps denoted d1), d2), d3), d4), d5), d6), d7). In an embodiment, the program code means at least comprise some of the steps 1-8 such as a majority of the steps such as all of the steps 1-8 of the general algorithm described in the section ‘General algorithm’ below.
A Computer Readable Medium
In a further aspect, a computer readable medium is provided, the computer readable medium storing a computer program comprising program code means for causing a data processing system to perform at least some of the steps of the method described above, in the detailed description of ‘mode(s) for carrying out the invention’ and in the claims, when said computer program is executed on the data processing system. In an embodiment, the program code means at least comprise the steps denoted d1), d2), d3), d4), d5), d6), d7). In an embodiment, the program code means at least comprise some of the steps 1-8 such as a majority of the steps such as all of the steps 1-8 of the general algorithm described in the section ‘General algorithm’ below.
Further objects of the invention are achieved by the embodiments defined in the dependent claims and in the detailed description of the invention.
As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well (i.e. to have the meaning “at least one”), unless expressly stated otherwise. It will be further understood that the terms “includes,” “comprises,” “including,” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being “connected” or “coupled” to another element, it can be directly connected or coupled to the other element or intervening elements maybe present, unless expressly stated otherwise. Furthermore, “connected” or “coupled” as used herein may include wirelessly connected or coupled. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items. The steps of any method disclosed herein do not have to be performed in the exact order disclosed, unless expressly stated otherwise.
The invention will be explained more fully below in connection with a preferred embodiment and with reference to the drawings in which:
The figures are schematic and simplified for clarity, and they just show details which are essential to the understanding of the invention, while other details are left out. Throughout, the same reference numerals are used for identical or corresponding parts.
Further scope of applicability of the present invention will become apparent from the detailed description given hereinafter. However, it should be understood that the detailed description and specific examples, while indicating preferred embodiments of the invention, are given by way of illustration only, since various changes and modifications within the spirit and scope of the invention will become apparent to those skilled in the art from this detailed description.
The proposed general scheme for noise PSD estimation is outlined in
While a standard low-latency noise reduction system normally divides the noisy signal in small frames in order to fulfil both stationarity and low-delay constraints, we propose here to use two potentially different frame sizes. One of them is used in the signal path and should fulfil normal low delay constraints. These time-frames we call the DFT1 analysis frames. The other one is used in the control path in order to estimate the noise PSD. These frames can (but need not) be chosen longer in size since they do not need to fulfil the low-delay constraint. These time-frames we call DFT2 frames. Let L1 and L2 be the length of the DFT1 and DFT2 analysis frame in samples, with L2≧L1. In
For notational convenience, we denote the set of DFT2 bin indices belonging to sub-band j, as Bj. For the DFT1 coefficients we will use the following frequency domain notation
X(j,m)=Z(j,m)+N(j,m), jε{0,K,J−1},
where X(j,m), Z(j,m) and N(j,m) are the noisy speech, clean speech and noise DFT1 coefficient, respectively, at a DFT1 frequency bin with index-number j and at a time-frame with index-number m.
For the DFT2 coefficients we will use a similar frequency domain notation, i.e.,
Y(k,m)=S(k,m)+W(k,m), kε{0,K,K−1},
where Y(k,m), S(k,m) and W(k,m) are the noisy speech, clean speech and noise DFT2 coefficient, respectively, at a DFT2 frequency bin with index-number k and at a time-frame with index-number m.
General Algorithm:
The purpose of this invention is to estimate the noise power spectral density (PSD), defined as
σN2(j,m)=E└|N(j,m)|2┘,
To do so, we propose the following algorithm.
The algorithm operates in the frequency domain, and consequently the first step is to transform the noisy input signal to the frequency domain.
Each noisy DFT2 periodogram bin |Y(k,m)|2 may contain signal components from the target signal (e.g. the speech signal in which one is eventually interested), and generally contains signal components from the background noise. It is possible to estimate the energy of the noise in each DFT2 bin by applying a gain to the noisy DFT2 periodogram, i.e.,
|W(k,m)|2=G(k,m)|Y(k,m)|2.
The gain function G(k,m) could be a function of several quantities, e.g. the so-called a posteriori SNR and the a-priori SNR, see below for details.
Assuming that the unknown noise PSD is constant within a sub band, the noise PSD level within the sub-band can be estimated as the average across the estimated (non-zero) noise energy levels |Ŵ(k,m)|2 computed in the previous step. To do so, let Ω(j,m) denote the set of DFT2 bin indices in sub-band j that have a gain function G(k,m)>0.
Other ways are possible for combining the DFT noise energy levels |Ŵ(k,m)|2 into sub-band noise level estimates |{circumflex over (N)}(j,m)|2. For example, one could compute a geometric mean value across the sub-band, rather than the arithmetic mean shown above.
The noise energy level |{circumflex over (N)}(j,m)|2 computed in this step can be seen as a first estimate of the noise PSD within the sub band. However, in many cases, this noise PSD level may be biased. For this reason, a bias compensation factor B(j,m) is applied to the estimate in order to correct for the bias. The bias compensation factor is a function of the applied gain functions G(k,m), kεBj. For example, it could be a function of the number of non-zero gain values G(k,m), kεBj, which is in fact the cardinality of the set Ω(j,m).
The bias factor B(j,m) generally depends on choices of L2 and K, and can e.g. be found off-line, prior to application, using the “training procedure” outlined in [Hendriks 08]. In one example of the proposed system, the values of B(j,m) are in the range 0.3-1.0.
The quantity |Ñ(j,m)|2 is an improved estimate of the noise PSD in sub-band j. Assuming that the noise PSD changes relatively slowly across time, the variance of the estimate can be reduced by computing an average of the estimate and those of the previous frames. This may be accomplished efficiently using the following first-order smoothing strategy.
The quantity
{circumflex over (σ)}N2(j,m)
is the final estimate of the noise PSD in sub band j. In order to be able to proceed with the next iteration of the algorithm, the noise PSD estimate for each DFT2 within sub band j bin is assigned this value (mathematically, this is correct under the assumption the true noise PSD is constant within a sub-band).
In a first example of the proposed system we consider the case K>J. Let the sampling frequency fs=8 kHz, and let the DFT1 and DFT2 analysis frames have lengths L1=64 samples and L2=640 samples, respectively. The lengths of the DFT analysis frame and the DFT2 analysis frame then correspond to 8 ms and 80 ms, respectively. The orders of the DFT2 and DFT transform are in this example set at K=1024 (=210) and J=64 (26), respectively.
The indices of the DFT2 bins corresponding to a sub-band with index-number j, are given by the index set
Bj={k1, . . . , k2}, where k1=(j−½)K/J and k2=(j+½)K/J,
where it is assumed that K and J are integer powers of 2.
In this example, sub band j consists of P=17 DFT2 spectral values. For example, the sub-band with index-number j=1 then consists of the DFT2 bins with index-numbers 8 . . . 24, and the centre frequency of this band is at the DFT2 bin with index-number k=16.
Another configuration would be one where L1=64 samples and L2=512 samples. The orders of the DFT and DFT2 transform can then be chosen as J=64 and K=512, respectively.
Steps 3 through 8 of the algorithm describes how to estimate the noise PSD for each sub-band j. In step 3 a gain G is applied to each of the DFT2 coefficients in the sub-band. After the average noise level in the band is computed in step 4, step 5 applies a bias compensation to compensate for the bias that is introduced by the gain function that is used.
A simplified use of the present embodiment of the algorithm is illustrated in
In
The present embodiment of the algorithm can e.g. advantageously be used in a hearing aid and other signal processing applications where an estimate of the noise PSD is needed and enough processing power is available to have K>J as is given in this example.
The block diagram of
In this example we consider the case K=J, i.e., there is no difference in spectral resolution between the DFT1 and DFT2. Let us again assume that the sampling frequency fs=8 kHz, and let the DFT1 analysis frame have a size of L1=64 samples and the DFT2 analysis frame a size of L2=64 samples. The orders of the DFT2 and DFT1 transform are in this example set at K=J=64, i.e., there is one DFT2 bin k per sub-band j.
In order to estimate the noise PSD for each sub-band j the steps 3 to 8 from the algorithm description should be followed. An important difference with respect to the previous example is that in step 4 the average noise level in the band is computed by taking the average across one spectral sample, which is, in fact, the spectral sample value itself.
The present embodiment of the algorithm can e.g. advantageously be used in signal processing applications where an estimate of the noise PSD is needed and processing power is constrained (e.g. due to power consumption limitations) such that K=J or when it is known beforehand that the noise PSD is rather flat across the frequency range of interest.
The invention is defined by the features of the independent claim(s). Preferred embodiments are defined in the dependent claims. Any reference numerals in the claims are intended to be non-limiting for their scope.
Some preferred embodiments have been shown in the foregoing, but it should be stressed that the invention is not limited to these, but may be embodied in other ways within the subject-matter defined in the following claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 08105346 | Sep 2008 | EP | regional |
| Number | Name | Date | Kind |
|---|---|---|---|
| 20050240401 | Ebenezer | Oct 2005 | A1 |
| 20080010063 | Komamura | Jan 2008 | A1 |
| Number | Date | Country |
|---|---|---|
| WO-2006097886 | Sep 2006 | WO |
| Entry |
|---|
| “Noise Tracking Using DFT Domain Subspace Decompositions”, IEEE Transactions on Audio, Speech, and Language Processing, vol. 16, No. 3, Mar. 2008, p. 541-553. |
| Doblinger, “Computationally Efficient Speech Enhancement by Spectral Minima Tracking in Subbands”, vol. 2, Eurospeech '95, Madrid, Spain, 4th European Conference on Speech Communication and Technology, pp. 1513-1516, Sep. 18-21, 1995. |
| Martin, “Noise Power Spectral Density Estimation Based on Optimal Smoothing and Minimum Statistics”, vol. 9, No. 5, pp. 504-512, IEEE Transactions on Speech and Audio Processing, Jul. 1, 2001. |
| Ephraim et al., “Speech Enhancement Using a Minimum Mean-Square Error Log-Spectral Amplitude Estimator”, vol. ASSP-33, No. 2, pp. 443-445, IEEE Transactions on Acoustics, Speech, and Signal Processing, Apr. 1985. |
| Ephraim et al., “Speech Enhancement Using a Minimum Mean-Square Error Short-Time Spectral Amplitude Estimator”, vol. ASSP-32, No. 6, pp. 1109-1121, IEEE Transactions on Acoustics, Speech, and Signal Processing, Dec. 1984. |
| Hendriks et al., “Noise Tracking Using DFT Domain Subspace Decompositions”, vol. 16, No. 3, pp. 541-553, IEEE Transactions on Audio, Speech, and Language Processing, Mar. 2008. |
| Sohn et al., “A Statistical Model-Based Voice Activity Detection”, vol. 6, No. 1, pp. 1-3, IEEE Signal Processing Letters, Jan. 1999. |
| Number | Date | Country | |
|---|---|---|---|
| 20100067710 A1 | Mar 2010 | US |
