Frequency domain noise attenuation utilizing two transducers

Abstract

Embodiments may find applications to ambient noise attenuation in cell phones, for example, where a second microphone is placed at a distance from the voice microphone so that ambient noise is present at both the voice microphone and the second microphone, but where the user's voice is primarily picked up at the voice microphone. Frequency domain filtering is employed on the voice signal, so that those frequency components representing mainly ambient noise are de-emphasized relative to the other frequency components. Other embodiments are described and claimed.

Description

FIELD

Embodiments of the present invention relate to signal processing, and more particularly, to digital signal processing to attenuate noise.

BACKGROUND

Cell phone conversations are sometimes degraded due to ambient noise. For example, ambient noise at the talker's location may affect the voice quality of the talker as perceived by the listener. It would be desirable to reduce ambient noise in such communication applications.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates two simplified views of a cell phone employing an embodiment of the present invention.

FIG. 2 illustrates an embodiment of the present invention.

DESCRIPTION OF EMBODIMENTS

FIG. 1 provides two simplified views of a cell phone employing an embodiment of the present invention. Unlike conventional cell phones, the cell phone of FIG. 1 has a microphone placed at a distance from the main microphone used for the voice. This microphone is indicated as “ambient microphone” in FIG. 1, whereas the microphone intended for the user's voice is indicated as “mouth microphone”. In the embodiment of FIG. 1, the ambient microphone on the back side of the cell phone. However, in other embodiments, the ambient microphone may be situated at other locations on the cell phone.

Generally stated, embodiments of the present invention make use of the two signals provided by the mouth and ambient microphones to process the signal from the mouth microphone so as to attenuate ambient noise. It is expected that ambient noise will be present at substantially the same power levels at the locations of the ambient and mouth microphones, but that the voice of the user will have a much higher power level at the location of the mouth microphone than for the ambient microphone. Embodiments of the present invention exploit this assumption to provide frequency domain filtering, where those frequency components identified has having mainly a voice contribution are emphasized relative to the other frequency components.

Embodiments of the present invention are not limited to cell phones, but may find applications in other systems.

FIG. 2 provides a high-level abstraction of some embodiments of the present invention. FIG. 2 comprises various modules (functional blocks), where a module may represent a circuit, a software or firmware module, or some combination thereof. Accordingly, FIG. 2 aids in a description of exemplary apparatus embodiments as well as exemplary method embodiments.

Referring to FIG. 2, signal a(t) is provided by transducer a, and signal m(t) is provided by transducer m. These signals are time domain signals, where the index t represents time. The signals may be voltage signals, or current signals. Transducer a and transducer b may be microphones, for example, but are not limited to merely microphones. For example, in application to a cell phone, transducer m may be the mouth microphone in FIG. 1 and transducer a may be the ambient microphone in FIG. 1, where for convenience identifying m with “mouth” and a with “ambient” may serve as a mnemonic.

A/D modules in FIG. 2 denote analog-to-digital converters, one A/D converter for signal a(t) and one A/D converter for signal m(t). The output of the A/D converter for signal a(t) may be represented by the discrete time series a(n), and the output of the A/D converter for signal m(t) may be represented by the discrete time series m(n), where n is a discrete time index. In practice, the symbol a(n), or m(n), for any discrete time index n represents a binary word in some kind of computer arithmetic representation, such as integer arithmetic or floating-point arithmetic. The particular implementation details are not important to an understanding of the embodiments, and for ease of discussion the symbol a(n), or m(n), may be viewed as representing a real number. Similar remarks apply to various other numerical symbols used to describe the embodiments. For example, some symbols will be introduced to represent complex numbers.

The BUF modules for the discrete time series a(n) and m(n) represent buffers to store a fixed number of samples of a(n) and m(n). The fixed number of samples may be taken to be the size of the analysis window applied to these discrete time series. WINDOW modules apply an analysis window to their respective discrete time series, where the analysis window is a set of weights, where each discrete time sample in a BUF module is multiplied by one of the weights.

For example, at some particular time, the samples of m(n) stored in its BUF module may be represented by m(n), n=n₀, n₀+1, . . . , n₀+N−1, where N is the number of samples. Denoting the set of window weights by W(i), i=0, 1 . . . , . . . N−1, the output of WINDOW module is the set of N numbers: {m(n₀)W(0), m(n₀+1)W(1), . . . , m(n₀+N−1)W(N−1)}. The above set of numbers after analysis windowing may be referred to as a frame. Frames may be computed at the rate of one frame for each N samples of m(n), or overlapping may be used, where frames are computed at the rate of one frame for each N/r samples of m(n), where r is an integer that divides into N. The resulting sequence of frames may be represented by m(f), where f is a discrete frame index. Similar remarks apply to the discrete time series a(n), where the resulting sequence of frames may be represented by a(f).

FFT modules in FIG. 2 refer to modules for performing a fast Fourier transform on a frame. More generally, a discrete Fourier transform (DFT) is applied, where a FFT merely denotes a particular algorithm for implementing a DFT. In other embodiments, other transforms may be applied. Such transforms map a time domain signal into a frequency domain signal. For each frame index f, the DFT of frame m(f) is denoted as M(k; f), where k is a frequency bin index belonging to a frequency bin index set {0, 1, . . . , K−1}. The DFT of frame a(f) is denoted as A(k; f). Often K=N, but various interpolation techniques may be employed so that K≠N for some embodiments.

DET module partitions, for each frame index f, the index set {0, 1, . . . , K−1} into disjoint partitions P(j; f), j=0, 1, . . . , J(f)−1, where j is a partition index and J(f) denotes the number of partitions for frame index f, where $\bigcup _{j=0}^{J\left(f\right)-1}P\left(j;f\right)=\left\{0,1,\dots ,K-1\right\}.$ For each partition there is one index k*(j; f )εP(j; f) such that |M(k*(j; f);f)+A(k*(j; f);f)| is a maximum over the partition P(j; f).

Embodiments may construct these partitions in various ways. For some embodiments, the partitions may be constructed as follows. For a given frame index f, all frequency bin indices k* are found for which |M(k*−1; f)+A(k*−1; f)|≦|M(k*; f)+A(k*; f)|, |M(k*+1; f)+A(k*+1; f)|<|M(k*; f)+A(k*; f)|  (1) Once the set of all such frequency bin indices is determined, each one indicating a local maximum of the function |M(k; f)+A(k; f)| in frequency bin space, the frequency bin index set is partitioned so that each partition boundary is half-way, or closest to half-way, between two adjacent such indices.

Other embodiments may construct partitions in other ways. For example, partitions may be constructed based upon local maximums of the function A(k; f). More generally, partitions may be constructed based upon local maximums of a functional of the functions A(k; f) and M(k; f). For example, in Eq. (1), the functional is the addition operator applied to the functions A(k; f) and M(k; f).

It should be noted that the statements in the previous paragraph regarding the frequency bin indices are interpreted in modulo K arithmetic. For example, k*−1 in the earlier displayed equation is to be read as (k*−1)mod(K). Similarly, the “half-way” frequency bin index between any two frequency bin indices for local maximums is interpreted with respect to modulo K arithmetic. Accordingly, the various partitions are contiguous if one imagines the frequency bin index set forming a circle, where 0 is adjacent to both 1 and K−1.

Other embodiments may choose the partitions in other ways, and may define the local maximum in different ways. For example, the relationship ≦ in Eq. (1) may be replaced with <, whereas the relationship <may be replaced with ≦.

It is convenient to denote the indices for the local maximums by k*(j; f), j=0, 1, . . . , J(f)−1. That is, for j=0, 1, . . . , J(f)−1, k*(j; f)εP(j; f) and |M(k*; f)+A(k*; f)| is a maximum over the partition P(j; f).

GAIN module makes use of the information provided by DET module to compute gains for each partition. In some embodiments, the gain for partition P(j; f), denoted by G(j; f ), is provided by a function F(R) of the ratio $R=\frac{A\left(k^{*}\left(j;f\right);f\right)}{M\left(k^{*}\left(j;f\right);f\right)}.$ For some embodiments, the function F(R) may be $F\left(R\right)=\{\begin{array}{cc}1& R\le T,\\ {10}^{-\alpha \log \left(R/T\right)}& R>T,\end{array}$ where T is a threshold. For some other embodiments, the function F(R) may be $F\left(R\right)=\{\begin{array}{cc}1& R\le T,\\ 0& R>T.\end{array}$ The above equations may be generalized so that the numeral 1 is replaced by some scalar, denoted as G₀, where G₀ is independent of j. That is, the function F(R) may be $F\left(R\right)=\{\begin{array}{cc}{G}_0& R\le T,\\ G_0{10}^{-\alpha \log \left(R/T\right)}& R>T,\end{array}\mathrm{or}\mathrm{may}\mathrm{be}F\left(R\right)=\{\begin{array}{cc}{G}_0& R\le T,\\ 0& R>T.\end{array}$

For some embodiments, the threshold T may be on the order of 1/10 to 1/100. In some other embodiments, it may also be higher, such as for example ½ or ¼. In practice, when an embodiment is used in a cell phone, it is expected that the mouth microphone is much closer to the speaker's mouth than the ambient microphone. Consequently, when the cell phone is in use and the user is speaking into the mouth microphone, it is expected that for a frequency bin k_m for which there is energy contribution from the user's voice, the magnitude of M(k_m; f) is much larger than the magnitude of A(k_m; f), whereas for a frequency bin k_a for which there is relatively little energy contribution from the user's voice, the magnitude of M(k_a; f) is not much larger than, or perhaps comparable to, the magnitude of A(k_a; f). Consequently, for cell phone applications, by setting the threshold to a relatively small number, the frequency bins containing mainly voice energy are easily distinguished from the frequency bins for which the user's voice signal has a relatively small energy content.

Multiplier 202 multiplies M(k; f ) by a gain for each frame index f and each frequency bin index k. The result of this product is denoted as {circumflex over (M)}(k; f ) in FIG. 2. Using a synthesis window on {circumflex over (M)}(k; f), a time domain signal {circumflex over (m)}(t) may be reconstructed. In applications in the cell phone of FIG. 1, it is expected that the voice signal in m(t) has a much larger power spectral density than that in a(t), and that ambient noise will be present in both m(t) and a(t) with comparable power spectral density. It is expected that for the proper choice of gain for each M(k; f), the reconstructed time domain signal {circumflex over (m)}(t) is a more pleasing reproduction of the actual voice of the user.

The gain used for multiplication may be G(j; f), where for each partition index j, each M(k; f) such that k belongs to P(j; f) is multiplied by G(j; f). However, it is expected that with this choice of gain, the resulting signal {circumflex over (m)}(t) may be of poor quality, with large amounts of so-called “musical noise”. This is expected because some frequency components may result in a ratio R that varies substantially from frame to frame, sometimes being above the threshold T, and at other times being below T. This results in some frequency components “popping” in and out when {circumflex over (m)}(t) is formed, resulting in “chirps” that quickly fade in and out.

This problem may be minimized in some embodiments by smoothing the computed gains G(j; f). For example, an “attack-release” smoothing method may be applied as follows. For each frame index f, and for each frequency bin index k, M(k; f) is multiplied by a smoothed gain Ĝ(k; f) to form the product {circumflex over (M)}(k; f)=M(k; f)Ĝ(k; f), where Ĝ(k; f) is given by $\stackrel{⋒}{G}\left(k;f\right)=\{\begin{array}{cc}{\beta }_{a}G\left(l;f\right)+\left(1-{\beta }_a\right)\stackrel{⋒}{G}\left(k;f-1\right),& \mathrm{for}G\left(l;f\right)>\stackrel{⋒}{G}\left(k;f\right),\\ {\beta }_{r}G\left(l;f\right)+\left(1-{\beta }_r\right)\stackrel{⋒}{G}\left(k;f-1\right),& \mathrm{for}G\left(l;f\right)\le \stackrel{⋒}{G}\left(k;f\right),\end{array}$ where G(l; f) is the gain for the partition P(l; f) to which k belongs, i.e., k ε P(l; f), and where β_a and β_r are positive numbers less than one.

The number β_a is an “attack” smoothing control weight, applied when the computed gain G(j; f) increases from one frame to the next, and the number β_r is a “release” control weight, applied when the gain G(j; f) decreases from one frame to the next. Typically, β_a is chosen relatively small, so that the smoothed gain Ĝ(k; f) slowly increases if G(j; f) increases from one frame to the next; and β_r is chosen close to one, so that the smoothed gain Ĝ(k; f) rapidly decreases if the gain G(j; f) decreases from one frame to the next. With this choice for these weights, it is expected that musical-noise components are attenuated because their corresponding gains G(j; f) do not have enough time to rise before they dip back down, whereas voice components most likely will not be seriously affected because their corresponding gains G(j; f) usually remain relatively large for many consecutive frames. For some embodiments, β_a may be adjusted during an initialization period, so that when the user starts speaking into the m microphone, the beginning of the utterance is not seriously affected by the slow rise time of the smoothed gain.

Other embodiments may smooth the gains G(j; f) using other types of smoothing algorithms.

Various modifications may be made to the disclosed embodiments without departing from the scope of the invention as claimed below. For example, is to be understood that some of the modules or functional blocks described in the embodiments may be grouped together into various larger modules, or some of the modules may comprise various sub-modules. Furthermore, various modules may be realized by application specific integrated circuits, processors running software, programmable field arrays, logic with firmware, or some combination thereof.

For some embodiments, the threshold value T is constant, but for other embodiments, the threshold value T may vary. For example, the threshold value may be a function of the frame index, the frequency bin index, or both.

It is to be understood that the scope of the invention is not limited by the placement of the first and second transducers relative to a speech source. Furthermore, it is to be understood that the scope of the invention is not limited to any particular distance, orientation, or directionality characteristic (or combination thereof) of the first and second transducers, where these characteristics may be selected to help differentiate between a first signal and a second signal, such as for example to differentiate ambient noise from a desired voice signal.

Throughout the description of the embodiments, various mathematical relationships are used to describe relationships among one or more quantities. For example, a mathematical relationship may express a relationship by which a quantity is derived from one or more other quantities by way of various mathematical operations, such as addition, subtraction, multiplication, division, etc. For example, the DFT or FFT may be performed on a frame of a time sampled signal. These numerical relationships and transformations are in practice not satisfied exactly, and should therefore be interpreted as “designed for” relationships and transformations. For example, it is understood that such transformations as a DFT or FFT cannot be done with infinite precision. One of ordinary skill in the art may design various working embodiments to satisfy various mathematical relationships or numerical transformations, but these relationships or numerical transformations can only be met within the tolerances of the technology available to the practitioner.

Accordingly, in the following claims, it is to be understood that claimed mathematical relationships or transformations can in practice only be met within the tolerances or precision of the technology available to the practitioner, and that the scope of the claimed subject matter includes those embodiments that substantially satisfy the mathematical relationships or transformations so claimed.

Claims

1. An apparatus comprising: a first transducer to provide a first signal; a second transducer to provide a second signal; a sampling module to provide a first sequence of frames from the first signal, and to provide a second sequence of frames from the second signal; a transform module to map each frame in the first and second sequences of frames to, respectively, a set of frequency components M(k; f) and a set of frequency components A(k; f), where k is a frequency bin index whose range is a frequency bin index set, and f is a frame index; a detector module to partition, for each frame index f, the frequency bin index set into disjoint partitions P(j; f), j=0, 1, . . . , J(f)−1, where j is a partition index and J(f) denotes the number of partitions for frame index f, such that for each partition P(j; f) there is one frequency bin index k*(j; f) belonging to P(j; f) such that a functional of A(k; f) and M(k; f) is a maximum over the partition P(j; f); a gain computation module to provide, for each frame index f, and for each partition P(j; f), a gain G(j; f), where G(j; f)=G0 if A⁡(k*⁡(j;f);f)M⁡(k*⁡(j;f);f)❘<T,where T is a threshold value less than one and G0 is independent of j, and where G(j; f)<G0 if A⁡(k*⁡(j;f);f)M⁡(k*⁡(j;f);f)>T;anda multiplier to provide, for each frame index f, and for each frequency bin index k, the product M(k; f)Ĝ(k; f), where Ĝ(k; f) is a function of the gain G(l; f) for the partition P(l; f) to which k belongs.

2. The apparatus as set forth in claim 1, wherein the functional for forming the partitions is |M(k*(j; f); f)+A(k*(j; f); f)|.

3. The apparatus as set forth in claim 1, wherein the gain computation module further provides for each partition P(j; f) a gain G(j; f) where log G(j; f)=log(G0)−a log(x/T) if x>T where

4. The apparatus as set forth in claim 1, wherein the gain computation module further provides for each partition P(j; f) a gain G(j; f) where G(j; f)=0 if x>T where

5. The apparatus as set forth in claim 1, wherein Ĝ(k; f) is also a function of the gain G(l*; f*) for the partition P(l*; f*) to which k belongs where f* is a frame index value other than the frame index value f.

6. The apparatus as set forth in claim 5, wherein for each frame index f, and for each frequency bin index k, the Ĝ(k; f) is given by:

7. The apparatus as set forth in claim 1, wherein the transform module performs a FFT.

8. The apparatus as set forth in claim 1, wherein the first transducer is a first microphone and the second transducer is a second microphone.

9. The apparatus as set forth in claim 8, wherein the apparatus is a cell phone.

10. A method comprising: sampling a first signal to provide a first sequence of frames, and sampling a second signal to provide a second sequence of frames; performing a transform of each frame in the first sequence of frames, where each transform of each frame in the first sequence of frames has elements M(k; f), where k is a frequency bin index ranging over a frequency bin index set, and f is a frame index; and performing the transform of each frame in the second sequence of frames, where each transform of each frame in the second sequence of frames has elements A(k; f); partitioning, for each frame index f, the frequency bin index set into disjoint partitions P(j; f), j=0, 1, . . . , J(f)−1, where j is a partition index and J(f) denotes the number of partitions for frame f, such that for each partition P(j; f) there is one frequency bin index k*(j; f) belonging to P(j; f) such that a functional of A(k; f) and M(k; f) is a maximum over the partition P(j; f); computing for each frame f, and for each partition P(j; f), a gain G(j; f), where G(j; f)=G0 if A⁡(k*⁡(j;f);f)M⁡(k*⁡(j;f);f)<T,where T is a threshold value less than one and G0 is independent of j, and where G(j; f)<G0 if A⁡(k*⁡(j;f);f)M⁡(k*⁡(j;f);f)<T;and forming the product M(k; f)Ĝ(k; f) for each frame index f, and for each frequency bin index k, where Ĝ(k; f) is a function of the gain G(l; f) for the partition P(l; f) to which k belongs.

11. The method as set forth in claim 10, wherein the functional for partitioning is |M(k*(j; f); f)+A(k*(j; f); f)|.

12. The method as set forth in claim 10, further providing for each partition P(j; f) a gain G(j; f), where log G(j; f)=log(G0)−a log(x/T) if x>T where

13. The method as set forth in claim 10, further providing for each partition P(j; f) a gain G(j; f), where G(j; f)=0 if x>T where

14. The method as set forth in claim 10, wherein Ĝ(k; f) is also a function of the gain G(l*; f*) for the partition P(l*; f*) to which k belongs where f* is a frame index value other than frame index value f.

15. The method as set forth in claim 14, wherein for each frame f, and for each frequency bin index k, the Ĝ(k; f) is given by:

16. The method as set forth in claim 10, further comprising: providing the first signal by talking into a first microphone; and providing the second signal by using a second microphone.

17. An apparatus comprising: a first transducer to provide a first signal; a second transducer to provide a second signal; a sampling module to provide a first sequence of frames from the first signal, and to provide a second sequence of frames from the second signal; a transform module to map each frame in the first and second sequences of frames to, respectively, a set of frequency components M(k; f) and a set of frequency components A(k; f), where k is a frequency bin index whose range is a frequency bin index set, and f is a frame index; a detector module to partition, for each frame index f, the frequency bin index set into disjoint partitions P(j; f), j=0, 1, . . . , J(f)−1, where j is a partition index and J(f) denotes the number of partitions for frame index f, and a gain computation module to provide, for each frame index f, and for each partition P(j; f), a gain G(j; f), where G(j; f)=G0 if A⁡(k*⁡(j;f);f)M⁡(k*⁡(j;f);f)<T,where k*(j; f) is a frequency bin index in partition P(j; f), and where T is a threshold value less than 0.5 and G0 is independent of j, and where G(j; f)<G0 if A⁡(k*⁡(j;f);f)M⁡(k*⁡(j;f);f)>T.

18. The apparatus as set forth in claim 17, wherein each partition P(j; f), j=0, 1, . . . , J(f)−1 comprises one and only one frequency bin index in the frequency bin index set.

19. The apparatus as set forth in claim 17, wherein each partition P(j; f) is such that there is one frequency bin index k*(j; f) belonging to P(j; f) such that |M(k*(j; f); f)+A(k*(j; f); f)| is a maximum over the partition P(j; f).

20. The apparatus as set forth in claim 17, wherein the gain computation module further provides the gain G(j; f) such that log G(j; f)=log(G0)−a log(x/T) if x>T where

21. The apparatus as set forth in claim 17, further comprising a multiplier to provide, for each frame index f, and for each frequency bin index k, the product M(k; f)Ĝ(k; f), where Ĝ(k; f) is a function of the gain G(l; f) for the partition P(l; f) to which k belongs.

22. The apparatus as set forth in claim 21, wherein Ĝ(k; f) is also a function of the gain G(l*; f*) for the partition P(l*; f*) to which k belongs where f* is a frame index value other than the frame index value f.

23. The apparatus as set forth in claim 17, wherein T<0.25.

24. The apparatus as set forth in claim 21, further comprising: a multiplier to provide, for each frame index f, and for each frequency bin index k, the product M(k; f)Ĝ(k; f), where Ĝ(k; f) is a function of the gain G(l; f) for the partition P(l; f) to which k belongs

25. The apparatus as set forth in claim 23, wherein Ĝ(k; f) is also a function of the gain G(l*; f*) for the partition P(l*; f*) to which k belongs where f* is a frame index value other than the frame index value f.