Method for wind noise reduction

Information

  • Patent Grant
  • 8433564
  • Patent Number
    8,433,564
  • Date Filed
    Monday, June 7, 2010
    14 years ago
  • Date Issued
    Tuesday, April 30, 2013
    11 years ago
Abstract
A noisy signal is picked up by a microphone, digitized by an Analog to Digital Converter and fed to a processor for analysis and wind noise reduction. Most of noise reduction methods are based on the assumption that the interfering noise is stationary or slowly varying compared with speech. This assumption allows “learning” the characteristics of the noise between speech pauses and, based on a noise estimate, to build different filters that reduce the noise. In the case of wind noise this basic assumption is not valid. Wind noise is highly non-stationary, its power and spectral characteristics vary greatly. Because wind noise is not stationary, regular noise reduction methods cannot be used to reduce wind noise. For reducing wind noise effects in a device, the presence of wind should be detected reliably and then a novel approach presented here must be applied to eliminate the wind noise.
Description
REFERENCES CITED


















US 2006/0120540 A1
June 2006
Henry Luo



EP 1 339 256 A2
March 2004
Roeck et al



EP 1 732 352 A1
December 2006
Hetherington et al



U.S. Pat. No. 7,174,023
February 2007
Ozawa



U.S. Pat. No. 5,288,955
February 1994
Staple et al.



US 2007/0003090
January 2007
Anderson










OTHER REFERENCES



  • [1] Thompson, S., “Directional Microphones Hearing Aids”, The Hearing Journal, November 2003, Vol. 56, No. 11



FIELD OF THE INVENTION

The present invention relates to means and methods of providing clear, high quality voice with a high signal-to-noise ratio, in voice communication systems, devices, telephones, and methods, and more specifically, to systems, devices, and methods that automate control in order to correct for variable wind noise levels and reduce or cancel the wind noise prior to sending the voice communication over cellular telephone communication links.


This invention is the field of processing signals in cell phones, Bluetooth headsets and similar devices. In general, the principles of the invention are applicable to any communication device which is operated in windy environments.


BACKGROUND OF THE INVENTION

Voice communication devices such as cell phones, wireless phones and devices other than cell phones have become ubiquitous; they show up in almost every environment. These systems and devices and their associated communication methods are referred to by a variety of names, such as but not limited to, cellular telephones, cell phones, mobile phones, wireless telephones in the home and the office, and devices such as Personal Data Assistants (PDAs) that include a wireless or cellular telephone communication capability. They are used at home, office, inside a car, a train, at the airport, beach, restaurants and bars, on the street, and almost any other venue. As might be expected, these diverse environments have relatively higher and lower levels of background, ambient, or environmental noise. For example, there is generally less noise in a quiet home than there is in a crowded bar. If this noise, at sufficient levels, is picked up by the microphone, the intended voice communication degrades and though possibly not known to the users of the communication device, uses up more bandwidth or network capacity than is necessary, especially during non-speech segments in a two-way conversation when a user is not speaking.


In an on-going cell phone call or other communication from an environment having relatively higher environmental noise, it is sometimes difficult for the party at the other end of the conversation to hear what the party in the noisy environment is saying. That is, the ambient or environmental noise in the environment often “drowns out” the cell phone user's voice, whereby the other party cannot hear what is being said or even if they can hear it with sufficient volume the voice or speech is not understandable. This problem may even exist in spite of the conversation using a high data rate on the communication network.


The term “wind noise” is used to describe several different ways that wind can be generated. For example, wind can cause a loose shutter to bang against a house or it can cause a flag to rustle and snap. In these cases, the wind has caused an object to move, and the motion makes a sound. In other cases, wind moving past an object can create a howling sound, even though the object does not vibrate. Here, the sound is caused by turbulence that is created in the moving air as it passes by the object. This turbulence, which cannot be seen, is very similar to the turbulence in a fast-moving stream as the water flows around and over large rocks. We have all experienced this kind of wind noise while inside a house during a windstorm. The sound of the howling wind originates in the turbulence of air motion past the walls and roof.


The form of wind noise that most interferes with our ability to hear and communicate is the noise generated by air flow around our own head. Here the sound is generated within centimeters of our ears, and may be heard at quite a high level because of this close proximity [1]


It is known art to reduce wind noise by mechanical means. Such means alone, however, do not eliminate the wind noise to a satisfactory level.


Therefore, wind noise has been studied extensively and many solutions have been proposed for hearing aids, Bluetooth headsets etc.


Current wind noise reduction solutions use high-pass filters or subtract an estimate of the wind noise from the noisy signal. An efficient wind noise reduction can be achieved only if can be detected reliably and consistently.


Wind noise exhibits some properties and features that are common to other types of noise encountered in our daily lives. Depending on the wind speed, direction, physical obstructions like hats, caps, hand etc the characteristics of wind noise vary greatly. For these reasons, it is difficult to detect the presence of wind noise and cancel it when compared to other environmental noises.


However, certain factors make wind noise unique. Wind noise predominantly is a low-frequency phenomenon. Many of the known art technologies detect wind noise using the property of low correlation of the wind noise.


It is known art to reduce wind noise by mechanical means such as foam, scrims etc. To be sufficiently effective, the mechanical means must be thick which might make the device look bulky. This can be undesirable.


Several attempts to detect wind noise are known in the related art. US patent US2002/037088, assigned to Dickel et al, detects wind noise by computing the correlation between signals received at the two microphones. Turbulence created at the two microphones, without any obstructions, causes signals with low correlation. However, our studies showed that obstructions in the vicinity of the microphone result the correlation to be high.


European patent EP 1 339 256 A2, assigned to Roeck et al, uses several of the well know wind noise properties like high energy content at low frequencies, low auto-correlation at two microphones and high-magnitudes. However, this approach also suffers from the same drawbacks discussed above.


European patent application EP 1 732 352 A1, assigned to Hetherington et al, uses multiple microphones where power levels in different microphones are compared. When the power level of the sound received at the second microphone is less than the power level of the sound received at the first microphone by a predefined value, wind noise may be present. However, this approach requires one of the microphones to be directional with high directivity index and the other microphone to be Omni-directional with low directivity index.


U.S. Pat. No. 7,174,023 granted to Ozawa uses a multi-microphone approach. This approach uses passing the “difference signals” from multiple microphones through a low pass filter to extract wind noise for analysis and synthesis. However, our studies and recordings of wind noise under conditions show that wind noise is sometimes concentrated in higher frequency regions as well.


U.S. Pat. No. 5,288,955 granted to Staple et al talks about an arrangement in a bullet-shaped housing having a rounded front portion. However, this is a hardware approach.


US patent 2007/0003090 granted to Anderson talks about using a mesh made with either nylon or metal having a single or plurality of layers. This also is a hardware approach.


US patent US 2006/0120540 A1 granted to Luo uses one microphone and two microphones. The patent talks about hearing aids but it does not cover Bluetooth headsets and cell phones, where the introduction of the second microphone could sometimes be difficult.


Hence there is a need in the art for a method of wind noise reduction or cancellation that is robust, suitable for mobile use, and inexpensive to manufacture. The increased traffic in cellular telephone based communication systems has created a need in the art for means to provide a clear, high quality signal with a high signal-to-noise ratio.


It is an objective of the present invention to provide methods and devices that overcome disadvantages of prior art wind noise detection and reduction.


The requirements of a wind noise reduction system for speech enhancement are a) Intelligibility, naturalness of the enhanced signal, b) Improvement of the signal-to-noise ratio, c) Short signal delay and d) Computational simplicity


There are several methods for performing noise reduction, but all can be categorized as types of filtering. In the related art, speech and noise are mixed into one signal channel, where they reside in the same frequency band and may have similar correlation properties. Consequently, filtering will inevitably have an effect on both the speech signal and the background noise signal. Distinguishing between voice and background noise signals is a challenging task. Speech components may be perceived as noise components and may be suppressed or filtered along with the noise components.


It is an objective of the present invention to provide methods and devices that overcome disadvantages of prior art wind noise detection and reduction schemes. The methods should be computationally inexpensive, ability to detect and reduce low, medium and high levels of wind noise.


SUMMARY OF THE INVENTION

Communication devices are used in different environments and are subjected to different environmental noises, in particular wind noise. Wind noise is highly non-stationary. Its power and spectral characteristics vary greatly. For applications like professional recordings, news broadcast etc., it is possible to mitigate the effects of wind noise using high quality microphones coupled with wind screens (Metal or foam based). However, these solutions cannot be directly applied to mobile devices (cell phones, Bluetooth headsets). To cope with this problem we can process the signal in a Digital Signal Processor. The noisy signal is picked up by the microphone, digitized by an Analog to Digital Converter and fed to the processor for analysis and noise reduction.


Most of noise reduction algorithms are based on the assumption that the interfering noise is stationary (HVAC, projector noise, etc) or slowly varying compared with speech (Car noise, Street noise). This assumption allows “learning” the characteristics of the noise between speech pauses and, based on a noise estimate, to build different filters that reduce the noise. In the case of wind noise this basic assumption is not valid. Wind noise is highly non-stationary, its power and spectral characteristics vary greatly. Because of its high non-stationary, regular noise reduction algorithms cannot be used to reduce wind noise. For reducing wind noise effects in a device, the signal has to be processed in a number of frequency bins.


The present invention provides a novel system and method for monitoring the wind noise in the environment in which a cellular telephone is operating and cancels it before it is transmitted to the other party so that the party at the other end of the voice communication link can more easily hear what the cellular telephone user is transmitting.


The present invention preferably employs noise reduction and or cancellation technology that is operable to attenuate or even eliminate pre-selected portions of an audio spectrum. By monitoring the wind noise in a location in which the cellular telephone is operating and applying noise reduction and/or cancellation protocols at the appropriate time via analog and/or digital signal processing, it is possible to significantly reduce wind noise to which a party to a cellular telephone call might be subjected.


In one aspect of the invention, the invention provides a system and method that enhances the convenience of using a cellular telephone or other wireless telephone or communications device, even in a location having relatively high amounts of wind noise.


In another aspect of the invention, the invention provides a system and method for canceling wind noise before it is transmitted to another party.


In yet another aspect of the invention, the invention monitors wind noise via a microphone and thereafter cancels the monitored wind noise.


In still another aspect of the invention, an enable/disable switch is provided on a cellular telephone device to enable/disable wind noise reduction.


These and other aspects of the present invention will become apparent upon reading the following detailed description in conjunction with the associated drawings. The present invention overcomes shortfalls in the related art with an adaptive wind noise cancellation algorithm. These modifications, other aspects and advantages will be made apparent when considering the following detailed descriptions taken in conjunction with the associated drawings.





BRIEF DESCRIPTION OF THE DRAWINGS


FIG. 1 is diagram of an exemplary embodiment of the wind noise reduction scheme as discussed in the current invention.



FIG. 2 is a diagram of an exemplary embodiment of the system which finds the ratio between low frequency energy and total energy and then makes a decision if the incoming signal is wind or not.



FIG. 3 is a diagram of an exemplary embodiment of the system which takes the decision and does the spectral correction to reduce the overall effect of wind noise.



FIG. 4
a is a diagram of a speech file corrupted with wind noise.



FIG. 4
b is a diagram of the ratio of low frequency energy to the total frequency energy for the signal as described in FIG. 4a.



FIG. 5
a is a diagram of a speech file corrupted with street noise.



FIG. 5
b is a diagram of the ratio of low frequency energy to the total frequency energy for the signal as described in FIG. 5a.



FIG. 6
a is a diagram of a noisy file before processing where wind noise interferes with speech.



FIG. 6
b is a diagram of a same file after processing using the wind noise reduction technology discussed in the current invention.





DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

The following detailed description is directed to certain specific embodiments of the invention. However, the invention can be embodied in a multitude of different ways as defined and covered by the claims and their equivalents. In this description, reference is made to the drawings wherein like parts are designated with like numerals throughout.


Unless otherwise noted in this specification or in the claims, all of the terms used in the specification and the claims will have the meanings normally ascribed to these terms by workers in the art.


The present invention provides a novel and unique background noise or environmental noise reduction and/or cancellation feature for a communication device such as a cellular telephone, wireless telephone, cordless telephone, recording device, a handset, and other communications and/or recording devices. While the present invention has applicability to at least these types of communications devices, the principles of the present invention are particularly applicable to all types of communication devices, as well as other devices that process or record speech in noisy environments such as voice recorders, dictation systems, voice command and control systems, and the like. For simplicity, the following description employs the term “telephone” or “cellular telephone” as an umbrella term to describe the embodiments of the present invention, but those skilled in the art will appreciate the fact that the use of such “term” is not considered limiting to the scope of the invention, which is set forth by the claims appearing at the end of this description.


Hereinafter, preferred embodiments of the invention will be described in detail in reference to the accompanying drawings. It should be understood that like reference numbers are used to indicate like elements even in different drawings. Detailed descriptions of known functions and configurations that may unnecessarily obscure the aspect of the invention have been omitted.


Let a windowed speech signal and noise signal be represented by s(k) and n(k) respectively. The sum of the two is then denoted by x (k),

x(k)=s(k)+n(k)  (1)


Taking the Fourier Transform of both sides of equation (1) gives










X


(



j





w


)


=


S


(



j





w


)


+

N


(



j





w


)







(
2
)






Where







x


(
k
)






F
·
T




X


(



j





w


)







(
3
)







In FIG. 1, the input signal is processed by block 111 where the FFT of the input signal is calculated. Blocks 112 and 113 do the wind noise reduction on each bin of the FFT or in each spectral band. It is known in the art that spectral processing can be done at maximum resolution in a per FFT bin base or on spectral bands when combining several bins into bands. Block 114 calculates the IFFT of the signal which is the desired output.



FIG. 2 illustrates the wind noise detection mechanism. Block 211 performs the FFT of the input signal. Block 212 computes the low frequency energy of the input noisy signal, ELF. Block 213 calculates the Total energy of the input signal, ETOT. Block 214 computes the ratio of energies calculated at block 212 and 213 respectively and outputs the signal called ER. Block 215 exponentially averages the energy ratio, ERAVG using the following equation

ERAVG=α(ERAVG)+(1−α)ER  (4)

The value of α can be chosen to be in the range 0.75 to 0.95.


If the energy ratio average is greater than a particular threshold wind is said to be present. Otherwise wind is said to be absent.


The threshold can be selected based on the analysis of different types of noises (see FIG. 4b and FIG. 5b).


In FIG. 3, block 311 decides if the incoming frame of signal is wind or not. If the decision is made as wind, block 312 estimates the energy of the frame (EF) and averages it (EFAVG). Again, the average equation (4) is used with similar range of values for β.

E(k)FAVG=β(E(k)FAVG)+(1−β)E(k)F  (5)

The value of β can be chosen to be in the range 0.75 to 0.95.


Equation (5) is calculated for each bin or spectral band “k” and it is used as an estimator of the wind noise.


Equation (5) can also be activated based on a Voice Activity Detector (VAD) in order to cover for situations where wind is not present constantly. For example when a person moves from a windy environment to a quiet environment.


Taking equation (2) into account, the noise spectrum is generally averaged for the conversation, so that the listener is not affected by varying noise levels. To obtain the estimate of the noise spectrum the magnitude |N(ejω)| of N(ejω) is replaced by its average value μ (e jω) taken during the regions estimated as “noise only”.

μ(ejw)=E{|N(ejw)|}

The Power Spectral Density of the Signal is calculated by subtracting the current Noise Estimator from the noisy observation is

Ŝ(ejw)=X(ejw)−μ(ejw)

Where μ(ejw) is the average value of the noise spectrum. Due to random variations of noise, spectral subtraction can result in negative estimates of the short-time magnitude or power spectrum. The magnitude and power spectrum are non-negative variables, and any negative estimates of these variables should be mapped into non-negative values.


E(k)FAVG is used as the estimator of μ(ejw).


In block 314, the SNR per channel is computed by subtracting the average noise power estimator from the power spectral density of the current frame. The gains are linear estimators based on the SNR per band. The gain estimations are given by:

gain[band]=K*a_priori_SNR[band]+LIMITER

Where “K” and “LIMITER” are constants obtained by maximizing the SNRI (Signal to Noise Ratio Improvement) over a Data Base of different speakers and noises. The LIMITER value controls the amount of noise left versus speech distortion level.


The present invention can be implemented by finding the gains per bin or per spectral band. In case the spectral band approach is used, the gains are calculated per band and they are expanded (duplicated) to cover all the FFT bins. These FFT gains are multiplied with the N FFT bins of the noisy signal to get the corrected spectrum in block 315.



FIG. 4
a is a diagram of a speech file corrupted with wind noise.



FIG. 4
b is a diagram of the ratio of low frequency energy to the total frequency energy for the signal as described in FIG. 4a. The low frequency energy is typically calculated for frequencies less than 150 Hz. When there is speech, the low frequency energy is low. Hence the energy ratio is also low. When there is only noise and no speech, the low frequency energy is high. Hence the energy ratio is high. If the energy ratio exceeds a pre-defined threshold for more than duration of ‘N’ seconds, it is classified as wind noise.



FIG. 5
a is a diagram of a speech file corrupted with street noise.



FIG. 5
b is a diagram of the ratio of low frequency energy to the total frequency energy for the signal as described in FIG. 5a. A suitable threshold, based on different windy conditions, is chosen to classify the incoming noisy signal as windy or not.



FIG. 6
a is a diagram of a noisy file before processing where wind noise interferes with speech.



FIG. 6
b is a diagram of a same file after processing using the wind noise reduction technology.


As described hereinabove, the invention has the advantages of improving the signal-to-noise ratio by reducing noise in various noisy conditions, enabling the conversation to be pleasant. While the invention has been described with reference to a detailed example of the preferred embodiment thereof, it is understood that variations and modifications thereof may be made without departing from the true spirit and scope of the invention. Therefore, it should be understood that the true spirit and the scope of the invention are not limited by the above embodiment, but defined by the appended claims and equivalents thereof.


Unless the context clearly requires otherwise, throughout the description and the claims, the words “comprise,” “comprising” and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is to say, in a sense of “including, but not limited to.” Words using the singular or plural number also include the plural or singular number, respectively. Additionally, the words “herein,” “above,” “below,” and words of similar import, when used in this application, shall refer to this application as a whole and not to any particular portions of this application.


The above detailed description of embodiments of the invention is not intended to be exhaustive or to limit the invention to the precise form disclosed above. While specific embodiments of, and examples for, the invention are described above for illustrative purposes, various equivalent modifications are possible within the scope of the invention, as those skilled in the relevant art will recognize. For example, while steps are presented in a given order, alternative embodiments may perform routines having steps in a different order. The teachings of the invention provided herein can be applied to other systems, not only the systems described herein. The various embodiments described herein can be combined to provide further embodiments. These and other changes can be made to the invention in light of the detailed description.


All the above references and U.S. patents and applications are incorporated herein by reference. Aspects of the invention can be modified, if necessary, to employ the systems, functions and concepts of the various patents and applications described above to provide yet further embodiments of the invention.


These and other changes can be made to the invention in light of the above detailed description. In general, the terms used in the following claims, should not be construed to limit the invention to the specific embodiments disclosed in the specification, unless the above detailed description explicitly defines such terms. Accordingly, the actual scope of the invention encompasses the disclosed embodiments and all equivalent ways of practicing or implementing the invention under the claims.


While certain aspects of the invention are presented below in certain claim forms, the inventors contemplate the various aspects of the invention in any number of claim forms. Accordingly, the inventors reserve the right to add additional claims after filing the application to pursue such additional claim forms for other aspects of the invention.

Claims
  • 1. A method of reducing a noise component from a speech signal, the method comprising the steps of: a) processing a frame of a windowed speech signal x(k), which is a combination of a speech signal s(k) and a noise signal n(k) such that x(k)=s(k)+n(k);b) applying a Fourier Transform (F.T.) to x(k) to obtain X(ejw)=S(ejw)+N(ejw), where X(ejw) is a spectrum of the frame of the windowed speech signal, where S(ejw) is a spectrum the speech signal, where N(ejw) is a spectrum of the noise signal, and where X(ejw) includes one or more frequency bands;c) obtaining an energy ratio (ER) of low frequency energy (ELF) to total energy (ETOT) with both ELF and ETOT obtained from the F.T. in part b, above;d) calculating an exponential average of the energy ratio (ER—AVG) using the equation ER—AVG=α(ER—AVG)+(1−α)ER
  • 2. A method of reducing a wind noise portion within a communication signal, the method comprising: a) subjecting a frame of a communication signal to a wind decision process to derive a ratio (ER) of low frequency energy (ELF) to total frequency energy (ETOT), wherein the frame of the communication signal is a combination of a speech signal and an input noise signal,wherein the wind decision process finds low frequency energy, where low frequency energy is defined as frequencies less than 150 Hz, and finds total frequency energy for the frame of the communication signal;and wherein the frame of the communication signal is classified as wind noise if the ratio of low frequency energy to total frequency energy is 0.1 to 0.6 until a duration threshold is met or exceeded, where the duration threshold is a value between 0.01 and 2.0 seconds;b) if the frame of the communication signal is classified as wind noise, calculating E(k)F, where E(k)F is an energy of the frame of the communication signal; and
US Referenced Citations (38)
Number Name Date Kind
4628529 Borth et al. Dec 1986 A
4630305 Borth et al. Dec 1986 A
5170433 Elliott et al. Dec 1992 A
5288955 Staple et al. Feb 1994 A
6122531 Nicholls et al. Sep 2000 A
6321197 Kushner et al. Nov 2001 B1
6411927 Morin et al. Jun 2002 B1
6529872 Cerisara et al. Mar 2003 B1
6658385 Gong et al. Dec 2003 B1
6691091 Cerisara et al. Feb 2004 B1
7174023 Ozawa Feb 2007 B2
7243068 Barker et al. Jul 2007 B2
7283956 Ashley et al. Oct 2007 B2
7305099 Gustavsson Dec 2007 B2
7330738 Kang et al. Feb 2008 B2
7464029 Visser et al. Dec 2008 B2
7617099 Yang et al. Nov 2009 B2
7725315 Hetherington et al. May 2010 B2
7885420 Hetherington et al. Feb 2011 B2
7895036 Hetherington et al. Feb 2011 B2
7895039 Braho et al. Feb 2011 B2
7949522 Hetherington et al. May 2011 B2
8165872 Leblanc et al. Apr 2012 B2
8296136 Nongpiur Oct 2012 B2
20010001141 Sih et al. May 2001 A1
20020165712 Souilmi et al. Nov 2002 A1
20030055635 Bizjak Mar 2003 A1
20040165736 Hetherington et al. Aug 2004 A1
20060069557 Barker et al. Mar 2006 A1
20060120540 Luo Jun 2006 A1
20060206320 Li Sep 2006 A1
20070003090 Anderson Jan 2007 A1
20070005350 Amada Jan 2007 A1
20070021958 Visser et al. Jan 2007 A1
20080037811 Gustavsson Feb 2008 A1
20080154585 Yoshioka Jun 2008 A1
20080189100 LeBlanc et al. Aug 2008 A1
20090112584 Li et al. Apr 2009 A1
Foreign Referenced Citations (2)
Number Date Country
EP 1 339 256 Aug 2003 CH
EP 1 732352 Dec 2005 DE
Non-Patent Literature Citations (1)
Entry
Thompson, S., “Directional Microphones Hearing Aids”, The Hearing Journal, Nov. 2003, vol. 56, No. 11.
Related Publications (1)
Number Date Country
20110004470 A1 Jan 2011 US
Provisional Applications (1)
Number Date Country
61222781 Jul 2009 US