1. Technical Field
This invention relates to acoustics, and more particularly, to a system that enhances the perceptual quality of a processed voice.
2. Related Art
Many hands-free communication devices acquire, assimilate, and transfer a voice signal. Voice signals pass from one system to another through a communication medium. In some systems, including some used in vehicles, the clarity of the voice signal does not depend on the quality of the communication system or the quality of the communication medium. When noise occurs near a source or a receiver, distortion garbles the voice signal, destroys information, and in some instances, masks the voice signal so that it is not recognized by a listener.
Noise, which may be annoying, distracting, or results in a loss of information, may come from many sources. Within a vehicle, noise may be created by the engine, the road, the tires, or by the movement of air. A natural or artificial movement of air may be heard across a broad frequency range. Continuous fluctuations in amplitude and frequency may make wind noise difficult to overcome and degrade the intelligibility of a voice signal.
Many systems attempt to counteract the effects of wind noise. Some systems rely on a variety of sound-suppressing and dampening materials throughout an interior to ensure a quiet and comfortable environment. Other systems attempt to average out varying wind-induced pressures that press against a receiver. These noise reducers may take many shapes to filter out selected pressures making them difficult to design to the many interiors of a vehicle. Another problem with some speech enhancement systems is that of detecting wind noise in a background of a continuous noise. Yet another problem with some speech enhancement systems is that they do not easily adapt to other communication systems that are susceptible to wind noise.
Therefore there is a need for a system that counteracts wind noise across a varying frequency range.
A voice enhancement logic improves the perceptual quality of a processed voice. The system learns, encodes, and then dampens the noise associated with the movement of air from an input signal. The system includes a noise detector and a noise attenuator. The noise detector detects a wind buffet by modeling. The noise attenuator then dampens the wind buffet.
Alternative voice enhancement logic includes time frequency transform logic, a background noise estimator, a wind noise detector, and a wind noise attenuator. The time frequency transform logic converts a time varying input signal into a frequency domain output signal. The background noise estimator measures the continuous noise that may accompany the input signal. The wind noise detector automatically identifies and models a wind buffet, which may then be dampened by the wind noise attenuator.
Other systems, methods, features and advantages of the invention will be, or will become, apparent to one with skill in the art upon examination of the following figures and detailed description. It is intended that all such additional systems, methods, features and advantages be included within this description, be within the scope of the invention, and be protected by the following claims.
The invention can be better understood with reference to the following drawings and description. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention. Moreover, in the figures, like referenced numerals designate corresponding parts throughout the different views.
A voice enhancement logic improves the perceptual quality of a processed voice. The logic may automatically learn and encode the shape and form of the noise associated with the movement of air in a real or a delayed time. By tracking selected attributes, the logic may eliminate or dampen wind noise using a limited memory that temporarily stores the selected attributes of the noise. Alternatively, the logic may also dampen a continuous noise and/or the “musical noise,” squeaks, squawks, chirps, clicks, drips, pops, low frequency tones, or other sound artifacts that may be generated by some voice enhancement systems.
In
The wind noise detector 102 may separate the noise-like segments from the remaining signal in a real or in a delayed time no matter how complex or how loud an incoming segment may be. The separated noise-like segments are analyzed to detect the occurrence of wind noise, and in some instances, the presence of a continuous underlying noise. When wind noise is detected, the spectrum is modeled, and the model is retained in a memory. While the wind noise detector 102 may store an entire model of a wind noise signal, it also may store selected attributes in a memory.
To overcome the effects of wind noise, and in some instances, the underlying continuous noise that may include ambient noise, the noise attenuator 104 substantially removes or dampens the wind noise and/or the continuous noise from the unvoiced and mixed voice signals. The voice enhancement logic 100 encompasses any system that substantially removes or dampens wind noise. Examples of systems that may dampen or remove wind noise include systems that use a signal and a noise estimate such as (1) systems which use a neural network mapping of a noisy signal and an estimate of the noise to a noise-reduced signal, (2) systems which subtract the noise estimate from a noisy-signal, (3) systems that use the noisy signal and the noise estimate to select a noise-reduced signal from a code-book, (4) systems that in any other way use the noisy signal and the noise estimate to create a noise-reduced signal based on a reconstruction of the masked signal. These systems may attenuate wind noise, and in some instances, attenuate the continuous noise that may be part of the short-term spectra. The noise attenuator 104 may also interface or include an optional residual attenuator 106 that removes or dampens artifacts that may result in the processed signal. The residual attenuator 106 may remove the “musical noise,” squeaks, squawks, chirps, clicks, drips, pops, low frequency tones, or other sound artifacts.
In the time and frequency spectral domain, the continuous noise 208 and a wind buffet 202 may be curvilinear. The continuous noise and wind buffet may appear to be formed or characterized by the curved lines shown in
SNR=σWB−σCN (Equation 1)
Any method may approximate the linearity of a wind buffet. In the signal-to-noise domain, an offset or y-intercept 302 and an x-intercept or pivot point may characterize the linear model 302. Alternatively, an x or y-coordinate and a slope may model the wind buffet. In
To detect a wind buffet, a line may be fitted to a selected portion of the low frequency spectrum in the SNR domain. Through a regression, a best-fit line may measure the severity of the wind noise within a given block of data. A high correlation between the best-fit line and the low frequency spectrum may identify a wind buffet. Whether or not a high correlation exists, may depend on a desired clarity of a processed voice and the variations in frequency and amplitude of the wind buffet. Alternatively, a wind buffet may be identified when an offset or y-intercept of the best-fit line exceeds a predetermined threshold (e.g., >3 dB).
To limit a masking of voice, the fitting of the line to a suspected wind buffet signal may be constrained by rules. Exemplary rules may prevent a calculated offset, slope, or coordinate point in a wind buffet model from exceeding an average value. Another rule may prevent the wind noise detector 102 from applying a calculated wind buffet correction when a vowel or another harmonic structure is detected. A harmonic may be identified by its narrow width and its sharp peak, or in conjunction with a voice or a pitch detector. If a vowel or another harmonic structure is detected, the wind noise detector may limit the wind buffet correction to values less than or equal to average values. An additional rule may allow the average wind buffet model or its attributes to be updated only during unvoiced segments. If a voiced or a mixed voice segment is detected, the average wind buffet model or its attributes are not updated under this rule. If no voice is detected, the wind buffet model or each attribute may be updated through any means, such as through a weighted average or a leaky integrator. Many other rules may also be applied to the model. The rules may provide a substantially good linear fit to a suspected wind buffet without masking a voice segment.
To overcome the effects of wind noise, a wind noise attenuator 104 may substantially remove or dampen the wind buffet from the noisy spectrum by any method. One method may add the wind buffet model to a recorded or modeled continuous noise. In the power spectrum, the modeled noise may then be subtracted from the unmodified spectrum. If an underlying peak or valley 902 is masked by a wind buffet 202 as shown in
To minimize the “music noise,” squeaks, squawks, chirps, clicks, drips, pops, low frequency tones, or other sound artifacts that may be generated in the low frequency range by some wind noise attenuators, an optional residual attenuator 106 (shown in
Further improvements to voice quality may be achieved by pre-conditioning the input signal before the wind noise detector processes it. One pre-processing system may exploit the lag time that a signal may arrive at different detectors that are positioned apart as shown in
Alternatively, multiple wind noise detectors 102 may be used to analyze the input of each of the microphones 502 as shown in
B(f,i)>B(f)Ave+c (Equation 2)
To detect a wind buffet, a wind noise detector 708 may fit a line to a selected portion of the spectrum in the SNR domain. Through a regression, a best-fit line may model the severity of the wind noise 202, as shown in
Alternatively, a wind buffet may be identified by the analysis of time varying spectral characteristics of the input signal that may be graphically displayed on a spectrograph. A spectrograph may produce a two dimensional pattern called a spectrogram in which the vertical dimensions correspond to frequency and the horizontal dimensions correspond to time.
A signal discriminator 710 may mark the voice and noise of the spectrum in real or delayed time. Any method may be used to distinguish voice from noise. In
To overcome the effects of wind noise, a wind noise attenuator 712 may dampen or substantially remove the wind buffet from the noisy spectrum by any method. One method may add the substantially linear wind buffet model to a recorded or modeled continuous noise. In the power spectrum, the modeled noise may then be removed from the unmodified spectrum by the means described above. If an underlying peak or valley 902 is masked by a wind buffet 202 as shown in
To minimize the “musical noise,” squeaks, squawks, chirps, clicks, drips, pops, low frequency tones, or other sound artifacts that may be generated in the low frequency range by some wind noise attenuators, an optional residual attenuator 714 may also be used. The residual attenuator 714 may track the power spectrum within a low frequency range. When a large increase in signal power is detected an improvement may be obtained by limiting the transmitted power in the low frequency range to a predetermined or calculated threshold. A calculated threshold may be equal to or based on the average spectral power of that same low frequency range at a period earlier in time.
At act 1106, a continuous or ambient noise is measured. The background noise estimate may comprise an average of the acoustic power in each frequency bin. To prevent biased noise estimations at transients, the noise estimation process may be disabled during abnormal or unpredictable increases in power at act 1108. The transient detection act 1108 disables the background noise estimate when an instantaneous background noise exceeds an average background noise by more than a predetermined decibel level.
At act 1110, a wind buffet may be detected when the offset exceeds a predetermined threshold (e.g., a threshold >3 dB) or when a high correlation exits between a best-fit line and the low frequency spectrum. Alternatively, a wind buffet may be identified by the analysis of time varying spectral characteristics of the input signal. When a line fitting detection method is used, the fitting of the line to the suspected wind buffet signal may be constrained by some optional acts. Exemplary optional acts may prevent a calculated offset, slope, or coordinate point in a wind buffet model from exceeding an average value. Another optional act may prevent the wind noise detection method from applying a calculated wind buffet correction when a vowel or another harmonic structure is detected. If a vowel or another harmonic structure is detected, the wind noise detection method may limit the wind buffet correction to values less than or equal to average values. An additional optional act may allow the average wind buffet model or attributes to be updated only during unvoiced segments. If a voiced or mixed voice segment is detected, the average wind buffet model or attributes are not updated under this act. If no voice is detected, the wind buffet model or each attribute may be updated through many means, such as through a weighted average or a leaky integrator. Many other optional acts may also be applied to the model.
At act 1112, a signal analysis may discriminate or mark the voice signal from the noise-like segments. Voiced signals may be identified by, for example, (1) the narrow widths of their bands or peaks; (2) the resonant structure that may be harmonically related; (3) their harmonics that correspond to formant frequencies; (4) characteristics that change relatively slowly with time; (5) their durations; and when multiple detectors or microphones are used, (6) the correlation of the output signals of the detectors or microphones.
To overcome the effects of wind noise, a wind noise is substantially removed or dampened from the noisy spectrum by any act. One exemplary act 1114 adds the substantially linear wind buffet model to a recorded or modeled continuous noise. In the power spectrum, the modeled noise may then be substantially removed from the unmodified spectrum by the methods and systems described above. If an underlying peak or valley 902 is masked by a wind buffet 202 as shown in
To minimize the “musical noise,” squeaks, squawks, chirps, clicks, drips, pops, low frequency tones, or other sound artifacts that may be generated in the low frequency range by some wind noise processes, a residual attenuation method may also be performed before the signal is converted back to the time domain. An optional residual attenuation method 1118 may track the power spectrum within a low frequency range. When a large increase in signal power is detected an improvement may be obtained by limiting the transmitted power in the low frequency range to a predetermined or calculated threshold. A calculated threshold may be equal to or based on the average spectral power of that same low frequency range at a period earlier in time.
A “computer-readable medium,” “machine-readable medium,” “propagated-signal” medium, and/or “signal-bearing medium” may comprise any means that contains, stores, communicates, propagates, or transports software for use by or in connection with an instruction executable system, apparatus, or device. The machine-readable medium may selectively be, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium. A non-exhaustive list of examples of a machine-readable medium would include: an electrical connection “electronic” having one or more wires, a portable magnetic or optical disk, a volatile memory such as a Random Access Memory “RAM” (electronic), a Read-Only Memory “ROM” (electronic), an Erasable Programmable Read-Only Memory (EPROM or Flash memory) (electronic), or an optical fiber (optical). A machine-readable medium may also include a tangible medium upon which software is printed, as the software may be electronically stored as an image or in another format (e.g., through an optical scan), then compiled, and/or interpreted or otherwise processed. The processed medium may then be stored in a computer and/or machine memory.
As shown in the first sequence of
In the second sequence, an averaging of the acoustic power in each frequency bin during unvoiced segments derives the background noise estimate. To prevent biased noise estimates, noise estimates may not occur when abnormal or unpredictable power fluctuations are detected.
In the third sequence, the unmodified spectrum is digitized, smoothed by a window, and transformed into the complex spectrum by an FFT. The unmodified spectrum exhibits portions containing noise-like segments and other portions exhibiting a regular harmonic structure.
In the fourth sequence, a sound segment is fitted to separate lines to model the severity of the wind and continuous noise. To provide a more complete explanation, an unvoiced, fully voiced, and mixed voiced sample are shown. The frequency bins in each sample were converted into the power-spectral domain and logarithmic domain to develop a wind buffet and continuous noise estimate. As more windows are processed, the average wind noise and continuous noise estimates are derived.
To detect a wind buffet, a line is fitted to a selected portion of the signal in the SNR domain. Through a regression, best-fit lines model the severity of the wind noise in each illustration. A high correlation between one best-fit line and the low frequency spectrum may identify a wind buffet. Alternatively, a y-intercept that exceeds a predetermined threshold may also identify a wind buffet. To limit the masking of voice, the fitting of the line to a suspected wind buffet signal may be constrained by the rules described above.
To overcome the effects of wind noise, the modeled noise may be dampened in the unmodified spectrum. In
From the foregoing descriptions it should be apparent that the above-described systems may condition signals received from only one microphone or detector. It should also be apparent, that many combinations of systems may be used to identify and track wind buffets. Besides the fitting of a line to a suspected wind buffet, a system may (1) detect the peaks in the spectra having a SNR greater than a predetermined threshold; (2) identify the peaks having a width greater than a predetermined threshold; (3) identify peaks that lack a harmonic relationships; (4) compare peaks with previous voiced spectra; and (5) compare signals detected from different microphones before differentiating the wind buffet segments, other noise like segments, and regular harmonic structures. One or more of the systems described above may also be used in alternative voice enhancement logic.
Other alternative voice enhancement systems include combinations of the structure and functions described above. These voice enhancement systems are formed from any combination of structure and function described above or illustrated within the attached figures. The logic may be implemented in software or hardware. The term “logic” is intended to broadly encompass a hardware device or circuit, software, or a combination. The hardware may include a processor or a controller having volatile and/or non-volatile memory and may also include interfaces to peripheral devices through wireless and/or hardwire mediums.
The voice enhancement logic is easily adaptable to any technology or devices. Some voice enhancement systems or components interface or couple vehicles as shown in
The voice enhancement logic improves the perceptual quality of a processed voice. The logic may automatically learn and encode the shape and form of the noise associated with the movement of air in a real or a delayed time. By tracking selected attributes, the logic may eliminate or dampen wind noise using a limited memory that temporarily or permanently stores selected attributes of the wind noise. The voice enhancement logic may also dampen a continuous noise and/or the squeaks, squawks, chirps, clicks, drips, pops, low frequency tones, or other sound artifacts that may be generated within some voice enhancement systems and may reconstruct voice when needed.
While various embodiments of the invention have been described, it will be apparent to those of ordinary skill in the art that many more embodiments and implementations are possible within the scope of the invention. Accordingly, the invention is not to be restricted except in light of the attached claims and their equivalents.
This application is a continuation of U.S. application Ser. No. 10/688,802, “System for Suppressing Wind Noise,” filed Oct. 16, 2003 now U.S. Pat. No. 7,895,036, which is a continuation in-part of U.S. application Ser. No. 10/410,736, “Method and Apparatus for Suppressing Wind Noise,” filed Apr. 10, 2003 now U.S. Pat. No. 7,885,420. The disclosure of each of these applications is incorporated herein by reference.
Number | Name | Date | Kind |
---|---|---|---|
4486900 | Cox et al. | Dec 1984 | A |
4531228 | Noso et al. | Jul 1985 | A |
4630304 | Borth et al. | Dec 1986 | A |
4811404 | Vilmur et al. | Mar 1989 | A |
4843562 | Kenyon et al. | Jun 1989 | A |
4845466 | Hariton et al. | Jul 1989 | A |
4959865 | Stettiner et al. | Sep 1990 | A |
5012519 | Adlersberg et al. | Apr 1991 | A |
5027410 | Williamson et al. | Jun 1991 | A |
5056150 | Yu et al. | Oct 1991 | A |
5146539 | Doddington et al. | Sep 1992 | A |
5251263 | Andrea et al. | Oct 1993 | A |
5313555 | Kamiya | May 1994 | A |
5400409 | Linhard | Mar 1995 | A |
5426703 | Hamabe et al. | Jun 1995 | A |
5426704 | Tamamura et al. | Jun 1995 | A |
5442712 | Kawamura et al. | Aug 1995 | A |
5479517 | Linhard | Dec 1995 | A |
5485522 | Solve et al. | Jan 1996 | A |
5495415 | Ribbens et al. | Feb 1996 | A |
5502688 | Recchione et al. | Mar 1996 | A |
5526466 | Takizawa | Jun 1996 | A |
5550924 | Heif et al. | Aug 1996 | A |
5568559 | Makino | Oct 1996 | A |
5574824 | Slyh et al. | Nov 1996 | A |
5584295 | Muller et al. | Dec 1996 | A |
5586028 | Sekine et al. | Dec 1996 | A |
5617508 | Reavers | Apr 1997 | A |
5651071 | Lindemann et al. | Jul 1997 | A |
5677987 | Seki et al. | Oct 1997 | A |
5680508 | Liu | Oct 1997 | A |
5692104 | Chow et al. | Nov 1997 | A |
5701344 | Wakui | Dec 1997 | A |
5727072 | Raman | Mar 1998 | A |
5752226 | Chan et al. | May 1998 | A |
5809152 | Nakamura et al. | Sep 1998 | A |
5839101 | Vähätalo et al. | Nov 1998 | A |
5859420 | Borza | Jan 1999 | A |
5878389 | Hermansky et al. | Mar 1999 | A |
5920834 | Sih et al. | Jul 1999 | A |
5933495 | Oh | Aug 1999 | A |
5933801 | Fink et al. | Aug 1999 | A |
5949888 | Gupta et al. | Sep 1999 | A |
5982901 | Kane et al. | Nov 1999 | A |
6011853 | Koski et al. | Jan 2000 | A |
6108610 | Winn | Aug 2000 | A |
6122384 | Mauro | Sep 2000 | A |
6130949 | Aoki et al. | Oct 2000 | A |
6163608 | Romesburg et al. | Dec 2000 | A |
6167375 | Miseki et al. | Dec 2000 | A |
6173074 | Russo | Jan 2001 | B1 |
6175602 | Gustafsson et al. | Jan 2001 | B1 |
6192134 | White et al. | Feb 2001 | B1 |
6199035 | Lakaniemi et al. | Mar 2001 | B1 |
6208268 | Scarzello et al. | Mar 2001 | B1 |
6230123 | Mekuria et al. | May 2001 | B1 |
6252969 | Ando | Jun 2001 | B1 |
6289309 | deVries | Sep 2001 | B1 |
6405168 | Bayya et al. | Jun 2002 | B1 |
6415253 | Johnson | Jul 2002 | B1 |
6434246 | Kates et al. | Aug 2002 | B1 |
6453285 | Anderson et al. | Sep 2002 | B1 |
6507814 | Gao | Jan 2003 | B1 |
6510408 | Hermansen | Jan 2003 | B1 |
6587816 | Chazan et al. | Jul 2003 | B1 |
6615170 | Liu et al. | Sep 2003 | B1 |
6643619 | Linhard et al. | Nov 2003 | B1 |
6647365 | Faller | Nov 2003 | B1 |
6687669 | Schrögmeier et al. | Feb 2004 | B1 |
6711536 | Rees | Mar 2004 | B2 |
6741873 | Doran et al. | May 2004 | B1 |
6766292 | Chandran et al. | Jul 2004 | B1 |
6768979 | Menéndez-Pidal et al. | Jul 2004 | B1 |
6782363 | Lee et al. | Aug 2004 | B2 |
6822507 | Buchele | Nov 2004 | B2 |
6859420 | Coney et al. | Feb 2005 | B1 |
6882736 | Dickel et al. | Apr 2005 | B2 |
6910011 | Zakarauskas | Jun 2005 | B1 |
6937980 | Krasny et al. | Aug 2005 | B2 |
6959276 | Droppo et al. | Oct 2005 | B2 |
7043030 | Furuta | May 2006 | B1 |
7047047 | Acero et al. | May 2006 | B2 |
7062049 | Inoue et al. | Jun 2006 | B1 |
7072831 | Etter | Jul 2006 | B1 |
7092877 | Ribic | Aug 2006 | B2 |
7117145 | Venkatesh et al. | Oct 2006 | B1 |
7117149 | Zakarauskas | Oct 2006 | B1 |
7158932 | Furuta | Jan 2007 | B1 |
7165027 | Kellner et al. | Jan 2007 | B2 |
7313518 | Scalart et al. | Dec 2007 | B2 |
7373296 | Van De Par et al. | May 2008 | B2 |
7386217 | Zhang | Jun 2008 | B2 |
20010028713 | Walker | Oct 2001 | A1 |
20020037088 | Dickel et al. | Mar 2002 | A1 |
20020071573 | Finn | Jun 2002 | A1 |
20020094100 | Kates et al. | Jul 2002 | A1 |
20020094101 | De Roo et al. | Jul 2002 | A1 |
20020176589 | Buck et al. | Nov 2002 | A1 |
20030040908 | Yang et al. | Feb 2003 | A1 |
20030147538 | Elko | Aug 2003 | A1 |
20030151454 | Buchele | Aug 2003 | A1 |
20030216907 | Thomas | Nov 2003 | A1 |
20040078200 | Alves | Apr 2004 | A1 |
20040093181 | Lee | May 2004 | A1 |
20040138882 | Miyazawa | Jul 2004 | A1 |
20040161120 | Petersen et al. | Aug 2004 | A1 |
20040165736 | Hetherington et al. | Aug 2004 | A1 |
20040167777 | Hetherington et al. | Aug 2004 | A1 |
20050114128 | Hetherington et al. | May 2005 | A1 |
20050238283 | Faure et al. | Oct 2005 | A1 |
20050240401 | Ebenezer | Oct 2005 | A1 |
20060034447 | Alves et al. | Feb 2006 | A1 |
20060074646 | Alves et al. | Apr 2006 | A1 |
20060100868 | Hetherington et al. | May 2006 | A1 |
20060115095 | Giesbrecht et al. | Jun 2006 | A1 |
20060116873 | Hetherington et al. | Jun 2006 | A1 |
20060136199 | Nongpiur et al. | Jun 2006 | A1 |
20060251268 | Hetherington et al. | Nov 2006 | A1 |
20060287859 | Hetherington et al. | Dec 2006 | A1 |
20070019835 | Ivo de Roo et al. | Jan 2007 | A1 |
20070033031 | Zakarauskas | Feb 2007 | A1 |
Number | Date | Country |
---|---|---|
2158847 | Sep 1994 | CA |
2157496 | Oct 1994 | CA |
2158064 | Oct 1994 | CA |
1325222 | Dec 2001 | CN |
0 076 687 | Apr 1983 | EP |
0 629 996 | Dec 1994 | EP |
0 629 996 | Dec 1994 | EP |
0 750 291 | Dec 1996 | EP |
1 450 353 | Aug 2004 | EP |
1 450 354 | Aug 2004 | EP |
1 669 983 | Jun 2006 | EP |
64-039195 | Feb 1989 | JP |
6282297 | Oct 1994 | JP |
6319193 | Nov 1994 | JP |
6349208 | Dec 1994 | JP |
2002-261530 | Sep 2000 | JP |
2001-215992 | Aug 2001 | JP |
2001-350498 | Dec 2001 | JP |
138806 | Jun 1998 | KR |
WO 0041169 | Jul 2000 | WO |
WO 0156255 | Aug 2001 | WO |
WO 0173761 | Oct 2001 | WO |
Number | Date | Country | |
---|---|---|---|
20110026734 A1 | Feb 2011 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 10688802 | Oct 2003 | US |
Child | 12902503 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 10410736 | Apr 2003 | US |
Child | 10688802 | US |