2. Technical Field
This disclosure relates to signal processing systems. In particular, this disclosure relates to a system that enhances periodic signal components.
3. Related Art
Audio signal processing systems may capture sound, reproduce sound, and convey sound. Audio systems may be susceptible to noise that can corrupt, mask, or otherwise detrimentally affect signal content. Wind, rain, background noise, such as engine noise, electromagnetic interference, and other noise sources may contribute noise to a signal captured, reproduced, or conveyed to other systems. When the noise level of sound increases, intelligibility of the signal may decrease.
Some systems may attempt to minimize noise by detecting sound through multiple microphones. The signals from each microphone may be combined to limit the noise. In some applications, multiple microphones may not be available. Other systems may use noise filters to selectively attenuate sound. The noise filters may indiscriminately eliminate or minimize desired signal content. Therefore, there is a need for a system that enhances signals.
A signal enhancement system reinforces signal content and improves the signal-to-noise ratio of a signal. The system detects, tracks, and reinforces non-stationary periodic signal components of a signal. The periodic signal components may represent vowel sounds or other voiced sounds. The system may detect, track, and attenuate quasi-stationary signal components in the signal.
Other systems, methods, features and advantages of the invention will be, or will become, apparent to one with skill in the art upon examination of the following figures and detailed description. It is intended that all such additional systems, methods, features and advantages be included within this description, be within the scope of the invention, and be protected by the following claims.
The invention can be better understood with reference to the following drawings and description. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention. Moreover, in the figures, like-referenced numerals designate corresponding parts throughout the different views.
The signal enhancement system detects and tracks one or more fundamental frequency components in a signal. The system may reinforce the tracked frequency components. The system may improve the intelligibility of information in a speech signal or other audio signals. The reinforced signal may have an improved signal-to-noise ratio (SNR).
In
The enhancement system 100 may accept input from the input sources 106. The input sources 106 may include digital signal sources or analog signal sources, such as a microphone 108. The microphone 108 may be connected to the enhancement system 100 through a sampling system 110. The sampling system 110 may convert analog signals from the microphone 108 into digital form at a selected sampling rate.
The sampling rate may be selected to capture any desired frequency content. For speech, the sampling rate may be approximately 8 kHz to about 22 kHz. For music, the sampling rate may be approximately 22 Hz to about 44 kHz. Other sampling rates may be used for speech and/or music.
The digital signal sources may include a communication interface 112, other circuitry or logic in the system in which the enhancement system 100 is implemented, or other signal sources. When the input source is a digital signal source, the signal enhancement system 100 may accept the digital signal samples with or without additional preprocessing.
The signal enhancement system 100 may also connect to post-processing logic 104. The post-processing logic 104 may include an audio reproduction system 114, digital and/or analog data transmission systems 116, or video processing logic 118. Other post-processing logic also may be used.
The audio reproduction system 114 may include digital to analog converters, filters, amplifiers, and other circuitry or logic. The audio reproduction system 114 may be a speech and/or music reproduction system. The audio reproduction system 114 may be implemented in a cellular telephone, wireless telephone, digital media player/recorder, radio, stereo, portable gaming device, or other devices employing sound reproduction.
The video processing system 118 may include circuitry and/or logic that provides a visual output. The signal used to prepare the visual output may be enhanced by the processing performed by the enhancement system 100. The video processing system 118 may control a television or other entertainment device. Alternatively, the video processing system 118 may control a computer monitor or liquid crystal display (LCD).
The transmission system 116 may provide a network connection, digital or analog transmitter, or other transmission circuitry and/or logic. The transmission system 116 may communicate enhanced signals generated by the enhancement system 100 to other devices. In a car phone, for example, the transmission system 116 may communicate enhanced signals from the car phone to a base station or other receiver through a wireless connection such as a ZigBee, Mobile-Fi, Ultrawideband, Wi-fi, or a WiMax protocol.
The enhancement system 100 may process quasi-stationary or non-stationary signals. Non-stationary signals may vary in frequency and/or amplitude relatively quickly over time. Voice is one example of a non-stationary signal.
With few exceptions, even the fundamental frequency component in a speaker's voice changes during speech. The change in fundamental frequency may vary by as much as approximately 50 percent per 100 ms or more. To the human ear, however, the speaker's voice may have a relatively constant pitch.
Quasi-stationary signals may change in frequency and/or amplitude less frequently than non-stationary signals. Quasi-stationary signals may arise from machine noise, a controlled human voice, or from other sources. Slowly changing engine noise or alternator whine are examples of quasi-stationary signals.
In
Speech signals may include a fundamental frequency component from approximately 70 Hz to about 400 Hz. Male speech may include a fundamental frequency component between approximately 70 Hz to about 200 Hz. Female speech may include a fundamental frequency component between approximately 200 Hz to about 400 Hz. A child's speech may include a fundamental frequency component between approximately 250 Hz to about 400 Hz.
The enhancement system 100 may process input signals that include speech from both male and female voices, either separately or simultaneously and overlapping. In such systems, the maximum pitch period may approximately correspond to the period of the fundamental frequency of the female voice. The maximum pitch period may be approximately about 1/300 Hz (approximately 3.3 ms), or it may be another pitch period associated with female voice.
Alternatively, the enhancement system 100 may process male speech. In such implementations, the maximum pitch period may correspond to the period of the fundamental frequency of a male voice. The maximum pitch period may be approximately 1/150 Hz (approximately 6.6 ms), or may be another pitch period.
The delay logic 204 may delay the input signal by the number of signal samples corresponding to the maximum pitch period. The number of signal samples may be given by:
NSS=MPP*f
where NSS is the number of signal samples, MPP is the maximum pitch period and fs is the sampling rate. Assuming an MPP of about 3.3 ms and a sampling rate of about 8 kHz, NSS=approximately 27 samples. In
The delayed input signal may be received by the filter 206. The filter 206 may include a filter output 208 that carries a filtered output signal, labeled y(n) or y. The filter 206 may track one or more frequency components in the input signal based on the delayed input signal. The filter 206 may track the fundamental frequencies in the input signal as the pitch changes during voiced speech.
The filter 206 may reproduce, replicate, approximate or otherwise include the tracked frequency content in the filtered output signal. The filter 206 may be a finite impulse response filter (FIR) or other type of digital filter. The filter coefficients may be adaptive. The filter 206 may be adapted by a normalized least mean squares (NLMS) technique or other type of adaptive filtering technique, such as recursive least squares (RLS) or proportional LMS. Other tracking logic, including other filters may also be used.
The filter 206 may converge to the fundamental frequency in the input signal. The range of fundamental frequencies f0 over which the filter 206 converges may be given by the following equations:
where ΔF0MAX is the period for the maximum pitch (expressed in terms of samples), fs is the sampling frequency (in units of Hz), and L is the length of the filter 206 (in units of samples). The filter length L may increase or decrease to increase or decrease the frequency extent over which the filter 206 tracks frequency components.
In the example above, the maximum pitch was approximately 300 Hz and the delay logic 204 may use a 27 sample delay. A filter length L of 64 samples may yield a filter 206 that tracks fundamental frequency content over a frequency range of approximately 88 Hz to about 296 Hz as follows:
The filter 206 may adapt over time. The filter 206 may adapt by evaluating an error signal “e” on a sample-by-sample basis. Alternatively, the filter 206 may adapt based on blocks of samples, or some other basis. In adapting, the filter 206 may change one or more of its filter coefficients. The filter coefficients may change the response of the filter 206. The filter coefficients may adapt the filter 206 so that the filter 206 may minimize the error signal e.
The error estimator 210 may generate the error signal e. The error estimator 210 may be an adder, comparator, or other circuitry or logic. The error estimator 210 may compare the input signal “x” with the filtered output signal “y”.
As the filter 206 converges to the fundamental frequency in the input signal, the error signal may decrease. As the error signal decreases, the filtered output signal “y” may more closely resemble the input signal x delayed by an integer multiple of the signal's fundamental frequencies. The gain control logic 212 may respond to the error signal.
The optional gain control logic 212 may include a multiplier 214 and a gain parameter 216. The gain control logic 212 may attenuate, amplify, or otherwise modify the filtered output signal.
The reinforcement logic 218 may reinforce frequency content in the input signal “x” with the gain controlled signal Ay. The reinforcement logic 218 may be an adder or other circuitry and/or logic. The reinforcement logic 218 may produce the enhanced output signal according to the equation below:
s=x+Ay
When the error signal e increases, the gain control logic 212 may reduce the gain A. When the gain is reduced, the filtered output signal may contribute less to the enhanced output signal. The relationship between the error signal e and the gain A may be continuous, stepped, linear, or non-linear.
In one implementation, the enhancement system 100 may establish one or more error thresholds. As the error signal exceeds an upper threshold, the gain control logic 212 may reduce the gain A to about zero. The upper threshold may be set to the input signal so that if e>x, then the gain A may be set to about zero. As the error signal falls below a lower threshold, the gain control logic 212 may increase the gain A to a value of about one.
When the error signal e exceeds the upper threshold, the filter control logic 220 may reset the filter 206. When the filter 206 is reset, the control logic 220 may reset the filter coefficients to about a value of zero, may re-initialize the filter coefficients, or may take other actions. The control logic 220 may also dynamically modify the filter length, may modify the delay implemented by the delay logic 204, or may modify other characteristics of the enhancement system 100. The control logic 220 also may modify the enhancement system 100 to adapt to changing environments in which the enhancement system 100 is used, to adapt the enhancement system 100 to a new speaker, or to other applications.
The filter control logic 220 also may control how quickly the filter 206 adapts, whether the filter adapts, or may monitor or control other filter characteristics. The control logic 220 may expect quickly changing frequency and amplitude components in the input signal. The control logic 220 may also expect or determine over time that particular frequency components in the input signal are prevalent.
The control logic 220 also may determine that the input signal x has changed in frequency content, amplitude, or other characteristics from what is expected or from what has been determined. In response, the control logic 220 may stop the filter 206 from adapting to the new signal content, may slow the rate of adaptation, or may take other actions. The control logic 220 may exercise control over the filter 206 until the input signal characteristics return to an expected signal, until a predetermined time has elapsed, until instructed to release control, or until another time or condition is met.
The delay logic 204 may prevent the filtered output signal from precisely duplicating the current input signal x. Thus, the filtered output signal may closely track the selected periodicities in the input signal x. When the current input signal x is reinforced by the filtered output signal y to produce the output signal s, periodic signal components may combine constructively, and random noise components may combine destructively. Therefore, the periodic signal components may be enhanced more than the noise.
The delay introduced by the delay logic 204 and the filter 206 may be approximately one cycle of a fundamental frequency component tracked by the filter 206. The delay may correspond to the glottal pulse delay for voice sounds, such as vowels. When the filtered output signal is added to the input signal, the delay may allow the fundamental frequency components to add in-phase or approximately in-phase.
When added in-phase, the resulting gain in the fundamental frequency content in the enhanced output signal may be approximately 6 dB or more. The noise in the input signal and the filtered output signal may tend to be out of phase. When the input signal and the filtered output signal are added, the noise may increase less than the enhanced frequency content, for example by 3 dB or less. The enhanced output signal may have an increased signal-to-noise ratio.
The input signal x that the enhancement system 100 processes may include multiple fundamental frequencies. For example, when two speakers are speaking at the same time, input signal may include two non-stationary fundamental frequencies. When multiple fundamental frequencies are present, the filter 206 may continue to adapt and converge to provide a filtered out signal y, which may be a delayed version of the input signal. The reinforcement logic 218 may reinforce one or more of the fundamental frequencies present in the input signal.
At any instance in time, the coefficients 300 may be analyzed to determine a fast estimate of the fundamental frequencies in the input signal, with good temporal resolution. The coefficients 300 may begin to peak at about coefficient 304 (the fifth filter coefficient), coefficient 306 (the sixth filter coefficient), and coefficient 308 (the seventh filter coefficient). By searching for a coefficient peak or an approximate coefficient peak and determining a corresponding coefficient index, c, a fast approximation of the fundamental frequency, fa, may be made where:
In
The control logic 220 may store historical data on many characteristics of the input signal, including the fundamental frequency of the input signal as it changes over time. The control logic 220 may examine the historical data as an aid in determining whether the characteristics of the input signal have unexpectedly changed. The control logic 220 may respond by exercising adaptation control over the filter 206 or by taking other actions.
The input signal may be delayed and filtered (Act 510). The enhancement system 100 may generate an error signal and may responsively adapt the filter 206 (Act 512). The enhancement system 100 may control the gain of the filtered output signal (Act 514). The enhancement system 100 may add the input signal and the gain controlled signal (Act 516) to produce an enhanced output signal. The enhancement system 100 also may determine fundamental frequency estimates (Act 518). The enhancement system 100 may employ the frequency estimates to exercise adaptation control over the filter 206 (Act 520).
A signal input 606 may connect to the first stage 602. The signal input 606 may connect to the delay logic 608. The delay logic may implement a delay that corresponds to the period of a maximum quasi-stationary frequency that may be suppressed by the first stage 602.
The maximum quasi-stationary frequency may be selected according to known or expected characteristics of the environment in which the enhancement system 600 is used. The filter control logic 610 may dynamically modify the delay to adapt the first stage 602 to the environment. The filter control logic 610 also may control the quasi-stationary filter 612.
The filter 612 in the first stage may include signal component tracking logic such as a NLMS adapted FIR filter or RLS adapted FIR filter. The filter 612 in the first stage may adapt slowly, for example with a sampling rate of about 8 kHz, a filter length of about 64, and an NLMS step size larger than about 0 and less than approximately 0.01. This may allow attenuation of quasi-stationary periodic signals while minimally degrading typical speech signals. The first stage filtered output 614 may provide a filtered output signal that approximately reproduces the quasi-stationary signal component in the input signal.
The suppression logic 616 and slow filter adaptation may allow non-stationary signal components to pass through the first stage 602 to the second stage 604. On the other hand, the suppression logic 616 may suppress quasi-stationary signal components in the input signal. The suppression logic 616 may be implemented as arithmetic logic that subtracts the filtered output signal from the input signal.
The replicated quasi-stationary signal content in the filtered output signal may be removed from the input signal. The output signal produced by the first stage 602 may be shown by the equation below:
x2=e1=x−y1
where e1 is the first stage output signal, x is the input signal, and y1 is the first stage filtered output. The first stage output 618 may be connected to the second stage 604. The second stage 604 may process the signal x2 with the adaptive filter 206. The filter 206 may adapt quickly, for example with a sampling rate of 8 about kHz, a filter length of about 64, and an NLMS step size larger than approximately 0.6 and less than about 1.0. This may allow the adaptive filter 206 to track the fundamental frequencies in typical speech signals.
The second stage 604 may enhance non-stationary signal components in the first stage output signal. The non-stationary signal components may be present in the input signal as a result of speech, music, or other signal sources. The second stage 604 may process the first stage output signal as described above.
The enhancement system 600 may employ a first suppression stage 602 followed by a second enhancement stage 604. The enhancement system 600 may be employed to reinforce non-stationary signal content, such as voice content. In environments that introduce slowly changing signal components, the enhancement system 600 may remove or suppress the slowly changing signal components. In a car phone, for example, the first stage 602 may remove or suppress engine noise, road noise, or other noises, while the second stage 604 enhances non-stationary signal components, such as male or female voice components.
The signal enhancement system 100 may enhance periodic signal content, increase SNR, and/or decrease noise in an input signal. When applied to a voice signal, the enhancement system 100 may reinforce fundamental speech frequencies and may strengthen vowel or other sounds. The enhancement system 100 may enhance other signals, whether they are audible or inaudible.
The overall delay introduced by the delay logic 204 or 608 and the filter 206 or 612 also may be approximately an integer number (one or greater) of cycles of the tracked pitch period. Delaying by additional cycles may allow the input signal to change to a greater degree than by waiting one cycle. Adding the longer delayed filtered signal to the current input signal may produce special effects in the output signal, such as reverberation, while still enhancing fundamental frequency components.
Performance of the periodic enhancement system 100 of
The sub-band periodic signal enhancement system 700 may include a filter bank 706 having a plurality of band-pass filters 710. The filter bank 706 may divide the input signal spectrum into a plurality of sub-band frequencies. Processing each sub-band frequency range individually may provide greater control and flexibility when processing the input signal x(n). Because each band-pass filter 710 may process a specific frequency range, the amount of signal enhancement in each sub-band may be more accurately controlled. When the signal-to-noise ratio across the frequency spectrum is non-uniform, the sub-band periodic signal enhancement system 700 may permit more accurate control of the amount of signal enhancement in each sub-band. This may permit harmonic signals to be enhanced more evenly across the entire spectrum of interest.
Important harmonic frequencies may be present in lower-frequency regions of a speech signal. Dividing the frequency range into sub-bands and processing each sub-band separately may permit improved isolation of individual harmonic components. This may increase the signal-to-noise ratio of the output signal. Dividing the frequency range into sub-bands may also permit improved isolation of weak harmonic signals in the presence of stronger harmonic signals or out-of-band noise signals.
Each band-pass filter 710 may provide band-divided output signals x1(k) . . . xN(k) to respective periodic signal enhancement circuits 720. The subscript N may denote the number of sub-bands. Each periodic signal enhancement circuit 720 may be the same as or similar to the periodic signal enhancement system 100 of
Each band-pass filter 710 may be similar to or identical in construction to the other band-pass filters in the filter bank 706. Alternatively, any specific band-pass filter may be different than an adjacent or other filter and may differ with regard to frequency response, bandwidth, band-pass range, or other parameters. For example, if the range of input frequencies varies from about 0 Hz to about 5,000 Hz, ten band-pass filters 710 may be used. The first band-pass filter may divide the input frequency into signals in the range from about 0 Hz to about 500 Hz, the second band-pass filter may divide the input frequency into signals in the range from about 500 Hz to 1000 Hz, etc. However, the frequency range processed may be greater or less than 5,000 Hz, and the number of band-pass filters may change. The choice of the number of band-pass filters 710 may be based on economic and complexity criteria. For clarity, five band-pass filters 710 are shown, but a greater number of filters or a fewer number of filters may be used.
The band-pass filter bank 706 may satisfy the perfect reconstruction principle. The perfect reconstruction principle stipulates that an original signal may be fully recovered from a plurality of corresponding sub-band signals by using an appropriate synthesis filter bank. Each band-pass filter 710 may be a finite impulse response (FIR) filter and may be implemented as an FFT (fast Fourier transform) filter. For example, each sub-band filter 710 may have about five to about ten taps and a similar number of filter coefficients.
The sub-band filter bank 706 may have a non-uniform bandwidth depending upon the specific application and type of input signal processed. The sub-band filter bank 706 may include Gammatone-type band-pass filters. Gammatone-type filters may model the human auditory system where the bandwidth and frequency response of each band-pass filter may approximate a basilar membrane.
Each band-pass filter 706 may be implemented as a separate hardware or software component in the sub-band periodic signal enhancement system 700. The computational complexity of the sub-band periodic enhancement system 700 may depend upon the number of sub-bands. To reduce cost and complexity, the number of separate band-pass filters 706 may be reduced by down-sampling the input signal. Down-sampling by a selected factor may require a corresponding increase in the sampling rate. The more complex the frequency response of each sub-band adaptive filter, the greater the length of the adaptive filter needed. For a filter bank 706 having N sub-bands, the input signal may be down-sampled by a factor of N. As a result, the length of each adaptive filter may be 1/N of the length of an equivalent adaptive filter without employing down-sampling. Accordingly, the computational load may not be significantly increased by using a plurality of sub-bands.
A synthesis filter bank 740 having a plurality of sub-band synthesis filters 750 may receive the respective output signals y1(k) . . . yN(k). The synthesis filter bank may reconstruct the sub-band signals and may include a summing circuit 760 to combine the sub-band signals and generate an output signal y(n). The synthesis filter bank 750 may also satisfy the perfect reconstruction principle. Each sub-band synthesis filter 750 may be similar to or identical to the other sub-band synthesis filters in the synthesis filter bank 740. Alternatively, any specific sub-band synthesis filter may be different than an adjacent or other filter and may differ with regard to frequency response, bandwidth or other parameters. Each sub-band synthesis filter 750 may be a finite impulse response (FIR) filter and may be implemented as an inverse FFT (fast Fourier transform) filter. For example, each sub-band synthesis filter 750 may have about five to about ten taps and a similar number of filter coefficients.
The periodic signal enhancement circuit 720 may include a delay circuit 830 that receives the sub-band signal xi(k) and an adaptive filter system 840. The adaptive filter system 840 may include the adaptive filter 206, the filter control logic 220, the error estimator 210, the gain control logic 212 and the reinforcement logic 218. A multiplier circuit 844 may scale the output of the adaptive filter system 840 by a control parameter Gi (846). The control parameter Gi may control an amount of periodic signal enhancement in each sub-band. The value of Gi may vary from about 0 to about 1. The amount of periodic signal enhancement in each sub-band may be adjusted depending upon the background noise spectrum, microphone characteristics, and channel impulse response. A summing circuit 852 may sum or combine the scaled output of the multiplier circuit 844 and the output x(k) of the band-pass filter.
The tonal noise detector 910 may detect the presence of tonal noise in a sub-band range. The tonal noise detector 910 may monitor the maximum value of the adaptive filter coefficients. If the maximum value of the coefficients is greater than a predetermined value for a predetermined period of time T, the tonal noise detector 910 may detect the presence of tonal noise. The pitch of speech, and in particular the pitch of vowel sounds, may be expected to vary within a particular period of time. A vowel sound may remain constant, but only for a fairly short period of time. In contrast, a tonal noise sound may remain constant for a longer period of time. The predetermined period of time T may be set to a value larger than the expected length of vowel sounds occurring in natural speech.
When the tonal noise detector 910 detects tonal noise, the tonal noise detector may direct the adaptive filter system 840 to stop adaptation of the filter coefficients. Alternatively, the tonal noise may be minimized. To minimize the tonal noise, a multiplier circuit 844 may scale the output of the adaptive filter system 840 by a negative component Bi of the tonal noise. The negative component Bi of the tonal noise may range from about minus one to about zero. A value of minus one may indicate that the tonal noise is minimized or cancelled. A value of zero may indicate that output of the adaptive filter system 840 may be unmodified.
The signal enhancement systems 200, 600, 700 and 720 may be implemented in hardware, software, or a combination of hardware and software. The enhancement systems 200, 600, 700 and 720 may take the form of instructions stored on a machine-readable medium such as a disk, flash card, or other memory. The enhancement systems 200, 600, 700 and 720 may be incorporated into communication devices, sound systems, gaining devices, signal processing software, or other devices and programs.
While various embodiments of the invention have been described, it will be apparent to those of ordinary skill in the art that many more embodiments and implementations are possible within the scope of the invention. Accordingly, the invention is not to be restricted except in light of the attached claims and their equivalents.
1. Priority Claim This application is a continuation-in-part of and claims the benefit of priority from U.S. application Ser. No. 10/973,575, filed Oct. 26, 2004, which is incorporated by reference.
Number | Date | Country | |
---|---|---|---|
Parent | 10973575 | Oct 2004 | US |
Child | 11757768 | Jun 2007 | US |