Patent Application
20240312474
The present invention relates generally to audio devices, and in particular to wireless headsets, with air microphones and vibration sensors to achieve voice quality enhancement in noisy conditions. The invention relates to voice quality enhancement methods.
The use of headsets wirelessly connected to host devices like smartphones, computers, laptops, gaming consoles, smart TVs, smart watches, augmented reality (AR) systems, virtual reality (VR) systems, tablets or any device that can wirelessly connect to headsets is becoming increasingly popular. Whereas consumers used to be tethered to their electronic device with wired headsets, wireless headsets are gaining more traction due to the enhanced user experience, providing the user more freedom of movement, enhanced portability and comfort of use. Further momentum for wireless headsets has been gained by certain smartphone manufacturers abandoning the implementation of the 3.5 mm audio jack in the smartphone, and promoting voice communications and music listening wirelessly, for example by using Bluetooth® technology.
In many environments, people are exposed to loud noises. For example, people visit music festivals where the sound levels are typically above the level where hearing damage may occur. Factory workers, construction builders, and professionals working in the music industry are frequently exposed to loud sound levels. The examples also include environments such as airplanes, offices, public transportations, and sports arenas. More and more people are wearing ear plugs to reduce the sound level arriving at their ear drums, and thus to achieve a desired sound level including avoiding hearing loss which typically results from exposure to loud sound levels for a longer duration of time.
Passive ear plugs are widely used as hearing protection device providing noise reduction by physically blocking sound from entering the ear canal. The problem with the passive ear plugs is that communication is impossible because the ear canal is blocked. As a result, many people remove their hearing protecting ear plugs when they wish to communicate, be it via a (smart)phone or directly orally to a person nearby. Combining the technology used in wireless headsets with hearing protection measures is one way to solve the communication problem in loud environments. For professionals working in loud environments, the combination of wireless headsets and hearing protection measures improves the quality of life of professionals because in addition for communications, they may use their headset to listen to their favorite music or podcast while working.
Wireless headsets typically have one or more air microphones to pick up the voice of the user. Air microphones pick up airborne acoustic waves and convert them into electrical signals. By using one or more air microphones, voice can be detected and captured. Once captured and converted into electrical signals the voice signals can be processed and/or analyzed. However, the air microphones (MIC) may pick up the loud environmental noise as well. If the noise level is high and dominates the audio signal that is picked up, the voice may not be audible.
For picking up voice more efficiently, a vibration sensor may be applied to pick up the voice. Vibration sensors may pick up the mechanical vibrations in the human skull caused by the vocal cords. Vibrations can be picked up via the skin (Skin Surface Microphones), from the bones (Bone Conduction microphone), or from other tissues in the user's head. The vibration sensor can for example be implemented by an accelerometer which may use MEMS technology. However, the vibration sensors cannot completely suppress the environmental noise. In addition to bone vibrations, it will also be sensitive to vibrations caused by the air waves from the noise which act upon the housing of the headset. Many vibration sensors have a high-frequency transfer function in order to compensate for the low-pass filtering of the voice signal caused by audio waves traversing through the human bone and tissue. As a result, high frequencies are emphasized to equalize the low-passed voice signal. Yet, the high-pass filtering becomes quite noticeable for the loud environmental noise that reaches the vibration sensor. Therefore, even a vibration sensor may perform poorly when picking up the voice signal in the presence of loud noise. Wireless headsets with improved voice pick-up performance in loud environmental noise conditions are therefore desirable.
The Background section of this document is provided to place embodiments of the present invention in technological and operational context to assist those of skill in the art in understanding their scope and utility. Unless explicitly identified as such, no statement herein is admitted being prior art merely by its inclusion in the Background section.
The following presents a simplified summary of the disclosure in order to provide a basic understanding to those of skill in the art. This summary is not an extensive overview of the disclosure and is not intended to identify key/critical elements of embodiments of the invention or to delineate the scope of the invention. The sole purpose of this summary is to present some concepts disclosed herein in a simplified form as a prelude to the more detailed description that is presented later.
The term “air microphone”, or “microphone” not used in a context of using a condensed medium, used in this document may mainly relate to the devices that detect audio signals travelling through gaseous medium such as air.
The term “vibration sensor” used in this document may mainly relate to the sensor devices that detect audio signals travelling through condensed medium such as solid, liquid, human tissue, human bones, etc.
The term “headset” used in this document includes earphones, headphones, earbuds, earplugs and any audio device which can be worn over or inside the ears to facilitate audio communication.
The term “voice pickup” refers to a process for capturing the voice in an audio signal, including enhancing the clarity and intelligibility of the voice within the audio signal. Sounds will be picked-up by air microphone or vibration sensors of a headset. Sounds will contain voice and other sounds, referred to as noise. Noise includes environmental noise or background noise. Voice pickup may be improved by suppressing the noise within the electrical audio signal.
A first aspect of the present invention relates to a method of improving voice pickup in a headset. The method of the first aspect may comprise a step of picking up sounds using one or more vibration sensors. Vibration sensors in headsets will detect vibration signals generated by the vocal cords propagating through the bones and muscles of a user. The one or more vibration sensors will process the picked-up sounds into a vibration sensor audio signal. Sounds picked up by the one or more vibration sensors may be processed and converted into electrical audio signals. The processing in the vibration sensor may further include echo cancellation, noise suppression, automatic gain control, voice activity detection, personal voice recognition, or any type of processing suitable for voice pickup. The processing may be applied to various properties of picked-up sounds such as amplitude, frequency, timbre, phase, and envelop. The vibration sensors in headsets are designed to capture vibration signals coming from vocal cords propagating inside the user's body. Vibrations sensors are less sensitive to other sounds/noise.
The method of the first aspect may further include the step of picking up sounds using one or more air microphones. Having multiple air microphones in a headset may be beneficial for realizing advanced audio processing techniques such as Enhanced Voice Pickup, Noise Suppression (NS), Active Noise Cancellation (ANC), spatial audio, and 3D sound.
The method of the first aspect may further include the step of processing sounds picked up by the one or more air microphones into a processed air microphone audio signal. The audio signal is the electrical output of the one or more air microphones. The processing may be any technique which can be used to pick-up sounds and create a suitable electrical audio signal. The processing may be applied to various properties of picked-up sounds such as amplitude, frequency, timbre, phase, and envelop.
Embodiments of the invention relate to methods and devices for reducing noise in the obtained output audio signal by enhancing the signal-to-noise ratio (SNR) of the voice that is picked up, wherein the voice signal is the desired signal of the SNR.
Embodiments of the method may further include the step of generating an output audio signal by combining the air microphone audio signal and the vibration sensor audio signal. Air microphones detect sound waves travelling in the air by converting the pressure variations of sound waves into electrical signals. In a noisy environment, air microphones will collect audio signals that have low SNR. As a result, the processed air microphone audio signal may predominantly contain noise signal. A vibration sensor may collect audio signals which have higher SNR than the air microphone. By combining the audio signals captured by different sources, the resultant output audio signal can have higher SNR than the SNR of the audio signal picked-up by the vibration sensor.
Combining of audio signals refers to any mathematical (addition, subtraction, linear combination, multiplication, integration, etc.) combination of properties of respective electrical audio signals. Preferably combining of audio signals refers to addition or subtraction of amplitudes of the audio signal. The audio output will have a duration corresponding to a certain duration of live sounds. The combined air microphone and vibration sensor audio signals span the same duration live sounds and have been captured at generally the same time and location.
Preferably the combining is embodied by subtracting the air microphone audio signal and the vibration sensor audio signal. to obtain a subtracted audio output audio signal. The subtraction may be carried out by using a subtractor. The subtracting step surprisingly results in reducing the noise portion of the processed vibration sensor audio signal. In this noisy environment, the low SNR signal of air microphone may be used to suppress the noise in the vibration sensor signal. As a result, the audio output audio signal has higher SNR.
In embodiments, the method of the first aspect may comprise the step of adjusting any of the properties of the audio signals that are input to the subtractor, preferably of the air microphone audio signal, and most preferably the amplitude of the audio signal, using an adaptive filter. An adaptive filter may adjust, by amplification (amplification is being used here to not only refer to an increase in signal but also a decrease, e.g. an amplification value between 0 and 1) the air microphone audio signal so that subsequent subtraction using that adjusted air microphone audio signal results in a higher SNR in the output audio signal.
In embodiments, the method of the first aspect may further comprise configuring, e.g. weighing, the adaptive filter based on the subtracted output audio signal, preferably on the SNR of the subtracted output audio signal. The method can be implemented by a feedback control loop by connecting the subtracted output audio signal to the adaptive filter for controlling the processed air microphone audio signal. In embodiments, increasing or decreasing the amplitude of the air microphone audio signal that is subtracted from the vibration sensor audio signal results in more or less SNR in the resultant subtracted output audio signal and the adaptive filter is adapted according to increase the SNR. This allows optimizing the subsequent subtraction based on the “live” output audio signals.
In embodiments, the adaptive filter allows different filter settings for different frequency bands. Each filter setting can be determined individually based on a frequency band in the output audio signal.
In embodiments, the method of the first aspect may further comprise a step of monitoring a noise level in any of the audio signals. The noise level in the audio signal will be dependent on the (environmental and/or background) noise in the sounds picked up by the headset. The SNR of voice in the audio signals will decrease if noise increases. The noise can vary and is dependent e.g. on where the user is and moves to during use of a headset. By continuously monitoring the noise level in the audio signals, further method steps can be carried out.
In embodiments, when the monitored noise level is high, the method of the first aspect includes outputting the subtracted output audio signal. When the noise level decreases, a second output audio signal is provided by, preferably linearly, combining the subtracted output audio signal with the air microphone audio signal and/or the vibration sensor audio signal. When the noise level decreases, the amount of subtracted output audio signal in the second output audio signal decreases. At low noise levels, the second output audio signal predominantly comprises the vibration sensor audio signal and/or the air microphone audio signal. In a low noise environment, the vibration sensor audio signal and the air microphone audio signal may both have a relatively high SNR. By linearly combining the subtracted output audio signal with the vibration sensor/air microphone audio signal into the second output audio signal, that second output audio signal will have good SNR at any noise level. When the noise level changes, the second output audio signal will transition to a linear combination of the subtracted output audio signal and the vibration sensor/air microphone audio signal. When the noise level varies smoothly, the change in the linear combination of the second output audio signal will transition similarly smoothly.
In embodiments, combining the subtracted output audio signal with the vibration sensor/air microphone audio signals may be carried out by using a combiner. The combiner can create a linear combination. Preferably a normalized linear combination is made. In preferred embodiments, the combiner may be configured to generate an output depending on the monitored noise level, wherein weight factors may be applied when combining the subtracted output audio signal and the processed vibration sensor audio signal and/or the processed air microphone audio signal, wherein the weight factors are dependent on the monitored noise level.
In embodiments, the monitoring of the noise level may use spectral analysis for determining the measured noise level. This allows analyzing audio signals in the frequency domain for identifying noise sources using the spectral characteristics of different noise sources and/or particular frequency behaviors (e.g., harmonics).
Monitoring the noise level allows determining changes in noise level and embodiments of the method can implement changes in subtracting the air microphone audio signal from the vibration sensor audio signal and/or changes in the combining of subtracted output audio signal and the air microphone/vibration sensor audio signal. In an embodiments, when the noise level is low such that the air microphone audio signal contains a strong voice component, the final output may not contain the subtracted output audio signal but only the air microphone audio signal or the vibration sensor audio signal.
In embodiments, the combination of vibration sensor audio signals may use voice detection for combining the subtracted output audio signal and the air microphone/vibration sensor audio signals. The voice detection may include spectral analysis of audio signals for detecting a voice component.
In embodiments, the method of the first aspect may further comprises the step of wirelessly sending the vibration sensor audio signal and/or the air microphone audio signal from a first wireless headset, preferably via a host device, to a second wireless headset, wherein the remaining steps (e.g., generating step) of the methods are carried out. In alternative embodiments, the method of the first aspect may further comprises the step of wireless sending the vibration sensor audio signal and/or the air microphone audio signal from a first wireless headset to a host device (e.g., a smartphone), wherein the remaining steps (e.g. generating step) of the methods are carried out.
A second aspect of the present invention relates to a headset comprising, one or more vibration sensors, and one or more air microphones and a subtractor for audio signals obtained with the one or more vibration sensors and the one or more air microphones.
The one or more vibration sensors are arranged to pick-up sounds and to convert the sounds into a vibration sensor audio signal. The one or more vibration sensor can have processors for processing the audio signal. The vibration audio signal is fed to the subtractor. The one or more air microphones are arranged to pick-up sounds and to convert the sounds into a air microphone audio signal. The one or more air microphones can have processors for processing the air microphone audio signal. The air microphone signal is fed to the subtractor.
According to embodiments of the invention, the subtractor is arranged to receive the vibration audio signal and the air microphone audio signal. The subtractor is arranged to subtract the air microphone audio signal from the vibration audio signal. The subtractor is arranged to obtain a subtracted output audio signal. The vibration audio signal will have, in high noise environments, have a relative high SNR for the voice in the picked-up sounds. The air microphone audio signal has low SNR with respect to the voice in the picked up sounds. Subtracting the air microphone audio signal from the vibration audio signal will reduce the noise present in the vibration audio signal even further, thereby increasing the SNR with respect to voice.
In embodiments, the headset may be a wireless headset. In embodiments, the headset may be earphones, headphones, earbuds, earplugs and any audio device which can be worn over or inside the ears to facilitate audio communication. The headset can comprise a microprocessor configured to process audio signals.
In embodiments, the headset may be configured to carry out the methods of the first aspect. One skilled in the art would appreciate that more hardware components may be used for carrying out the steps of the methods of the first aspect. These hardware components may include antenna, audio codec, Digital-to-Analog (D/A) converter, speaker, radio transceiver, Digital Signal Processor (DSP), Tensilica processor, digital microphones, Power Management Unit (PMU), battery, and any component suitable for carrying out the methods of the present invention.
In embodiments, the headset further comprises a adaptive filter. The adaptive filter is arranged to filter the air microphone audio signal. Filtering can encompass amplification (including amplification factors between 0 and 1), preferably of the amplitude of the air microphone audio signal. The output of the one or more air microphone is fed to the subtractor. In embodiments the adaptive filter is also connected to the output of the subtractor. By providing the subtracted output audio signal to the adaptive filter, a feedback loop is created, wherein the adaptive filtering of the air microphone audio signal that is fed to the subtractor is based on the subtracted output audio signal. The adaptive filter can be set to increase the SNR in the subtracted output audio signal. In embodiments that adaptive filter is set to bring the level of the noise (=non-voice signal in the audio signal) in the air microphone audio signal to a similar level as the noise in the vibration sensor audio signal. By subsequently subtracting the adaptively filtered air microphone audio signal from the vibration sensor audio signal, the noise present in the vibration sensor audio signal is further reduced, increasing the SNR.
In embodiments the headset further comprises a combiner. The combiner is arranged to combine the subtracted output audio signal with the vibration sensor audio signal and/or with the air microphone audio signal. Preferably the combiner is arranged to linearly combine the two signals. The combiner allows combining the subtracted output audio signal with one of the original audio signals.
In embodiments the headset further comprises a noise level sensor arranged to obtain a noise level signal. The noise level sensor allows determining whether the headset is being used in a noisy environment or in a low noise environment. In case of a low noise environment, the headset can be configured to output the vibration sensor audio signal or the air microphone audio signal as the output for further reproduction. In case of low noise, the SNR of the vibration sensor audio signal and/or the air microphone audio signal is high enough for high quality reproduction of voice signals. Switching to a 100% vibration sensor audio signal or air microphone audio signal can be implemented by the combiner that sets weighing of the subtracted output audio signal to zero and weighing of the vibration sensor audio signal or air microphone audio signal to 100%. In that manner the second audio output signal of the combiner will be one of the original picked up audio signals.
In embodiments, the combiner is arranged to combine the subtracted output audio signal with the vibration sensor audio signal and/or air microphone audio signal dependent on the noise level signal. In case of increasing noise, the combiner can increase the weight of the subtracted output audio signal in the second output audio signal. In case of decreasing noise, the combiner can lower the weight of the subtracted output audio signal in the second output audio signal. In this manner a gradual transition between subtracted output audio signal and one of the original audio signals, dependent on the noise level in the picked-up sounds, can be obtained.
In embodiments, a voice activity or noise level detection circuit may be used to gradually switch the voice audio path from a voice audio path wherein the noise is suppressed by the adaptive filter to a voice audio path directly originating from the air microphone(s) and/or vibration sensor(s) depending on the noise level.
By exploiting the low signal-to-noise conditions on the air microphone(s), the noise can be suppressed in the vibration sensor signal using an adaptive filter and a subtractor. In high signal-to-noise conditions, the voice signal can be picked up directly from the air microphone and/or vibration sensor without the use of the adaptive filter. A voice activity or noise level detection circuit can be used to gradually switch from a voice audio path where noise is suppressed by an adaptive filter to a voice audio path directly originating from the air microphone and/or vibration sensor when the noise level is diminishing.
The above and the following present a basic understanding to those of skill in the art. This summary is not an extensive overview of the disclosure and is not intended to identify key/critical elements of embodiments of the invention or to delineate the scope of the invention. The sole purpose of this summary is to present some concepts disclosed herein in a simplified form as a prelude to the more detailed description that is presented later.
The present invention will now be described more fully hereinafter with reference to the accompanying drawings, showing several embodiments of the invention. However, this invention should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Like numbers refer to like elements throughout. Embodiments will be described, by way of example only, with reference to the accompanying schematic drawings in which corresponding reference symbols indicate corresponding parts, and in which:
For simplicity and illustrative purposes, the present invention is described by referring mainly to exemplary embodiments thereof. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be readily apparent to one of ordinary skill in the art that the present invention may be practiced without limitation to these specific details. In this description, well known methods and structures have not been described in detail so as not to unnecessarily obscure the present invention.
Electronic devices, such as mobile phones and smartphones, are in widespread use throughout the world. Although the mobile phone was initially developed for providing wireless voice communications, its capabilities have been increased tremendously. Modern mobile phones can access the worldwide web, store a large amount of video and music content, include numerous applications (“apps”) that enhance the phone's capabilities (often taking advantage of additional electronics, such as still and video cameras, satellite positioning receivers, inertial sensors, and the like), and provide an interface for social networking. Many smartphones feature a large screen with touch capabilities for easy user interaction. In interacting with modern smartphones, wearable headsets are often preferred for enjoying private audio, for example voice communications, music listening, or watching video, thus not interfering with or disturbing other people sharing the same area. Because it represents such a major use case, embodiments of the present invention are described herein with reference to a smartphone, or simply “phone” as the host device. However, those of skill in the art will readily recognize that embodiments described herein are not limited to mobile phones, but in general apply to any electronic device capable of providing audio content.
Hearing loss used to be a human defect connected with aging. Yet, today many people, including the young, are experiencing hearing problems like hearing loss and tinnitus caused by be exposure to high sound levels for longer duration of time. Youngsters are visiting music festivals where sound levels are above the safe threshold for avoiding hearing loss. But also professionals are frequently working in environments where they are exposed to loud noises for longer durations of time. People working in factories with loud machines, construction builders, and even truck drivers driving in their cabins for long hours are likely to develop problems with hearing. More and more people purchase hearing protection devices like ear plugs that can be placed inside the ear canal and suppress most of the environmental noise. The problem with these ear plugs is that it isolates the user from its environment, thus preventing him to communicate efficiently. As a result, people take out their ear plugs when they need to communicate, thus jeopardizing their hearing capabilities.
A microprocessor 270 may control the radio signals, applying audio processing (for example voice processing such as echo cancellation or noise suppression) on the signals exchanged with radio transceiver 250, or may control other devices and/or signal paths within the headset 12. Microprocessor 270 may be a separate circuit, or may be integrated into another component present in the headset, for example radio transceiver 250. Microprocessor 270 may include a dedicated Digital Signal Processor (DSP), for example one based on a Tensilica processor.
Audio codec 260, connected to the microprocessor 270, includes a Digital-to-Analog (D/A) converter, the output of which may connect to a speaker 210. One or more air microphones 220 may be added to pick up airborne acoustic waves, for example emanating from the voice of the headset user. To support different audio functions like Enhanced Voice Pickup, Noise Suppression (NS) and Active Noise Cancellation (ANC), more than one air microphone 220 may be embedded in headset 12.
For example, two or more air microphones 220a and 220b may be located at the outside of the ear piece to form an array allowing beam-forming (BF) for enhanced voice pickup. A third air microphone 220c may be located inside the ear piece in front of the speaker to allow audio feedback in an ANC application. Audio codec 260 may include Analog-to-Digital (A/D) converters that receive analog input signals from air microphones 220a, 220b, and 220c and convert them into digital audio signals. Codec 260 collects the picked-up sounds and provides one or more digital microphone audio signal(s) to the microprocessor 270. Alternatively, digital air microphones may be used, which do not require A/D conversion and may provide digital audio signals directly to the audio codec 260 or to the microprocessor 270.
Tubes (not shown) in the housing of headset 12 may serve as air channels to feed the outside airborne acoustic waves towards the air microphones 220.
One or more vibration sensors may be connected to the microprocessor 270. The vibration sensor picks up the sounds and provides a digital vibration sensor audio signal to the microprocessor 270. The vibration sensor is not exposed to the outside air, but is located in the housing of headset 12 for optimal pickup of the headset user's voice arriving at headset 12 through bone conduction and/or through skin.
A digital interface may be provided between vibration sensor 230 and microprocessor 270. If the interface is analog, the output of the vibration sensor 230 may need to be connected to the codec 260 for A/D conversion.
Power Management Unit (PMU) 240 may provide a stable voltage and current supplied to all electronic circuitry. The headset 12 may be powered by a battery 290 which typically provides a 3.7V voltage and may be of the coin cell type. The battery 290 can be a primary battery but is preferably a rechargeable battery. Recharging circuitry may be included in the PMU 240.
Many other components, like sensors, may be added to headset 12 but are not shown since they do not affect the invention presented.
Airwaves 32 emanating from the sawing machine 30 excite both the air microphone 220 and the vibration sensor 230. Airwaves 340 emanating from the user's mouth excite both the air microphone 220 and the vibration sensor 230. Acoustic waves 320 generated by the user's vocal cords traversing human bone and skin mainly excite the vibration sensor 230. It will be clear that the air microphone 220 and vibration sensor 230 may pick up acoustic signals emanated by the user's vocal cords and emanated by the interfering machine 30. Airborne acoustic waves may impinge on the headset housing, thus impacting the vibration sensor 230. Vibration sensor 230 thus may not only pickup the user's voice through bone conduction. As a result, in loud environments where the sound waves 32 reach high levels, the voice pickup by either the air microphone 220 and/or the vibration sensor 230 may be challenged.
In a very loud environment, the Signal-to-Noise (SNR) ratio—where the intended signal is the voice signal and the noise is for example the sound 32 from the sawing machine 30—is very low on the air microphone 220, i.e. the noise is dominant in the air microphone signal. In the vibration sensor output audio signal, the SNR may be low as well, but probably not as low as in the air microphone signal.
A circuit arrangement can be built that allows to subtract the noise signal from the vibration sensor output. A microprocessor 270 can be arranged to subtract the signals or a dedicated subtractor 460 can be provided. Subtracting the air microphone audio signal from the vibration sensor audio signal will further improve the SNR in the signal from the vibration sensor 230. Such an arrangement 400 is shown in
The output of the subtractor 460 is also provided to the radio transceiver 250 which will carry the signal wirelessly to the host device 19 over the radio link 14.
The voice acoustic signals arrive at the vibration sensor 230 both via bone conduction 320 and via the air 340. The air waves 340 carrying the voice also arrive at the air microphone 220. Air waves 32 carrying the loud machine noise arrive both at the vibration sensor 230 and the air microphone 220.
Possibly, a codec(s) 260 is (are) present (not shown in
In the microprocessor 270 an adaptive filter arrangement 450 can be made. In embodiments a dedicated adaptive filter 450 can be present to create a suitable signal that can be subtracted from the vibration sensor audio signal in a subtractor 460.
In embodiments, the output of the subtractor, the output audio signal 470, is used to control the coefficients of adaptive filter 450, thus changing the filter's transfer function.
The adaptive filter 450 can be a Finite Impulse Response (FIR) filter whose filter coefficients are calculated based on the subtractor (460) output using a Least Mean Square (LMS) algorithm as described in the article “Adaptive Noise Cancelling: Principles and Applications,” by B. Widrow et al, published in Proceedings of the IEEE, Vol. 63, No. 12, December 1975. To allow for variations in noise levels, a Normalized Least Mean Square (NLMS) algorithm can be applied. These type of adaptive filters are common practice in echo cancellers applied in all kinds of audio communication products (including the headset 12 as shown in
The arrangement 400 shown in
An arrangement 500 that can operate both in low SNR and high SNR conditions is shown in
In the multipliers 582, 584, and 586, the received signals are multiplied with a certain weight WA, WB, and WC, respectively. Multiplier outputs are subsequently added together in adder 560. Adder 560 and multiplier 582, 584, 586 operate as a linear combiner.
The weight levels are determined in control circuitry 590 which uses as input the audio signals on 512, 532, and 552 from the vibration sensor 230, the subtractor 460, and the air microphone 220, respectively. In embodiments the subtracted output audio signal 532 is combined with at least one of the audio signals of the vibration sensor 230 or the air microphone 220. Under low SNR conditions on the air microphone 220, most weight is placed on the subtractor output, i.e. the signal on 532 (large WB, small WA and/or small WC). Under high SNR conditions, little or no weight is placed on the subtractor output 532. In that circumstance, most weight is placed on the vibration sensor output 512 and/or air microphone output 552. Combining the output of the vibration sensor 230 and the air microphone 220 may be beneficial to restore some of the voice high-frequency content not present in the vibration sensor (due to the low-pass filtering caused by the human bone and tissue).
An example of the variation in the weights W as the SNR varies is shown in
When the SNR on the MIC 220 is high, the adaptive filter 450 may adapt to the headset user's voice rather than to the environmental noise. As a result, a voice signal component may be subtracted from the voice, which is experienced as a pinched-off voice. To prevent this from happening, the adaptive filter 450 may only adapt when the SNR level is low and the acoustic signal on the MIC 220 is predominantly environmental noise. If the SNR level rises above a level where the voice, not the noise, becomes dominant, the adaptive filter 450 may stop updating its filter coefficients. Instead, it may freeze the coefficients and use them also when the SNR level further rises.
An arrangement 700 that controls the updates of the filter coefficients is shown in
where c is a convergence parameter and ε is the error signal, which may be the output 470 of the subtractor 460. The convergence parameter u determines how quickly the filter coefficients will be updated. If μ is too small, the filter 450 will only adapt very slowly; when μ is too large, the system may become unstable and may never converge to the proper filter coefficients. By opening the switch, one may set μ to zero, in which case C(j+1)=C(j) and the filter coefficients are frozen to their latest value. However, one could also make a more gradual control, where the convergence parameter μ is inversely proportional to the SNR level at MIC 220.
Alternatively, one may use the energy Ex(j) in X(j) to control the convergence parameter u. In case of the Normalized LMS Widrow-Hoff algorithm, the update is normalized by the energy in latest M audio samples from the air microphone 220, changing equation 1 into:
Ex(j) is a representation of the signal strength of the air microphone 220 over the last M audio samples. x(k) is the signal strength at time k. If Ex(j) is large, there is a lot of environmental noise and the coefficients C(j) may be updated; if Ex(j) is small, there is little environmental noise and the coefficients C(j) may not be updated. By setting a threshold TH on Ex(j) above which the coefficients are updated, a proper control under different SNR conditions may be achieved. In the simplest form the condition can be defined as:
Compared to the noise suppression arrangement 500 in
When the headset is taken off (and/or turned off), the filter coefficients C should be stored in non-volatile memory. When the headset is turned on and/or placed on the ear of the user, the adaptive filter 450 can use the stored coefficients C as initial setting. This will speed up the convergence when the user directly enters an area with loud environmental noise.
Preferably, the MIC 220 should pick up as little voice from the voice airwaves 340 as possible. If the MIC 220 only mostly picks up the environmental noise airwaves 32, the adaptive filter in the arrangement 400 will be able to adapt to the noise only and subtract it from the signal picked up in the vibration sensor 230 irrespective of the SNR in the vibration sensor. One way of reducing the voice pickup by the MIC 220 is by applying beam forming. Beam forming in headsets is usually provided to improve voice pickup by applying two air microphones 220a and 220b to form an (end-fire) array, see
For the noise suppression arrangements 500 or 700, we may use the opposite concept and create a null in the direction of the mouth. This can simply be achieved with the same air microphone array, now by delaying the MIC 220a closest to the mouth using delay in 814. In this inverse-beam-forming (IBF) configuration, sounds from the mouth are suppressed and a logarithmic gain response depending on the direction angle 854 as shown in
In
For the detection of voice and/or noise levels and setting the weight levels in control unit 590 and making switching decisions in control units 790 and 990, several detection methods can be used with analysis in the time and/or in the frequency domain. Control units 590, 790, and 990 use the MIC 220 signal(s) and/or the vibration sensor 230 signal as input to make decisions on combining weights, switch settings, and/or coefficient updates. The simplest way is to consider power levels of the incoming signals. Peak detection and/or root-mean-square methods can be used in the time domain. Alternatively, or in addition, the signals can be mapped into the frequency domain to apply spectral analysis. For example, voice detection can be applied by analyzing the spectral characteristics of the signals and look for voiced components. By spectral analysis, one can also find out whether noise levels are high because of wind noise. Wind noise may not affect the vibration sensor, and should therefore not be subtracted. In that case, the control unit should give maximal weight to the vibration sensor output (WA=1). More complicated circuits may therefore combine the weighted combining shown in
A slightly different use scenario 1000 involving two users is shown in
An arrangement 1100 integrated in the headset 1012 of the second user that can operate both in low SNR and high SNR conditions is shown in
Various operations in the digital domain have been described like adders, subtractors, filters, delays, and so on. Several other audio operations may be added to the embodiments shown in this invention in order to improve the voice pick-up function. For example echo cancellation, active noise cancellation, and other audio enhancement functions may be added and improve the quality of the audio signal once the loud environment noise has been suppressed by the arrangements discussed. Further features may be added to suppress the effect of wind.
All these operations can be carried out in different places in the wireless headset configuration 200 shown in
Embodiments of the current invention present numerous advantages over the prior art. When a talker using a headset is present in a noisy environment, combining the sound pickup by an air microphone with a vibration sensor, and using an adaptive filter to suppress environmental noise, the speech quality experienced by a far-end listener is greatly improved. When environmental noise diminishes, the system gradually switches over to voice pickup by a vibration sensor and/or air microphone directly without using the suppression method provided by the adaptive filter. Alternatively, when environmental noise diminishes, the adaptive filter coefficients in filter 450 are not updated anymore, while subtraction of noise still happens in subtractor 460.
The present invention may, of course, be carried out in other ways than those specifically set forth herein without departing from essential characteristics of the invention. The present embodiments are to be considered in all respects as illustrative and not restrictive, and all changes coming within the meaning and equivalency range of the appended claims are intended to be embraced therein.
This application claims the benefit of provisional patent application No. 63/452,765, filed Mar. 17, 2023, which is hereby incorporated by reference in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| 63452765 | Mar 2023 | US |
