Patent Application
20200020313
An aspect of the disclosure here relates to audio processing for headphones. Other aspects are also described.
Headphones, as a single headphone for one ear, or a set of two with one headphone for each ear, are in popular use for listening to music, speech during a mobile phone call, or other audio. When using a headphone of any type, whether in the ear, over the ear or around the ear, the user is acoustically cut off from the surrounding environment. The user experiences a loss of high-frequency sound components due to passive attenuation of the ear cup or earbud.
Various versions of an audio processing system having headphones are presented herein. In one aspect of the disclosure here, an audio processor is configured for a transparency effect, and for occlusion effect mitigation. Some versions also have reduced sensitivity to wind noise.
The headphone has a driver (one or more earpiece acoustic transducers or speakers), an external microphone and an internal microphone. The driver and the internal microphone are located within a headphone housing so as to face (or be on a straight path to) an aural canal of the ear against which the headphone housing is fitted. The headphone also has an accelerometer within the headphone housing to receive vibration through bone conduction.
The audio processor (which may be a digital audio processor integrated within the headphone housing) is to analyze signals from the internal microphone, the external microphone and the accelerometer to detect wind noise. The audio processing has a first filter that is to reduce gain of lower frequencies relative to higher frequencies of the signal from the external microphone, in a feedforward path. The gain of the lower frequencies is reduced relative to the higher frequencies, responsive to detecting increased wind noise.
The audio processing is to also adjust a second filter that filters the signal from the accelerometer. The second filter is in a feedback path. The second filter may be adjusted to compensate for the reduced gain of the lower frequencies relative to the higher frequencies in the first filter. The adjusting of the second filter may mitigate the occlusion effect (that is caused by positioning of the headphone relative to the aural canal.)
Outputs of the feedforward path and the feedback path are combined to produce an input signal for the driver. These outputs are combined so that the driver produces sound in the aural canal that not only has transparency, or contains the sound of the surrounding environment which is external to the headphone, but also with reduced wind noise (i.e., reduced relative to the wind noise that is in the sound external to the headphone as might be captured for example by the external microphone.)
Another aspect of the disclosure here is a method of audio processing for transparency with occlusion effect mitigation for headphones. The method includes analyzing signals from an internal microphone, an external microphone and an accelerometer of a headphone, to detect wind noise. The method includes reducing gain of a first filter in lower frequencies relative to higher frequencies, where the first filter is to filter the signal from the external microphone (and not the signal from the accelerometer) in a so-called feedforward path. The gain reduction is responsive to detecting increased wind noise.
The method also includes adjusting a second filter that is in a so-called feedback path, in which the second filter is to filter the signal from the accelerometer (and not the signal from the external microphone.) Adjusting the second filter is also based on detecting the increased wind noise. The adjusting of the second filter may be designed to compensate for the reduced gain of the lower frequencies relative to the higher frequencies in the first filter. The adjusting the second filter may mitigate the occlusion effect (that is caused by positioning of the headphone relative to an aural canal.)
The method includes combining output of the feedforward path and output of the feedback path to produce a signal for the driver. As a result the driver produces sound in the aural canal that has transparency with reduced wind noise, relative to sound external to the headphone.
The above summary does not include an exhaustive list of all aspects of the present invention. It is contemplated that the invention includes all systems and methods that can be practiced from all suitable combinations of the various aspects summarized above, as well as those disclosed in the Detailed Description below and particularly pointed out in the claims filed with the application. Such combinations have particular advantages not specifically recited in the above summary.
Several aspects of the disclosure here are illustrated by way of example and not by way of limitation in the figures of the accompanying drawings in which like references indicate similar elements. It should be noted that references to “an” or “one” aspect in this disclosure are not necessarily to the same aspect, and they mean at least one. Also, in the interest of conciseness and reducing the total number of figures, a given figure may be used to illustrate the features of more than one aspect of the disclosure, and not all elements in the figure may be required for a given aspect.
Several aspects of the disclosure with reference to the appended drawings are now explained. Whenever the shapes, relative positions and other aspects of the parts described are not explicitly defined, the scope of the invention is not limited only to the parts shown, which are meant merely for the purpose of illustration. Also, while numerous details are set forth, it is understood that some aspects of the disclosure may be practiced without these details. In other instances, well-known circuits, structures, and techniques have not been shown in detail so as not to obscure the understanding of this description.
To aid the Patent Office and any readers of any patent issued on this application in interpreting the claims appended hereto, applicant wishes to note that they do not intend any of the appended claims or claim elements to invoke 35 U.S.C. 112(f) unless the words “means for” or “step for” are explicitly used in the particular claim.
Since headphones muffle external sound, some headphones are equipped with a transparency feature that uses an external microphone and amplification to bring external sounds into the aural canal, so that the wearer can hear and be aware of surroundings. However, there is an occlusion effect with headphones, where sounds such as a headphone wearer's speech (i.e., voice), breath, heartbeat and footfalls are delivered by bone conduction to the aural canal, and are perceived as prominent or over-emphasized, thus modifying the headphone wearer's experience. Speech, for example, may be perceived as booming, with lower frequencies emphasized due to the occlusion effect.
Headphones with a transparency feature also suffer from wind noise picked up by the external microphone, for example during windy conditions, walking in the street, bicycling, etc. The wind noise is picked up by the external microphone, which is directly exposed to wind, and is amplified by the transparency feature. Headphones with a transparency feature and an occlusion effect mitigation feature (that uses an internal microphone) may make the amplified wind noise even worse in the aural canal, especially in low frequencies, e.g., 80 Hz-600 Hz. Indeed, to achieve transparent external sounds, the transparency feature has to compensate for the voice occlusion cancellation from the feedback ANC filter, by amplifying the frequencies between 80 Hz and 600 Hz. In other words, the amplified wind noise in the low frequencies is a consequence of the occlusion effect mitigation, where the feedback ANC (with internal microphone) cancels too much of the external sounds. In various examples described herein, the occlusion effect is better suppressed via an accelerometer, and external sounds are faithfully reproduced at the eardrum, without amplifying the low frequencies of the wind noise. Some versions of the headphones use active control of digital audio filters as wind conditions change.
Ideally, when transparency processing is applied during obstruction, which plays back through a speaker in the earbud 302 a processed version of the external sound as it is picked up by an external microphone, the external sound is subjected to the transparency gain 310 which may be tuned to be flat across all frequencies, so that sound pressure as a function of time in the aural canal 104 is approximately equal to what the sound pressure as a function of time would be in the aural canal 104 without the obstruction from the earbud 302. Both types of audio processing for transparency, namely the accelerometer-absent version and the examples given below of the proposed transparency method with occlusion effect mitigation that uses an accelerometer, when properly tuned, can produce an effect that is close to the ideal transparency gain 310, in the absence of bone conduction and absence of wind noise.
In one version, the feedforward filter 702 is implemented with a high pass filter. This could also include amplification (of the high frequency components in the passband of the high pass filter.) With the combination of the high pass filter and the amplification, the feedforward filter 702 in that version would not amplify the low frequencies, but would amplify the higher frequencies, in the signal from the external microphone 602, to compensate for passive attenuation of higher frequencies by the headphone obstruction of the aural canal 104, and thus would deemphasize wind noise passed to the aural canal 104. The accelerometer 706 and filter 704 are tuned to offset or mitigate the occlusion effect, so that this version of the system shown in
In one scenario, a wind detector 908, e.g, as part of or whose function is performed by the filter coefficient controller 906, analyzes signals from the external microphone 602, the internal microphone and the accelerometer 706, and detects wind noise, and changes in wind noise. For example, the wind detector 908 could perform a fast Fourier transform or other spectral analysis of the signals and look for a spectral signature of wind noise. Or, the wind detector 908 could determine that the internal microphone 610 signal resembles a passively high-pass filtered version of the external microphone signal (e.g., contains substantial low-frequency sound), but also determine that the sound differs from the low-frequency vibration picked up by the accelerometer which is more likely speech, breath, heartbeat or footfalls. Based on that, the filter coefficient controller 906 could deduce that the low-frequency sound being picked up by the external microphone is likely wind. Other forms of signature matching, difference analysis, frequency and amplitude analysis, etc., may be developed and used in the filter coefficient controller 906, in keeping with the teachings herein.
When the wind detector 908 detects presence of wind noise or increased wind noise, the filter coefficient controller 906 reduces the gain of the lower frequencies in the feedforward filter 902 relative to the higher frequencies. Conversely, when the wind detector 908 detects absence of wind noise, or decreased wind noise, the filter coefficient controller 906 increases the gain of the lower frequencies in the feedforward filter 902 relative to the higher frequencies. This could be done with, for example, a stepwise gain function, or linear or nonlinear adjustment of gain relative to amplitude of wind noise. The filtering could be implemented with a variable, adjustable high pass filter, a shelf filter, multiple selectable filters, or various other filters. This could be accompanied by amplification in the feedforward path, to compensate for passive attenuation of higher frequencies by the headphone obstruction of the aural canal 104.
Meanwhile, the filter coefficient controller 906 also adjusts the feedback filter 904 based on detecting wind noise, absence of wind noise, increase in wind noise or decrease wind noise, etc., to compensate for the change in gain of the lower frequencies relative to the higher frequencies in the feedforward filter 902. The feedback filter 904 is also adjusted to compensate for the occlusion effect (that is also caused by the headphone obstruction the aural canal.) More specifically, the feedback path produces a correction signal to reduce booming of the wearer's voice that is produced in the aural canal through the bone conduction 404, and otherwise perform occlusion effect mitigation.
Outputs of the feedforward path and the feedback path are combined, for example in the summer 606, to produce a signal for the driver 608. With the filters 902, 904 tuned by the filter coefficient controller 906, the driver 608 produces sound in the aural canal 104 that has transparency (external sound is reproduced) with reduced wind noise, e.g., relative to the wind noise that is picked up by the external microphone 602, and also has occlusion effect mitigation. In some versions, the audio processing combines the signal from the internal microphone and the signal from the accelerometer, for a single input to the feedback filter 904, in the feedback path, as shown by the dashed lines in
The driver 608 and internal microphone 610 may be positioned to face the entrance of the aural canal, or insert into the aural canal 104, and the accelerometer is positioned to receive vibration through bone conduction, e.g., by being physically coupled to the wearer's ear or cheek. The external microphone may be positioned to directly receive sound external to the headphone and the aural canal 104. In one version, the headphone is a single unit, for use with a single ear of a listener. In another version, headphones for a listener have one headphone for one ear and another headphone for the other ear, and each headphone may have its separate copy of the audio processing and other components described above. This may be desirable when the headphones are a pair of wireless earbuds where each can be used by itself without the other. In other instances, such as in a pair of bridged headphones, some of the hardware described above may be shared by both headphones, e.g., the accelerometer is positioned in only the left headphone.
In yet another version, a headset has one headphone for one ear, another headphone for the other ear, and a further microphone that is outside of the headphone housings and positioned in front of the mouth, e.g. on a boom, on another rigid structure that is coupled to the headset, or on a cable that tether the headset to for example a portable device such as a smartphone or a tablet computer. The further microphone in that case could be used to acoustically pick up the wearer's voice and ambient sounds, either by itself or as part of a pickup beamformer.
The filters and other digital audio processing described above can be implemented with one or more processors (generically referred to here as “a processor”), for example a digital signal processor that is executing the appropriate software (instructions) that is stored in memory. The processor and memory may be entirely within a headphone housing, or the operations may be “distributed” as two or more processor-memory combinations, e.g., one processor-memory is housed within the headphone housing and another is housed within for example a smartphone or a tablet computer that may be carried by the wearer and that is in wireless or wired communication with the processor-memory that in the headphone
Some versions of the audio processing systems described above with reference to
While certain aspects have been described and shown in the accompanying drawings, it is to be understood that such are merely illustrative of and not restrictive on the broad invention, and that the invention is not limited to the specific constructions and arrangements shown and described, since various other modifications may occur to those of ordinary skill in the art. For example, while
