This disclosure relates generally to audio recording by portable electronic devices with built-in microphones.
A port of a built-in microphone of a portable electronic device has a fixed location on the device housing. Generally, the built-in microphone port is visually unobtrusive and a user may inadvertently interfere with an audio recording by placing a hand or finger over the microphone port, rubbing or tapping on the microphone port, subjecting the microphone to unintended wind noise, or subjecting the microphone to too much background noise. More than one microphone port on the device housing increases the chances of a user unintentionally creating microphone interference.
Sometimes these types of microphone interference might be remedied easily by the user. Unfortunately, the user may be unaware of the interference until the user plays back the recorded audio. At playback time, however, it is too late to remedy the microphone interference.
Thus, there is an opportunity to reduce unintentionally-created microphone interference during audio recording. The various aspects, features and advantages of the disclosure will become more fully apparent to those having ordinary skill in the art upon careful consideration of the following Drawings and accompanying Detailed Description.
A method and apparatus for detecting microphone interference includes a first built-in microphone producing a first microphone signal and a second built-in microphone producing a second microphone signal. A first filter bank creates a first high-frequency-band signal and a first low-frequency-band signal from the first microphone signal. A second filter bank creates a second high-frequency-band signal and second low-frequency-band signal from the second microphone signal. A first measurement calculator determines a high-frequency-band energy value from the first high-frequency-band signal and the second high-frequency-band signal when the first high-frequency-band signal's magnitude exceeds a predetermined first threshold and the second high-frequency-band signal's magnitude exceeds a predetermined second threshold. A second measurement calculator calculates a low-frequency-band energy value from the first low-frequency-band signal and the second low-frequency-band signal when the first low-frequency-band signal's magnitude exceeds a predetermined third threshold and the second low-frequency-band signal's magnitude exceeds a predetermined fourth threshold. A logic control block, coupled to the first measurement calculator and the second measurement calculator, detects microphone interference and produces an output signal indicating microphone occlusion or wind noise.
Optionally, a first saturation counter can determine a first saturation count signal from the first microphone signal and a second saturation counter can determine a second saturation count signal from the second microphone signal. The logic control block, when coupled to the first saturation counter and the second saturation counter, can also detect microphone interference in the form of mechanical microphone interference or microphone overload.
The output of the logic control block can be used to try to mitigate the microphone interference. In the examples described below, the output of the logic control block is coupled to a user interface to suggest, to the user of the apparatus, ways to mitigate the interference. In other embodiments, the output of the logic control block could be sent to one or more signal processors to try to mitigate the interference without the user being aware of the interference.
microphones 111, 115 and displaying a notice 191 regarding possible microphone interference.
In this example, the “front” side is determined by a camera 120. Thus, a “front” microphone 111 faces the same direction as the camera 120. This particular designation for “front” is merely a matter of expedience to enable a user to quickly distinguish between the two built-in microphones in this particular example. As a matter of nomenclature, though, either microphone could be considered a “first” microphone with the other microphone being designated a “second” microphone. As shown here, an electronic display 130 is positioned on the electronic device 100 opposite the camera 120. Note, however, that this is merely a matter of configuration and that the electronic display 130 could be been positioned facing the same direction as the camera 120 (e.g., in a web-cam configuration).
In this example, the two built-in microphones 111,115 are closely-spaced and matched. For example, both microphones 111,115 are omnidirectional condenser microphones having matched frequency responses and facing opposite directions. Note that both microphones could alternately be directional capacitive microphones or other types of microphones. Also, the frequency responses could be electronically corrected to match.
When the microphone interference detection apparatus and method detects potential microphone interference, the electronic device 100 provides an annunciation intended to guide the user to mitigate the detected microphone interference. As shown in
Other types of interference can also be detected. Examples include microphone overload caused by background noise that is too loud for the microphones 111, 115 to handle, and wind noise caused by air pressure and velocity fluctuations near the microphones 111, 115.
Audio signal components from the front microphone that are above the cutoff frequency for the first high-pass filter are provided to a first threshold block 241, audio signal components from the rear microphone that are above the cutoff frequency for the second high-pass filter are provided to a second threshold block 245, audio signal components from the front microphone that are below the cutoff frequency for the first low-pass filter are provided to a third threshold block 243, and audio signal components from the rear microphone that are below the cutoff frequency for the second low-pass filter are provided to a fourth threshold block 247. In this example, the cutoff frequency for both the first and second high-pass filters is about 400 Hz and the cutoff frequency for both the first and second low-pass filters is about 300 Hz. If the high and low band filters 231, 235 were replaced with audio crossovers, the crossover frequency could be between 300-400 Hz.
For each sampling time period, if the signal magnitude for each signal to each threshold block 241, 243, 245, 247 is below a predetermined threshold, then the signal is not passed to the next stage of the microphone interference detection apparatus. By avoiding the further calculations, the apparatus can save signal processing power when the probability of microphone interference is low (and/or the probability of accurate microphone interference detection is low). In this embodiment, the first and second threshold blocks 241, 243 both use equivalent threshold values, and the third and fourth threshold blocks 243, 247 both use equivalent threshold values. Of course, other embodiments may use different threshold values for each threshold block, the same threshold value for all of the threshold blocks, dynamically varying threshold values, and other variants.
For each of the four signals, if the signal amplitude passes the corresponding threshold, the signal energy during a particularlar sampling time period is calculated as: E=Σi=1Nxi2 Thus, the first energy calculator 251 calculates the energy of the upper-frequency-band signal from the front microphone 111 as E1HIGH, the second energy calculator 255 calculates the energy of the upper-frequency-band signal from the rear microphone 115 as E2HIGH, the third energy calculator 253 calculates the energy of the lower-frequency-band signal from the front microphone 111 as E1LOW, and the fourth energy calculator 257 calculates the energy of the lower-frequency-band signal from the rear microphone 115 as E2LOW.
A first measurement calculator 262 calculates the difference of the high-band energies and normalizes the results to a high-frequency-band energy value as follows:
M
HIGH=|(E1HIGH−E2HIGH)/(E1HIGH+E2HIGH)|
A second measurement calculator 266 calculates the difference of the low-band energies and normalizes the results to a low-frequency-band energy value as follows:
M
LOW=|(E1LOW−E2LOW)/(E1LOW+E2LOW)|.
The high and low frequency band energy values can be calculated using alternate methodologies, such as the energy of the difference between the signals (rather than the difference of the energies of the signals). Also, it is not necessary to normalize the high and low frequency band energy values by (E1HIGH+E2HIGH) and (E1LOW+E2LOW), respectively.
After that, a first smoothing block 272 smoothes out the resulting M1 signal using a simple smoothing function: MHIGH(n)=αMHIGH(n)+(1−α)MHIGH(n−1). A second smoothing block 276 does the same thing with the MLOW signal from the second measurement calculator 266. Thus, MLOW(n)=αMLOW(n)30 (1−α)MLOW(n−1). Although the value for α is shown as the same for both smoothing blocks 272, 276, the values for α could be different for MHIGH and MLOW.
The two smoothed signals MHIGH(n) and MLOW(n) are provided to a logic control block 280. Although the generation of the smoothed signals MHIGH(n) and MLOW(n) are shown as occurring outside of the logic control block 280, an alternate implementation could place one or more threshold blocks, energy calculators, measurement calculators, or smoothing blocks within the logic control block.
A first saturation count block 291 from the front microphone's ADC 221 provides two saturation counts S1H, S1L, and a second saturation count block 295 from the rear microphone's ADC 225 provides two more saturation counts S2H, S2L the logic control block 280. Each saturation count signal reflects the number of times that an incoming digital signal crosses a predetermined threshold in a given time period. The S1H and S2H saturation counts reflect the number of times that the incoming first and second microphone signals cross a “high” conversion threshold in a given time period. For example, if the ADC maximum positive output is 1 and maximum negative output is −1, then the S1H and S2H saturation counts reflect the number of times, in a given time period, that the saturation count blocks 291, 295 detect the incoming digital signal equaling (or almost equaling) a 1 or −1. Of course, different threshold values (including variable threshold values) can be used instead of the examples given. The S1L and S2L saturation counts reflect the number of times that the incoming first and second microphone signals cross a “low” conversion threshold (“low” simply being lower than the “high” conversion threshold) in the given time period.
A fifth input 297 to the logic control block 280 is a reset signal. This reset signal triggers a reset of the logic control block 280 and can reflect when the electronic device 100 is audibly alerting the user (e.g., incoming phone call ring tone, various beeps for audible feedback to user interactions, or when the electronic device is providing speech instructions to the user) such that these known audible alerts are ignored.
The output signal 299 of the logic control block 280 is provided to other components (not shown) of the electronic device 100 so that the electronic device can interact with the user to mitigate any detected microphone interference using, for example, the electronic display 130 or a loudspeaker (not shown). Preferably, the output signal 299 exhibits a prioritization of microphone interference causes and a hysteresis setting so that instructions can be provided to the user in an orderly fashion. For example, the types of interference that could be detected can have a priority (which will be shown with reference to
If the Reset signal is not high (e.g., Reset=0), then the logic control block 280 calculates 315 a high saturation difference value MSH=(S1H−S2H)/(S1H+S2H) where the calculation is aborted if (S1H+S2H)=0 to protect the calculation from “division by zero” issues. Decision block 321 determines if the magnitude of the high saturation difference value MSH is greater than a predetermined high saturation count threshold TSH, which can be determined experimentally. Thus, when S1H>>S2H, then MSH tends to be near 1, and when S1H is close in value to S2H, then |MSH| ends to be near 0.
If the high saturation difference value MS has a magnitude that is greater than the high saturation count threshold TSH, then decision block 330 determines if S2L is less than a low saturation count threshold TSL. If S2L<TSL, then the logic control block 280 provides an output signal 299 indicating that mechanical microphone interference has been detected 335 at the front microphone 111. In other words, there is a high saturation count (above a high saturation count threshold) at the front microphone and a low saturation count (below a low saturation count threshold) at the rear microphone. Then the flow returns to the start 301 to obtain the next set of values for MHIGH, MLOW, S1L, S1H, S2L, S2H, and Reset.
If either of decision blocks 321, 330 are “NO”, decision block 323 determines if the magnitude of the high saturation difference value MSH is less than a negative of the predetermined high saturation count threshold (i.e., −TSH). Thus, when S1H<<S2H, then MSH tends to be near −1. If the output of decision block 323 is “YES”, then decision block 325 determines if S1L is less than the low saturation count threshold TSL. If S1L<TSL, then the logic control block 280 provides an output signal 299 indicating that mechanical microphone interference has been detected 327 at the rear microphone 115. Then the flow returns to the start 301 to obtain the next set of values for MHIGH, MLOW, S1L, S1H, S2L, S2H, and Reset.
If the output of decision block 323 is “NO”, then decision blocks 342, 347 check whether either high saturation count signal (e.g., S1H or S2H) is greater than a third saturation count threshold TS3.The third saturation count threshold TS3 can be set equal to one of the previous saturation count thresholds (e.g., TSH or TSL) or may be determined independently though experimentation. If S1H>TS3, as determined by block 342, then the logic control block 280 provides an output signal 299 indicating that microphone overload has been detected 345 at the front microphone. If S2H>TS3 as determined by block 347, then the logic control block 280 provides an output signal 299 indicating that microphone overload has been detected 349 at the rear microphone. If microphone overload interference has been detected at either microphone the flow returns to the start 301 to obtain the next set of values for MHIGH, MLOW, S1L, S1H, S2L, S2H, and Reset
If decision blocks 342, 347 do not determine S1H>TS3 or S2H>TS3, then decision block 352 checks whether |MHIGH|>|MLOW| or MHIGH>THIGH, where THIGH is a high-frequency-band energy threshold that can be determined experimentally. In other words, if the magnitude of the normalized difference between the high-band energy of the front microphone and the high-band energy of the rear microphone is greater than the magnitude of the normalized difference between the low-band energy of the front microphone and the low-band energy of the rear microphone, and the magnitude of the normalized difference between the high-band energy of the front microphone and the high-band energy of the rear microphone is greater than a high- band energy difference threshold, then the output signal 299 indicates that the logic control block 280 has detected 354 that the front microphone is experiencing occlusion. As mentioned previously, different high-frequency-band energy values can be calculated instead of the “normalized difference-of-the-energies” high-frequency-band energy values described in detail in this paragraph. Of course, the value of the corresponding threshold Thigh would change if the high-frequency-band energy values were calculated differently.
If the output of decision block 352 is “NO”, then decision block 356 checks whether |MHIGH>|MLOW| and MHIGH>−THIGH. If the output of decision block 356 is “YES”, then the output signal 299 indicates that the logic control block 280 has detected 358 that the rear microphone is experiencing occlusion. After detection 354,358 of occlusion at either microphone, the flow then returns to the start 301 to obtain the next set of values for MHIGH, MLOW, S1H, S1L, S2H, S2L, and Reset.
If decision block 356 does not result in a detection of microphone occlusion, decision block 360 checks if the |MLOW|>TLOW, where TLOW is a low-frequency-band energy threshold that may be determined experimentally. If |M2|>T2 then the output signal 299 indicates that the logic control block 280 has detected 365 wind noise at the microphones 111, 115. In other words, if the magnitude of the normalized difference between the low-band energy of the front microphone and the low-band energy of the rear microphone is greater than the low-band energy threshold, then the output signal 299 indicates that the logic control block 280 has detected 365 that a microphone is experiencing wind noise. As mentioned previously, different low-frequency-band energy values can be calculated instead of the “normalized difference-of-the-energies” low-frequency-band energy values described in detail in this paragraph. Of course, the value of the corresponding threshold TLOW would change if the high-frequency-band energy values were calculated differently. Although, in this implementation, the wind noise detection has not been separated into wind noise detection on specific microphones, it can easily be done by checking the value MLOW against positive or negative version of the threshold TLOW (as explained with respect to threshold THIGH).The flow then returns to the start 301 to obtain the next set of values for MHIGH, MLOW, S1L, S1H, S2L, S2H, and Reset.
Thus, the input signals MHIGH, MLOW, S1L, S1H, S2L, S2H, and Reset are evaluated on a priority basis to detect different types of possible microphone interference. A Reset signal has the highest priority, mechanical microphone interference has a next priority, microphone overload has a third priority, microphone occlusion has a fourth priority, and wind noise has a fifth priority. These detection decisions are not used directly to compensate for the detected microphone interference, but instead are used to provide a signal to the user interface of the electronic device so that the user can be aware that microphone interference may be occurring (at the time it is occurring). Also, the output signal 299 may exhibit hysteresis so that the types of detected microphone interference can be presented to the user in an orderly fashion (and not confuse or overwhelm the user).
As mentioned previously,
Thus, the microphone interference detection apparatus and method provides a mechanism to alert a user of an electronic device regarding possible audio recording interference. Because, sometimes the user is not aware of the audio interference until later playback of the recorded audio, this microphone interference detection apparatus and method gives amateur audio (and audiovisual) recorders an opportunity to mitigate potential audio interference. In other embodiments, the output of the microphone interference detection apparatus and method could be sent to one or more signal processors to try to mitigate the interference without the user being aware of the interference. The microphone interference detection apparatus and method can be integrated into a recording device and is designed to provide a methodical presentation of detected microphone interference.
While this disclosure includes what are considered presently to be the embodiments and best modes of the invention described in a manner that establishes possession thereof by the inventors and that enables those of ordinary skill in the art to make and use the invention, it will be understood and appreciated that there are many equivalents to the embodiments disclosed herein and that modifications and variations may be made without departing from the scope and spirit of the invention, which are to be limited not by the embodiments but by the appended claims, including any amendments made during the pendency of this application and all equivalents of those claims as issued.
It is further understood that the use of relational terms such as first and second, top and bottom, and the like, if any, are used solely to distinguish one from another entity, item, or action without necessarily requiring or implying any actual such relationship or order between such entities, items or actions. Much of the inventive functionality and many of the inventive principles are best implemented with or in software programs or instructions. It is expected that one of ordinary skill, not withstanding possibly significant effort and many design choices motivated by, for example, available time, current technology, and economic considerations, when guided by the concepts and principles disclosed herein will be readily capable of generating such software instructions and programs with minimal experimentation. Therefore, further discussion of such software, if any, will be limited in the interest of brevity and minimization of any risk of obscuring the principles and concepts according to the present invention.
As understood by those in the art, logic control block 280 includes a processor that executes computer program code to implement the methods described herein. Embodiments include computer program code containing instructions embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other computer-readable storage medium, wherein, when the computer program code is loaded into and executed by a processor, the processor becomes an apparatus for practicing the invention. Embodiments include computer program code, for example, whether stored in a storage medium, loaded into and/or executed by a computer, or transmitted over some transmission medium, such as over electrical wiring or cabling, through fiber optics, or via electromagnetic radiation, wherein, when the computer program code is loaded into and executed by a computer, the computer becomes an apparatus for practicing the invention. When implemented on a general-purpose microprocessor, the computer program code segments configure the microprocessor to create specific logic circuits.
Number | Date | Country | |
---|---|---|---|
Parent | 12822176 | Jun 2010 | US |
Child | 14566281 | US |