Patent Application
20130202132
The present teachings relate to systems for, and methods of, compensating for a varying distance between an electronic loudspeaker in a mobile electronic device and a user's ear(s).
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present teachings and together with the description, serve to explain the principles of the present teachings. In the figures:
Techniques compensate for the effect of a varied distance, and relative movement, between an electronic loudspeaker in a device and a user's ear. In general, as a distance between a loudspeaker and a user's ear increases, the sound pressure of detected audio decreases (correspondingly, as distance decreases, detected sound pressure increases). Certain embodiments compensate for this effect by adjusting a gain of a loudspeaker amplifier in proportion to the distance. Certain embodiments also allow a user to intuitively and efficiently adjust a gain of the loudspeaker in the mobile device by activating a volume set mode. When in the volume set mode, the user may move the mobile device toward or away from his or her head and the gain level will be adjusted in inverse proportion to the distance. The device may be a mobile device or a cellular telephone according to certain embodiments. In some embodiments, the device may be a speakerphone.
According to various embodiments, a method compensates for movement of a loudspeaker relative to a user's head, where the loudspeaker is present in a mobile device. The method includes producing, by the device, an electrical signal representative of audio and determining, by the device, a distance between the device and the user's head. The method also includes automatically setting, by the device, a gain of the electrical signal in accordance with the distance. The method further includes outputting, via the loudspeaker of the device, the electrical signal with the gain.
Reference will now be made in detail to exemplary embodiments of the present teachings, which are illustrated in the accompanying drawings. Where possible the same reference numbers will be used throughout the drawings to refer to the same or like parts.
Processor 104 may be further coupled to display 106 and other user interface 108 elements. Display 106 may be, by way of non-limiting example, a liquid crystal display, which may include a touchscreen. Other user interface 108 elements may be, by way of non-limiting example, a full or partial physical keyboard or keypad. In embodiments where display 106 is a touchscreen, display 106 may be combined with user interface 108 so as to display an active full or partial keyboard or keypad. That is, user interface 108 may include a full or partial virtual keyboard or keypad.
Processor 104 may be further coupled to loudspeaker 114 by way of amplifier 112. Loudspeaker 114 may be, by way of non-limiting example, a loudspeaker of a cellular telephone or audio system. Loudspeaker 114 may be capable of producing sound suitable for a speakerphone mode or a private telephone mode. Amplifier 112 may include a preamplification stage and a power amplification stage. In some embodiments, amplifier 112 may include one or both of a digital-to-analog converter and decoding (e.g., compression, decompression, and/or error correction decoding) circuitry.
Processor 104 may be further coupled to microphone 118 by way of amplifier 116. Microphone 118 may be, by way of non-limiting example, a microphone of a cellular telephone. Microphone 118 may be capable of receiving and converting to electricity sound captured by the cellular telephone. Amplifier 116 may include a preamplification stage. In some embodiments, amplifier 116 may include one or both of an analog-to-digital converter and encoding (e.g., error correction and/or compression encoding) circuitry.
Processor 104 may be further coupled to sensor system 120. Sensor system 120 may be any of several various types. By way of non-limiting example, sensor system 120 may be infrared, acoustic, or photographic. If infrared, sensor system 120 may include an infrared emitter (e.g., a high-power light emitting diode) and an infrared receiver (e.g., an infrared sensitive diode). If acoustic, sensor system 120 may include an ultrasonic transducer or separate ultrasonic emitters and receivers. In some embodiments, microphone 118 may perform ultrasonic reception. If photographic, sensor system 120 may include a camera utilizing, e.g., optics and a charge coupled device. In some embodiments in which sensor system 120 is photographic, one or both of sensor system 120 and processor 104 may employ facial recognition, known to those of skill in the art, capable of determining when a human face is within a depth of field of sensor system 120. Regardless as to the particular technology used by sensor system 120, sensor system 120 may include interpretive circuitry that is capable of converting raw empirical measurements into electrical signals interpretable by processor 104.
Sensor system 120 may further include accelerometer 122, which detects applied linear force (e.g., in one, two or three linearly orthogonal directions). Accelerometer 122 may be, by way of non-limiting example, a micro-electromechanical system (MEMS), capable of determining the magnitude and direction of any acceleration. Sensor system 120 may also include a gyroscope (possibly as, or as part of, accelerometer 122) that detects applied rotational force (e.g., in one, two or three rotationally orthogonal directions). Sensor system 120 may further include a velocity sensor, which detects the velocity of objects relative to a face of the mobile device 102. The velocity sensor may be, by way of non-limiting example, an optical interferometer capable of determining the magnitude and direction of any velocity of the device relative to an object in front of the sensor. The velocity sensor may detect velocity only in a direction normal (i.e., perpendicular) to the face (e.g., display) of the mobile device, or in three orthogonal directions.
Mobile device 204 is capable of detecting a distance 206 between itself and user's head 208. To that end, mobile device 204 includes a sensor system (e.g., sensor system 120 of
For example, if the sensor system is infrared, it may detect an infrared signal sent from mobile device 204 and reflected off of user's head 208. Using techniques known to those of skill in the art, such a reflected signal may be used to determine distance 206. Analogously, if ultrasonic, the sensor system may detect an ultrasonic signal transmitted from mobile device 204 and reflected off of user's head 208. Using techniques known to those of skill in the art, such a reflected signal may be used to determine distance 206. If photographic, the sensor system may use facial recognition logic to determine that user's head 208 is within a depth of field and, using techniques known to those of skill in the art, determine distance 206. Additionally if photographic information is acquired by an autofocus camera, distance 206 can be determined to be the focal distance of the camera's optical system. The autofocus system in this example can focus on the closest object, or on the specific region of the user's head, depending on the autofocus algorithm employed.
Any of the aforementioned techniques (infrared, ultrasonic, photographic) may be used in combination with acceleration data (e.g., detected by accelerometer 122) to calculate additional distances using, by way of non-limiting example, dead reckoning, known to those of skill in the art. For example, if an infrared, ultrasonic, or photographic technique is used to determine an absolute distance at a given time, and a subsequent acceleration in a direction away from the user's head is detected over a particular time interval, then, as known to those of skill in the art, these parameters are sufficient to derive an estimate of the absolute distance at the end (or during) the time interval. Regardless of the specific technology used to determine distance 206, mobile device 204 is capable of such determination.
Sensor systems (e.g., a photographic sensor) can also be used to determine a proportional change in distance by comparing the relative size of features on a user's head (e.g., an eye, an ear, a nose, or a mouth) and determining the proportional change in distance accordingly based on a reference size of the feature. In this way, the proportional change in distance can be used to perform the gain adjustments described herein without having to determine an absolute distance between the mobile device and the user.
Thus, at block 300, a mobile device (e.g., mobile device 102 of
At block 302, the distance between the device and the user's head is determined. As discussed above in reference to
At block 304, the gain level is set in accordance to the distance determined at block 302. In some embodiments, the gain level (e.g., gain of amplifier 112 of
In the above table, note that with each doubling of distance comes an additional 6.02 dB of gain used to compensate for the perceived decrease in volume.
At block 306, the audio is output from the loudspeaker. This may be achieved by feeding the output of a power amplifier directly to the loudspeaker (e.g., loudspeaker 114 of
Flow from block 306 may return back to block 302 so that the gain is repeatedly adjusted. The repetitive adjustment may occur at periodic intervals (e.g., every 0.1 second, 0.5 second, or 1.0 second) as determined using a timer such as timer 124 of
Although an initial setting of 0 dB of gain for a distance of 1 cm is shown in the table above, the user may be more comfortable with another gain setting. Alternatively instead of an increase in gain as the distance is increased, the gain can be implemented as an increase in attenuation as distance is decreased. For example, in the case above, if the gain at 16 cm were to be 0 dB, the gain at 1 cm would then be −24.08 dB, or 24.08 dB of attenuation.
At block 400, the user provides a volume set activation request to a mobile device. The volume set activation request may be the user activating a physical or virtual (e.g., touchscreen) button on the mobile device. Alternately, or in addition, the volume set activation request may be a voice command recognized by the device. The mobile device receives the request and enters a volume adjustment mode, which the user controls as discussed presently. At block 402, the mobile device determines a distance to the user's head using any of the techniques disclosed herein (e.g., infrared, ultrasonic, and/or photographic; with or without dead reckoning).
At block 404, the mobile device adjusts an output gain for the loudspeaker in inverse proportion to the distance. Thus, the farther the mobile device from the user's head, the more the gain level is lowered. Note that the volume adjustment is made relative to the current gain set for the mobile device's loudspeaker. Thus, for example, a user may hold the mobile device 10 cm from the user's head and request activation of the volume set mode according to block 400. If the user brings the mobile device toward the user's head, the mobile device will increase the gain; if the user brings the mobile device away from the user's head, the mobile device will decrease the gain.
The proportionality of change in gain may be linear, quadratic, or another type of proportionality. For example, in some embodiments, each unit distance movement toward or away from the user's head (e.g., 1 cm) may result in an increase or decrease of gain by a fixed amount (e.g., 1 dB). As another example, in some embodiments, each unit distance movement toward or away from the user's head (e.g., 2 cm) may result in an increase or decrease of gain by an amount that is a function (e.g., a quadratic function) of the distance (e.g., 22=4 dB). Exponential proportionalities are also contemplated. For example, each unit distance movement (e.g., ×cm) may result in an increase or decrease of gain as an exponential function of the distance (e.g., 2x dB).
Other embodiments may adjust loudspeaker gain based on a change in relative distance. Thus, for example, some embodiments may use an initial distance from the user's head as a starting point. Each subsequent halving of the distance between the mobile device and the user's head may result in an increase of gain by a fixed amount (e.g., 6.02 dB), and each doubling of distance from the user's head may result in a decrease in gain by a fixed amount (e.g., 6.02 dB).
At block 406, the device outputs audio (e.g., by way of loudspeaker 114). The audio may be the typical audio output of a cellular telephone (e.g., a remote speaker's voice). Alternately, or in addition, internally-generated audio may be output at block 406 during the gain adjustment process. Such internally-generated audio may be a tone, a plurality of tones, a musical chord, or intermittent outputs of any of the preceding (e.g., 0.1 second tones at 0.5 second intervals). A user may utilize the output audio of block 406 to determine a desired level of output, which corresponds to an internal gain setting of the device. That is, the audio may serve as a feedback mechanism such that the user may accurately adjust the output volume.
At block 408, the device checks if it has received a volume set inactivation request from the user. Reception of such a request causes the device to store 410 its gain level at its current state set during the operations of block 404. This stored value becomes the updated “anchor” for an updated output gain table. In some embodiments, the volume set inactivation request may be the user activating a physical or virtual (e.g., touchscreen) button on the mobile device. In some embodiments, this may be the same button activated at block 400. The volume set inactivation request may also be a voice command recognized by the device. If no activation request has been received, the flow returns to step 402 so that the gain can repeatedly be adjusted.
In other embodiments, when the volume adjustment mode is activated, the adaptive gain control discussed in reference to
In some embodiments, the volume set activation request of block 400 is made by activating and holding down a button (whether physical or virtual). In such embodiments, the volume set inactivation request of block 408 may be made by releasing the same button. Thus, in such embodiments, the user employs the technique of
In addition, or in the alternative to the manual and/or automatic adjustment of audio output gain, the audio input gain can also be adjusted as discussed below.
Thus, at block 500, a mobile device (e.g., mobile device 102 of
At block 504, the distance between the device and the user's head is determined. As discussed above in reference to
At block 506, the mobile device sets a gain of an amplifier of the electrical signal. In some embodiments, the gain level (e.g., gain of amplifier 116 of
At block 508, audio filtering is modified to compensate for a so-called noise pumping effect. Specifically, if gain increases according to block 506, the noise within the captured audio also increases. Accordingly, if gain is increased by a certain number of decibels, a noise filter may be set to reduce noise by a corresponding or identical amount. The filter may be, by way of non-limiting example, a finite impulse response (FIR) filter set to filter noise at particular frequencies at which it occurs. Further details of a particular technique according to block 508 are discussed below in reference to
At block 510, an output signal is generated. The output signal may be the result of the gain adjustment of block 506 and the noise reduction of block 508 applied to the electrical signal received at block 502. In some embodiments, the output signal is an analog signal to be stored in the mobile device; in other embodiments, the output signal is transmitted, e.g., to a cellular tower.
Flow from block 510 may return back to block 504 so that the gain may be repeatedly adjusted. The repetitive adjustment may occur at periodic intervals (e.g., every 0.1 second, 0.5 second, or 1.0 second) as determined using a timer such as timer 124 of
At block 602, the frequency bands of the sound in association with block 600 are determined. This may be achieved using, for example, a Fourier transform or by dividing the audio spectrum into sub-bands. The frequency bands determined at block 602 represent the primary bands that contain the most noise. At block 604, audio filtering levels, or sub-band spectral suppression levels, are adjusted to reduce noise in the bands identified in block 602. The amount of reduction (or increase) may correspond with the amount of gain added (or reduced) at block 506 of
Thus, for example, if a particular band identified as containing of mostly noise has a typical suppression value of, for example, 20 dB, and an additional 6 dB of gain is imposed at block 506 of
The technique of
Thus, at block 700, a velocity of the mobile device (e.g., mobile device 102 of
Alternative techniques for determining device velocity can also be used when either distance or acceleration are sampled at a repetitive rate. For example if the distance or acceleration is sampled many times each second at a constant rate, a distance or acceleration time signal can be created. Because the velocity is the derivative of the distance time signal or the integral of the acceleration time signal, the velocity can be calculated in either the time or frequency domain. Suitable techniques include differentiating the distance signal in the time domain or integrating the acceleration signal in the time domain. An alternative technique is to convert the time signal into the frequency domain and either multiply each fast Fourier transform (FFT) bin value of the distance signal by the frequency of each FFT bin or divide each FFT bin value of the acceleration signal by the frequency of each FFT bin.
At block 702, the sound is adjusted to account for any Doppler shift caused by the velocity detected at block 700. In particular, the mobile device may include a look-up table or formula containing correspondences between velocity and pitch shift. After the velocity is determined at block 700, the corresponding pitch shift may be determined by such table or formula. The pitch shift may be adjusted in real-time using resampling technology to pitch shift or frequency scale, as is known in the art.
If direct velocity sensing, acceleration sensing, or proportional distance measurement are utilized, the Doppler shift compensation can be implemented without knowing the absolute distance between the mobile device and the user, just as the gain compensation can be implemented using only a proportional distance measure. In the cases of direct velocity sensing or acceleration sensing, this would not require any distance information to perform the Doppler shift. Thus the Doppler compensation can operate independent from a distance sensing operation.
In another embodiment, the method of compensating for a Doppler effect in
In some embodiments, both the uplink and down link audio can be modified simultaneously to compensate for amplitude modulation as well as Doppler shift in the uplink and down link audio signals.
The foregoing description is illustrative, and variations in configuration and implementation may occur to persons skilled in the art. Other resources described as singular or integrated can in embodiments be plural or distributed, and resources described as multiple or distributed can in embodiments be combined. The scope of the present teachings is accordingly intended to be limited only by the following claims.
The present application is related to co-pending U.S. utility patent application entitled “MOTION BASED COMPENSATION OF UPLINKED AUDIO,” bearing Ser. No. ______ by Robert A. Zurek et al., filed concurrently herewith, and the contents thereof are hereby incorporated by reference herein in its entirety.
