The present invention relates to personalized sound systems, including an ear-level device adapted to be worn on the ear, and the use of such systems to select hearing profiles to be applied using the sound system.
Ear-level devices, including headphones, earphones, head sets, hearing aids and the like, are adapted to be worn at the ear of a user and provide personal sound processing. U.S. patent application Ser. No. 11/569,449, entitled Personal Sound System Including Multi-Mode Ear-level Module with Priority Logic, published as U.S. Patent Application Publication No. US-2007-0255435-A1 is incorporated by reference as if fully set forth herein. In US-2007-0255435-A1, a multi-mode ear-level device is described in which configuration of the ear-level device and call processing functions for a companion mobile phone are described in detail.
It is widely understood that hearing levels vary widely among individuals, and it is also known that signal processing techniques can condition audio content to fit an individual's hearing response. Individual hearing ability varies across a number of variables, including thresholds of hearing, or hearing sensitivity (differences in hearing based on the pitch, or frequency, of the sound), dynamic response (differences in hearing based on the loudness of the sound, or relative loudness of closely paired sounds), and psychoacoustical factors such as the nature of and context of the sound. Actual injury or impairment, physical or mental, can also affect hearing in a number of ways. A widely used gauge of hearing ability is a profile showing relative hearing sensitivity as a function of frequency.
The most widespread employment of individual hearing profiles is in the hearing aid field, where some degree of hearing impairment makes intervention a necessity. This entails detailed testing in an audiologist or otologist office, employing sophisticated equipment and highly trained technicians. The result is an individually-tailored hearing aid, utilizing multiband compression to deliver audio content exactly matched to the user's hearing response. However, this process is typically expensive, time-consuming and cumbersome, and it plainly is not suitable for mass personalization efforts.
The rise of the Internet has offered the possibility for the development of personalization techniques that flow from on-line testing. Efforts in that direction have sought to generate user hearing profiles by presenting the user with a questionnaire, often running to 20 questions or more, and using the user input to build a hearing profile. Such tests have encountered problems in two areas, however. First, user input to such questionnaires has proved unreliable. Asked about their age alone, without asking for personal information, for example, users tend to be less than completely truthful. To the extent such tests can be psychologically constructed to filter out such bias, the test becomes complex and cumbersome, so that users simply do not finish the test.
Another testing regime is set out in U.S. Pat. No. 6,840,908, entitled System and Method for Remotely Administered, Interactive Hearing Tests, issued to Edwards and others on 11 Jan. 2005, and owned by the assignee of the present application. That patent presents a number of techniques for such testing, most particularly a technique called N-Alternative Forced Choice, in which a user is offered a number of audio choices among which to select one that sounds best to her. Also known as sound flavors, based on the notion of presenting sound and asking the user which one is preferred; this method can lack sufficient detail to enable the analyst to build a profile.
Although different forms of test procedures for generating a personalized hearing profile have been employed by the art, none has been deployed in a way to produce accurate results for a large number of consumers.
A personalized hearing profile is generated for an ear-level device comprising a memory, a microphone and a speaker, each coupled to a processor. Communication is established between the ear-level device and a companion device having a user interface. A frame of reference in the user interface is provided, where positions in the frame of reference are associated with sound profile data. A position on the frame of reference is determined in response to user interaction with the user interface, and certain sound profile data associated with the position. A chosen one of the following is transmitted to the ear level device: (a) certain sound profile data, whereby the ear level device is capable of generating sound through the speaker based upon the certain sound profile data to provide real-time feedback to the user, or (b) audio stream data generated using (1) an audio stream generated by the companion device, and (2) the certain sound profile data. The ear level device is thereby capable of generating sound through the speaker based upon the audio stream data to provide real-time feedback to the user. The determining and transmitting steps are repeated until detection of an end event.
In some examples the communication establishing step is carried out with a chosen one of a mobile phone, digital music player or computer as the companion device. In some examples the certain sound profile data is transmitted to the ear level device; and an audio stream is provided for the ear level device, which the ear level device can play on the speaker during execution of the sound profile program. In some examples the rendering step is carried out with the sound profile data comprising frequency band amplitude adjustment data and dynamic range adjustment data. In some examples the sound profile data includes a plurality of preset profiles associated with respective positions on the frame of reference, each preset profile comprising dynamic range compression data and frequency shaping data.
In some examples the user interface includes a graphical user interface executed using a display associated with the user interface, and a visual indicator is displayed on the display resulting from the user interaction with the graphical user interface, the visual indicator corresponding to a position on the frame of reference for the sound profile data. In some examples, with the exception of the visual indicator, the display is maintained free of visual indicia correlating location on the frame of reference to the sound profile data.
Other aspects and advantages of the present invention can be seen on review of the drawings, the detailed description, and the claims which follow.
The ear module 10 is adapted to operate in a plurality of modes, corresponding to modes of operating the ear module, such as a Bluetooth® mode earpiece for the phone 11, and the environmental mode. The ear module and the companion devices can execute a number of functions in support of utilization of the ear module in the network.
The ear module 10 includes a voice menu mode in which data indicating a function to be carried out by the ear module or by a companion device, such as a mobile phone 11, is selected in response to user input on the ear module 10. The user input can be for example the pressing of a button on the ear module 10.
In one embodiment described herein, the wireless audio links 14, 15 between the ear module 10 and the linked companion microphone 12, between the ear module 10 and the companion mobile phone 11 respectively, are implemented according to Bluetooth® compliant synchronous connection-oriented SCO channel protocol (See, for example, Specification of the Bluetooth System, Version 4.0, 17 Dec. 2009). Wireless link 16 couples the mobile phone 11 to a network service provider for the mobile phone service. The wireless configuration links 17, 18, 19 between the companion computer 13 and the ear module 10, the mobile phone 11, and the linked companion microphone 12, and optionally the other audio sources are implemented using a control channel, such as a modified version of the Bluetooth® compliant serial port profile SPP protocol or a combination of the control channel and SCO channels. (See, for example, BLUETOOTH SPECIFICATION, SERIAL PORT PROFILE, Version 1.1, Part K:5, 22 Feb. 2001).
Of course, a wide variety of other wireless communication technologies may be applied in alternative embodiments. The mobile phone 11, or other computing platform such as computer 13, preferably has a graphical user interface and includes for example a display and a program that displays a user interface on the display such that the user can select functions of the mobile phone 11 such as call setup and other telephone tasks, which can then be selectively carried out via user input on the ear module 10, as described in more detail below. Alternatively, the user can select the functions of the mobile phone 11 via a keyboard or touch pad suitable for the entry of such information. The mobile phone 11 provides mobile phone functions including call setup, call answering and other basic telephone call management tasks in communication with a service provider on a wireless telephone network or other network. In addition, and as discussed below, mobile phone 11, or other computing platform such as computer 13, can be used to allow the user to generate a personalized hearing profile for ear module 10.
The companion microphone 12 consists of small components, such as a battery operated module designed to be worn on a lapel, that house “thin” data processing platforms, and therefore do not have the rich user interface needed to support configuration of private network communications to pair with the ear module 10. For example, thin platforms in this context do not include a keyboard or touch pad practically suitable for the entry of personal identification numbers or other authentication factors, network addresses, and so on. Thus, to establish a private connection pairing with the ear module, the radio is utilized in place of the user interface.
The nonvolatile memory 54 stores audio data associated with various functions that can be carried out by the companion mobile phone. The nonvolatile memory 54 also stores computer programs and configuration data for controlling the ear module 10. These include providing a control program, a configuration file and audio data for the personalized hearing profiles, also called sound profiles. The programs are executed by the digital signal processor 52 in response to user input on the ear module 10. In addition, the nonvolatile memory 54 stores a data structure for a set of variables used by the computer programs for audio processing, where each mode of operation of the ear module may have one or more separate subsets of the set of variables, referred to as “presets” herein. In addition, memory 54 can store one or more individually generated sound profiles, as discussed below; further, one or more test sounds can be stored in memory 54 for use in creating the individually generated sound profiles.
The radio module 51 is coupled to the digital signal processor 52 by a data/audio bus 70 and a control bus 71. The radio module 51 includes, in this example, a Bluetooth® radio/baseband/control processor 72. The processor 72 is coupled to an antenna 74 and to nonvolatile memory 76. The nonvolatile memory 76 stores computer programs for operating the radio module 51 and control parameters as known in the art. The nonvolatile memory 76 is adapted to store parameters for establishing radio communication links with companion devices. The processing module 50 also controls the man-machine interface 48 for the ear module 10, including accepting input data from the one or more buttons 47 and providing output data to the one or more status lights 46.
In the illustrated embodiment, the data/audio bus 70 transfers pulse code modulated audio signals between the radio module 51 and the processing module 50. The control bus 71 in the illustrated embodiment comprises a serial bus for connecting universal asynchronous receive/transmit UART ports on the radio module 51 and on a processing module 50 for passing control signals.
A power control bus 75 couples the radio module 51 and the processing module 50 to power management circuitry 77. The power management circuitry 77 provides power to the microelectronic components on the ear module in both the processing module 50 and the radio module 51 using a rechargeable battery 78. A battery charger 79 is coupled to the battery 78 and the power management circuitry 77 for recharging the rechargeable battery 78.
The microelectronics and transducers shown in
The ear module 10 operates in a plurality of modes, including in the illustrated example, an environmental mode for listening to conversation or ambient audio, a phone mode supporting a telephone call, a companion microphone mode for playing audio picked up by the companion microphone which may be worn for example on the lapel of a friend, and a hearing profile generation mode for generating a personalized hearing profile based upon real-time feedback to the user. The hearing profile generation mode will be described below with reference to a companion mobile phone device; however, the hearing profile generation mode could be carried out with other appropriate companion devices having a graphical user interface or other user interface having a touch sensitive area for producing user input based on at least two dimensions of touch position on the interface. The signal flow in the device changes depending on which mode is currently in use. An environmental mode does not involve a wireless audio connection. The audio signals originate on the ear module 10. The phone mode, the companion microphone mode, and the hearing profile generation mode involve audio data transfer using the radio module 51. In the phone mode, audio data is both sent and received through a communication channel between the radio and the phone. In the companion microphone mode, the ear module receives a unidirectional audio data stream from the companion microphone. In the hearing profile generation mode, the ear module 10 receives a profile data stream and may receive an audio stream from the companion mobile phone 11.
The control circuitry in the device is adapted to change modes in response to commands exchanged by the radio, and in response to user input, according to priority logic. For example, the system can change from the environmental mode to the phone mode and back to the environmental mode, the system can change from the environmental mode to the companion microphone mode and back to the environmental mode. For example, if the system is operating in environmental mode, a command from the radio which initiates the companion microphone may be received by the system, signaling a change to the companion microphone mode. In this case, the system loads audio processing variables (including preset parameters and configuration indicators) that are associated with the companion microphone mode. Then, the pulse code modulated data from the radio is received in the processor and up-sampled for use by the audio processing system and delivery of audio to the user. At this point, the system is operating in a companion microphone mode. To change out of the companion microphone mode, the system may receive an environmental mode command via the serial interface from the radio. In this case, the processor loads audio processing variables associated with the environmental mode. At this point, the system is again operating in the environmental mode.
If the system is operating in the environmental mode and receives a phone mode command from the control bus via the radio, it loads audio processing variables associated with the phone mode. Then, the processor starts processing the pulse code modulated data for delivery to the audio processing algorithms selected for the phone mode and providing audio to the microphone. The processor also starts processing microphone data for delivery to the radio and transmission to the phone. At this point, the system is operating in the phone mode. When the system receives a environmental mode command, it then loads the environmental audio processing variables and returns to environmental mode.
The control circuitry also includes logic to change to the Function Selection and Control Mode in response to user input via the man-machine interface 48.
Read-only program memory 207 stores instructions, parameters and other data for execution by the processing section 203. In addition, a read/write memory 208 in the mobile phone stores instructions, parameters, personal hearing profiles and other data for use by the processing section 203. There may be multiple types of read/write memory on the phone 200, such as nonvolatile read/write memory 208 (flash memory or EEPROM for example) and volatile read/write memory 209 (DRAM or SRAM for example), as shown in
An input/output controller 210 is coupled to a touch sensitive display 211, to user input devices 212, such as a numerical keypad, a function keypad, and a volume control switch, and to an accessory port (or ports) 213. The accessory port or ports 213 are used for other types of input/output devices, such as binaural and monaural headphones, connections to processing devices such as PDAs, or personal computers, alternative communication channels such as an infrared port or Universal Serial Bus USB port, a portable storage device port, and other things. The controller 210 is coupled to the processing section 203. User input concerning call set up and call management, and concerning use of the personal hearing profile, user preference and environmental noise factors is received via the input devices 212 and optionally via accessories. User interaction is enhanced, and the user is prompted to interact, using the display 211 and optionally other accessories. Input may also be received via the microphone 205 supported by voice recognition programs, and user interaction and prompting may utilize the speaker 206 for various purposes.
In the illustrated embodiment, memory 208 stores a program for displaying a function selection menu user interface on the display 211, such that the user can select the functions to be carried out during the generation of personal hearing profiles discussed below.
The generation of a personalized hearing profile for ear module 10 will be discussed primarily with reference to FIGS. 1 and 4-12. The communication link 15 between ear module 10 and mobile phone 11, or other companion device including a graphical user interface, will typically be a dual audio and communication link for the personalized hearing profile generation.
Touching hearing profile icon 912 causes the sound profile program stored in mobile phone 900 to be accessed; the sound profile program then displays the screen image 914 shown in
Main region 922 can also include a default position 926; positioning visual indicator 924 at default position 926 resets the hearing profile to a factory set hearing profile, commonly called the factory preset, or other hearing profile designated as a default at the time of the frame of reference is rendered. If desired other ways for selecting the default hearing profile can be used; for example task bar 916 could include a touch-selectable icon for selecting the default hearing profile. As mentioned above, the indices or other markers of coordinates on frame of reference rendered in the graphical user interface are, in this example, not visually perceptible to the user. That is, personal sound screen image 920 does not include any visual representation of what positions on main region 922 of screen image 920 are associated with specific sound profile data in this example. This permits the user to select a hearing profile by simply moving visual indicator 924 over main region 922 while listening to a sound stream broadcast by ear module 10; the sound stream being heard by the user reflects the hearing profile corresponding to the current position of the visual indicator 924 in real-time. The lack of indices, other markers of coordinates or other data correlating to location on the frame of reference, can prevent user bias in selecting hearing profiles, and for some users improve the ability to select an appropriate hearing profile.
In this example the hearing profile is generated by manipulating frequency emphasis, often called frequency shaping or frequency boosting, which is a function of gain and audio frequency, and output gain/dynamic range compression, the latter sometimes referred to as simply dynamic range compression which is a different function of gain and audio frequency. Other hearing variables and hearing profile functions, such as time constants or noise reduction aggressiveness can also be used instead of or in conjunction with these two examples.
Frequency shaping is, in this example, manipulated by emphasizing, also called boosting, the volume for selected frequency ranges so that the selected frequency ranges become louder compared with the other frequency ranges. A familiar example of frequency shaping is provided by equalizers found with many sound systems. In one example, lower frequencies are emphasized or higher frequencies are emphasized with the amount of boosting also chosen. The six different patterns of frequency shaping for this example are illustrated in
Dynamic range compression is a common technique that reduces the dynamic range of an audio signal. Dynamic range compression is usually thought of as a way of reducing the volume of very loud sounds while leaving the volume of quieter sounds unaffected. In some cases very quiet sounds are made louder while louder sounds are unaffected. Dynamic range compression is typically referred to as a ratio. A ratio of 4:1 means that if a sound is 4 dB over a threshold sound level, it will be reduced to 1 dB over the threshold sound level.
One method for enhancing an audio signal by the control of frequency shaping and output gain/dynamic range compression is discussed below with reference to
The finite impulse response (FIR) filter 965 shapes the frequency characteristic of the signal. Other frequency shaping methods could be used (IIR filtering, FFT based modifications, etc.) with the same effect. One way of controlling the frequency characteristic is to provide a family of frequency shaping patterns to choose from that have a logical relationship.
The first block 967 in
The frequency shaping and the output gain/dynamic range compression components, shown in
The use of an essentially featureless two-dimensional graphic display 904 will commonly limit the number of hearing profile parameters to two. However, an additional hearing variable, such as time constants or noise reduction aggressiveness, or hearing profile function, could be accommodated on a two-dimensional graphic display. For example, a third variable may be accessed on a two-dimensional touchscreen type of graphic display by lightly tapping on visual indicator 924 with the initial two taps accessing the third variable and additional taps accessing the different levels for the third variable. Instead of requiring additional taps, the different levels for the third variable could be accessed based on the length of time the user leaves his or her finger or stylus on visual indicator 924. However, providing for a third hearing variable is not presently preferred because some of the simplicity provided by simply moving one's finger or stylus or cursor over an essentially featureless two-dimensional display to select a personal hearing profile would be lost. However, if the selection of the third hearing variable would not affect the desirability of the choice of the first two hearing variables, typically frequency emphasis and output gain/dynamic range compression, then a third hearing very well could be a useful addition.
Generating a personalized hearing profile for an ear-level device, such as ear module 10, can be carried out as follows. Communication between ear module 10 and a companion device, such as mobile phone 900, is initiated. See 970 in
The sound profile data typically comprises frequency shaping data and output gain/dynamic range compression data with the functions of output gain/dynamic range compression data mapped along a first coordinate axis and frequency shaping data mapped along a second coordinate axis. For example, the first and second coordinate axes can be defined by Cartesian-type coordinates, that is linear distances along straight lines, such as in
Ear module 10 simultaneously broadcasts an audio stream for hearing by the user, typically through the speaker of the ear module, during execution of the sound profile program; see 980. This permits the ear level device to generate sound through the speaker based upon the sound profile data corresponding to the current position of visual indicator 924 on main region 922 of screen image 920 to provide real-time feedback to the user. The user can continue to move visual indicator 924 to different chosen positions on main region 922; doing so changes the parameters of the sound profile used to generate sound through the speaker thereby changing the sound of the audio stream as it emanates from the speaker. Once an acceptable sound profile is found, which is typically determined by the sound emanating from the speaker, the user can stop moving visual indicator 924 and exit the sound profile program; see 982. The sound profile program will remain active until an end event, such as turning off mobile phone 900 or ear module 10 or by exiting the sound profile program in mobile phone 900. Also, the sound profile selected can be stored, and applied as a default profile or as a beginning profile in later interactions with the program.
In some examples the companion device transmits sound data to ear module 10 that has been generated using the hearing profile data. The procedure, see
In some cases the audio stream is generated by the ambient environment and captured by the microphone of the ear module 10. The audio stream may also be generated by a device, such as cell phone 900 or computer 13, spaced apart from the ear module 10. Further, the audio stream may be stored in ear module 10. If desired the selected sound profile may be stored one or more of mobile phone 900 and ear module 10. In some examples sound profiles for different circumstances can be generated and stored; examples include listening to music generated by a digital music player through the ear module 10, and listening to telephone conversations using ear module 10 and mobile phone 900, and using ear module 10 in a environmental mode to listen to conversations. These stored personal sound profiles, commonly called personal sound profile presets, and then be quickly accessed by the user according to the current listening situation. The ease by which a personal sound profile can be generated for the current listening environment, as well as ease by which preset personal sound profile can be generated and stored, provides distinct incentives to do so.
While the present invention is disclosed by reference to the preferred embodiments and examples detailed above, it is to be understood that these examples are intended in an illustrative rather than in a limiting sense. It is contemplated that modifications and combinations will readily occur to those skilled in the art, which modifications and combinations will be within the spirit of the invention and the scope of the following claims.
Any and all patents, patent applications and printed publication referred to above are incorporated by reference for all purposes.
Number | Name | Date | Kind |
---|---|---|---|
4061874 | Fricke et al. | Dec 1977 | A |
6011853 | Koski et al. | Jan 2000 | A |
6058197 | Delage | May 2000 | A |
6212496 | Campbell et al. | Apr 2001 | B1 |
6463128 | Elwin | Oct 2002 | B1 |
6532005 | Campbell | Mar 2003 | B1 |
6684063 | Berger et al. | Jan 2004 | B2 |
6813490 | Lang et al. | Nov 2004 | B1 |
6840908 | Edwards et al. | Jan 2005 | B2 |
6850775 | Berg | Feb 2005 | B1 |
6944474 | Rader et al. | Sep 2005 | B2 |
7181297 | Pluvinage et al. | Feb 2007 | B1 |
7190795 | Simon | Mar 2007 | B2 |
7328151 | Muesch | Feb 2008 | B2 |
20030078515 | Menzel et al. | Apr 2003 | A1 |
20040008849 | Moller | Jan 2004 | A1 |
20040136555 | Enzmann | Jul 2004 | A1 |
20050248717 | Howell et al. | Nov 2005 | A1 |
20060045281 | Korneluk et al. | Mar 2006 | A1 |
20070255435 | Cohen et al. | Nov 2007 | A1 |
20080025538 | Zad-Issa | Jan 2008 | A1 |
20080137873 | Goldstein | Jun 2008 | A1 |
20080165980 | Pavlovic et al. | Jul 2008 | A1 |
20090154741 | Woods et al. | Jun 2009 | A1 |
20090180631 | Michael et al. | Jul 2009 | A1 |
20100027824 | Atamaniuk et al. | Feb 2010 | A1 |
20100029337 | Kuhl et al. | Feb 2010 | A1 |
20110176686 | Zaccaria | Jul 2011 | A1 |
Number | Date | Country |
---|---|---|
10222408 | Nov 2003 | DE |
0705016 | Apr 1996 | EP |
1089526 | Apr 2001 | EP |
2000209698 | Jul 2000 | JP |
2001136593 | May 2001 | JP |
0124576 | Apr 2001 | WO |
0154458 | Jul 2001 | WO |
03026349 | Mar 2003 | WO |
2004110099 | Dec 2004 | WO |
2006105105 | Oct 2006 | WO |
Entry |
---|
International Search Report mailed Aug. 17, 2011 in PCT/US2011/036135. |
Lippmann, R. P. et al., Study of multichannel amplitude compression and linear amplification for persons with sensorineural hearing loss, J. Acoust. Soc. Am. 69(2), Feb. 1981, pp. 524-534. |
Number | Date | Country | |
---|---|---|---|
20110280409 A1 | Nov 2011 | US |