Patent Application
20090076636
The instant invention relates to an assistive listening system including a hearing aid and a wireless, handheld, programmable digital signal processing device.
Programmable, “at-ear”, hearing aids are well-known in the art. When using the term “at-ear”, the Applicant intends to include all types of hearing aids that are located in the vicinity of the ear, such as Completely-in-the-Canal (CIC) hearing aids, Mini-Canal (MC) hearing aids, In-the-Canal (ITC) hearing aids, Half-Shell (HS) hearing aids, In-the-Ear (ITE) hearing aids, Behind-the-Ear (BTE) hearing aids, and Open-fit Mini-BTE hearing aids.
Prior art programmable hearing aids typically include a small, low-power digital audio processing device, or digital signal processor (DSP), which locally receives an audio input from an on-board microphone, processes the audio input and outputs the audio directly to the wearer through a small speaker. A DSP is specifically designed to perform the audio signal analysis and computation required to deliver the clearest sound to the user. This analysis and computation involves reshaping the audio signals using mathematical equations (algorithms). Because of the size of a typical at-ear hearing aid, audio processing power is limited and thus functionality is typically limited to just one audio processing algorithm (fixed set of calculations) and often a single hearing profile. Modifications to the hearing profile (personalized adjustments) typically require a trip to an audiologist to connect the hearing aid to a special interface to make adjustments. An audiologist can change the variables for the fixed set of calculations, but cannot change the calculations which are built into the hardware of the DSP. This process is akin to changing the equalizer settings where the gain of certain frequency ranges is increased or decreased depending on the wearer's hearing loss.
Programmable hearing aids that include the ability to process audio signals according to multiple hearing profiles are also well known in the art. In these devices, the audiologist is able to program multiple profiles into the hearing aid memory, and the user is able to select a particular hearing profile by manually actuating a switch on the hearing aid corresponding to the desired setting. However, the underlying processing algorithm (fixed mathematical calculations) remains the same.
Some of these multiple-profile hearing aids include a separate handheld programming device that can selectively push a programming profile to the hearing aid at the direction of the user. Alternatively, the handheld programming device samples ambient sound with an on-board microphone, analyzes the audio signal and then automatically sends (pushes) a programming signal to the earpiece to tell the earpiece how to process the audio signal (automatically sets the hearing profile). These separate handheld devices do have digital signal processing capabilities and do process ambient audio, but the processed audio is not transmitted back to the earpiece. Only a programming signal is transmitted back to the hearing aid. The actual signal processing is still completed in the hearing aid based on the hearing profile determined by the handheld device.
Assistive listening systems having a wireless earpiece and a separate handheld or base unit are also well known in the art. Some of these prior art systems provide for digital processing in the separate device, while others are simply wireless repeaters for taking in audio signals from a source and transmitting it to the earpiece. However, one aspect of these prior art systems is that the systems that provide for digital signal processing (DSP) in the handheld unit remove the audio signal processing capabilities from the earpiece. Where the DSP capabilities are preserved in the earpiece, the handheld or base unit is simply being used as a signal repeater.
While the prior art programmable hearing aids and assistive listening devices have served the market for many years, demographics are rapidly changing such that many elderly people are now comfortable with electronic devices and computers, and society now generally embraces the concept of all people carrying and wearing listing devices, such as MP3 players. It is believed that there is an unmet need in the assistive listening industry for a versatile and powerful assistive listening system that combines the known benefits of at-ear hearing aids with the powerful programming and processing capabilities that are now available in advanced digital signal processors. By supplementing the audio processing functions of the hearing aid with a separate digital signal processing device, which can accommodate a larger audio processor, memory, input and output ports, the Applicant can significantly enhance the usability and overall functionality of hearing devices.
In one embodiment, the assistive listening system includes a functional hearing aid and a wireless, handheld, programmable digital signal processing device.
The hearing aid generally includes components of a programmable hearing aid, i.e. microphone, digital signal processor, speaker and power source. The hearing aid also includes an analog amplifier and a wireless ultra-wide band (UWB) transceiver for communicating with the separate handheld digital signal processor device.
The digital signal processing device generally includes a programmable digital signal processor, a UWB transceiver for communicating with the hearing aid, an LCD display, and a user input device (keypad). Other wireless transmission technologies are also contemplated.
The handheld device may be user programmable to accept different processing algorithms for processing sound signals received from the hearing aid. The handheld device may also be capable of receiving audio signals from multiple sources, and gives the user control over selection of incoming sources and selective processing of sound. In the context of being user programmable, the digital signal processing device includes a software platform that provides for the ability of the user to select, or plug-in, desired processing algorithms for application to selected incoming audio signals. The software platform provides the ability to load and save different processing algorithms, to apply those processing algorithms to one or more incoming sound sources, and/or to apply multiple processing algorithms in series, if desired. Finally, the embodiments are directed to a method associated with processing of the incoming sound signals.
More specifically, an embodiment is directed to a method of processing sound for the hearing impaired comprising receiving at a first input a first digital audio signal, receiving at a second input a second digital audio signal, processing the first digital audio signal according to a first signal processing algorithm, processing the second digital audio signal according to a second signal processing algorithm; mixing the first and second processed audio signals; and outputting the mixed first and second audio signals.
Alternately, in another embodiment, a method is directed to the steps of receiving at an input a digital audio signal, processing the digital audio signal according to a first signal processing algorithm to provide a once processed audio signal, processing the once processed digital audio signal according to a second signal processing algorithm to provide a twice processed audio signal, and outputting the twice processed audio signal.
Accordingly, among the embodiments are: an assistive listening system including both a ear hearing aid and a separate handheld digital signal processing device that supplements the functional signal processing of the hearing aid; a handheld digital signal processing device that can accept audio signals from a plurality of different sources; a handheld digital signal processing device that is wireless; a wireless handheld DSP device that is user programmable to apply different processing algorithms for processing sound signals received from the hearing aid or other audio source; and a method of processed sound for the hearing impaired wherein one or more processing algorithms are selectively applied to one ore more incoming audio signals.
Other objects, features and advantages of the invention shall become apparent as the description thereof proceeds when considered in connection with the accompanying illustrative drawings.
In the drawings which illustrate the best mode presently contemplated for carrying out the present invention:
Referring now to the drawings, the assistive listening system of the present invention is illustrated and generally indicated at 10 in
The user depicted in
Turning to
In accordance with the invention, the hearing aid 12 also includes an analog amplifier 24 and a wireless Ultra-Wide Band (UWB) transceiver 26 and antenna 28 for communicating with the separate handheld digital signal processor device 14.
The Applicant has chosen Ultra-Wide Band (UWB) wireless communication as the preferred wireless transmission technology for transmitting and receiving data between the hearing aid and the handheld device. UWB is known for its fast transfer speeds and ability to handle large amounts of data. While the Applicant has selected UWB as the preferred wireless transmission technology, it is to be understood that other wireless technologies, such as Infra Red, WiFi, Bluetooth® (Bluetooth is a registered trademark of Bluetooth Sig, Inc), etc. are also suitable for accomplishing the same purpose (although at lower data rates and greater latency).
Referring to
The programmable DSP 30 is preferably a high-power audio processing device, such as Analog Devices®, Blackfin® BF-538 DSP, although other similar devices would also be suitable for use in connection with the invention (Analog Devices® and Blackfin® are trademarks or registered trademarks of Analog Devices Corp.).
The UWB transceiver 32 is similar to the UWB transceiver 26 in the hearing aid and is capable of wireless communication with the UWB transceiver 26 in the hearing aid.
The LCD screen 36 is a standard component that is well known in the industry and will not be described in further detail.
The user input device 38 is preferably defined as a keypad input. However, the Applicant also contemplates the use of a touch-screen input (not shown), as well as other mechanical and electrical inputs, scroll wheels, and other touch-based input devices.
Where the input device 38 is a touch screen, the LCD and input device are combined into a single hardware unit. Touch-screen LCD devices are well known in the art, and will not be described in further detail.
The rechargeable battery system 40 includes a rechargeable battery 42, such as a conventional high capacity, lithium ion battery, and a power management circuit 44 to control battery charging and power distribution to the various components of the handheld DSP device 14.
In operation of the basic system 10, the hearing aid(s) 12 can independently operate without the handheld DSP device 14. The hearing aid 12 includes its own microphone 16, its own DSP 18 that can receive and process audio according to prior art processing methods, and its own speaker 20 for outputting audio directly to the wearer's ear.
An aspect of the present invention is a control and switching system 46 on-board the hearing aid 12 that monitors the wireless connection status of the handheld DSP device 14 and the power status of the hearing aid 12 and selectively routes the incoming audio from the hearing aid microphone 16 responsive to the status. When the hearing aid 12 is fully charged, and the handheld DSP device 14 is in communication range, the default operation is for the hearing aid 12 to route incoming audio from the on-board microphone wirelessly through the handheld DSP device 14 for processing (See FIGS. 2 and 2A—Mode A). More specifically, referring to
When the control system 46 senses that the handheld DSP device 14 is not available, i.e. either out of range or low battery, the hearing aid control system 46 automatically defaults to the DSP 18 on-board the hearing aid 12 so that the hearing aid 12 functions as a conventional hearing aid (FIGS. 2 and 2A—Mode B). More specifically, referring to
When the control system 46 senses that the hearing aid 12 power is low, regardless of wireless status of the handheld DSP 14, it will automatically default to the on-board DSP 18 to conserve power that is normally consumed by the wireless transceiver 26 (FIGS. 2 and 2A—Mode B).
The hearing aid control system 46 will further automatically switch to a conventional analog amplifier mode when the hearing aid power is critically low (FIGS. 2 and 2A—Mode C). More specifically, referring to
It is noted that switches 47A, 47B, 49A, 49B can be physical analog switches or software flags which determine where the signal is sourced from and sent to. It is also contemplated that the embodiment may further be implemented without an analog processing layer (Mode C).
Accordingly, it can be seen that the hearing aid control system 46 is effective for controlling the routing of audio signals received by the on-board microphone 16, and is further effective for automatically controlling battery management to extend the battery life and function of the hearing aid 12 to the benefit of the wearer.
Referring to
Further, an aspect of the binaural processing scheme in the present invention is that the control systems 46 can collectively perform load balancing where processing is first done in one hearing aid 12 and the other hearing aid 12 is in a low power transceiver mode, and then after a set period of time, the devices 12 swap modes in order to balance battery drain in each of the hearing aids (See
Yet another aspect of the invention is the ability of the handheld DSP device 14 to receive audio signals from other external sources. Turning to
The DSP device 14 further includes a T-coil sensor 72 for receiving signals from conventional telephones and American's with Disabilities Act (ADA) mandated T-coil loops in public buildings, or other facilities, which utilize T-coil loops to assist the hearing impaired. The T-coil sensor 72 shares the A/D converter 68 with the left microphone input 50.
In addition to the UWB transceiver 32 being used for communicating with the hearing aid 12, the UWB transceiver 32 is also capable of receiving incoming wireless audio signals from a plurality of different wireless audio sources. In this regard, the system 10 is configured to include a UWB wireless telephone adapter generally indicated at 74 (
We now turn to a category of devices we refer to as “intermittent” audio sources. By “intermittent”, we simply mean that sound emanating from the source is not constant, i.e. a telephone ringing as opposed to sound emanating from a television, or that the user may not be attendant to the sound source and may thus not immediately recognize the sound. Referring to
Similar to the concept of the wireless telephone adapter,
The wireless smoke alarm adapter 80 preferably includes a UWB transceiver 102, a microcontroller 104, and wired input 106 for series connection with a wired smoke alarm system (not shown). The UWB smoke alarm adapter 80 is preferably powered by the existing voltage in the wired smoke alarm line 106 and is configured to monitor the incoming signal voltage and wirelessly transmit an alarm signal to the DSP device 14 to alert the user that the smoke alarm is sounding. Wireless battery powered units (battery 108) are also contemplated. As indicated above, the handheld DSP device 14 is programmable to recognize each connected audio source, and in this regard, displays to the user on the LCD 36, a graphical representation 110 of a fire (or a smoke alarm) to visually identify to the user the source of the signal, as well as energizes the LCD backlight 98, and displays a text message 112 such as “SMOKE ALARM” or “FIRE”.
The wireless doorbell adapter 82 preferably includes a UWB transceiver 114, a microcontroller 116, and a wired input 118 for series connection with a wired doorbell system. The UWB doorbell adapter 82 is preferably powered by the existing voltage in the wired doorbell line and is configured to monitor the incoming signal voltage and wirelessly transmit a signal to the DSP device 14 to alert the user that the doorbell is ringing. Wireless battery powered units (battery 120) are also contemplated. As indicated above, the handheld DSP device 14 is programmable to recognize each connected audio source, and in this regard, displays to the user on the LCD 36, a graphical representation of a door bell to visually identify to the user the source of the signal as well as energizes the LCD backlight 98 and displays a text message such as “DOOR BELL”.
We now turn back to “constant” incoming audio sources and situations where the user is attendant to the source of the incoming sound. Referring to
Turning to
Although the primary use of the wireless microphone 78 is intended for personal conversation, it is possible to use the microphone 78 in any situation where the user wants to listen to a localized sound. For example, if the user were a guest at someone's home, and wanted to watch television, the user could simply place the wireless microphone 78 adjacent to the television speaker in order to better hear the television without the need for the more specialized wireless audio adapter. Similarly, if the user were making a pot of coffee and were awaiting the ready signal, the user could place the microphone 78 next to the coffee maker and then go about other morning activities while awaiting the coffee to be ready. The wireless microphones 78 thus allow the user significant freedom of movement that hearing persons often take for granted.
Turning to
It should be noted that the handheld DSP device 14 can also recognize the wireless audio sources from the wireless audio adapter 76, wireless telephone adapter 74, and wireless microphone 78 and can display a visual cue to identify the input source.
It can be appreciated that the above-noted wireless input devices 74, 76, 78, 80, 82, 146 are all configured to function with the handheld DSP device 14 of the present invention. However, there are many existing wireless devices that can also be advantageously utilized with the present invention. For example, there are a multitude of Bluetooth® enabled devices 156 (
The handheld DSP device 14 is preferably configured to recognize pairing with Bluetooth® enabled cell phones 156 such that the user can channel a cell phone call through the handheld DSP device 14. Referring briefly to
Turning now to audio output, as an alternative output to the hearing aid 12, the DSP device includes a conventional stereo audio out jack generally indicated at 162 (
As another alternative to the hearing aid 12, audio output can also be channeled through the Bluetooth® transceiver 158 to a conventional Bluetooth® headset 176 (
We will turn to a more detailed discussion of the operation of the programmable DSP device 14 and how incoming audio streams are processed. There are several aspects to how the incoming audio streams are processed. As explained hereinabove, prior art hearing aids include a DSP, but because of size and power constraints, the DSP's are typically low power devices and are limited in functionality to single processing algorithm. In many cases, these low-power DSP's are customized ASIC chips, which are fixed hardware designs that cannot be altered, other than to change selected operating parameters.
The high-power DSP 30 of the present handheld DSP device 14 is a microcontroller based (software-based) device that is user programmable to accept different processing algorithms for “enhancing” audio signals received from the hearing aid, as well as other input sources, and gives the user control over selection of incoming sources and selective processing of audio signals.
“Processing” is generally defined as performing any function on the audio signal, including, but not limited to multiplexing, demultiplexing, “enhancing”, “filtering”, mixing, volume adjustment, equalization, compression, etc.
“Audio signal enhancement” involves the processing of audio signal to improve one or more perceptual aspects of the audio signals for human listening. These perceptual aspects include improving or increasing signal to noise ratio, intelligibility, degree of listener fatigue, etc. Techniques for audio signal processing or enhancement are generally divided into “filtering” and “enhancement”, although filtering is considered to be a subset of enhancement, “Enhancing” is generally defined as applying an algorithm to restore, emphasize or correct desired characteristics of the audio signal. In other words, an enhancement algorithm modifies desirable existing characteristics of the audio signal. “Filtering” is generally defined as applying an algorithm to an audio signal to improve sound quality by evaluating, detecting, and removing unwanted characteristics of the audio signal. In other words, a filtering algorithm generally removes something from the signal. The importance of the distinction of these two types of processing algorithms will only become apparent in the context of the order of application of the algorithms as further explanation of the system unfolds.
In the context of being user programmable, the handheld DSP device 14 includes built-in Flash memory 178 for storing the operating system of the device 14 as well as built-in SD Ram 180 for data storage (preferably at least 64 Megabytes) which can be used to store customization settings and plug-in processing algorithms. Further, the handheld DSP device 14 includes a memory card slot 182, preferably an SD memory card or mini-SD memory card, to receive an optional memory card holding up to an additional 2 gigabytes of data. Still in the context of being user programmable, the handheld DSP device 14 includes an expansion connector 183 and also a separate USB interface 184 for communication with a personal computer to download processing algorithms. The system further includes a host software package that will be installed onto a computer system and allow the user to communicate with and transfer data to and from the various memory locations 178, 180, 182 within the handheld DSP device 14. Communication and data transfer to and from the memory locations 178, 180, 182 and with other electronic devices is accomplished using any of the available communication paths, including wired paths, such as the USB interface 184, or wireless paths , such as the Bluetooth® link, and the UWB link etc.
Referring now to
As will be described further hereinbelow, the DSP 30 has the ability to demultiplex the data stream and then separately process each of the types of input. Still referring to
The software system of the handheld DSP device 14 is based on a plug-in module platform where the operating software has the ability to access and process data streams according to different user-selected plug-ins. The concept of plug-in software modules is known in other arts, for example, with internet browser software (plug-in modules to enable file and image viewing) and image processing software (plug-in modules to enable different image filtering techniques). Processing blocks, generally indicated at 192, are defined within the plug-in software platform that will allow the user to select and apply pre-defined processing modules, generally indicated at 194, to a selected data stream. Plug-in processing modules 194 are stored in available memory 178, 180, 182 and are made available as selections within a basic drop-down menu interface that will prompt the user to select particular plug-in processing modules for processing of audio signals routed through different input sources. For purposes of this disclosure, the Applicant defines a processing module 194 as a plug-in module including a “processing algorithm” which is to be applied to the audio signal. The term “processing algorithm” is intended to include both filtering algorithms and enhancement algorithms.
Within the plug-in software system, the basic structure of all of the processing modules 194 is generally similar in overall programming, i.e. each module is capable of being plugged into the processing block of the software platform to be applied to the audio stream and process the audio stream. The difference between the individual processing modules 194 lies in the particular algorithm contained therein and how that algorithm affects the audio stream. As indicated above, we define filter modules 194F and enhancement modules 194E. As used herein, a “filter module” 194F is intended to mean a module that contains an algorithm that is classified as a filtering algorithm. As used herein an “enhancement module” 194E is intended to mean a module 194 that contains an algorithm that is classified as an enhancing algorithm.
Now turning to the motivation for separating “filtering algorithms” from “enhancement algorithms”, it is recognized by the Applicant that it is preferable to apply filters to the audio signal to improve the signal to noise ratio prior to applying enhancements. Accordingly, to simplify the user interface, and improve functionality of a device that would be programmed by those with only limited knowledge of audio processing, the Applicant's separated the selection and application of filter algorithms and enhancement algorithms into two sequential processing blocks. Referring to
During a setup mode, the user will scroll through a drop down menu of available input sources to select a particular input source, or multiple input sources. For example, if the user were sitting at home watching television with a family member, the user may select to have two inputs, namely a wireless audio adapter input 76 to receive audio signals directly from the television, as well as a wireless microphone input 78 to hear the other person seated in the room. All other inputs may be unselected so that the user is not distracted by unwanted noise. Alternately, if the user were at a restaurant with several companions, the user may have several wireless microphones 78 that are paired with the handheld DSP device 14 and then selected as input sources to facilitate conversation at the table. All other input sources could be unselected. Input source selection is thus easily configured and changed on the fly for different environments and hearing situations. Commonly used configurations will be stored as profiles within the user set-up so that the user can quickly change from environment to environment without having the reconfigure the system each time.
For each incoming audio source, the user can customize filtering and enhancement of each incoming audio source according the users' own hearing deficits and/or hearing preferences (See
In
As indicated above, the user may connect the handheld DSP device 14 to the user's computer, and using the device interface software, download into memory a plurality of different signal filter modules 194F available within the user software. It is further contemplated that the interface software will have the ability to connect to the internet and access an online database(s) of filters modules 194F that can be downloaded. In the future, as new filter modules 194F are developed, they can be made available for download and can be loaded onto the handheld DSP device 14.
For each incoming audio source, the user can further customize enhancement of each incoming audio source according the user's own hearing deficits and/or hearing preferences. Similar to the selection of available incoming audio sources and filter modules 19F, for each incoming audio source, the user will selectively apply desired enhancement modules 194E to improve the sound quality each of different audio source. In this regard, a plurality of software-based enhancement modules 194E is stored in memory for selective application to an incoming audio source. Referring to
Again, as indicated above, the user may connect the handheld DSP device 14 to the user's computer, and using the device interface software, download into memory 178, 180, 182 a plurality of different signal enhancement algorithms 194E available within the user software. It is further contemplated that the interface software will have the ability to connect to the internet and access an online database(s) of enhancement algorithms 194E that can be downloaded. In the future, as new enhancement algorithms 194E are developed, they can be made available for download and can be loaded onto the handheld DSP device 14.
Turing back to
After application of the master volume and equalization adjustments 196, the audio signal streams are mixed 190 into a single audio stream for output. After mixing, the single output stream is compressed (AGC) for final output to the user, whether through the wireless hearing aid link, wireless Bluetooth® link, or wired output.
Referring to
Preferably, the system continuously buffers the previous 30 seconds of audio output for selective replay by the user, although the system also preferably provides for the user to select the time segment of the replay buffer, i.e. 15 seconds, 20 seconds, 30 seconds, etc.
Accordingly, if the user cannot decipher a particular part of the previously heard output, the user can press an input key 38, (such as a dedicated replay key) which triggers the system to temporarily switch the output to replay of the buffered audio. The user can then better distinguish the audio the second time. As a further enhancement to the replay feature, the system is further configured to convert the replayed audio into text format (for speech) and to display the converted speech on the LCD screen 36 of the handheld DSP device 14. Speech to text conversion programs are well known in the art, and the operating system of the handheld DSP 14 is configured with a speech to text sub-routine that is employed during the replay function. It is preferred that the replay audio is buffered after application of all of filters 194 and enhancements 194 and after mixing 190 to the single audio output stream. The enhanced sounds, particularly voices may thus be better distinguished by both the user and by the speech to text program. As a further alternative, the system can be configured to employ the speech to text conversion sub-routine as a personal close-captioning service. In this regard, the speech to text conversion program is constantly running and will display converted text to the user at all times.
It is a further aspect of the system 10 that each of the audio signals can be separately buffered and stored in available memory. In this regard, the system is capable of replaying the audio from only signal source. For example, if the user had an audio signal from a television source and another audio signal from another person, the user could selectively replay the signal originating from the other person so as to be better able to distinguish the spoken words of the individual rather than having the audio mixed with the television source. Likewise, only that isolated audio signal could be converted to text so that the user was able to read the text of the conversation without having the distraction of the television dialogue interjected with the conversation.
Referring to
In the same context, referring to
It can therefore be seen that the instant invention provides an assistive listening system 10 including both a functional at-ear hearing aid 12, or pair of hearing aids 12, and a separate handheld digital signal processing device 14 that supplements the functional signal processing of the hearing aid 12, and further provides a control system 46 on board the hearing aid(s) that controls routing of incoming audio signals according to wireless transmission status and power status. The system 10 still further provides a handheld digital signal processing device 30 that can accept audio signal from a plurality of different sources and that includes a versatile plug-in software platform that provides for selective application of different signal filters and sound enhancement algorithms to selected sound sources.
While there is shown and described herein certain specific structure embodying the invention, it will be manifest to those skilled in the art that various modifications and rearrangements of the parts may be made without departing from the spirit and scope of the underlying inventive concept and that the same is not limited to the particular forms herein shown and described except insofar as indicated by the scope of the appended claims. For example, although a Blackfin™ digital signal processor is identified and described as the preferred device for processing, it is also contemplated that other devices, such as ASIC's, FPGA's, RISC processors, CISC processors, etc. could also be used to perform at least some of the calculations required herein. Additionally, although the invention focuses on the use of the present system for the hearing impaired, it is contemplated that individuals with normal hearing could also benefit from the present system. In this regard, there are potential applications of the present system in military and law enforcement situations, as well as for the general population in situations where normal hearing is impeded by excessive environment noise.
