Patent Application
20070160243
1. Field of the Invention
The present invention relates to a system and a method for separation of a user's voice from ambient sound by using at least one device to be worn at the user's ear or at least partly in the user's ear canal.
2. Description of Related Art
For communication purposes, in particular for wireless electronic communication between or from persons exposed to a noisy environment, such as workers in industrial plants, policemen, soldiers, firemen, etc., it is desirable to have a sound pick-up system which is capable at least to some extent to separate the user's voice from ambient noise, or generally ambient sound, in order to improve the intelligibility of the person's speech to the listener, who may be one of the other persons exposed to the noisy environment or who may be a remote person.
A common approach to achieve such separation of a person's voice is the use of a boom microphone, i.e. a microphone which is placed close to the mouth, carried by a headset, helmet or any other device worn by the person. Such microphone selectively emphasizes the near field around the mouth.
Other approaches are vibration pick-up devices which are in direct contact with the throat, picking up the vibrations of the vocal chord, or which are in direct contact with the meatus wall or the outer ear canal, picking up the vibrations of the head tissue (i.e. “bone conduction” microphones) or which are in direct contact with the cheek-bone.
Devices of these types are either fairly sensitive to acoustic noise masking the speech or certain speech sounds are poorly transmitted, especially the high frequency consonant sounds necessary for good intelligibility. Furthermore, for industrial applications boom microphones have the drawbacks that they limit the freedom of movement of the user and that, when combined with a hearing protection device, they will affect the stability and hence the attenuation of the hearing protection device. Bone-conduction microphones have the drawbacks that they have a very limited audio bandwidth which limits the intelligibility of the speech and that they often have to be pressed fairly hard which causes discomfort to the user.
U.S. Pat. No. 6,661,901 B1 relates to an active hearing protection system comprising an earplug with an outer microphone for picking up ambient sound and an inner microphone which is sealed with respect to ambient sound but is open towards the inner part of the user's ear canal. In an operation mode in which separation of the user's voice from ambient noise is desired only the inner microphone is activated while the outer microphone is not, with the signal from the inner microphone being processed by an electronics unit integrated within the earplug in order to make the user's voice highly natural and intelligible, either for the user himself or his external communication partners.
Another approach is based on a so-called “blind source separation” (BSS) algorithm in order to separate a person's voice from background noise by corresponding audio signal processing. In this respect, US 2003/0055535 A1 relates to the use of a BSS algorithm for separating the voice of an operator of a vehicle wheel alignment system with a voice audio interface from background noise by using a microphone array in order to avoid the necessity to use a headset. US 2005/0060142 A1 relates in a more general manner to the use of two spaced-apart microphones operated with a BSS algorithm for voice separation from background noise in audio applications. EP 1 509 065 A1 relates to binaural hearing aid system wherein the audio signals captured by the microphone of the right ear hearing aid and the microphone of the left ear hearing aid undergo a BSS algorithm, followed by additional signal processing, in order to increase the intelligibility of speech in background noise by the user of the hearing aid system.
It is an object of the invention to provide for a system and a method for separation of a person's voice from ambient sound wherein good intelligibility of speech is achieved while nevertheless discomfort to the person is to be avoided as far as possible.
According to the invention this object is achieved by a system as defined in claims 1 and 33, and by corresponding methods as defined in claims 36 and 37, respectively.
The invention is beneficial in that, by using a first microphone and a second microphone, wherein according to the solution of claims 1 and 36 the first microphone is oriented acoustically outwardly towards the environment and the second microphone oriented acoustically inwardly towards the user's ear canal and according to the solution of claims 33 and 37 the first microphone is located at the right ear and the second microphone is located at the left ear, and by processing the audio signals from the first and second microphone by a blind source separation algorithm, good separation of the user's voice from ambient sound with resulting high intelligibility of the user's speech is achieved without the need for additional restrictions regarding the location of the microphones, so that in particular the need for a boom microphone or a bone-conduction microphone can be avoided. In particular, by orienting the first microphone towards the environment and orienting the second microphone towards the user's ear canal or by locating the first microphone at the right ear and the second microphone at the left ear these two microphones pick up sufficiently differently mixed signals of the ambient sound and the user's voice so that a BSS algorithm will work efficiently.
Generally, blind source separation (also referred to as “independent component analysis” (ICA)) is a technique for separating mixed source signals (components) which are presumably statistically independent from each other. In its simplified form, blind source separation applies an “un-mixing” matrix of weights to the mixed signals, for example, multiplying the matrix with the mixed signals, to produce separated signals. The weights are assigned initial values, and then adjusted to maximize joint entropy of the signals in order to minimize information redundancy. This weight-adjusting and entropy-increasing process is repeated until the information redundancy of the signals is reduced to a minimum. Because this technique does not require information on the source of each signal, it is referred to as “blind source separation”. An introduction to blind source separation is found, for example, in US 2005/0060142 A1.
In the most simple case, BSS is applied to two different mixtures of two (acoustic) sources, wherein the two different mixtures are obtained by using two spaced apart microphones. Mixing of the two sources can be represented by a matrix A, with the BSS algorithm corresponding mathematically to finding the inverse matrix of A without knowing anything about the matrix nor about the sources, except that they are statistically independent. In the case of a person's voice mixed with background noise the latter assumption usually is valid. The mixtures of the two sources could be different with respect to amplitude and/or phase of the two sources. In other words, by picking up sound signals with two differently oriented microphones the signal of each of these microphones will correspond to a mixture which is different with regard to the difference in amplitude and/or phase of the two acoustic sources (i.e. user's voice on the one hand and ambient noise on the other hand). By orienting one of the microphones outwardly towards the environment and the other microphone inwardly to the ear canal, a particularly large difference between the two mixtures can be obtained in a simple and particularly comfortable manner, i.e. no boom microphones or bone-conduction microphones which would cause discomfort to the user need to be used.
According to one embodiment, the two microphones are part of a hearing protection device. With such a configuration, the microphones can be arranged such that the ambient sound reaching the inwardly oriented microphone is attenuated by the hearing protection device relative to ambient sound reaching the outwardly oriented microphone. Although such hearing protection device could be an earmuff, according to a preferred embodiment the hearing protection device is an earplug comprising a shell which is to be inserted at least partially into the user's ear canal. According to one embodiment, the shell is a customized hard shell having an elasticity from shore D85 to D65 and having an outer shape according to the measured inner shape of the user's outer ear and ear canal. According to an alternative embodiment, the shell is a generic soft shell capable of adapting to the shape of the user's outer ear and ear canal.
Preferably, the inwardly oriented microphone is located at or is open to the inner part of the shell which is to be inserted into the ear canal, with the inwardly oriented microphone preferably being located at the inner end of the shell or within a channel of the shell open to the inner end of the shell.
Preferably, the outwardly oriented microphone is located at or is open to the outer part of the shell which is not to be inserted into the ear canal, with the outwardly oriented microphone preferably being located at the outer end of the shell or within a channel of the shell open to the outer end of the shell.
If the hearing protection device is an earmuff, the first microphone is located at or is open to the outer side of the earmuff and the second microphone is located at or is open to the inner side of the earmuff.
In all embodiments, the device preferably comprises a speaker adapted to provide an external audio signal to the user's ear. Thereby bidirectional communication with the user is achieved. Preferably the speaker is located at or is open to a portion of the device which is to be worn within the user's ear canal, whereby the speaker is brought acoustically close to the user's ear drum so that good intelligibility of the sound provided by the speaker is achieved even in noisy environments.
Usually the device will be binaural, i.e. it will comprise one unit for the right ear and another unit for the left ear. The speaker and the microphones may be integrated in the same unit, i.e. in at least one of the units, or the speaker may be part of the unit for one ear and the microphones may be part of the unit for the other ear.
According to one embodiment, the device may be adapted to be worn completely within the user's ear canal, whereby the device can be more or less completely hidden from the views of other persons. According to other embodiments, the device may be any other kind of a wired or wireless headset.
Preferably the audio signal processing unit is integrated within the device. In an alternative embodiment, the audio signal processing unit may be adapted to be worn behind the user's ear or somewhere at the user's body. In this case, although the device becomes more handy, more space, and hence also more power, is available for the audio signal processing unit, which may reduce the costs an/or may improve the performance of the blind source separation. In such case, the audio signal processing unit may be connected to the microphone either by wires, which is the most simple solution, or via a wireless link, for example, a radio frequency link such as Bluetooth link, an inductive link or an infrared link, which solution would result in enhanced wearing comfort for the user.
As a further alternative, the audio signal processing unit could be designed to be located remote from the user and is connected to the microphones via a radio frequency link such as Bluetooth link. In this case, there would be even less restrictions regarding the size and power consumption of the audio signal processing unit, which might result in reduced costs and/or increased performance of the blind source separation.
In all embodiments, the system preferably comprises a radio frequency transmitter for transmitting the processed audio signal output of the audio signal processing unit to a remote radio frequency receiver in order to provide the user's voice to another person. According to one embodiment, the radio frequency transmitter is integrated within the audio signal processing unit, i.e. the radio frequency transmitter may be either integrated within the device or it may be adapted to be worn behind the use's ear or at the user's body or it could be even located remote from the user. However, in an alternative embodiment, the radio frequency transmitter could be remote from the audio signal processing unit, in which case the processed audio signal output of the audio signal processing unit would be provided to a radio frequency transmitter by wires or via a radio frequency link, an inductive link or an infrared link. In this case the location of the audio signal processing unit and the radio frequency transmitter can be optimized independently form each other.
In all embodiments, the blind source separation algorithm preferably works in the frequency domain, i.e. the algorithm is simultaneously carried out in different frequency bands/bins and the outcomes of these bands/bins are combined in an appropriate way.
In all embodiments, the blind source separation algorithm preferably works with the assumption that the sources are statistically independent, i.e. that the user's voice is independent of the ambient sound.
These and further objects, features and advantages of the present invention will become apparent from the following description when taken in connection with the accompanying drawings which, for purposes of illustration only, show several embodiments in accordance with the present invention.
The system of
Preferably the hard shell is designed such that it provides for an acoustic attenuation, averaged over the audible frequency range, of at least 10 dB when inserted into the user's ear canal.
Rather than having a customized hard shell, earplug 10 may have a generic soft shell which adapts to the shape of the users outer ear and ear canal due to its elasticity.
In
The transmitter T1 of the earplug 10 is adapted to transmit audio signals from the earplug 10 to a remote receiver R2 via a radio link, while the receiver R1 of the earplug 10 is adapted to receive audio signals from a remote transmitter T2. The audio signals received by the receiver R1 are demodulated and then undergo signal processing in the audio signal processing unit 24 as input to the speaker S in order to provide remote audio signals to the user. Such remote audio signals could be the speech of another person picked up by a microphone whose output is sent to the remote transmitter T2 by wires or via, for example, a mobile telephone or mobile radio device.
The audio signals provided by the microphones M1 and M2 are passed as input to the blind source separation unit 26, in which a processed audio signal output for the transmitter T1 is produced, with the processed audio signal output consisting completely or at least essentially of the user's voice which has been separated from the ambient sound by action of the blind source separation algorithm carried out in the BSS unit 26. Such BSS signal processing utilizes the fact that the sound mixtures picked up by the microphone M1 which is oriented towards the environment and M2 which is oriented towards the ear canal 12, respectively, consist—due to the different orientation of the microphones M1 and M2—of essentially different mixtures of the ambient sound and the user's voice, which are different regarding amplitude ratio of these two signal contributions or sources (i.e. ambient sound on the one hand and user's voice on the other hand) and regarding phase difference of these two signal contributions of the mixture.
The output signal of the BSS unit 26 is transmitted via the transmitter T1 to the remote receiver R2 which usually will be connected to a remote speaker for presenting the user's voice to another person. The remote speaker and the remote microphone connected to the remote transmitter T2 could be part of an earpiece or an earplug worn by another person, which may be similar or identical to the earplug 10. Thereby in-the-ear hearing protection devices with integrated communication function can be achieved. Such a system could be used by any persons who need to communicate in a noisy environment, such as workers, soldiers, firemen, etc. However, the remote receiver R2 also might serve for communication via a mobile telephone or a mobile radio device.
In general, the remote receiver R2/transmitter T2 could be a part of an interface of a standard wireless communication device, such as a mobile telephone device or a mobile radio device Preferably, the wireless link between the transmitter T1/receiver R1 and the remote receiver R2/remote transmitter T2 is a Bluetooth link. The remote receiver R2/transmitter T2 then could be a part of the Bluetooth interface of a standard wireless communication device, such as a mobile telephone device or a mobile radio device.
According to
In
In
In
Although in the embodiments shown is FIGS. 1 to 4 the speaker S is shown as being integrated in the same earplug 10 as the microphones M1 and M2, in other embodiments the speaker S may be provided at an earplug other than the earplug at which the microphones M1 and M2 are provided.
According to a modification of the embodiment of
An alternative modification of the embodiment of
According to a modification of the embodiment of
While
While the examples discussed so far relate to an application of the communication system to hearing protection earplugs, the invention also may be applied to earmuffs serving as a hearing protection device or generally to any kind of headset, i.e. also to hearing devices which do not provide for a hearing protection function. For example, the headset may consist of a device with a customized or a generic shell which is designed such that it can be worn completely in the ear canal 12 (i.e. as a “CIC device”) and which serves exclusively communication purposes, for example, for security persons, policemen, firemen, etc., i.e. for persons who are exposed to a noisy environment, with the noise level, however, being below a threshold value which would require the use of hearing protection devices. Such CIC device generally could have the construction of the earplug of
In general, the device need not be designed as an earplug but it may rather be any kind of headset.
By “earplug” any device is meant which is to be worn in the ear canal or in the concha, so that also earmolds, i.e. devices to be worn in the concha rather than in the ear canal, are included.
In
The housing 50, in addition to the speaker S and the microphone M2, also includes an outwardly oriented microphone M1, an audio signal processing unit 24 with a BSS unit 26, a RF transmitter unit T1, a RF receiver unit R1 and a power supply 56.
The functionality of the electronic components may correspond, for example, to that of the embodiment of
In
According to the embodiment shown in
According to
In
The speaker S and the microphone M2 are located at one end of a sound tube 52 which extends with its free end 68 into the user's ear canal. As in the embodiment of
The housing 50A with the electronic components 24, T1, R1, and power supply 56 included therein forms a communication unit which, on the one hand, receives external audio signals from a communication device, which are reproduced via the speaker S to the user's ear. On the other hand, the communication device extracts the user's voice by blind source separation of audio signals from the microphones M1 and M2 in the BSS unit 26, with the separated audio signals corresponding to the user's voice and being transmitted to the communication device.
As an alternative to the embodiment shown in
A modification of the embodiment of
While various embodiments in accordance with the present invention have been shown and described, it is understood that the invention is not limited thereto, and is susceptible to numerous changes and modifications as known to those skilled in the art. Therefore, this invention is not limited to the details shown and described herein, and includes all such changes and modifications as encompassed by the scope of the appended claims.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 11316384 | Dec 2005 | US |
| Child | 11563975 | Nov 2006 | US |
