Patent Application
20240422482
The disclosure relates to hearing devices, e.g. headsets or headphones or hearing aids or ear protection devices or combinations thereof, in particular to the pick up of a user's own voice. In the present context, a ‘target signal’ is generally (unless otherwise stated) the user's own voice.
The hearing device comprises at least two (first and second) input transducers (e.g. microphones and/or vibration sensors) located at or in or near an ear of the user. The at least two, e.g. first and/or second, input transducers may be located at or in an ear canal of the user. The locations of the first and second input transducers in the hearing device when mounted on the user may be selected to provide different acoustic characteristics of the first and second electric input signals.
An estimate of the user's own voice may be provided as a liner combination of electric input signals from the at least two input transducers, e.g. a) in the time domain by linear filtering and subsequent summation of filtered first and second electric input signals, or b) in the (e.g. DFT-) filter bank domain to apply complex (beamformer) weights to each of the first and second electric input signals and subsequent summation of the thus weighted first and second electric input signals. The linear filters (e.g. FIR-filters) as well as the complex (beamformer) weights may be estimated based on an optimization procedure, e.g. comprising a Minimum Variance Distortionless Response (MVDR) procedure.
In a first aspect, a hearing device is provided by the present application.
A hearing device adapted to be worn by a user and for picking up sound containing the user's own voice is provided. The hearing device comprises an input unit comprising first and second input transducers for converting sound to first and second electric input signals, respectively, representing said sound. The hearing device further comprises a processor configured to receive said first and second electric input signals and to provide a combined signal as a linear combination of the first and second electric input signals, wherein the combined signal comprises an estimate of the user's own voice. The hearing device is configured to provide that said first and second input transducers are located on said user at first and second locations, when worn by said user; wherein said first and second locations are selected (arranged) to provide that said first and second electric signals exhibit substantially different directional responses for sound from the user's mouth, as well as from sound from sound sources located in an environment around the user.
Thereby an improved quality of an own voice estimate may be provided.
The term ‘substantially different directional responses’ may e.g. be exemplified by a free-field response of an input transducer, e.g. a microphone, from a given sound source and a response of an input transducer in a situation where an acoustic propagation path of sound from the sound source to a given input transducer is occluded by one or more objects between said sound source and said input transducer. The ‘substantially different directional responses’ may be present in at least one frequency range of the first and second electric input signals, in a multitude of frequency ranges or in all frequency ranges of operation of the hearing device.
Substantially different directionally responses can e.g. be observed for far field sources by measuring the directional response of each of the first and second transducers and drawing the polar plot of each microphone. This is a standard measuring method.
The first and second locations may be selected (arranged) to provide that the first and second electric signals exhibit substantially different directional responses for air-borne sound from the environment. The sound sources located in an environment around the user may be located relative to the user to provide that the user is located in an acoustic far-field relative to sound from such sound sources, e.g. more than 1 m from the user.
The hearing device may comprise a processor connected to the input unit. The processor may comprise one or more beamformers, each providing a spatially filtered signal by filtering and summing the first and second (or more) electric input signals, wherein one of the beamformers is an own voice beamformer and wherein the spatially filtered signal comprises an estimate of the user's own voice.
The hearing device may comprise an in the ear (ITE-) part (e.g. an earpiece) that provides an open fitting between the first and second locations. The ITE-part may be configured to allow air and sound to propagate between the first and second locations. The ITE-part may comprise a guiding element comprising one or more openings that allows air and sound to pass.
In a second aspect, a hearing device is provided by the present application.
A hearing device adapted to be worn by a user and for picking up sound containing the user's own voice is furthermore. The hearing device comprises
The term ‘all other environmental acoustic signals than that originating from the user's own voice’ is intended not to include ‘body noises, e.g. chewing, etc.
The different SNR-environments can be verified by a standard measurement. For each input transducer (e.g. microphone) the frequency response for own voice as well as a far field diffuse noise are measured. The difference between these two measurements will provide the relative SNR, and the difference between the relative SNRs of the two input transducers (e.g. microphones) will provide the ΔSNROV.
The SNR-threshold THSNR, may be larger than or equal to 5-10 dB, such as larger than or equal to 20-30 dB (e.g. in a low frequency region, below a threshold frequency). The SNR-threshold THSNR may be frequency dependent, e.g. larger at relatively low frequencies than at relatively high frequencies. The SNR threshold criterion may be fulfilled at least in some frequency bands, e.g. below a threshold frequency, e.g. below 4 kHz, such as below 3 kHz. The SNR threshold criterion may e.g. be fulfilled with ΔSNROV of 13-25 dB in low end (which is dominated by OV), and with ΔSNROV of 20-30 dB in a mid-frequency range (dominated by passive damping), where a threshold frequency between low and mid frequency range may be around 1 kHz.
The first and second locations may (further) be defined by properties of the respective first and second electric input signals being different in that they
The first and second locations may be defined by properties of the respective first and second electric input signals being further different in that they exhibit a difference in spectral shaping ΔS(f), e.g. distortion, of a sound source signal S, e.g. an own voice signal, ΔS(f)=ΔS(f)1−ΔS(f)2 being larger than a spectral shaping threshold THΔS, where f is frequency. The individual spectral shaping measures ΔS(f)i, i=1, 2, may e.g. be determined as a sum over frequency, e.g. at a predefined number of frequencies, of a difference between the original sound source signal and the signal provided at the input transducer in question. The difference in spectral shaping ΔS(f) may e.g. be determined as a difference between the two measures ΔS(f)i, i=1, 2, i.e. ΔS(f)=ΔS(f)1−ΔS(f)2.
The hearing device may comprise a processor connected to said input unit. The processor may comprise one or more beamformers each providing a spatially filtered signal by filtering and summing said first and second (or more) electric input signals. One of the beamformers may be an own voice beamformer and wherein said spatially filtered signal comprises an estimate of the user's own voice.
The hearing device may comprise an in the ear (ITE-)part that fully or partially (acoustically) blocks (occludes) the ear canal between the first and second locations. The ITE-part may comprise a seal that is configured to fit in the ear canal of the user to at least partially (acoustically) seal the first location from the second location. The difference in SNR and/or level and/or spectral characteristics may be enhanced by a partial or full sealing between the first and second locations (acoustic environments). In particular at low frequencies, e.g. below 4 kHz or below 2.5 kHz.
In a third aspect, a hearing device is provided by the present application.
A hearing device adapted to be worn by a user and for picking up sound containing the user's own voice is provided. The hearing device comprises
In the present context, the term ‘microphone’ (unless specifically stated) is intended to mean an acoustic to electric transducer that converts air-borne vibrations to an electric signal. In other words, ‘microphone’ is not intended to cover underwater microphones (‘hydrophones’) of acoustic transducers for picking up surface acoustic waves of vibrations in solid matter (e.g. bone conduction).
The vibration sensor may comprise or be constituted by one or more of a bone conduction microphone, an accelerometer, a strain gage vibration sensor.
The hearing device may be configured to provide that the first input transducer is located in an ear canal of the user (when the hearing device is worn by the user).
The hearing device may be configured to provide that the first input transducer is located at a mastoid part of the temporal bone of the user (when the hearing device is worn by the user).
The first input transducer may be located at an ear of the user, e.g. in a mastoid part of the temporal bone.
The hearing device may be configured to provide that the second input transducer is located at or in an ear canal of the user (when the hearing device is worn by the user).
The hearing device may be configured to provide that the second input transducer is located between an ear canal and the mouth of the user (when the hearing device is worn by the user).
The hearing device may comprise more than two input transducers, e.g. three or more. The more than two input transducers may comprise one or more of a microphone and/or a vibration sensor. Any of the more than two input transducers may be located at or in the ear canal, or between an ear canal and the mouth of the user, or on a bony part at the ear of the user, e.g. in a mastoid part of the temporal bone.
The hearing device may comprise a processor connected to the input unit. The processor may comprise one or more beamformers each providing a spatially filtered signal by filtering and summing first and second (or more) electric input signals. One of the beamformers may be an own voice beamformer wherein the spatially filtered signal comprises an estimate of the user's own voice.
In a fourth aspect, a hearing device is provided by the present application.
The hearing device is adapted to be worn by a user and for picking up sound containing the user's own voice. The hearing device comprises
If the first input transducer is not in (direct or indirect) contact with vibrating matter, a possible “air channel” (e.g. the airborne part of the transmission channel) from the vibrating matter (e.g. bone/tissue) to the first input transducer may e.g. be between 0 and 10 mm.
The term ‘primarily originates from’ may in the present context be taken to mean ‘to more than 50%’, e.g. ‘to more than 70%’, such as ‘to more than 90% originates from’.
The hearing device may comprise an in the ear (ITE-)part that fully or partially (acoustically) blocks (occludes) the ear canal between the first and second acoustic environments.
The term ‘acoustically different from each other’ may in the present context be taken to mean, that the first and second acoustic environments are separated by one or more objects that prohibit or diminish exchange of acoustic energy between them.
The term ‘acoustically different from each other’ may in the present context be taken to mean, e.g. ‘at least partially isolated from each other’, e.g. in that the two acoustic environments are separated by an object, e.g. comprising a seal, for attenuating acoustic transmission between the first and second acoustic environments.
The term ‘acoustically different from each other’ may in the present context be taken to mean that a ‘Transition region’ between the first and second acoustic environments (cf. e.g.
The hearing device may comprise a processor connected to the input unit.
The processor may be configured to receive the first and second electric input signals and to provide a combined signal as a linear combination of the first and second electric input signals, wherein the combined signal comprises an estimate of the user's own voice.
The processor may comprise one or more beamformers each providing a spatially filtered signal by filtering and summing the first and second (or more) electric input signals. One of the beamformers may be an own voice beamformer and wherein the spatially filtered signal comprises an estimate of the user's own voice.
The hearing device may be configured to provide a transitional region between the first and second acoustic environments. The hearing device may comprise an object which fully or partially occludes the ear canal (e.g. an ITE-part (e.g. an earpiece) when the hearing device is worn by the user. The object may e.g. comprise a sealing element. The sealing element may be partially open (i.e. e.g. comprise one or more openings allowing a certain exchange of air and sound with the environment to decrease a sense of occlusion by the user).
In a fifth aspect, a hearing device is provided by the present application.
A hearing device adapted to be worn by a user and for picking up sound containing the user's own voice is provided. The hearing device may comprise.
The hearing device may comprise a processor connected to said input unit. The processor may comprise one or more beamformers each providing a spatially filtered signal by filtering and summing said first and second (or more) electric input signals. One of the beamformers may be an own voice beamformer and wherein said spatially filtered signal comprises an estimate of the user's own voice.
The hearing device may be configured to provide that the second input transducer is capable of picking up predominantly airborne sound. The airborne sound may include sound from the environment, including from the user's mouth.
In a sixth aspect of the present application, A hearing device adapted to be worn by a user and for picking up sound containing the user's own voice, the hearing device comprising an input unit comprising first and second input transducers for converting sound to first and second electric input signals, respectively, representing said sound;
The spectral distortion of the second electric input signal may be smaller than the spectral distortion of the first electric input signal, at least in a frequency range comprising the user's own voice. The difference in spectral responses between the first electric input signal and the second electric signal may e.g. be measured as a difference between the first and second electric input signal at one or more frequencies, e.g. at one or more frequencies which are relevant for speech, e.g. at 1 KHz and/or 2 kHz, or at (one or more of, such as all, of) 100 Hz, 500 Hz, 1 kHz, 2 kHz, and 4 kHz, etc. (possibly averaged over time, e.g. 1 s or more). If the difference between the first and second electric input signals at one or more, e.g. at least two, frequencies (which are relevant for speech) are larger than a threshold difference, the first and second electric signals are taken to exhibit substantially different spectral responses for sound from the user's mouth, i.e. e.g. if the difference between ΔOV(k1)=MAG(IN1OV(K1))−MAG(IN2OV(k1)) and ΔOV(k2)=MAG(IN1OV(k2))−MAG(IN2OV(k2)) is larger than a threshold value, e.g. larger than 3 dB, such as larger than 6 dB, where k1 and k2 are different frequencies spanning a frequency range, e.g. between 100 Hz and 2.5 kHz, or between 1 kHz and 2 kHz, and IN1OV, IN2OV are the first and second electric input signals, when the user speaks, and MAG is magnitude.
The first location may be selected to exploit conduction of sound from the user's mouth through the head (skull) of the user. Conduction of sound from the user's mouth through the head of the user may e.g. be constituted by or comprise bone conduction (e.g. in combination with skin and/or tissue (flesh). The first input transducer may comprise or be constituted by a vibration sensor, e.g. an accelerometer.
The second location may be selected to exploit air conduction of sound from the user's mouth. Conduction of sound from the user's mouth to the second location be constituted by or comprise propagation through air. The second input transducer may comprise or be constituted by a microphone.
In the present context a ‘microphone’ is taken to mean an input transducer that is specifically configured to convert vibration of sound in air to an electric signal representative thereof.
The hearing device may comprise an in the ear (ITE-)part that fully or partially (acoustically) blocks (occludes) the ear canal between the first and second locations.
In a seventh aspect, a hearing device is provided by the present application.
A hearing device configured to be located at or in an ear of a user, and to pick up sound containing the user's own voice may furthermore be provided. The hearing device may comprise:
The application may comprise a transmitter configured to wirelessly transmit the estimate of the user's own voice to an external device or system.
The application may comprise a voice control interface configured to control functionality of the hearing device based on the estimate of the user's own voice. The application may e.g. comprise a keyword detector, e.g. wake-word detector and/or a command word detector.
It is intended that the following features can be combined with a hearing device according to any of the abovementioned aspects.
The hearing device may be configured to provide that the first input transducer may be located in an ear canal of the user facing the eardrum and the second input transducer may be located at or in the ear canal of the user facing the environment. The first and second input transducers may be located in an ITE-part adapted for being located fully or partially in the ear canal of the user.
The hearing device according may comprise an output unit comprising an output transducer, e.g. a loudspeaker or a vibrator, for converting an electric signal representing sound to an acoustic signal representing said sound.
The hearing device may be configured to provide that the output transducer plays into the (or a) first acoustic environment.
The hearing device may be configured to provide that the output transducer is located in the hearing device between the first and second input transducers.
The hearing device may comprise a housing adapted to be located at or in an ear (e.g. at or in an ear canal) of the user, whereon or wherein said first input transducer and/or said output transducer is/are supported or located.
The hearing device may comprise an earpiece wherein said earpiece (e.g. a housing of the earpiece) is configured to contribute to an at least partial sealing between (the) first and second acoustic environments and/or (the) first and second locations.
The hearing device (e.g. the housing or the earpiece) may comprise a sealing element configured to contribute to the at least partial sealing between (the) first and second acoustic environments and/or (the) first and second locations.
The hearing device may comprise a transmitter, e.g. a wireless transmitter, configured to transmit the estimate of the user's own voice or a processed version thereof to another device or system, e.g. to a telephone or a computer.
The hearing device may comprise a keyword detector or an own voice detector configured to receive the estimate of the user's own voice or a processed version thereof. This may be used to detect a keyword (e.g. a wake-word) for a voice-controlled application to ensure that a particular spoken keyword originates from the wearer of the hearing device.
The hearing device may comprise a processor for processing the first and second electric input signals and providing a processed signal. The processor may be configured to apply one or more processing algorithms to processing the first and second electric input signals, or signals derived therefrom, e.g. an own voice signal or a beamformed signal representing sound from the environment, e.g. voice (e.g. from a speaker, e.g. a communication partner).
An estimate of the user's own voice may be provided as a liner combination of electric input signals from the at least two input transducers, e.g. a) in the time domain by linear filtering and subsequent summation of filtered first and second electric input signals, or b) in the (e.g. DFT-) filter bank domain to apply complex (beamformer) weights to each of the first and second electric input signals and subsequent summation of the thus weighted first and second electric input signals. The linear filters (e.g. FIR-filters) as well as the complex (beamformer) weights may be estimated based on an optimization procedure, e.g. comprising a Minimum Variance Distortionless Response (MVDR) procedure.
The processor may comprise a beamformer block configured to provide one or more beamformers each being configured to filter the first and second electric input signals, and to provide a spatially filtered (beamformed) signal. The one or more beamformers may comprise an own voice beamformer comprising predetermined or adaptively updated own voice filter weights, wherein an estimate of the user's own voice is provided in dependence on said own voice filter weights and said first and second electric input signals.
The processor may be configured to receive the first and second electric input signals and to provide a combined signal as a linear combination of the first and second electric input signals, wherein the combined signal comprises an estimate of the user's own voice.
The hearing device may comprise one or more further input transducers for providing one or more further electric signals representing sound in the environment of the user. The hearing device may comprise at least one of said one or more further input transducers is located off-line compared to said first and second input transducers.
The first and second input transducer may comprise at least one microphone. The first and second input transducer may comprise at least one vibration sensor, e.g. an accelerometer.
The hearing device may be constituted by or comprise a hearing aid, a headset, an earphone, an ear protection device or a combination thereof.
The hearing device may be constituted by or comprise an air-conduction type hearing aid, a bone-conduction type hearing aid, a cochlear implant type hearing aid, or a combination thereof.
The hearing device or a system comprising a hearing device as described above, in the section ‘detailed description of drawings’ or in the claims below may comprise first and second earpieces, adapted for being located at or in first and second ears, respectively, of the user. Each of the first and second earpieces may comprise at least two input transducers, e.g. microphones. Each of the first and second earpieces may each comprise antenna and transceiver circuitry configured to allow an exchange of data, e.g. including audio data, between them.
The input unit may comprise respective analogue to digital converters and/or analysis filter bank as appropriate for the application in question.
An input transducer may be constituted by or comprise a microphone (for sensing airborne sound), or a vibration sensor (e.g. for sensing bone-conducted vibration), e.g. an accelerometer. The first and second input transducer may comprise at least one microphone. The first and second input transducers may be microphones. The second input transducer may e.g. be constituted by or comprise a microphone. The first input transducer may e.g. be constituted by or comprise a vibration sensor (e.g. an accelerometer). The first and/or second input transducer may e.g. be located outside the ear canal, e.g. in or at Pinna, or behind an ear (Pinna). The first and/or second input transducer may e.g. be located at or in the ear canal. The second input transducer may e.g. be located between an ear canal opening and the user's mouth. The first and second input transducers may e.g. be located in a horizontal plane (when the user is wearing the hearing device and is in an upright position). The first and second input transducers may e.g. be located along a line following an ear canal of the user.
The first and second input transducers may comprise an eardrum-facing input transducer and an environment-facing input transducer. The first input transducer may be located in an ear canal of the user facing the eardrum and the second input transducer may be located at or in the ear canal of the user facing the environment. In the present context, the term ‘an input transducer facing the environment’ is intended to mean that it mainly receives acoustically transmitted sound from the environment (e.g. in that it has an inlet directed towards the environment (e.g. away from the ear drum, e.g. towards the mouth of the user). Likewise, the term ‘an input transducer facing the eardrum’ is intended to mean that it mainly receives sound from a (residual) volume close to the eardrum, e.g. in that it has an inlet directed towards the ear drum. Such location will particularly expose the first input transducer to bone conducted sound from the skull of the user (mainly due to the user's own voice). The so-called residual volume may constitute or form part of a first acoustic environment, or to characterize a first location of the first input transducer.
The hearing device may comprise one or more further input transducers for providing one or more electric signals representing sound. The one or more further input transducers may located in the first acoustic environment or at the first location and/or in the second acoustic environment or the second location. The one or more further input transducers may be located at or in the ear canal or in pinna or outside pinna. The one or more further transducers may e.g. be located on a support structure (e.g. a boom arm) extending towards the user's mouth.
At least one of the one or more further input transducers may be located off-line compared to said first and second input transducers. The location of the first and second input transducers in the hearing device define a first (microphone) axis. The first (microphone) axis may be substantially parallel to a first axis combining the first and second ear canals (or eardrums) of the user (or substantially parallel to a longitudinal axis of the ear canal (e.g. from the ear canal opening towards the eardrum). The at least one of the one or more further input transducers may be located in a direction of the first axis. However, the at least one of the one or more further input transducers may be located in a direction from the ear canal opening towards the mouth of the user (and thus (possibly) off-line relative to the first and second input transducers). The location of the second and at least one of the one or more further input transducers in the hearing device may define a second (microphone) axis substantially in a direction towards the mouth of the user.
The hearing device my comprise an output unit comprising an output transducer, e.g. a loudspeaker, for converting an electric signal representing sound to an acoustic signal representing said sound. The output unit may comprise a digital to analogue converter and/or a synthesis filter bank as appropriate for the application in question. The output transducer may comprise a loudspeaker, a vibrator of a bone conduction hearings device and/or a multi electrode array of a cochlear implant type hearing device. The output transducer may be arranged in the hearing device at a first location configured to play into the first acoustic environment. The output transducer may be located in the hearing device between the first and second input transducers.
The hearing device may comprise an ITE part adapted for being fully or partially inserted int an ear canal of the user, e.g. an earpiece. The ITE-part/earpiece may e.g. comprise a housing, adapted to be located at or in an ear of the user, whereon or wherein said first input transducer and/or said output transducer is/are supported or located.
The ITE-part/earpiece may be configured to contribute to an at least partial sealing between the first and second acoustic environments or the first and second locations. The earpiece may be configured to constitute an at least partial sealing between the first and second acoustic environments. The hearing device, e.g. the ITE-part/earpiece, may comprise a sealing element configured to contribute to said at least partial sealing between the first and second acoustic environments.
The hearing device may comprise a receiver, e.g. a wireless receiver, for receiving a signal representative of sound from another device or system. The hearing device may comprise a transmitter, e.g. a wireless transmitter, configured to transmit a signal picked up by said first and second input transducers or a processed version thereof (e.g. the user's own voice) to another device or system. The hearing device may comprise antenna and transceiver circuitry configured to establish a wireless audio link between the hearing device and another device, e.g. a telephone or a computer, The wireless audio link may be based on Bluetooth, e.g. Bluetooth Low Energy, or similar technology.
The hearing device may comprise a processor for processing said first and second electric input signals and providing a processed signal. The processed signal may be adapted to compensate for the user's hearing impairment. The processed signal may be presented to the user via an output transducer.
The processor may comprise a beamformer block configured to provide one or more beamformers each being configured to filter said first and second electric input signals, and to provide a spatially filtered (beamformed) signal. The one or more beamformers may comprise an own voice beamformer comprising predetermined or adaptively updated own voice filter weights, wherein an estimate of the user's own voice is provided in dependence on the own voice filter weights and the first and second (or more) electric input signals. The one or more beamformers comprises an MVDR beamformer (MVDR =minimum variance distortionless response).
A hearing device or a hearing system may comprise first and second earpieces, adapted for being located at or in first and second ears, respectively, of the user. Each of the first and second hearing devices may comprise at least two input transducers, e.g. microphones. The first and second earpieces may comprise antenna and transceiver circuitry configured to allow an exchange of data, e.g. including audio data, between them.
A hearing device may comprise a hearing aid, e.g. a hearing instrument adapted for being located at the ear or fully or partially in the ear canal of a user, a headset, an earphone, an ear protection device or a combination thereof. A hearing device may be constituted by or comprise an air-conduction type hearing aid, a bone-conduction type hearing aid, a cochlear implant type hearing aid, or a combination thereof. The hearing device (or hearing devices of a binaural hearing system) may e.g. comprise or be implemented in connection with a carrier adapted to be worn on the head of the user, e.g. a spectacle frame.
The hearing device, e.g. a hearing aid, may be adapted to provide a frequency dependent gain and/or a level dependent compression and/or a transposition (with or without frequency compression) of one or more frequency ranges to one or more other frequency ranges, e.g. to compensate for a hearing impairment of a user. The hearing device may comprise a signal processor for enhancing the input signals and providing a processed output signal, e.g. being adapted to compensate for a hearing impairment of a user, e.g. the user of the hearing device.
The hearing device, e.g. a hearing aid or a headset, etc., may comprise an output unit for providing a stimulus perceived by the user as an acoustic signal based on a processed electric signal. The output unit may comprise a number of electrodes of a cochlear implant (for a CI type hearing aid) or a vibrator of a bone conducting hearing aid. The output unit may comprise an output transducer. The output transducer may comprise a receiver (loudspeaker) for providing the stimulus as an acoustic signal to the user (e.g. in an acoustic (air conduction based) hearing aid). The output transducer may comprise a vibrator for providing the stimulus as mechanical vibration of a skull bone to the user (e.g. in a bone-attached or bone-anchored hearing aid).
The hearing device comprises an input unit for providing an electric input signal representing sound. The input unit may comprise an input transducer, e.g. a microphone or a vibration sensor, for converting an input sound to an electric input signal.
The hearing device may comprise a directional microphone system adapted to spatially filter sounds from the environment, and thereby e.g. to enhance a target acoustic source among a multitude of acoustic sources in the local environment of the user wearing the hearing device (or suppress signal(s) from one or more a specific directions). The directional system is adapted to detect (such as adaptively detect) from which direction a particular part of the microphone signal originates (e.g. noise or target parts). This can be achieved in various different ways as e.g. described in the prior art. In hearing devices, a microphone array beamformer is often used for spatially attenuating background noise sources and/or (possibly simultaneously) to provide target signal (e.g. from a communication partner or the user him-or herself) with an improved signal quality. Many beamformer variants can be found in literature. The minimum variance distortionless response (MVDR) beamformer is widely used in microphone array signal processing. Ideally the MVDR beamformer keeps the signals from the target direction (also referred to as the look direction) unchanged, while attenuating sound signals from other directions maximally. The generalized sidelobe canceller (GSC) structure is an equivalent representation of the MVDR beamformer offering computational and numerical advantages over a direct implementation in its original form.
The hearing device may comprise a memory. The memory may be configured to store one or more sets of (e.g. pre-determined, or updated during use) beamformer weights, or, correspondingly, filter coefficients of linear filters, e.g. FIR-filters, sec e.g.
The hearing device may comprise antenna and transceiver circuitry (e.g. a wireless receiver) for wirelessly receiving a direct electric input signal from another device, e.g. from an entertainment device (e.g. a TV-set), a communication device, a wireless microphone, or another hearing device, e.g. a bearing aid. The direct electric input signal may represent or comprise an audio signal and/or a control signal and/or an information signal.
In general, a wireless link established by antenna and transceiver circuitry of the hearing device can be of any type. The wireless link may be used under power constraints, e.g. in that a head set or a hearing device is constituted by or comprise a portable (typically battery driven) device. The wireless link may be a link based on near-field communication, e.g. an inductive link based on an inductive coupling between antenna coils of transmitter and receiver parts. The wireless link may be based on far-field, electromagnetic radiation. The wireless link may e.g. be configured to transfer an electromagnetic signal in the radio frequency range (3 kHz to 300 GHz). The wireless link may e.g. be configured to transfer an electromagnetic signal in a frequency range of light (e.g. infrared light 300 GHz to 430 THz, or visible light, e.g. 430 THz to 770 THz). The wireless link based on far-field, electromagnetic radiation may e.g. be based on Bluetooth technology (e.g. Bluetooth Low-Energy technology).
The hearing device may have a maximum outer dimension of the order of or less than 0.15 m (e.g. a headset). The hearing device may have a maximum outer dimension of the order of or less than 0.04 m (e.g. a hearing instrument).
The hearing device may be or form part of a portable (i.e. configured to be wearable) device, e.g. a device comprising a local energy source, e.g. a battery, e.g. a rechargeable battery. The hearing device may e.g. be a low weight, easily wearable, device, e.g. having a total weight less than 100 g, e.g. less than 20 g.
The hearing device, e.g. a hearing aid, may comprise a forward or signal path between an input unit (e.g. an input transducer, such as a microphone or a microphone system and/or direct electric input (e.g. a wireless receiver)) and an output unit, e.g. an output transducer. The signal processor may be located in the forward path. The signal processor may be adapted to provide a frequency dependent gain according to a user's particular needs. The hearing device may comprise an analysis path comprising functional components for analyzing the input signal (e.g. determining a level, a modulation, a type of signal, an acoustic feedback estimate, etc.). Some or all signal processing of the analysis path and/or the signal path may be conducted in the frequency domain. Some or all signal processing of the analysis path and/or the signal path may be conducted in the time domain.
The hearing device may comprise an analogue-to-digital (AD) converter to digitize an analogue input (e.g. from an input transducer, such as a microphone) with a predefined sampling rate, e.g. 20 kHz. The hearing device may comprise a digital-to-analogue (DA) converter to convert a digital signal to an analogue output signal, e.g. for being presented to a user via an output transducer.
The hearing device, e.g. the input unit, and or the antenna and transceiver circuitry comprise(s) a TF-conversion unit for providing a time-frequency representation of an input signal. The time-frequency representation may comprise an array or map of corresponding complex or real values of the signal in question in a particular time and frequency range. The TF conversion unit may comprise a filter bank for filtering a (time varying) input signal and providing a number of (time varying) output signals each comprising a distinct frequency range of the input signal. The TF conversion unit may comprise a Fourier transformation unit for converting a time variant input signal to a (time variant) signal in the (time-)frequency domain. The frequency range considered by the hearing aid from a minimum frequency fmin to a maximum frequency fmax may comprise a part of the typical human audible frequency range from 20 Hz to 20 kHz, e.g. a part of the range from 20 Hz to 12 kHz. Typically, a sample rate fs is larger than or equal to twice the maximum frequency fmax, fs≥2fmax. A signal of the forward and/or analysis path of the hearing device may be split into a number NI of frequency bands (e.g. of uniform width), where NI is e.g. larger than 5, such as larger than 10, such as larger than 50, such as larger than 100, such as larger than 500, at least some of which are processed individually. The hearing device may be adapted to process a signal of the forward and/or analysis path in a number NP of different frequency channels (NP≤NI). The frequency channels may be uniform or non-uniform in width (e.g. increasing in width with frequency), overlapping or non-overlapping.
The hearing device, e.g. a hearing aid, may be configured to operate in different modes, e.g. a normal mode and one or more specific modes, e.g. selectable by a user, or automatically selectable. A mode of operation may be optimized to a specific acoustic situation or environment. A mode of operation may include a low-power mode, where functionality of the hearing device is reduced (e.g. to save power), e.g. to disable wireless communication, and/or to disable specific features of the hearing device.
The hearing device may comprise a number of detectors configured to provide status signals relating to a current physical environment of the hearing device (e.g. the current acoustic environment), and/or to a current state of the user wearing the hearing device, and/or to a current state or mode of operation of the hearing device. Alternatively. or additionally, one or more detectors may form part of an external device in communication (e.g. wirelessly) with the hearing device. An external device may e.g. comprise another hearing device, a remote control, and audio delivery device, a telephone (e.g. a smartphone), an external sensor, etc.
One or more of the number of detectors may operate on the full band signal (time domain). One or more of the number of detectors may operate on band split signals ((time-) frequency domain), e.g. in a limited number of frequency bands.
The number of detectors may comprise a level detector for estimating a current level of a signal of the forward path. The detector may be configured to decide whether the current level of a signal of the forward path is above or below a given (L-)threshold value. The level detector operates on the full band signal (time domain). The level detector operates on band split signals ((time-)frequency domain).
The hearing device may comprise a voice activity detector (VAD) for estimating whether or not (or with what probability) an input signal comprises a voice signal (at a given point in time). A voice signal is in the present context taken to include a speech signal from a human being. It may also include other forms of utterances generated by the human speech system (e.g. singing). The voice activity detector unit is adapted to classify a current acoustic environment of the user as a VOICE or NO-VOICE environment. This has the advantage that time segments of the electric microphone signal comprising human utterances (e.g. speech) in the user's environment can be identified, and thus separated from time segments only (or mainly) comprising other sound sources (e.g. artificially generated noise). The voice activity detector may be adapted to detect as a VOICE also the user's own voice. Alternatively, the voice activity detector may be adapted to exclude a user's own voice from the detection of a VOICE.
The hearing device may comprise an own voice detector for estimating whether or not (or with what probability) a given input sound (e.g. a voice, e.g. speech) originates from the voice of the user of the system. A microphone system of the hearing device may be adapted to be able to differentiate between a user's own voice and another person's voice and possibly from NON-voice sounds.
The number of detectors may comprise a movement detector, e.g. a vibration sensor, e.g. an acceleration sensor. The movement detector is configured to detect movement of the user's facial muscles and/or bones, e.g. due to speech or chewing (e.g. jaw movement) and to provide a detector signal indicative thereof.
The hearing device may comprise a classification unit configured to classify the current situation based on input signals from (at least some of) the detectors, and possibly other inputs as well. In the present context ‘a current situation’ is taken to be defined by one or more of
The classification unit may be based on or comprise a neural network, e.g. a trained neural network.
The hearing device may further comprise other relevant functionality for the application in question, e.g. compression, feedback control, noise reduction, etc.
The hearing device may comprise a hearing instrument, e.g. a hearing instrument adapted for being located at the ear or fully or partially in the ear canal of a user. The hearing device may e.g. comprise a headset, an earphone, an ear protection device or a combination thereof. The headset may be adapted to be worn by a user and comprise an input transducer (e.g.
microphone) to (e.g. wireless) transmitter path and a (e.g. wireless) receiver to output transducer (e.g. loudspeaker) path. The headset may be adapted to pick up a user's own voice and transmit it via the transmitter to a remote device or system. Likewise, the headset may be adapted to receive a sound signal from a remote device or system and present it to the user via the output transducer.
In an aspect, use of a hearing device as described above, in the ‘detailed description of embodiments’ and in the claims, is moreover provided. Use may be provided in a system comprising audio distribution. Use may be provided in a system comprising one or more hearing devices (e.g. hearing instruments), headsets, ear phones, active ear protection systems, etc., e.g. in handsfree telephone systems, teleconferencing systems (e.g. including a speakerphone), public address systems, karaoke systems, classroom amplification systems, etc.
In an aspect, a method of operating a hearing device adapted to be worn by a user and for picking up sound containing the user's own voice is furthermore provided by the present application. The method may comprise.
The method may further comprise
In a further aspect, a method of operating a hearing device adapted to be worn by a user and for picking up sound containing the user's own voice is provided. The method may comprise.
In a further aspect, a method of operating a hearing device adapted to be worn by a user and for picking up sound containing the user's own voice is provided. The method may comprise.
In a further aspect, a method of operating a hearing device adapted to be worn by a user and for picking up sound containing the user's own voice, the hearing device comprising an ear piece adapted for being located at least partially in an ear canal of the user, is provided. The method may comprise
In a further aspect, a method of operating a hearing device adapted to be worn by a user and for picking up sound containing the user's own voice is provided. The method may comprise converting sound to first and second electric input signals, respectively, representing said sound using first and second input transducers;
In a further aspect, a method of operating a hearing device adapted to be located at or in an ear of a user, and to pick up sound containing the user's own voice may furthermore be provided. The method may comprise:
It is intended that some or all of the structural features of the device described above, in the ‘detailed description of embodiments’ or in the claims can be combined with embodiments of the method(s), when appropriately substituted by a corresponding process and vice versa. Embodiments of the method(s) have the same advantages as the corresponding devices.
The method may e.g. comprise
The method may e.g. comprise
In a further aspect, a hearing system comprising a hearing device as described above, in the ‘detailed description of embodiments’, and in the claims, AND an auxiliary device is moreover provided.
The hearing system is adapted to establish a communication link between the hearing device and the auxiliary device to provide that information (e.g. control and status signals, possibly audio signals) can be exchanged or forwarded from one to the other.
The auxiliary device may comprise a remote control, a smartphone, or other portable or wearable electronic device, such as a smartwatch or the like.
The auxiliary device may be constituted by or comprise a remote control for controlling functionality and operation of the hearing device(s). The function of a remote control is implemented in a smartphone, the smartphone possibly running an APP allowing to control the functionality of the hearing device via the smartphone (the hearing device(s) comprising an appropriate wireless interface to the smartphone, e.g. based on Bluetooth or some other standardized or proprietary scheme).
The auxiliary device may be constituted by or comprise an audio gateway device adapted for receiving a multitude of audio signals (e.g. from an entertainment device, e.g. a TV or a music player, a telephone apparatus, e.g. a mobile telephone or a computer, e.g. a PC) and adapted for selecting and/or combining an appropriate one of the received audio signals (or combination of signals) for transmission to the hearing device.
The auxiliary device may be constituted by or comprise another hearing device. The hearing system may comprise two hearing devices, e.g. hearing aids, adapted to implement a binaural hearing system, e.g. a binaural hearing aid system.
The auxiliary device may comprise a speakerphone (comprising a number of input transducers and a number of output transducers, e.g. for use in an audio conference situation), e.g. comprising a beamformer filtering unit, e.g. providing multiple beamforming capabilities.
The aspects of the disclosure may be best understood from the following detailed description taken in conjunction with the accompanying figures. The figures are schematic and simplified for clarity, and they just show details to improve the understanding of the claims, while other details are left out. Throughout, the same reference numerals are used for identical or corresponding parts. The individual features of each aspect may each be combined with any or all features of the other aspects. These and other aspects, features and/or technical effect will be apparent from and elucidated with reference to the illustrations described hereinafter in which:
The figures are schematic and simplified for clarity, and they just show details which are essential to the understanding of the disclosure, while other details are left out. Throughout, the same reference signs are used for identical or corresponding parts.
Further scope of applicability of the present disclosure will become apparent from the detailed description given hereinafter. However, it should be understood that the detailed description and specific examples, while indicating preferred embodiments of the disclosure, are given by way of illustration only. Other embodiments may become apparent to those skilled in the art from the following detailed description.
The detailed description set forth below in connection with the appended drawings is intended as a description of various configurations. The detailed description includes specific details for the purpose of providing a thorough understanding of various concepts. However, it will be apparent to those skilled in the art that these concepts may be practiced without these specific details. Several aspects of the apparatus and methods are described by various blocks, functional units, modules, components, circuits, steps, processes, algorithms, etc. (collectively referred to as “elements”). Depending upon particular application, design constraints or other reasons, these elements may be implemented using electronic hardware, computer program, or any combination thereof.
The electronic hardware may include micro-electronic-mechanical systems (MEMS), integrated circuits (e.g. application specific), microprocessors, microcontrollers, digital signal processors (DSPs), field programmable gate arrays (FPGAs), programmable logic devices (PLDs), gated logic, discrete hardware circuits, printed circuit boards (PCB) (e.g. flexible PCBs), and other suitable hardware configured to perform the various functionality described throughout this disclosure, e.g. sensors, e.g. for sensing and/or registering physical properties of the environment, the device, the user, etc. Computer program shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.
The disclosure relates to hearing devices, e.g. headsets or headphones or hearing aids ear protection devices or combinations thereof, in particular to the pick up of a user's own voice. In the present context, a ‘target signal’ is generally (unless otherwise stated) the user's own voice.
In the present application, an own voice capturing system that captures the voice of the user and transfers it to an application (e.g. locally in the hearing device or to in external device or system) is provided. The capturing is achieved by using at least two input transducers, e.g.
microphones. The conventional use of the at least two microphones is to use spatial filtering (e.g. beamforming) or source separation (e.g. BSS) on the external sounds from the environment in order to separate unwanted acoustical signals (‘noise’) from wanted acoustical signals. In a ‘normal mode’ hearing aid application, target signals are typically arriving from the frontal direction (e.g. to pick up the voice of a communication partner). In a headset application (or in a hearing aid with a telephone mode or a voice interface), target signals are typically arriving from a direction towards the mouth of the user (to pick up the user's own voice).
By placing input transducers (e.g. microphones) of the hearing device in or at the ear canal of the user of the hearing device and e.g. by (partially) sealing the ear canal to the outside offers some interesting opportunities, e.g. for own voice estimation. The input transducers (e.g. microphones) inside the ear canal will pick up own voice signals (OV). The quality of the signal (OV) will depend primarily of the seal of the ear canal. The present application provides a combination of in-ear input transducers (e.g. microphones or vibration sensors) with standard input transducers (e.g. microphones) located outside the (possibly) sealed off part of the ear canal, e.g. completely outside the ear canal (e.g. at or in or behind pinna or further towards the user's mouth). The use of binaural in-ear microphones may also improve signal quality. The two types of locations of the input transducers provide wanted acoustical signals (own voice) that are highly correlated. In the sealed use case, the two types of input transducers (e.g. microphones, or an (external) microphone and an (internal) vibration sensor) also provide noise signals that tend to be uncorrelated.
According to the present disclosure, an estimate of the user's own voice is provided from a linear combination of signals created by input transducers located in different acoustic environments (e.g. relying on bone conduction and air conduction, respectively). The possible symmetry of binaural detection (with regard to the location of the mouth) using input transducers at both ears of the user could greatly aid the quality of own-voice estimation. The environmental noise (unwanted noise) will not exhibit these symmetries. Hence an algorithm may distinguish wanted from unwanted acoustical signals by investigating correlation between the two sources, experienced by input transducers located in two different acoustic environments e.g. located outside and inside an ear canal of the user, e.g. outside and inside a seal of the ear canal. The present disclosure may e.g. rely on standard beamforming procedures, e.g. the MVDR formalism, to determine linear filters or beamformer weights to extract the user's voice from the electric input signals.
The hearing device comprises at least two (first and second) input transducers (e.g. microphones or vibration sensors) located at or in or near an ear of the user. The first and/or second input transducers may be located at or in an ear canal of the user, or elsewhere on the head of the user. The first and second input transducers provide first and second electric input signals, respectively. The following
In the embodiments of
In the embodiments of
In the embodiments of
An ear canal opening may be used as a reference point for the location of the input transducers (e.g. microphones) of the hearing device, e.g. the first input transducer may be located on the internal side of the ear canal opening (and or on a bony part of the head), termed ‘a 1st acoustic environment’ in
The ear canal opening is in the present context taken to be defined by (e.g. a center point of) a typically oval cross section where the ear canal joins the outer ear (pinna), cf. e.g.
It is the intention that the configurations of
The embodiments of a hearing device (HD) illustrated in
In the embodiments of
In
The loudspeaker (SPK) is located in the earpiece (HD) to play sound towards the eardrum into the residual volume (‘Ear canal (residual volume)’). A loudspeaker outlet (‘SPK outlet’) directs the sound towards the eardrum. Instead of (or in addition to the loudspeaker), the hearing device (HD) may comprise a vibrator for transferring stimuli as vibrations of skull-bone or a multi-electrode array for electric stimulation of the hearing nerve.
In the embodiments of
The first microphone (M1) may be substituted by a vibration sensor e.g. located at the same position as the first microphone, or in direct or indirect contact with the skin in the soft or bony part of the ear canal (the vibration sensor, e.g. comprising an accelerometer, being particularly adapted to pick up bone conducted sound). In another embodiment, the first microphone (M1) may be substituted (or supplemented) by a vibration sensor located outside the ear canal at a location suited to pick up bone conducted sound from the user's mouth, e.g. at an ear of the user in a mastoid part of the temporal bone, or e.g. near the bony part of the ear canal, cf. e.g.
In the embodiment of
The embodiment of a hearing device shown in
In an embodiment, the earpiece has only two microphones (M1, M2), e.g. located as outlined in
The second microphone (M2) may in another embodiment be located in the ear canal away from its opening (‘Ear canal opening’) in a direction towards the eardrum, e.g. confined to the soft (non-bony) part of the ear canal, e.g. less than 10 mm from the opening (cf. e.g.
In general, the second microphone (M2) may be located a distance away from the first microphone (M1), e.g. in the same physical part of the hearing device (e.g. an earpiece) as the first microphone (as e.g. shown in
The hearing device of
The distance between the first and second input transducers, e.g. microphones (M1, M2), may be in the range from 5 mm to 100 mm, such as between 10 mm and 50 mm, or between 10 mm and 30 mm.
The hearing device (HD) may comprise three or more input transducers, e.g. microphones, e.g. one or more located on a boom arm pointing towards the user's mouth (such microphone(s) being e.g. located in the 2nd acoustic environment). Two of the at least three microphones may be located around and just outside, respectively, the ear canal opening, e.g. 10-20 mm outside (in the 2nd acoustic environment). Two of the at least three microphones may e.g. be located in the ear canal relatively close to the ear drum, e.g. in the 1st or 2nd acoustic environment.
The first microphone may be located at or in the ear canal. The first microphone may be located closer to the ear drum than the second microphone. The second microphone may be located closer to the ear drum than a third microphone, etc.
The first and second microphones may be located at or in the ear canal of the user so that they experience first and second acoustic environments, wherein the first and second acoustic environments are at least partially acoustically isolated from each other when the user wears the hearing device, e.g. a headset. In the below table, internal and external may refer to first and second, respectively.
The first (internal) input transducer signal has the advantage of a good SNR (some of the noise from the environment has been filtered out by the directional properties of the outer ear and head and possibly torso), and the noise source (cf. ‘Noise’ in the table) will hence be more localized (point like), which facilitates its attenuation by a null (or minimum) of the beamformer in the direction away from the ear (e.g. perpendicular to the side of the head, and definitely not in a direction of the mouth, so the chance of (accidentally) attenuating the target signal is minimal). The spectral shape (coloring) of the signal from the first input transducer may, however, depending on the actual location (depth) in the ear canal and the degree of sealing of the first input transducer be poorer (e.g. confined to lower frequencies, e.g. less than 2 or 3 kHz) and thus sounding un-natural, if listened to. The first electric input signal from the first (internal) input transducer may experience a boost in dependence on leakage and residual volume. This boost is therefore difficult to “calibrate”.
The second (‘external’ (or ‘less internal’)) input transducer signal has the advantage of a good spectral shape that makes it more pleasant for a (far end listener) to listen to, but it has the downside of being ‘polluted’ by noise from the environment (which may be at least partially removed by spatial filtering (beamforming) and optionally post-filtering). But compared to the first input transducer, the second input transducer may experience a more diffuse noise distribution.
The hearing device may preferably comprise a beamformer, e.g. an MVDR beamformer, configured to provide an estimate of the user's voice based on beamformer weights applied to the first and second electric input signals. A property of an MVDR beamformer is that it will always provide a beamformed signal that exhibits an SNR that is larger than or equal to any of the input signals (it does not destroy SNR). In the present case, the ‘external’ (second) input transducer may preferably be the reference microphone for which a ‘distortionless response’ is provided by the MVDR-beamformer.
The filter weights (w) of the MVDR-beamformer may be adaptively determined. Typically, the noise field (e.g. represented by a noise covariance matrix Cv) is updated during speech pauses of the user (no OWN-voice), or speech pauses in general (no voice). The transfer functions dov,i from the user's mouth to each of the at least two microphones (i=1, . . . , M, M≥2) may be determined in advance of use of the hearing device or be adaptively determined during use (e.g. when the hearing device is powered up or repeatedly during use), when the user's own voice is present (and preferably when the noise level is Sbelow a threshold value). The transfer functions dov,i from the user's mouth to each of the at least two microphones (i=1, . . . , M, M≥2) may be represented by a look vector dov=(dov,1 . . . , dov,M)T, where superscript T indicates transposition.
In case the first input transducer is in acoustic communication with the environment, the MVDR-beamformer may rely on a predetermined look vector (e.g. determined in advance of use of the hearing device). In case the first input transducer is occluded (substantially (acoustically) sealed off from the environment), the look vector of the MVDR-beamformer may be adaptively updated.
The hearing device of
In the embodiment of
In the embodiment of
The embodiment of
The embodiment of
Now referring to
The hearing device (HD), e.g. the beamformer, further comprises a spatial filter controller SCU configured to apply at least a first set (p=1) of beamformer weights (w1p, w2p, . . . , wNp) (or linear filters, e.g. FIR-filters) to the multitude of electric input signals (IN1, IN2, . . . , INN). The first set of beamformer weights (p=1) (or linear filters) is applied to provide spatial filtering of an external sound field (e.g. from a sound source located at the user's mouth), cf. signals (Y1, Y2, . . . , YN). The hearing device further comprises a memory MEM accessible from the spatial filter controller SCU. The spatial filter controller SCU is configured to adaptively select an appropriate set of beamformer weights (signal wip) (or linear filters) among two or more sets (p=1, 2, . . . ) of beamformer weights (or linear filters) stored in the memory (including the first set of beamformer weights (or linear filters)). At a given point in time, an appropriate set of beamformer weights (or linear filters) may e.g. be selected from sets of different beamformer weights (or linear filter coefficients) stored in the memory or such appropriate (updated) beamformer weights (or linear filters) may be adaptively determined, e.g. dependent of a change in source location (e.g. in a case where the user's own voice is NOT of interest). The beamformer weights (or filter coefficients of linear filters, e.g. FIR-filters) may be determined by any method known in the art, e.g. using the MVDR procedure.
The part of a hearing device illustrated in
The embodiment of
An adaptive (own voice) beamformer may comprise a first set of beamformers C1 and C2, wherein the adaptive beamformer filter is configured to provide a resulting directional signal (comprising an estimate of the user's own voice) YBF(k)=C1(k)−β(k)C2(k), where β(k) is an adaptively updated adaptation factor. This is illustrated in
The beamformers C1 and C2 may comprise
In this case, the target direction is the direction of the user's mouth (the target sound source is equal to the user's own voice).
Y(k)=C1(k)−β(k)C2(k).
It should be noted that the sign in front of β(k) might as well be +, if the sign(s) of the beamformer weights constituting the delay-and-subtract beamformer C2 are appropriately adapted. The beamformed signal YBF is expressed as YBF=YOV=(wC1(k)−β(k)·wC2(k))H·IN(k), where bold face (x) indicates a vector, e.g. IN(k)=(IN1(k), IN2(k)), in case of two electric input signals, as illustrated in
The beamformer (BFU) may e.g. be adapted to work optimally in situations where the microphone signals consist of a point-noise target sound source in the presence of additive noise sources. Given this situation, the scaling factor β(k) (β in
The adaptation factor β(k) may be expressed as
where * denotes the complex conjugation and ⋅
denotes the statistical expectation operator, which may be approximated in an implementation as a time average, k is the frequency index, and c is a constant (e.g. 0). The expectation operator
⋅
may be implemented using e.g. a first order IIR filter, possibly with different attack and release time constants. Alternatively, the expectation operator may be implemented using a FIR filter.
In a further embodiment, the adaptive beamformer processing unit is configured to determine the adaptation parameterβopt(k) from the following expression
where wC1 and wC2 are the beamformer weights for the delay and sum C1 and the delay and subtract C2 beamformers, respectively, Cv is the noise covariance matrix, and H denotes Hermitian transposition.
The adaptive beamformer (BF) may e.g. be implemented as a generalized sidelobe canceller (GSC) structure, e.g. as a Minimum Variance Distortionless Response (MVDR) beamformer, as is known in the art.
The beamformers C1(k) and C2(k) (defined by respective sets of complex beamformer weights (w11(k), w12(k)) and (w21(k), w22(k))), as illustrated in
A binaural hearing system comprising first and second hearing devices (e.g. hearing aids, or first and second earpieces of a headset) as described above may be provided. The first and second hearing devices may be configured to allow the exchange of data, e.g. audio data, and with another device, e.g. a telephone, or a speakerphone, a computer (e.g. a PC or a tablet). Own voice estimation may be provided based on signals from microphones in the first and second hearing devices. Own voice detection may be provided in both hearing devices. A final own voice detection decision may be based on own voice detection values from both hearing devices or based on signals from microphones in the first and second hearing devices.
The hearing system (HS) according to the present disclosure comprises first and second hearing devices HD1, HD2 (e.g. first and second hearing aids of a binaural hearing aid system, or first and second ear pieces of a headset) configured to be worn on the head of a user comprising a head worn carrier, here embodied in a spectacle frame.
The hearing system comprises left and right hearing devices and a number of microphones and possibly vibration sensors mounted on the spectacle frame. Glasses or lenses (LE) of the spectacles may be mounted on the cross bar (CB) and nose sub-bars (NSB1, NSB2). The left and right hearing devices (HD1, HD2) comprises respective BTE-parts (BTE1, BTE2), and further comprise respective ITE-parts (ITE1, ITE2). The hearing system may further comprise a multitude of input transducers, here shown as microphones, and here configured in three separate microphone arrays (MAR, MAL, MAF) located on the right, left side bars and on the (front) cross bar, respectively. Each microphone array (MAR, MAL, MAF) comprises a multitude of microphones (MICR, MICL, MICF, respectively), here four, four and eight, respectively. The microphones may form part of the hearing system (e.g. associated with the right and left hearing devices (HD1, HD2), respectively, and contribute to localise and spatially filter sound from the respective sound sources of the environment around the user (and possibly in the estimation of the user's own voice). In an embodiment, all microphones of the system are located on the glasses and/or on the BTE part and/or in the ITE-part. The hearing system (e.g. the ITE-parts) may e.g. comprise electrodes for picking up body signals from the user, e.g. forming part of sensors for monitoring physiological functions of the user, e.g. brain activity or eye movement activity or temperature.
However, as taught by the present disclosure, for own voice estimation, it may be advantageous to locate a first input transducer (e.g. a microphone or a vibration sensor) in the (preferably partially occluded part of the) ear canal. It might alternatively, or additionally, be advantageous to locate a first input transducer (e.g. a vibration sensor) on the mastoid bone, e.g. in the form of a vibration sensor contacting the skin of the user covering the mastoid bone, possibly forming part of the BTE-part, or located on a specifically adapted carrier part of the spectacle frame.
Other sensors (not shown) may be located on the spectacle frame (camera, radar, etc.).
The BTE-and ITE parts (BTE and ITE) of the hearing devices are electrically connected, either wirelessly or wired, as indicated by the dashed connection between them in
Instead of a spectacle frame, the carrier may be a dedicated frame for carrying the first and second hearing devices and for appropriately locating the first and second (and possible further) input transducers on the head (e.g. at the respective ears) of the user.
In the embodiment of a hearing device in
parameter settings defining such programs, or parameters of algorithms, e.g. optimized parameters of a neural network, e.g. beamformer weights of one or more (e.g. an own voice) beamformer(s)) and/or hearing aid configurations, e.g. input source combinations (MBTE1, MBTE2, MBTE3, M1, M2, M3, WLR1, WLR2), e.g. optimized for a number of different listening situations or modes of operation. One mode of operation may be a communication mode, where the user's own voice is picked up by microphones of the hearing aid (e.g. M1, M2, M3) and transmitted to another device or system via one of the wireless interfaces (WLR1, WLR2). The substrate further comprises a configurable signal processor (DSP, e.g. a digital signal processor, e.g. including a processor (e.g. PRO in
The hearing system (here, the hearing device HD) may further comprise a detector unit comprising one or more inertial measurement units (IMU), e.g. a 3D gyroscope, a 3D accelerometer and/or a 3D magnetometer, here denoted IMU1 and located in the BTE-part (BTE). Inertial measurement units (IMUs), e.g. accelerometers, gyroscopes, and magnetometers, and combinations thereof, are available in a multitude of forms (e.g. multi-axis, such as 3D-versions), e.g. constituted by or forming part of an integrated circuit, and thus suitable for integration, even in miniature devices, such as hearing devices, e.g. hearing aids. The sensor IMU1 may thus be located on the substrate (SUB) together with other electronic components (e.g. MEM, FE, DSP). One or more movement sensors (IMU) may alternatively or additionally be located in or on the ITE part (ITE) or in or on the connecting element (IC), e.g. used to pick up sound from the user's mouth (own voice).
The hearing device (HD) further comprises an output unit (e.g. an output transducer) providing stimuli perceivable by the user as sound based on a processed audio signal from the processor or a signal derived therefrom. In the embodiment of a hearing device in
The electric input signals (from input transducers MBTE1, MBTE2, MBTE3, M1, M2, M3, IMU1) may be processed in the time domain or in the (time-) frequency domain (or partly in the time domain and partly in the frequency domain as considered advantageous for the application in question).
The hearing device (HD) exemplified in
The input signals (S-IN, M-IN) to the headset (HD) may be presented in the (time-) frequency domain or converted from the time domain to the (time-) frequency domain by appropriate functional units, e.g. included in receiver unit (Rx) and input unit (IU) of the headset. A headset according to the present disclosure may e.g. comprise a multitude of time to time time-frequency conversion units (e.g. one for each input signal that is not otherwise provided in a time-frequency representation, e.g. analysis filter bank units (A-FB) of
The headset (HD) is configured to provide an estimate of the user's own voice as disclosed in the present application. The MSP-signal processing unit (G2) may e.g. comprise an own voice beamformer as described in the present disclosure (see e.g.
It is intended that the structural features of the devices described above, either in the detailed description and/or in the claims, may be combined with steps of the method, when appropriately substituted by a corresponding process.
As used, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well (i.e. to have the meaning “at least one”), unless expressly stated otherwise. It will be further understood that the terms “includes,” “comprises,” “including,” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will also be understood that when an element is referred to as being “connected” or “coupled” to another element, it can be directly connected or coupled to the other element but an intervening element may also be present, unless expressly stated otherwise. Furthermore, “connected” or “coupled” as used herein may include wirelessly connected or coupled. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items. The steps of any disclosed method is not limited to the exact order stated herein, unless expressly stated otherwise.
It should be appreciated that reference throughout this specification to “one embodiment” or “an embodiment” or “an aspect” or features included as “may” means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosure. Furthermore, the particular features, structures or characteristics may be combined as suitable in one or more embodiments of the disclosure. The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects.
The claims are not intended to be limited to the aspects shown herein but are to be accorded the full scope consistent with the language of the claims, wherein reference to an element in the singular is not intended to mean “one and only one” unless specifically so stated, but rather “one or more.” Unless specifically stated otherwise, the term “some” refers to one or more.
Accordingly, the scope should be judged in terms of the claims that follow.
This application is a Continuation of copending application Ser. No. 17/574,773 filed Jan. 13, 2022, which is a Divisional of U.S. application Ser. No. 16/826,017, filed on Mar. 20, 2020 (now U.S. Pat. No. 11,259,127 issued Feb. 22, 2022), the entire contents of which are hereby incorporated by reference into the present application.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16826017 | Mar 2020 | US |
| Child | 17574773 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 17574773 | Jan 2022 | US |
| Child | 18821613 | US |
