Patent Application
20170365247
This application claims priority from Korean Patent Application No. 10-2016-0075165, filed on Jun. 16, 2016 in the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference in its entirety.
Aspects of the exemplary embodiments relate to an electronic device, an echo signal cancelling method thereof and a non-transitory computer readable recording medium, and more particularly, to an electronic device capable of cancelling a residual echo additionally from a signal in which an echo is canceled through an adaptive filter firstly, an echo signal cancelling method thereof and a non-transitory computer readable recording medium.
With respect to an electronic device such as a communication terminal, a voice of a far-end speaker is echoed by a surrounding environment and input through a near-end microphone. The phenomenon that the sound produced by a device is input to the device again is called an acoustic echo.
Recently, the development of the electronic device which operates by recognizing a voice has been increased. In addition, many display devices such as a smart TV often include a microphone to perform a voice recognition function. Accordingly, there is a growing need to develop a method to cancel an acoustic echo, which is available not only for a communication terminal but for electronic devices in a variety of fields.
There is an increasing necessity of a method to cancel an echo without using phase information because a vibration of an air pressure according to a speaker positioned near a microphone causes a confusion on a phase of a signal. According to the necessity, the method to convert a signal to a frequency domain has been developed, however, the efficiency of an adaptive filter is still low because of a use of a complex coefficient.
In addition, a residual echo still exists even after a sound is processed by the adaptive filter. Further, a dispersion value of a noise required for cancelling the residual echo may not be estimated appropriately.
An aspect of the exemplary embodiment has been made to address the problems described above and to provide an electronic device capable of cancelling an echo with an adaptive filter (AF) and cancelling a residual echo with a residual echo suppressor (RES), an echo signal cancelling method thereof and a non-transitory computer readable recording medium.
According to an exemplary embodiment, there is provided an electronic device including a speaker configured to output a sound corresponding to a reference signal, a microphone configured to generate a microphone signal by obtaining a received sound and a filter configured to cancel an echo signal of the reference signal from the microphone signal, and the filter includes the first filter configured to estimate an echo signal of the reference signal and cancel the estimated echo signal from the microphone signal and the second filter configured to generate an adaptive gain to cancel a residual echo from the microphone signal in which the estimated echo signal is canceled, and generate an output signal by using the generated adaptive gain and the microphone signal in which the estimated echo signal is canceled.
According to an exemplary embodiment, there is provided an echo signal cancelling method of an electronic device including generating a microphone signal by obtaining a sound received by the electronic device, estimating an echo signal of a reference signal which is output from the electronic device and cancelling the estimated echo signal from the microphone signal, generating an adaptive gain to cancel a residual echo from the microphone signal in which the estimated echo signal is canceled and generating an output signal by using the generated adaptive gain and the microphone signal in which the estimated echo signal is canceled.
According to an exemplary embodiment, there is provided a non-transitory computer readable recording medium comprising a program to execute an echo signal cancelling method of an electronic device, the method including generating a microphone signal by obtaining a sound received by the electronic device, estimating an echo signal of a reference signal which is output from the electronic device and cancelling the estimated echo signal from the microphone signal, generating an adaptive gain to cancel a residual echo from the microphone signal in which the estimated echo signal is canceled, and generating an output signal by using the generated adaptive gain and the microphone signal in which the estimated echo signal is canceled.
According to yet another exemplary embodiment, an electronic device includes: a speaker configured to output a sound corresponding to a reference signal; a microphone configured to generate a microphone signal by obtaining the sound corresponding to the reference signal; and a filter configured to cancel an echo signal of the reference signal from the microphone signal, in which the filter includes: a first filter configured to estimate the echo signal of the reference signal and to cancel the estimated echo signal from the microphone signal; and a second filter configured to generate an adaptive gain to cancel a residual echo from the microphone signal in which the estimated echo signal is canceled, and generate an output signal by using the generated adaptive gain and the microphone signal, in which the estimated echo signal is canceled.
As described above, according to the diverse exemplary embodiments, the electronic device may cancel an echo signal occurred from the signal which is output from the electronic device itself, in two stages. In addition, the electronic device may cancel a residual echo by estimating an appropriate dispersion value of the residual echo.
Hereinafter, exemplary embodiments are described in detail with reference to the accompanying drawings. In describing the exemplary embodiments, well-known element structures and technologies are not described in detail since they would obscure the exemplary embodiments in unnecessary detail. Further, all terms used in the description are general terms that are widely used in consideration of their functions in the exemplary embodiments, but may differ depending on intentions of a person skilled in the art to which the exemplary embodiments belong, custom, or appearance of new technology. Accordingly, they should be defined on the basis of the contents of the whole description of the exemplary embodiments.
The term such as “first” and “second” used in various exemplary embodiments may modify various elements regardless of an order and/or importance of the corresponding elements, and does not limit the corresponding elements. The terms are used only to distinguish one element from other elements. For example, a first element may be named a second element without departing from the scope of right of various exemplary embodiments, and similarly, a second element may be named a first element. The term of and/or includes combination or one of a plurality of related items recited.
The terms used herein are solely intended to explain a specific exemplary embodiment, and not to limit the scope of the present disclosure. A singular expression includes a plural expression, unless otherwise specified. The terms, “include”, “comprise”, etc. of the description are used to indicate that there are features, numbers, operations, elements, parts or combination thereof, and they should not exclude the possibilities of combination or addition of one or more features, numbers, operations, elements, parts or combination thereof.
Referring to
The electronic device 100 may receive an utterance of a user s(t) through the microphone 120. However, the microphone 120 receives not only an utterance of a user but an echo signal of the reference signal h(t)*x(t), which is output from the speaker 110. The microphone 120 may generate a microphone signal d(t) by obtaining the received sound.
A purpose of the electronic device 100 according to an exemplary embodiment is to cancel the echo signal of the reference signal h(t)*x(t) from the microphone signal d(t) and obtain the signal of a near-end speaker s(t) only. Referring to
According to the above, the electronic device 100 according to an exemplary embodiment may obtain a signal of a near-end speaker which is aimed to be received by cancelling the echo signal in two stages. The echo signal cancelling method in two stages will be described below in detail.
The speaker 110 may output a sound. For example, the speaker 110 may output the sound corresponding to a reference signal by processing the reference signal (or an audio signal).
The microphone 120 may obtain a sound around the electronic device 100. In addition, the microphone 120 may generate a microphone signal (or an audio signal) by processing the obtained sound.
The filter 130 may cancel an echo signal of the reference signal from the microphone signal. The filter 130 may be composed of circuits performing respective functions and may perform a signal processing by using CPU, DSP and the like.
The filter 130 may include the first filter 131 which cancels an acoustic echo signal output from the speaker 110 by using an adaptive filter (AF), and the second filter 133 which cancels a residual echo which remains even after the processing of the first filter 131.
In addition, the filter 130 may further include compositions such as a converter 134 which converts a signal into a frequency domain, a magnitude obtainer 135 which obtains a spectral magnitude from the signal converted into the frequency domain, a phase obtainer 136 which obtains a phase from the signal converted into the frequency domain, an inverter 137 which converts a signal into a time domain, and a synthesizer 138 which synthesizes the magnitude and the phase.
All the compositions performing respective functions in the above may be included in the filter 130, or some of the functions may be performed in an additional composition.
The first filter 131 may estimate the echo signal of the reference signal and cancel the echo signal of the reference signal which is estimated from the microphone signal. The first filter 131 may be called an acoustic echo canceller (AEC).
For example, the first filter 131 may generate an estimated echo signal which traces the echo signal of the reference signal by renewing an adaptive filter (AF) coefficient. The first filter 131 may cancel a noise occurred by a voice signal of the electronic device 100 itself by cancelling the estimated echo signal generated by the microphone signal.
The first filter 131 may update an echo signal which is estimated successively to trace the echo signal of the reference signal. The signal which goes through the first filter 131 and in which the echo signal of the reference signal estimated from the microphone signal is canceled is called an error signal. For example, if the estimation is perfect, the estimated echo signal may coincide with an actual echo signal of the reference signal. However, the estimation may not be perfect in a practical operation, and thus the error signal which went through the first filter 131 may include a residual echo.
The second filter 133 may generate an output signal by cancelling a residual echo from the microphone signal (that is, the error signal) in which the estimated echo signal is canceled. For example, the second filter 133 may generate an adaptive gain to cancel the residual echo from the microphone signal in which the estimated echo signal is canceled. In addition, the second filter 133 may generate an output signal by using the microphone signal in which the generated adaptive gain and the estimated echo signal are canceled. A specific operation of the second filter 133 will be described below again.
The processor 140 may control an overall composition of the electronic device 100. An exemplary embodiment of
The communicator 150 may transmit the output signal which is generated through the filter 130 to an external device. For example, if the electronic device 100 is realized as a smartphone and performs a phone call on a speaker phone mode, the communicator 150 may cancel an echo signal of a voice of a person at the other end of the line which is output from the speaker 110, and transmit the output signal in which the voice of the user only remains, to the device of the person at the other end of the line.
The communicator 150 may communicate with an external device through a wired and wireless network. For example, the communicator 150 may include a variety of communication modules such as a near-field wireless communication module (not illustrated) and a wireless communication module (not illustrated). Herein, the near-field wireless communication module refers to the module which communicates with an external device located nearly, according to a near-field wireless communication method such as Bluetooth, Zigbee and the like. In addition, the wireless communication module refers to the module which is connected to an external network according to a wireless communication protocol such as WiFi, IEEE and the like, and performs a communication. In addition to the above, the wireless communication module may further include a mobile communication module which accesses a mobile communication network according to a variety of mobile communication standards such as 3rd generation (3G), 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE) and LTE Advanced (LTE-A) and which performs a communication.
The communicator 150 may let the voice recognizer 160 to communicate with an external server and the like. For example, the communicator 150 may transmit a result of a voice recognition which is processed in the voice recognizer 160 to an external server, and receive corresponding information from an external device.
The voice recognizer 160 may perform a voice recognition by using an output signal generated in the filter 130. The voice recognizer 160 may generate a control signal capable of controlling the electronic device 100 itself or an external device according to the result of the voice recognition.
For example, the voice recognizer 160 may detect and recognize a trigger word from the generated output signal and generate a result of a word and a sentence voice recognition from an output signal after the trigger word is recognized.
In the exemplary embodiment of
The microphone 120 may obtain a received sound and generate a microphone signal d(n). The microphone signal d(n) may include the echo signal of the reference signal. The filter 130 may perform an operation to cancel the echo signal of the reference signal included in the microphone signal. The electronic device 100 may preserve a voice signal of a near-end speaker by cancelling the echo signal of the reference signal.
The converter 134-1 and 134-2 may convert the reference signal x(n) and the microphone signal d(n) into a frequency domain respectively. In addition, the filter 130 may divide each of the microphone signal and the reference signal, which are converted into the frequency domain, into a magnitude and a phase. The magnitude obtainer 135-1 and 135-2 may respectively receive the microphone signal and the reference signal which are converted into the frequency domain, and obtain the magnitude only. The signal in which the phase is canceled and only the magnitude remains may be called spectral magnitude. The electronic device 100 according to an exemplary embodiment may solve a problem such as a release of an estimated value occurred due to a rapid change of a phase, by using the signal which is converted into a frequency domain and also of which phase is divided and only the magnitude remains. In addition, the phase obtainer 136 may receive the microphone signal converted into the frequency domain and obtain the phase only.
An exemplary embodiment of
The first filter 131 may receive a spectrum magnitude of the converted reference signal, X(l,k), and a spectrum magnitude of the microphone signal, D(l,k). The first filter 131 or AEC may include a plurality of adaptive filters which operate in different frequency domains. Each of the plurality of adaptive filters may generate an estimated echo signal of frequency domains assigned respectively. Such AEC is called sub band AEC.
The first filter 131 may trace an echo signal of a reference signal while updating adaptive filter coefficient H(l,k). The first filter 131 generates an estimated echo signal of the reference signal F(l,k), by multiplying a reference signal of a previous time frame by an adaptive filter coefficient of a present time frame. In addition, the first filter 131 may cancel the echo signal of a reference signal estimated in a microphone signal. The signal that an echo signal estimated in the microphone signal is canceled is called error signal E(l,k). A detailed operation of the first filter 131 may be explained again in
The second filter 133 may generate adaptive gain G(l,k) to cancel a residual echo from the microphone signal in which the estimated echo signal is canceled (that is, the error signal E(l,k)). The error signal E(l,k) may include both the residual echo and a voice signal of a near-end speaker. If the adaptive gain is multiplied by the error signal, only the voice signal of the near-end speaker is output. That is, the relation of the error signal and the adaptive gain is as follows.
E=Signal+Noise,G·E=Signal
The second filter 133 may estimate a dispersion value of the residual echo and generate the adaptive gain by using all frequency components of the dispersion value of the estimated residual echo and the error signal (the microphone signal in which an echo signal estimated in the first filter 131 is canceled). The second filter 133 may generate an adaptive gain by using one or more frequency components of the microphone signal in which the estimated echo signal is canceled. A detailed operation of the second filter 133 may be explained again in
The synthesizer 138 may synthesize a spectrum magnitude that the error signal is multiplied by the adaptive gain, with a phase of the microphone signal. In addition, the inverter 137 may convert the synthesized signal into a time domain.
Referring to
A functional block expressed as sqrt( ) corresponds to the magnitude obtainer 135, and obtains the reference signal and a spectrum magnitude of the microphone signal. A functional block expressed as ei a tan corresponds to the phase obtainer 136, and obtains a phase component of the microphone signal. RLS functional block corresponds to the first filter 131. It is called RLS functional block because the RLS functional block adapts a filter coefficient by using Recursive Least Squares (RLS) algorithm. RLS functional block may generate an estimated echo signal of the reference signal F(l,k) by using a spectrum magnitude of an input reference signal X(l,k) and an adaptive filter coefficient.
F(l,k)=Σi=0L-1Hi(l,k)X(l−i,k)
In the above equation, L is a length of a filter.
Error signal E(l,k) corresponds to the value that the estimated echo signal of the reference signal is canceled from the microphone signal. The error signal may include a voice signal of a near-end speaker and a residual echo.
RES gain functional block corresponds to the second filter 133. The RES gain functional block may estimate a dispersion value of a residual echo by using the reference signal, the microphone signal, the estimated echo signal of the reference signal and the error signal. In addition, the RES gain functional block may generate an adaptive gain from a dispersion value of an estimated residual echo.
The electronic device 100 may multiply the generated the adaptive gain G(l,k) by the error signal E(l,k) and obtain only a voice signal of a near-end speaker in which a residual echo is canceled. In addition, the electronic device 100 may use the phase obtained from the microphone signal as a phase of the voice signal of the near-end speaker.
The electronic device 100 may generate an output signal by multiplying the adaptive gain G(l,k), the error signal E(l,k) and a phase of the microphone signal ejφd(l,k), and converting the same into a time domain.
Referring to
F(l,k)=ΣiHi(l,k)X(l−i,k)
E(l,k)=D(l,k)−F(l,k)
The first filter 131 may output the adaptive filter coefficient, the estimated echo signal and the error signal, so the second filter 131 may use the adaptive filter coefficient, the estimated echo signal and the error signal to cancel a residual echo.
The second filter 133 may have to estimate a dispersion value of the residual echo first, in order to calculate the adaptive gain. According to an exemplary embodiment, the dispersion value of the residual echo may be calculated through information regarding an adaptive filter which reflects a reference signal and an acoustic environment.
The dispersion value of the residual echo λd, may be calculated through the following equation.
λd(l,k)=α2(l,k)λd(l−1,k)+(1−α2(l,k))·|c(l,k)X2(l,k)−F2(l,k)|
In the above equation, α(l,k) is a decaying coefficient of the adaptive filter coefficient H(l,k), c(l,k) is a coupling coefficient between a reference signal and a microphone signal, X(l,k) is the reference signal, F(l,k) is an estimated echo signal of the reference signal, l is a frame index, and k is a frequency component index.
First, the second filter 133 may calculate the coupling coefficient between the reference signal and the microphone signal c(l,k) as the following equation.
< > calculation corresponds to the calculation drawing a kind of an average by a frame unit.
After calculating the coupling coefficient c(l,k), the second filter 133 may calculate total energy of the reference signal c(l,k)X2(l,k) received in the microphone 120, by multiplying the coupling coefficient by a square of the reference signal. In addition, the second filter 133 may calculate a residual reference energy c(l,k)X2(l,k)−F2(l,k) which remains in the error signal, by cancelling a square of the estimated echo signal F2(l,k) from the total energy of the reference energy.
The second filter 133 may reflect a residual echo energy of a previous frame by using a decaying coefficient of adaptive filter coefficient α(l,k).
The second filter 133 may calculate the decaying coefficient of adaptive filter coefficient α(l,k) by using the adaptive filter coefficient H(l,k). Components of the adaptive filter coefficient H(l,k) may be expressed as the following recursion on the assumption that the components decrease regularly.
|Hi(l,k)|=α(l,k)|Hi-1(l,k)|
The second filter 133 may calculate the decaying coefficient by using a recursive relation as the following equation.
α(l,k)=(|Hmin(l,k)|/|Hmax(l,k)|)1/(imin−imzx)
In the above equation, Hmin is a minimum filter coefficient, Hmax is a maximum filter coefficient, and each of imin and imax are corresponding indexes.
The second filter 133 may generate the adaptive gain G(l,k) by using the dispersion value of the residual echo λd and the error signal E(l,k). In addition, the second filter 133 may generate an output signal by multiplying the generated adaptive gain by the error signal.
The electronic device 100 may multiply the generated output signal by the phase of the microphone signal, convert the same into a time domain, and output the converted signal.
{circumflex over (Y)}(l,k)=G(l,k)E(l,k)ejφ
The electronic device 100 may estimate the echo signal of the reference signal by using the adaptive filter, and cancel the estimated echo signal from the microphone signal in S820. However, in the microphone signal in which the estimated echo signal is canceled by using the adaptive filter, a residual echo remains, and thus the electronic device 100 may generate an adaptive gain to cancel the residual echo in S830. In addition, the electronic device 100 may generate an output signal by using the microphone signal in which the generated adaptive gain and the estimated echo signal are canceled, in S840.
In order for the error signal, that is, the estimated echo signal to cancel the residual echo remains in the signal which is canceled from the microphone signal, the electronic device 100 may use a residual echo suppressor in S940. In detail, the residual echo suppressor may calculate the dispersion value of the residual echo from the information of the reference signal and the adaptive filter (acoustic environment), and calculate the adaptive gain to cancel the residual echo by using the calculated dispersion value of the residual echo.
Through the two stages of cancelling the echo signal, the electronic device 100 may obtain an output signal. In addition, the electronic device 100 may convert the output signal into the time domain again in S950. Successively, the electronic device 100 may output the output signal which is converted into the time domain, in S960.
The other exemplary embodiments of the echo signal cancelling method of the electronic device 100 are duplicated with the explanation of the electronic device 100, and thus will be omitted.
As described above, the electronic device 100 may cancel the echo signal in two stages. In addition, the electronic device 100 may obtain the adaptive gain value to cancel the residual echo effectively, by calculating the dispersion value of the residual echo by using the reference signal and the acoustic environmental information.
The above explained methods may be realized in a program instruction formation which can be performed through a variety of computer means and recorded in a computer readable medium. The above computer readable medium may include one of or a combination of a program instruction, a data file, and a data structure and the like. The program instruction recorded in the medium may be planned and composed especially for the exemplary embodiment, or well-known and usable for those skilled in a computer software. The computer readable recording medium may include the examples such as a hard disk, a floppy disk, a magnetic media such as a magnetic tape, an optical media such as CD-ROM and DVD, a magneto-optical media such as a floptical disk, and a hardware apparatus especially configured to store and execute a program instruction, such as ROM, RAM and flash memory. An example of the program instruction may include a machine cod formed by a compiler, as well as a high-level language code which can be executed by a computer using an interpreter. The above hardware apparatus may be composed to be operated as more than one software module to perform an operation of the exemplary embodiment, and so is the reverse thereof.
While the present disclosure has been shown and described with reference to various exemplary embodiments and drawings thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein. Thus, the scope of the present disclosure should not be limited by the explained exemplary embodiments, and should be defined by the appended claims and their equivalents.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2016-0075165 | Jun 2016 | KR | national |
