Embodiments of the present disclosure relate to the field of signal processing technologies and, in particular, to an echo cancellation method and apparatus.
In an audio system, due to coupling between a speaker and a microphone, generation of an acoustic echo is inevitable. The acoustic echo subjected to a channel delay is transmitted back to a peer-end speaker, which thus affects audio call quality. Especially in a case of a hands-free call, an echo with too much energy will cause serious interference to a far-end communicator.
During the hands-free call, a voice of a peer end, after reaching a local end, is played by the speaker of the local end, and then picked up by the microphone of the local end and sent to the peer end, and this sound is called an echo. When the speaker of the local end plays the voice sent by the peer end: if the communicator at the local end does not speak, the sound collected by the microphone at the local end is a pure echo, and this stage is called a pure-echo stage; if the communicator at the local end speaks while the speaker is playing the voice of the peer end, the sound collected by the microphone at the local end includes the voice played by the speaker and the voice of the communicator at the local end, the sound at the local end is called a near-end voice, and this stage is called a double talk stage. A current common echo cancellation algorithm includes two modules: an adaptive filtering module and a non-linear programming module, where a linear echo is removed through adaptive filtering and a remaining non-linear echo is suppressed by the non-linear programming module. In a non-linear programming process, in order to ensure that the echo in the pure-echo stage is cancelled cleanly, a non-linear suppression parameter is usually set relatively large.
In a first aspect, an embodiment of the present disclosure provides an echo cancellation method, which is applied to an electronic device having a first sound pickup apparatus and a second sound pickup apparatus, and the method includes:
performing, with use of a first near-end signal d1(k), adaptive filtering on a second near-end signal d2(k) that is delayed, to obtain a first filtering signal e2(k), where the first near-end signal d1(k) is a signal picked up by the first sound pickup apparatus, and the second near-end signal d2(k) is a signal picked up by the second sound pickup apparatus;
performing non-linear echo signal cancellation processing on a target signal according to the first filtering signal e2(k) to obtain a signal to be transmitted EE1(k), where the target signal is the first near-end signal d1(k) or the second near-end signal d2(k);
sending the signal to be transmitted EE1(k).
In a feasible design, when the target signal is the first near-end signal d1(k), the performing the non-linear echo signal cancellation processing on the target signal according to the first filtering signal e2(k) to obtain the signal to be transmitted EE1(k) includes:
constructing a non-linear suppression parameter Para according to the first filtering signal e2(k) and the first near-end signal d1(k); and
performing the non-linear echo signal cancellation processing on the first near-end signal d1(k) according to the non-linear suppression parameter Para to obtain the signal to be transmitted EE1(k).
In a second aspect, an embodiment of the present disclosure provides an echo cancellation apparatus, which is applied to an electronic device having a first sound pickup apparatus and a second sound pickup apparatus, and the echo cancellation apparatus includes:
a filtering module, configured to perform, with use of a first near-end signal d1(k), adaptive filtering on a second near-end signal d2(k) that is delayed, to obtain a first filtering signal e2(k), where the first near-end signal d1(k) is a signal picked up by the first sound pickup apparatus, and the second near-end signal d2(k) is a signal picked up by the second sound pickup apparatus;
an echo cancellation module, configured to perform non-linear echo signal cancellation processing on a target signal according to the first filtering signal e2(k) to obtain the signal to be transmitted EE1(k), where the target signal is the first near-end signal d1(k) or the second near-end signal d2(k); and
In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor, a memory, and a computer program stored on the memory and runnable on the processor, where when the processor executes the program, the method of the first aspect or various feasible implementations of the first aspect is implemented.
In a fourth aspect, an embodiment of the present disclosure provides a readable storage medium having instructions stored thereon, where when the instructions run on an electronic device, the electronic device is enabled to execute the method of the first aspect or various feasible implementations of the first aspect.
In a fifth aspect, an embodiment of the present disclosure provides a computer program product, where when the computer program product runs on an electronic device, the electronic device is enabled to execute the method described in the first aspect or various feasible implementations of the first aspect.
In order to explain the embodiments of the present disclosure or the technical solutions in the prior art more clearly, the drawings to be used in the embodiments or the description of the prior art will be introduce briefly in the following. Obviously, the drawings in the following description are some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained from these drawings without paying creative labor.
To describe the purposes, technical solutions and advantages of embodiments of the present disclosure more clearly, the technical solutions in the embodiments of the present disclosure are clearly and comprehensively described in the following with reference to the accompanying drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of embodiments of the present disclosure, not all embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those of ordinary skill in the art without any creative effort are all within the protection scope of the present disclosure.
At present, when a local end (also referred as a callee end) electronic device plays a voice of a peer end, a speaker of the local end picks up the voice and sends it to the peer end, the sound is referred as an acoustic echo. The acoustic echo includes a direct echo and an indirect echo, where the direct echo is also referred as a linear echo, which refers to a sound picked up by a microphone of an electronic device when a speaker of the electronic device plays the voice of the peer end; the indirect echo is also referred as a non-linear echo, which refers to a set of echoes that enter the microphone after successive reflections or multiple reflections from different paths when the speaker of the electronic device plays the voice of the peer end. Generally speaking, in a hands-free mode, due to reasons such as a poor material or structure of the speaker, a non-linear transmission path is very easy to produce, thereby the non-linear echo is produced. The acoustic echo is transmitted to the peer end after a channel delay, which affects quality of voice communications. Especially in the hands-free mode, an over-capacity echo will cause interference to a communicator on the peer end in terms of semantic understanding, which greatly affects communication experience. With an innovation of a communication technology, people's requirements for the quality of voice communications continue to improve, and echo cancellation has become a focus of the voice communications.
Common echo cancellation algorithms include at least an adaptive filtering module and a non-linear programming module, where the adaptive filtering module is configured to remove a linear echo, and a remaining non-linear echo is suppressed by the non-linear programming module. Exemplarily, reference may be made to
y(k)=hT*x(k) (1)
ŷ(k)=ĥT*x(k) (2)
d(k)=y(k)+v(k) (3)
e(k)=d(k)−ŷ(k) (4)
where * represents a convolution, hT=[h0, h1, h2, . . . hM−1]T represents a real echo channel, and ĥT=[ĥ0, ĥ1, ĥ2, . . . ĥM−1]T represents an echo channel simulated by the adaptive filter. It can be known from the above formulas that: when the echo channel simulated by the adaptive filter is equal to the real echo channel, the adaptive filter can completely cancel the echo, retaining only the near-end voice.
In the above
In order to ensure that the voice can be retained as much as possible while the echo is cancelled completely, the non-linear programming module and a double talk detection (DTD) device are required to work together. In the pure-echo stage, a suppression coefficient of the non-linear parameter is increased, and a suppression coefficient of the non-linear parameter in the double talk stage is decreased to protect the voice. However, it is difficult to guarantee the accuracy of double talk detection, and it follows that NLP may not be able to cancel the pure echo or cause a great damage to the voice in the double talk stage, even if the accuracy of the double talk detection can reach 100% so that the echo can be effectively suppressed in a pure-echo node and the voice can be much protected in a double talk node, however, due to voice protection, since the non-linear programming cannot accurately distinguish the echo and the voice that are mixed together, the echo in the double talk stage cannot be suppressed extraordinarily.
In view of this, the embodiments of the present disclosure provide an echo cancellation method and apparatus, two microphones perform adaptive filtering on each other, so that a voice signal in a sound signal picked up by one of the microphones is removed to obtain a pure-echo signal. By using the pure-echo signal to cancel a non-linear echo, a purpose of cancelling the non-linear echo is achieved.
It should be noted that in
In addition, it should also be noted that, although the architecture described in
101, perform, with use of a first near-end signal d1(k), adaptive filtering on a second near-end signal d2(k) that is delayed, to obtain a first filtering signal e2(k).
The first near-end signal d1(k) is a signal picked up by the first sound pickup apparatus, and the second near-end signal d2(k) is a signal picked up by the second sound pickup apparatus.
Exemplarily, there are two microphones (mic) on the electronic device at a local end. During a voice call, the two microphones pick up sounds, the sound picked up by the first sound pickup apparatus is called the first near-end signal d1(k), and the sound picked up by the second sound pickup apparatus is called the second near-end signal d2(k). After that, the second near-end signal d2(k) of the second sound pickup apparatus is appropriately time-delayed, and the adaptive filtering on the second near-end signal d2(k) that is delayed is performed with use of the first near-end signal d1(k) to filter out the voice in the second near-end signal d2(k), so as to obtain the first filtering signal e2(k) for which only a pure echo is retained.
In an implementation, when the electronic device is in a hands-free call mode, the method described in this embodiment is executed.
102, perform, according to the first filtering signal e2(k), non-linear echo signal cancellation processing on a target signal subjected to a linear echo cancellation, to obtain a signal to be transmitted EE1(k).
The target signal is the first near-end signal d1(k) or the second near-end signal d2(k) subjected to the linear echo cancellation.
Exemplarily, the electronic device may perform echo cancellation processing on the first near-end signal d1(k) according to the first filtering signal e2(k) to obtain the signal to be transmitted EE1(k); or, perform, with use of the first filtering signal e2(k), echo cancellation processing on the second near-end signal d2(k) to obtain the signal to be transmitted EE1(k); or, the electronic device performs two-stage Wiener filtering on the first near-end signal d1(k) by using the first filtering signal e2(k) as an echo estimation, to obtain the signal to be transmitted EE1(k); or, the electronic device performs two-stage Wiener filtering on the second near-end signal d2(k) by using the first filtering signal e2(k) as an echo estimation, to obtain the signal to be transmitted EE1(k).
103, send the signal to be transmitted EE1(k).
Exemplarily, the electronic device sends the signal to be transmitted EE1(k) to the electronic device at the peer end, so that the electronic device at the peer end plays it.
The echo cancellation method according to the embodiments of the present disclosure, which are applied to an electronic device having a first sound pickup apparatus and a second sound pickup apparatus, are used in a voice communication process to: perform, with use of a first near-end signal d1(k), adaptive filtering on a second near-end signal d2(k) that is delayed, to obtain a first filtering signal e2(k), where the first filtering signal e2(k) is an echo signal with a voice being filtered out and with only an echo being retained; then determine, with use of the first filtering signal e2(k), a signal to be transmitted EE1(k); and send same to a peer-end electronic device. In this process, two microphones perform adaptive filtering on each other, so that a voice signal in a sound signal picked up by one of the microphones is removed to obtain a pure-echo signal. By using the pure-echo signal to cancel a non-linear echo in the sound signal picked up by any one of the microphones, a purpose of cancelling the non-linear echo is achieved.
In
Next, description will be made in detail below by taking an example where the electronic device may perform a delay process on the second near-end signal d2(k) picked up by the second sound pickup apparatus, then perform, with use of the first near-end signal d1(k), the adaptive filtering on the second near-end signal d2(k) that is delayed to obtain the first filtering signal e2(k), and then perform, with use of the first filtering signal e2(k), non-linear echo cancellation processing on the first near-end signal d1(k) or the second near-end signal d2(k) to obtain the signal to be transmitted EE1(k).
First, the adaptive filtering.
Please refer to
e
2(k)=d2(k)−ŷ2(k)=d2(k)−d1(k)×ĥ2 (5)
where k represents a k-th frame signal, when a sampling rate is 8000 Hz, each frame signal has M=160 sampling points, the first near-end signal d1(k) represents the k-th frame signal picked up by the first sound pickup apparatus, and the second near-end signal d2(k) represents the k-th frame signal picked up by the second sound pickup apparatus, ŷ2 represents an analog echo signal from the first sound pickup apparatus to the second sound pickup apparatus, ĥ2 represents a path from the first sound pickup apparatus to the second sound pickup apparatus. Since the path may be considered to be linear for a near-end voice signal, the voice signal in the second near-end signal d2(k) may be cancelled cleanly through the adaptive filtering, retaining only the echo signal. The adaptive filter may be a frequency domain normalized least mean square (FDNLMS) adaptive filter.
Second, construct the non-linear suppression parameter Para.
Exemplarily, the electronic device transform the second near-end signal d2(k) before the adaptive filtering into a frequency domain by using a fast Fourier transform (FFT). During a transforming process, the electronic device determines a first intermediate signal E2(k) according to the first filtering signal e2(k). In addition, the electronic device transforms a time domain signal after the adaptive filtering, i.e., the first filtering signal e2(k), into the frequency domain by using the FFT transform, and during the transforming process, determines a second intermediate signal D2(k) according to the second near-end signal d2(k). After that, the electronic device determines a first frequency domain signal YY(k) according to the first intermediate signal E2(k). After that, the electronic device determines a second frequency domain signal XX(k) according to the second intermediate signal D2(k). Finally, the electronic device constructs the non-linear suppression parameter Para according to the first frequency domain signal YY(k) and the second frequency domain signal XX(k).
In the above embodiments, the first intermediate signal is represented as follows:
where e2(k−1) is a previous frame signal of e2(k);
the second intermediate signal is represented as follows:
where d2(k−1) is a previous frame signal of d2(k);
the first frequency domain signal YY(k) is represented as follows:
YY(k)=first M+1 elements of E2(k) (8)
where the element may be understood as a frequency point, or it may be understood as a sampling point in the frequency domain after a short-time Fourier transform (short-time Fourier transform, STFT).
The second frequency domain signal XX(k) is represented as follows:
XX(k)=first M+1 elements of D2(k) (9)
the non-linear suppression parameter Para is represented as follows:
Para=[abs(XX(k))−abs(YY(k))]/abs(XX(k)) (10);
where FFT represents a fast Fourier transform, and abs represents a modulus of a complex number.
According to the above, it can be seen that: a difference between the first frequency domain signal YY(k) and the second frequency domain signal XX(k) is that a former with the voice signal being filtered out and only the echo signal being retained. This is because a voice frequency point in a sound picked up by the first sound pickup apparatus are highly correlated with a voice frequency point in a sound picked up by the second sound pickup apparatus, therefore, after adaptive filtering, it is possible to remove the voice signal well while retaining the echo signal, and the echo signal contains the non-linear echo.
In the above embodiments, for an echo frequency point in the second frequency domain signal XX(k), a difference between the non-linear suppression parameter Para and 0 is less than a first threshold, for a voice frequency point in the second frequency domain signal XX(k), a difference between the non-linear suppression parameter Para and 1 is less than a second threshold.
Exemplarily, according to the formula (10), it can be seen that: for the echo frequency point, a value in a vector of the first frequency domain signal YY(k) is almost equal to a corresponding value in a vector of the second frequency domain signal XX(k), so that the difference between the non-linear suppression parameter Para of the echo frequency point and 0 is smaller than the first threshold, that is, the non-linear suppression parameter Para of the echo frequency point is close to 0; for the voice frequency point, since a voice component in the first frequency domain signal YY(k) has been filtered out, the difference between the non-linear suppression parameter Para of the voice frequency point and 1 is smaller than the second threshold, that is, the non-linear suppression parameter Para of the voice frequency point is close to 1.
Finally, the adaptive filtering and the non-linear programming.
According to the above process of determining the non-linear suppression parameter Para, it can be seen that: the value of the non-linear suppression parameter Para at the echo frequency point is close to 0 and the value at the voice frequency point is close to 1, at this time, if the non-linear suppression parameter Para is directly multiplied by the microphone signal in the frequency domain, the echo frequency point will be effectively suppressed and the voice frequency point will not be damaged as much as possible. In order to make an echo cancellation effect better, it is necessary to perform the adaptive filtering before the signal to be transmitted EE1(k) is determined according to the non-linear suppression parameter Para and third frequency domain signal ZZ(k), that is, before the non-linear filtering.
Taking the echo cancellation of the first near-end signal d1(k) as an example, the electronic device needs to perform adaptive filtering on the first near-end signal d1(k) first, where the adaptive filtering takes the downlink signal x(k) as a reference signal, and perform, with use of the downlink signal x(k), the adaptive filtering on the first near-end signal d1(k) to obtain a second filtering signal e1(k). The second filtering signal e1(k) is represented as follows:
e
1(k)=d1(k)−ŷ1(k)=x(k)×(h1T−ĥ1T)+v1(k) (11)
where the first near-end signal d1(k) includes the voice signal v1(k) and the echo signal y1(k), ŷ1(k) is an analog echo signal obtained by simulating y1(k), x(k) is a downlink signal played by the speaker, h1T is an echo path from the speaker to the first sound pickup apparatus, ĥ1T is an echo estimation of the echo path from the speaker to the first sound pickup apparatus, v1(k) is a voice signal picked up by the first sound pickup apparatus.
According to the formula (11), the linear echo in the first near-end signal d1(k) can be cancelled, and thus the second filtering signal e1(k) can be obtained, the second filtering signal e1(k) still contains many non-linear echoes. In order to cancel the non-linear echoes, the electronic device determines a third intermediate signal E1(k) according to the second filtering signal e1(k), and the third intermediate signal is represented as follows:
where e1(k−1) is a previous frame signal of e1(k).
After transforming the second filtering signal e1(k) into the frequency domain, the electronic device determines a third frequency domain signal ZZ(k) according to the third intermediate signal E1(k), and the third frequency domain signal is represented as follows:
ZZ(k) is equal to first M+1 elements of E1(k).
Then, the electronic device determines the signal to be transmitted EE1(k) according to the non-linear suppression parameter Para and the third frequency domain signal ZZ(k), and the signal to be transmitted EE1(k) is represented as follows:
EE
1(k)=ZZ(k)gpara (13)
In addition, when the electronic device determines the signal to be transmitted EE1(k) according to the non-linear suppression parameter Para and the third frequency domain signal ZZ(k), a voice type may also be considered, and the non-linear suppression parameter Para is calculated according to the voice type. In this process, the electronic device determines the voice type of the first near-end signal d1(k), where the voice type includes a pure-echo type and a double talk voice type; and determines a parameter n according to the voice type, where the parameter n corresponding to the pure-echo type is greater than the parameter n corresponding to the double talk voice type, and the parameter n is used to indicate a suppression intensity of a non-linear echo. Finally, the signal to be transmitted EE1(k) is represented as follows:
EE
1(k)=ZZ(k)gpara{circumflex over ( )}n (14)
In the formula (14), n is determined by a double talk detector. If the first near-end signal d1(k) is the pure echo, then the value of n is relatively large, and the suppression is enhanced. If a determination result indicates that the first near-end signal d1(k) is the double talk voice, that is, echo+voice, then the value of n is 1 or other smaller values. If a determination result indicates that there is no downlink signal x(k), then no suppression is performed, that is, no non-linear echo cancellation is performed.
In the above embodiments, the element may be understood as a frequency point, or may be understood as a sampling point in the frequency domain after a short-time Fourier transform (STFT).
In the following, the method described in the embodiment of the present disclosure is verified by using an example where the sound pickup apparatus specifically is a microphone, a test signal comes from a 3GPP standards database, including a pure-voice stage, a pure-echo stage, and a double talk stage, and a test is performed in an anechoic room by simulating a call scenario through simulation of the base station and artificial mouths.
It should be noted that, although in the above embodiments, construction of one non-linear suppression parameter para is used to describe the echo cancellation method provided in the embodiment of the present disclosure in detail, in other feasible implementations, two non-linear suppression parameters may also be constructed, for example, the electronic device constructs one non-linear suppression parameter by using the first near-end signal before and after filtering, and constructs another non-linear suppression parameter by using the second near-end signal before and after filtering, and then performs the non-linear echo cancellation on the first near-end signal or the second near-end signal by using the two non-linear suppression parameters.
Moreover, in addition to the adaptive filtering, the embodiment of the present disclosure may also use Wiener filtering to cancel the non-linear echo. In this manner, when the target signal is the second near-end signal d2(k), when determining the signal to be transmitted EE1(k) according to the first filtering signal e2(k), the electronic device performs, with use of the first filtering signal e2(k), first Wiener filtering on the second near-end signal d2(k) to obtain a first Wiener result; determines a voice type of the second near-end signal d2(k) according to the first Wiener result, where the voice type includes a pure-echo type and a double talk voice type; determines a Wiener filtering intensity according to the voice type, where the Wiener filtering intensity corresponding to the pure-echo type is greater than the Wiener filtering intensity corresponding to the double talk voice type; performs second Wiener filtering on the second near-end signal d2(k) is performed according to the Wiener filtering intensity to obtain a second Wiener result, and obtains the signal to be transmitted EE1(k) according to the second Wiener result.
Exemplarily, reference may be made to
The following are apparatus embodiments of the present disclosure, which may be used to execute the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.
a filtering module 11, configured to perform, with use of a first near-end signal d1(k), adaptive filtering on a second near-end signal d2(k) that is delayed, to obtain a first filtering signal e2(k), where the first near-end signal d1(k) is a signal picked up by the first sound pickup apparatus, and the second near-end signal d2(k) is a signal picked up by the second sound pickup apparatus;
an echo cancellation module 12, configured to perform non-linear echo signal cancellation processing on a target signal according to the first filtering signal e2(k) to obtain a signal to be transmitted EE1(k), where the target signal is the first near-end signal d1(k) or the second near-end signal d2(k); and a sending module 13, configured to send the signal to be transmitted EE1(k).
In a feasible design, when the target signal is the first near-end signal d1(k), the echo cancellation module 12 is specifically configured to: construct a non-linear suppression parameter Para according to the first filtering signal e2(k) and the first near-end signal d1(k); and perform the non-linear echo signal cancellation processing on the first near-end signal d1(k) according to the non-linear suppression parameter Para to obtain the signal to be transmitted EE1(k).
In a feasible design, the echo cancellation module 12 is specifically configured to determine a first intermediate signal E2(k) according to the first filtering signal e2(k), the first intermediate signal
where e2(k−1) is a previous frame signal of e2(k); determine a second intermediate signal D2(k) according to the second near-end signal d2(k), the second intermediate signal
where d2(k−1) is a previous frame signal of d2(k); determine a first frequency domain signal YY(k) according to the first intermediate signal E2(k), where the first frequency domain signal YY(k) is equal to first M+1 elements of E2(k); determine a second frequency domain signal XX(k) according to the second intermediate signal D2(k), where the second frequency domain signal XX(k) is equal to first M+1 elements of D2(k); and construct the non-linear suppression parameter Para according to the first frequency domain signal YY(k) and the second frequency domain signal XX(k), Para=[abs(XX(k))−abs(YY(k))]/abs(XX(k)); where FFT represents a fast Fourier transform, abs represents a modulus of a complex number.
In a feasible design, the echo cancellation module 12 is configured to: perform, with use of a downlink signal x(k), the adaptive filtering on the first near-end signal d1(k) to obtain a second filtering signal e1(k), e1(k)=x(k)×(h1T−ĥ1T)+v1(k), where x(k) is a downlink signal played by a speaker, h1T is an echo path from the speaker to the first sound pickup apparatus, ĥ1T is an echo estimation of the echo path from the speaker to the first sound pickup apparatus, v1(k) is a voice signal picked up by the first sound pickup apparatus; determine a third intermediate signal E1(k) according to the second filtering signal e1(k), the third intermediate signal
where e1(k−1) is a previous frame signal of e1(k); determine a third frequency domain signal ZZ(k) according to the third intermediate signal E1(k), where the third frequency domain signal ZZ(k) is equal to first M+1 elements of E1(k); and determine the signal to be transmitted EE1(k) according to the non-linear suppression parameter Para and the third frequency domain signal ZZ(k).
a double talk determining module 14, configured to determine a voice type of the first near-end signal d1(k), where the voice type includes a pure-echo type and a double talk voice type; and determine a parameter n according to the voice type, where the parameter n corresponding to the pure-echo type is greater than the parameter n corresponding to the double talk voice type, and the parameter n is used to indicate a suppression intensity of a non-linear echo.
In a feasible design, the echo cancellation module 12 is configured to determine an n-th power of the non-linear suppression parameter Para; and determine the signal to be transmitted EE1(k) according to the n-th power of the non-linear suppression parameter Para and the third frequency domain signal ZZ(k), EE1(k)=ZZ(k)g para{circumflex over ( )}n, where g represents dot multiplication.
In a feasible design, for an echo frequency point in the second frequency domain signal XX(k), a difference between the non-linear suppression parameter Para and 0 is less than a first threshold, for a voice frequency point in the second frequency domain signal XX(k), a difference between the non-linear suppression parameter Para and 1 is less than a second threshold.
In a feasible design, the echo cancellation module 12 is configured to: perform, with use of the first filtering signal e2(k), first Wiener filtering on the second near-end signal d2(k) to obtain a first Wiener result; determine a voice type of the second near-end signal d2(k) according to the first Wiener result, where the voice type includes a pure-echo type and a double talk voice type; determine a Wiener filtering intensity according to the voice type, where the Wiener filtering intensity corresponding to the pure-echo type is greater than the Wiener filtering intensity corresponding to the double talk voice type; perform second Wiener filtering on the second near-end signal d2(k) according to the Wiener filtering intensity to obtain a second Wiener result, and obtain the signal to be transmitted EE1(k) according to the second Wiener result.
at least one processor 21 and a memory 22;
the memory 22 stores computer executable instructions;
the at least one processor 21 executes the computer executable instructions stored in the memory 22, so that the at least one processor 21 executes the echo cancellation method as described above.
In an implementation, the electronic device 200 further includes a communication component 23. The processor 21, the memory 22, and the communication component 23 may be connected via a bus 24.
An embodiment of the present disclosure also provides a readable storage medium having computer executable instructions stored thereon, where when the computer executable instructions are executed by a processor, the echo cancellation method as described above is implemented.
An embodiment of the present disclosure also provides a computer program product, where when the computer program product runs on an electronic device, the electronic device is enabled to execute the echo cancellation method as described above.
A person of ordinary skill in the art may understand that: all or part of the steps in the above method embodiments may be completed by hardware related to a program instruction. The aforementioned computer program may be stored in a computer readable storage medium. When the program is executed, the steps included in the above method embodiments are implemented; and the foregoing readable storage medium includes: a read only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk or other media on which program codes can be stored.
Embodiments of the present disclosure provide an echo cancellation method and apparatus, two microphones perform adaptive filtering on each other, so that a voice signal in a sound signal picked up by one of the microphones is removed to obtain a pure-echo signal. By using the pure-echo signal to cancel a non-linear echo in the sound signal picked up by any one of the microphones, a purpose of cancelling the non-linear echo is achieved.
In a first aspect, an embodiment of the present disclosure provides an echo cancellation method, which is applied to an electronic device having a first sound pickup apparatus and a second sound pickup apparatus, and the method includes:
performing, with use of a first near-end signal d1(k), adaptive filtering on a second near-end signal d2(k) that is delayed, to obtain a first filtering signal e2(k), where the first near-end signal d1(k) is a signal picked up by the first sound pickup apparatus, and the second near-end signal d2(k) is a signal picked up by the second sound pickup apparatus;
performing non-linear echo signal cancellation processing on a target signal according to the first filtering signal e2(k) to obtain a signal to be transmitted EE1(k), where the target signal is the first near-end signal d1(k) or the second near-end signal d2(k);
sending the signal to be transmitted EE1(k).
In a feasible design, when the target signal is the first near-end signal d1(k), the performing the non-linear echo signal cancellation processing on the target signal according to the first filtering signal e2(k) to obtain the signal to be transmitted EE1(k) includes:
constructing a non-linear suppression parameter Para according to the first filtering signal e2(k) and the first near-end signal d1(k); and
performing the non-linear echo signal cancellation processing on the first near-end signal d1(k) according to the non-linear suppression parameter Para to obtain the signal to be transmitted EE1(k).
In a feasible design, before the performing the non-linear echo signal cancellation processing on the first near-end signal d1(k) according to the non-linear suppression parameter Para to obtain the signal to be transmitted EE1(k) includes:
determining a first intermediate signal E2(k) according to the first filtering signal e2(k), the first intermediate signal
where e2(k−1) is a previous frame signal of e2(k);
determining a second intermediate signal D2(k) according to the second near-end signal d2(k), the second intermediate signal
where d2(k−1) is a previous frame signal of d2(k);
determining a first frequency domain signal YY(k) according to the first intermediate signal E2(k); where the first frequency domain signal YY(k) is equal to first M+1 elements of E2(k);
determining a second frequency domain signal XX(k) according to the second intermediate signal D2(k), where the second frequency domain signal XX(k) is equal to first M+1 elements of D2(k); and
constructing the non-linear suppression parameter Para according to the first frequency domain signal YY(k) and the second frequency domain signal XX(k), Para [abs(XX(k))−abs(YY(k))]/abs(XX(k)); where FFT represents a fast Fourier transform, abs represents a modulus of a complex number.
In a feasible design, the performing the non-linear echo signal cancellation processing on the first near-end signal d1(k) according to the non-linear suppression parameter Para to obtain the signal to be transmitted EE1(k) includes:
performing, with use of a downlink signal x(k), the adaptive filtering on the first near-end signal d1(k) to obtain a second filtering signal e1(k), e1(k)=x(k)×(h1T−ĥ1T)+v1(k), where x(k) is a downlink signal played by the speaker, h1T is an echo path from the speaker to the first sound pickup apparatus, ĥ1T is an estimation of the echo path from the speaker to the first sound pickup apparatus, v1(k) is a voice signal picked up by the first sound pickup apparatus;
determining a third intermediate signal E1(k) according to the second filtering signal e1(k), the third intermediate signal
where e1(k−1) is a previous frame signal of e1(k);
determining a third frequency domain signal ZZ(k) according to the third intermediate signal E1(k), where the third frequency domain signal ZZ(k) is equal to first M+1 elements of E1(k); and
determining the signal to be transmitted EE1(k) according to the non-linear suppression parameter Para and the third frequency domain signal ZZ(k).
In a feasible design, the above method further includes:
determining a voice type of the first near-end signal d1(k), where the voice type includes a pure-echo type and a double talk voice type; and
determining a parameter n according to the voice type, where the parameter n corresponding to the pure-echo type is greater than the parameter n corresponding to the double talk voice type, and the parameter n is used to indicate a suppression intensity of a non-linear echo.
In a feasible design, the determining the signal to be transmitted EE1(k) according to the non-linear suppression parameter Para and the third frequency domain signal ZZ(k) includes:
determining an n-th power of the non-linear suppression parameter Para; and
determining the signal to be transmitted EE1(k) according to the n-th power of the non-linear suppression parameter Para and the third frequency domain signal ZZ(k), EE1(k)=ZZ(k)g para{circumflex over ( )}n, where g represents dot multiplication.
In a feasible design, for an echo frequency point in the second frequency domain signal XX(k), a difference between the non-linear suppression parameter Para and 0 is less than a first threshold, for a voice frequency point in the second frequency domain signal XX(k), a difference between the non-linear suppression parameter Para and 1 is less than a second threshold.
In a feasible design, when the target signal is the second near-end signal d2(k), the determining the signal to be transmitted EE1(k) according to the first filtering signal e2(k) includes:
performing, with use of the first filtering signal e2(k), first Wiener filtering on the second near-end signal d2(k) to obtain a first Wiener result;
determining a voice type of the second near-end signal d2(k) according to the first Wiener result, where the voice type includes a pure-echo type and a double talk voice type;
determining a Wiener filtering intensity according to the voice type, where the Wiener filtering intensity corresponding to the pure-echo type is greater than the Wiener filtering intensity corresponding to the double talk voice type;
performing second Wiener filtering on the second near-end signal d2(k) according to the Wiener filtering intensity to obtain a second Wiener result, and obtaining the signal to be transmitted EE1(k) according to the second Wiener result.
In a second aspect, an embodiment of the present disclosure provides an echo cancellation apparatus, which is applied to an electronic device having a first sound pickup apparatus and a second sound pickup apparatus, and the echo cancellation apparatus includes:
a filtering module, configured to perform, with use of a first near-end signal d1(k), adaptive filtering on a second near-end signal d2(k) that is delayed, to obtain a first filtering signal e2(k), where the first near-end signal d1(k) is a signal picked up by the first sound pickup apparatus, and the second near-end signal d2(k) is a signal picked up by the second sound pickup apparatus;
an echo cancellation module, configured to perform non-linear echo signal cancellation processing on a target signal according to the first filtering signal e2(k) to obtain the signal to be transmitted EE1(k), where the target signal is the first near-end signal d1(k) or the second near-end signal d2(k); and
a sending module, configured to send the signal to be transmitted EE1(k).
In a feasible design, when the target signal is the first near-end signal d1(k), the echo cancellation module is specifically configured to: construct a non-linear suppression parameter Para according to the first filtering signal e2(k) and the first near-end signal d1(k); and perform the non-linear echo signal cancellation processing on the first near-end signal d1(k) according to the non-linear suppression parameter Para to obtain the signal to be transmitted EE1(k).
In a feasible design, the echo cancellation module is specifically configured to: determine a first intermediate signal E2(k) according to the first filtering signal e2(k), the first intermediate signal
where e2(k−1) is a previous frame signal of e2(k); determine a second intermediate signal D2(k) according to the second near-end signal d2(k), the second intermediate signal
where d2(k−1) is a previous frame signal of d2(k); determine a first frequency domain signal YY(k) according to the first intermediate signal E2(k), where the first frequency domain signal YY(k) is equal to first M+1 elements of E2(k); determine a second frequency domain signal XX(k) according to the second intermediate signal D2(k), where the second frequency domain signal XX(k) is equal to first M+1 elements of D2(k); and construct the non-linear suppression parameter Para according to the first frequency domain signal YY(k) and the second frequency domain signal XX(k), Para [abs(XX(k))−abs(YY(k))]/abs(XX(k)); where FFT represents a fast Fourier transform, abs represents a modulus of a complex number.
In a feasible design, the echo cancellation module is configured to: perform, with use of a downlink signal x(k), the adaptive filtering on the first near-end signal d1(k) to obtain a second filtering signal e1(k), e1(k)=x(k)×(h1T−ĥ1T)+v1(k), where x(k) is a downlink signal played by the speaker, h1T is an echo path from the speaker to the first sound pickup apparatus, ĥ1T is an estimation of the echo path from the speaker to the first sound pickup apparatus, v1(k) is a voice signal picked up by the first sound pickup apparatus; determine a third intermediate signal E1(k) according to the second filtering signal e1(k), the third intermediate signal
where e1(k−1) is a previous frame signal of e1(k); determine a third frequency domain signal ZZ(k) according to the third intermediate signal E1(k), where the third frequency domain signal ZZ(k) is equal to first M+1 elements of E1(k); and determine the signal to be transmitted EE1(k) according to the non-linear suppression parameter Para and the third frequency domain signal ZZ(k).
In a feasible design, the above apparatus further includes:
a double talk determining module, configured to determine a voice type of the first near-end signal d1(k), where the voice type includes a pure-echo type and a double talk voice type; and determine a parameter n according to the voice type, where the parameter n corresponding to the pure-echo type is greater than the parameter n corresponding to the double talk voice type, and the parameter n is used to indicate a suppression intensity of a non-linear echo.
In a feasible design, the echo cancellation module is configured to: determine an n-th power of the non-linear suppression parameter Para; and determine the signal to be transmitted EE1(k) according to the n-th power of the non-linear suppression parameter Para and the third frequency domain signal ZZ(k), EE1(k)=ZZ(k)g para{circumflex over ( )}n, where g represents dot multiplication.
In a feasible design, for an echo frequency point in the second frequency domain signal XX(k), a difference between the non-linear suppression parameter Para and 0 is less than a first threshold, for a voice frequency point in the second frequency domain signal XX(k), a difference between the non-linear suppression parameter Para and 1 is less than a second threshold.
In a feasible design, the echo cancellation module is configured to: perform, with use of the first filtering signal e2(k), first Wiener filtering on the second near-end signal d2(k) to obtain a first Wiener result; determine a voice type of the second near-end signal d2(k) according to the first Wiener result, where the voice type includes a pure-echo type and a double talk voice type; determine a Wiener filtering intensity according to the voice type, where the Wiener filtering intensity corresponding to the pure-echo type is greater than the Wiener filtering intensity corresponding to the double talk voice type; perform second Wiener filtering on the second near-end signal d2(k) according to the Wiener filtering intensity to obtain a second Wiener result, and obtain the signal to be transmitted EE1(k) according to the second Wiener result.
In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor, a memory, and a computer program stored on the memory and runnable on the processor, where when the processor executes the program, the method of the first aspect or various feasible implementations of the first aspect is implemented.
In a fourth aspect, an embodiment of the present disclosure provides a readable storage medium having instructions stored thereon, where when the instructions run on an electronic device, the electronic device is enabled to execute the method of the first aspect or various feasible implementations of the first aspect.
In a fifth aspect, an embodiment of the present disclosure provides a computer program product, where when the computer program product runs on an electronic device, the electronic device is enabled to execute the method described in the first aspect or various feasible implementations of the first aspect.
The echo cancellation method and apparatus according to embodiments of the present disclosure, which are applied to an electronic device having a first sound pickup apparatus and a second sound pickup apparatus, are used in a voice communication process to: perform, with use of a first near-end signal d1(k), adaptive filtering on a second near-end signal d2(k) that is delayed, to obtain a first filtering signal e2(k), where the first filtering signal e2(k) is an echo signal with a voice being filtered out and with only an echo being retained; then determine, with use of the first filtering signal e2(k), a signal to be transmitted EE1(k); and send same to a peer-end electronic device. In this process, two microphones perform adaptive filtering on each other, so that a voice signal in a sound signal picked up by one of the microphones is removed to obtain a pure-echo signal. By using the pure-echo signal to cancel a non-linear echo in the sound signal picked up by any one of the microphones, a purpose of cancelling the non-linear echo is achieved.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present disclosure, but not to limit it; although the present disclosure has been illustrated in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: the technical solutions recorded in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently substituted; and these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present disclosure.
Number | Date | Country | Kind |
---|---|---|---|
201911241302.7 | Dec 2019 | CN | national |
This application is a continuation of International Application PCT/CN2020/134336, filed on Dec. 7, 2020, which claims priority to Chinese Patent Application No. 201911241302.7, filed on Dec. 6, 2019, both of the aforementioned applications are incorporated by reference herein.
Number | Date | Country | |
---|---|---|---|
Parent | PCT/CN2020/134336 | Dec 2020 | US |
Child | 17832747 | US |