Signal processing apparatus, signal processing method, and signal processing program

Abstract

To remove only noise components without removing desired signal components, a signal processing apparatus includes a noise decorrelator that removes noise signals having correlation between at least two input signals, in each of which a desired signal and a noise signal coexist, by receiving the at least two input signals from at least two channels, and a residual noise remover that removes residual noise included in an output signal of the noise decorrelator based on a phase difference between the output signal of the noise decorrelator and at least one input signal included in the at least two input signals.

Description

TECHNICAL FIELD

The present invention relates to a technique of acquiring a desired signal from a mixed signal in which the desired signal and noise coexist.

BACKGROUND ART

In the above technical field, patent literature 1 discloses a technique of reducing residual noise when removing noise components included in input signals, by calculating the phase difference between at least two of input signals of multiple channels and enhancing the phase difference.

CITATION LIST

Patent Literature

Patent literature 1: International Publication No. 2007/025265

Patent literature 2: International Publication No. 2005/024787

Patent literature 3: Japanese Patent No. 4765461

Patent literature 4: Japanese Patent No. 4282227

Non-patent literature 1: Handbook of Speech Processing, Chapter 47, Adaptive Beamforming and Postfiltering, Springer, 2008

SUMMARY OF THE INVENTION

Technical Problem

In the technique described in the above literature, however, although the phase difference is enhanced to reduce residual noise, desired signal components may be unwantedly removed together with noise components.

The present invention enables to provide a technique of solving the above-described problem.

Solution To Problem

One aspect of the present invention provides a signal processing apparatus comprising:

a noise decorrelator that removes noise signals having correlation between at least two input signals, in each of which a desired signal and a noise signal coexist, by receiving the at least two input signals from at least two channels; and

a residual noise remover that removes residual noise included in an output signal of the noise decorrelator based on a phase difference between the output signal of the noise decorrelator and at least one input signal included in the at least two input signals.

Another aspect of the present invention provides a signal processing method comprising:

removing noise signals having correlation between at least two input signals, in each of which a desired signal and a noise signal coexist, by receiving the at least two input signals from at least two channels; and

removing residual noise included in an output signal in the removing the noise signals, based on a phase difference between the output signal in the removing the noise signals and at least one input signal included in the at least two input signals.

Still other aspect of the present invention provides a signal processing program for causing a computer to execute a method, comprising:

removing noise signals having correlation between at least two input signals, in each of which a desired signal and a noise signal coexist, by receiving the at least two input signals from at least two channels; and

removing residual noise included in an output signal in the removing the noise signals, based on a phase difference between the output signal in the removing the noise signals and at least one input signal included in the at least two input signals.

Advantageous Effects of Invention

According to the present invention, it is possible to remove only noise components without removing desired signal components.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing the arrangement of a signal processing apparatus according to the first embodiment of the present invention;

FIG. 2 is a block diagram showing the arrangement of a residual noise remover according to the first embodiment of the present invention;

FIG. 3 is a block diagram showing the arrangement of a signal processing apparatus according to the second embodiment of the present invention;

FIG. 4 is a block diagram showing the arrangement of a residual noise remover according to the second embodiment of the present invention;

FIG. 5 is a block diagram showing the arrangement of a phase difference-based noise remover according to the second embodiment of the present invention;

FIG. 6 is a flowchart illustrating a processing sequence by the signal processing apparatus according to the second embodiment of the present invention;

FIG. 7 is a block diagram showing the arrangement of a residual noise remover according to the third embodiment of the present invention;

FIG. 8 is a block diagram showing an example of a correction calculator according to the third embodiment of the present invention;

FIG. 9 is a block diagram showing the arrangement of a residual noise remover according to the fourth embodiment of the present invention;

FIG. 10 is a block diagram showing the arrangement of a noise re-remover according to the fourth embodiment of the present invention;

FIG. 11 is a block diagram showing the arrangement of a residual noise remover according to the fifth embodiment of the present invention; and

FIG. 12 is a block diagram showing the arrangement of an amplitude-based noise remover according to the fifth embodiment of the present invention.

DESCRIPTION OF THE EMBODIMENTS

Preferred embodiments of the present invention will now be described in detail with reference to the drawings. It should be noted that the relative arrangement of the components, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless it is specifically stated otherwise. Note that “speech signal” in the following explanation indicates a direct electrical change that occurs in accordance with speech or another audio and transmits the speech or the other audio, and is not limited to speech.

[First Embodiment]

A signal processing apparatus 100 according to the first embodiment of the present invention will be described with reference to FIGS. 1 and 2. As shown in FIG. 1, the signal processing apparatus 100 includes a noise decorrelator 101 and a residual noise remover 102. As shown in FIG. 2, the residual noise remover 102 includes suppression coefficient calculators 201₁ to 201_M and a suppressor 202.

The noise decorrelator 101 receives, from at least two channels, at least two input signals X₁ to X_M in each of which a desired signal and a noise signal coexist. The noise decorrelator 101 removes noise components commonly included in the input signals, that is, noise components having correlation between the channels, thereby outputting X₀.

The residual noise remover 102 receives the output signal X₀ of the noise decorrelator 101 and at least one of the at least two input signals X₁ to X_M. The residual noise remover 102 removes a noise component included in X₀ based on the difference (phase difference) between the phase of the output signal X₀ and the phase of at least one of the input signals X₁ to X_M, thereby outputting S₀.

The suppression coefficient calculators 201₁ to 201_M calculate suppression coefficients W₁ to W_M based on the phase differences between the input signal X₀ and the input signals X₁ to X_M, respectively. The suppressor 202 removes a residual noise component included in the input signal X₀ using at least one of the suppression coefficients W₁ to W_M.

With the above arrangement, it is possible to remove only noise components without removing desired signal components.

[Second Embodiment]

A signal processing apparatus 300 according to the second embodiment of the present invention will be described next with reference to FIGS. 3 to 6. Note that FIG. 6 is a flowchart illustrating processing by the signal processing apparatus according to this embodiment.

(Overall Arrangement)

FIG. 3 is a block diagram showing the arrangement of the signal processing apparatus 300 according to this embodiment. In this embodiment, the signal processing apparatus 300 is a system for acquiring a desired signal from mixed signals of multiple channels, in each of which a desired signal and noise coexist. The desired signal will be described as a speech signal below. However, the technical scope of the present invention is not limited to this.

The signal processing apparatus 300 includes a noise decorrelator 301 and a residual noise remover 302. The noise decorrelator 301 receives two or more multi-channel input signals X₁ to X_M, and mainly removes noise components included in two or more channels, that is, noise components having correlation between the channels, thereby outputting X₀.

The residual noise remover 302 receives the output signal X₀ of the noise decorrelator 301 and at least one of the multi-channel input signals X₁ to X_M. The residual noise remover 302 removes a noise component included in X₀ based on the difference (phase difference) between the phase of X₀ and the phase of at least one of X₁ to X_M, thereby outputting S₀.

(Noise Decorrelator)

The multi-channel input signals X₁ to X_M are modeled, as given by:X₁(f,t)=S(f,t)+N_C1(f,t)+N_i1(f,t)   (1-1)x_M(f,t)=S(f,t)+N_CM(f,t)+N_iM(f,t)   (1-M)

wherein X₁ to X_M represent the complex spectra of the input signals, each of which is obtained by performing frequency analysis such as discrete Fourier transform for a signal in the time domain of a corresponding channel, f represents the index of a frequency, and t represents the index of time. In the following explanation, f and t will be omitted except when necessary. Furthermore, S represents the complex spectrum of a desired speech component, N_c1 to N_cM respectively represent noise components included in two or more channels of channels 1 to M, that is, the complex spectra of noise components having correlation between the channels, N_i1 to N_iM respectively represent noise components independently included in respective channels 1 to M, that is, the complex spectra of noise components having low correlation between the channels.

The noise decorrelator 301 mainly removes the noise components N_c1 to N_cM having correlation between the channels using a technique such as an adaptive noise canceller (for example, a method described in patent literature 2: International Publication No. 2005/024787) or an adaptive beamformer (a method described in non-patent literature 1: Handbook of Speech Processing, Chapter 47, Adaptive Beamforming and Postfiltering, Springer, 2008, such as a generalized side-lobe canceller or minimum variance beamformer). Removal processing in the noise decorrelator 301 may be either processing in a frequency domain or processing in a time domain, as a matter of course. If processing of removing noise components having correlation between the channels is performed in the time domain, conversion into the signal X₀ in the frequency domain is performed by frequency analysis after the processing. The noise decorrelator 301 outputs X₀ given by:X₀=S+N_i0   (2)

where N,_i0 represents residual noise after the processing of the noise decorrelator 301, and mainly indicates noise components having no correlation between the channels. Note that if the difference (phase difference or amplitude difference) among N_c1 to N_cM of the channels is known in advance, a method which does not require an adaptive operation such as a fixed beamformer which directs null toward a specific space can be used.

(Residual Noise Remover)

FIG. 4 shows the arrangement of the residual noise remover 302. The residual noise remover 302 includes a phase difference-based noise remover 421. The phase difference-based noise remover 421 receives the output signal X₀ of the noise decorrelator 301 and at least one of the multi-channel input signals X₁ to X_M. The noise remover 421 removes a noise component included in X₀ based on the difference (phase difference) between the phase of X₀ and that of at least one of the signals X₁ to X_M, thereby outputting S₁. The residual noise remover 302 outputs S₁as S₀.

(Phase Difference-Based Noise Remover)

FIG. 5 shows the arrangement of the phase difference-based noise remover 421. The phase difference-based noise remover 421 includes suppression coefficient calculators 501₁ to 501_M, a suppression coefficient integrator 502, and a suppressor 503.

(Suppression Coefficient Calculator)

The suppression coefficient calculators 501₁ to 501_M calculate suppression coefficients W₁ to W_M using the output signal X₀ of the noise decorrelator 301 and the multi-channel input signals X₁ to X_M, respectively. Operations for channels 1 to M are the same, and thus the suppression coefficient calculator 501₁ will be described.

A phase component exp{−jθ_X0} of X₀ input to the suppression coefficient calculator 501₁ is obtained by normalizing equation (2) using an amplitude component |X₀| of X₀, given by:

$\begin{array}{cc}\frac{X_0}{X_0}=\frac{X_0e^{-j{\theta }_{X0}}}{X_0}=e^{-j{\theta }_{X0}}=\frac{S}{\lceil X_0}+\frac{N_{i0}}{X_0}& \left(3\right)\end{array}$ where θ_X0 represents the phase of X₀.

Similarly, a phase component exp{−jθ_X1} of the input signal X₁ of channel 1 is obtained by normalizing equation (1-1) using an amplitude component |X₁| of X₁, given by:

$\begin{array}{cc}\frac{X_1}{X_1}=\frac{X_1e^{-j{\theta }_{X1}}}{X_1}=e^{-j{\theta }_{X1}}=\frac{S}{X_1}+\frac{N_{C1}}{X_1}+\frac{N_{i1}}{X_1}& \left(4\right)\end{array}$ where θ_X1 represents the phase of X₁.

Using the phase component exp{−jθ_X0} of X₀ and the phase component exp{−jθ_X1} of X₁, the suppression coefficient W₁ is calculated by:

$\begin{array}{cc}{W}_1=\mathrm{Real}\left[{e^{-j{\theta }_{X0}}\left(e^{-j{\theta }_{X1}}\right)}^{*}\right]\frac{X_1}{X_0}& \left(5\right)\end{array}$

where Real[] represents an operator for extracting only the real part of a complex number, and * represents a complex conjugate. If |X₀| is nearly equal to |X₁|, a correction term |X₁|/|X₀| of equation (5) can be eliminated. Substituting equations (3) and (4) into equation (5) yields:

$\begin{array}{cc}{W}_1=\mathrm{Real}\left[\left(\frac{S}{X_0}+\frac{N_{i0}}{X_0}\right){\left(\frac{S}{X_1}+\frac{N_{C1}}{X_1}+\frac{N_{i1}}{X_1}\right)}^{*}\right]\frac{X_1}{X_0}& \left(6\right)\end{array}$ The complex spectra S, N_i0, N_C1, and N_i1 are classified into amplitude components and phase components to take a complex conjugate, as given by:

$\begin{array}{cc}{W}_1=\mathrm{Real}\left[\left(\frac{S}{X_0}{e}^{-j{\theta }_S}+\frac{N_{i0}}{X_0}{e}^{-j{\theta }_{\mathrm{Ni}0}}\right)\left(\frac{S}{X_1}{e}^{j{\theta }_S}+\frac{N_{C1}}{X_1}{e}^{j{\theta }_{\mathrm{NC}1}}+\frac{N_{i1}}{X_1}{e}^{j{\theta }_{\mathrm{Ni}1}}\right)\right]\frac{X_1}{X_0}& \left(7\right)\end{array}$

Further arrangement yields:

$\begin{array}{cc}{W}_1=\mathrm{Real}\left[\frac{{S}^2}{X_0X_1}+E_{S1}+E_{N1}\right]\frac{X_1}{X_0}& \left(8\right)\end{array}$ where

$\begin{array}{cc}{E}_{S1}=\frac{\begin{array}{c}SN_{C1}e^{-j\left({\theta }_S-{\theta }_{\mathrm{NC}1}\right)}+\\ SN_{i1}e^{-j\left({\theta }_S-{\theta }_{\mathrm{Ni}1}\right)}+N_{i0}Se^{-j\left({\theta }_{\mathrm{Ni}0}-{\theta }_S\right)}\end{array}}{X_0X_1}& \left(9\right)\\ E_{N1}=\frac{N_{i0}N_{C1}e^{-j\left({\theta }_{\mathrm{Ni}0}-{\theta }_{\mathrm{NC}1}\right)}+N_{i0}N_{i1}e^{-j\left({\theta }_{\mathrm{Ni}0}-{\theta }_{\mathrm{Ni}1}\right)}}{X_0X_1}& \left(10\right)\end{array}$

If the speech component S and noise components N_i0, N_C1, and N_i1 have no correlation, each phase component randomly takes values between −1 to 1 for the real and imaginary parts in the numerator of each of equations (9) and (10). As a result, the estimated values of E_S1 and E_N1 are zero and are negligible. Consequently, equation (8) can be approximately written by:

$\begin{array}{cc}{W}_1\approx \mathrm{Real}\left[\frac{{S}^2}{X_0X_1}\right]\frac{X_1}{X_0}=\frac{{S}^2}{X_0X_1}\frac{X_1}{X_0}=\frac{{S}^2}{{X_0}^2}& \left(11\right)\end{array}$

Note that based on equation (5), equation (11) is rewritten by:

$\begin{array}{cc}{W}_1=\mathrm{Real}\left[e^{-j\left({\theta }_{X0}-{\theta }_{X1}\right)}\right]\frac{X_1}{X_0}\approx \frac{{S}^2}{{X_0}^2}& \left(12\right)\end{array}$ Therefore, W₁ is based on the phase difference (θ_X0-θ_X1) between X₀ and X₁.

Similarly, the suppression coefficient calculator 501_M calculates the suppression coefficient W_M by:

$\begin{array}{cc}{W}_M=\mathrm{Real}\left[{e^{-j{\theta }_{X0}}\left(e^{-j{\theta }_{\mathrm{XM}}}\right)}^{*}\right]\frac{X_M}{X_0}\approx \frac{{S}^2}{{X_0}^2}& \left(13\right)\end{array}$

The suppression coefficient calculators 501₁ to 501_M output W₁ and W_M calculated according to equations (5) and (13), respectively. Note that since |S| and |X₀| take positive numbers, and |S|≤|X₀|, W₁ to W_M may be restricted to fall within the range from 0 to 1, and then output.

(Suppression Coefficient Integrator)

The suppression coefficient integrator 502 receives the suppression coefficients W₁ to W_M from the suppression coefficient calculators 501₁ to 501_M, and outputs an integrated suppression coefficient W_S1. For example, the integrated suppression coefficient W_S1 is obtained by:

$\begin{array}{cc}{W}_{S1}=\mathrm{Ave}\left[W_1,\dots ,W_M\right]\approx \frac{{S}^2}{{X_0}^2}& \left(14\right)\end{array}$

where Ave represents an averaging operator. Note that an averaging operation need not be performed using all the suppression coefficients W₁ to W_M. A suppression coefficient largely different from the average value of all the coefficients may be eliminated, and then an averaging operation may be performed again. Alternatively, an averaging operation may be performed using only the suppression coefficients of channels each of which takes a value falling within a predetermined range, or an averaging operation may be performed using only the suppression coefficients of predetermined channels. Without performing an averaging operation, the suppression coefficient of a predetermined channel may be used or the suppression coefficient of a channel having the maximum value of the suppression coefficients W₁ to W_M may be used so as not to remove a desired speech component.

The suppression coefficient integrator 502 receives the suppression coefficients W₁ to W_M for each frequency f for every time t. Therefore, instead of the averaging operation for only the channels, as given by equation (14), an averaging operation may be performed for near-by frequencies f and close times t.

(Suppressor)

The suppressor 503 receives the integrated suppression coefficient W_S1 and the signal X₀ from the noise decorrelator 301, and removes residual noise included in X₀.

$\begin{array}{cc}{S}_1=\sqrt{W_{S1}}{X}_0\approx \frac{S}{X_0}X_0e^{-j{X}_0}=Se^{-j{X}_0}& \left(15\right)\end{array}$

As indicated by equation (15), the output signal S₁ of the suppressor 503 includes the amplitude component of the desired speech signal as an amplitude component, and the phase component of the signal X₀ from the noise decorrelator 301 as a phase component.

FIG. 6 is a flowchart for explaining a noise removal method according to this embodiment. In step S601, input signals input from a plurality of channels are used to remove noise components having correlation, thereby obtaining one output signal. For example, for simplicity, for M=2, Nc1 and Nc2 are eliminated from equations (1-1) and (1-2), thereby solving S. Since Nc1 and Nc2 have correlation, Nc2 can be written using Nc1. Since Ni1 and Ni2 have no relationship, they remain in an output.

In step S603, suppression coefficients for suppressing noise remaining in the output signal obtained in step S601 are calculated using the phase component of the output signal and the phase components of the input signals.

In step S605, an integrated suppression coefficient is obtained using the average of the suppression coefficients.

The process advances to step S607 to remove the residual noise using the integrated suppression coefficient.

According to this embodiment, the noise decorrelator 301 removes noise components having correlation between the channels, thereby obtaining X₀. X₀ has low correlation with noise components included in the multi-channel input signals X₁ to X_M except for a speech component. Therefore, residual noise can be removed by obtaining a noise suppression coefficient based on the difference between the phase of X₀ and the phase of at least one of X₁ to X_M. According to this embodiment, as indicated by equation (15), it is possible to remove only the noise components without removing the desired speech components.

[Third Embodiment]

A signal processing apparatus according to the third embodiment of the present invention will be described with reference to FIGS. 7 and 8. The signal processing apparatus according to this embodiment is the same as that shown in FIG. 3 according to the second embodiment except that the residual noise remover 302 shown in FIG. 3 is replaced by a residual noise remover 702 shown in FIG. 7. Therefore, only the residual noise remover 702 will be described.

FIG. 7 shows the arrangement of the residual noise remover 702. The residual noise remover 702 includes correctors 722₁ to 722_M and a phase difference-based noise remover 421. The phase difference-based noise remover 421 performs the same operation as that of the phase difference-based noise remover shown in FIG. 4, and is denoted by the same reference, and a description thereof will be omitted.

(Corrector)

The correctors 722₁ to 722_M respectively receive multi-channel input signals X₁ to X_M, and correct the input signals, thereby outputting them. Instead of equation (1-1) to (1-M), the input signals X₁ to X_M are given by:X₁=G₁S+N_C1+N_i1   (16-1)X_M=G_MS+N_CM+N_iM   (16-M)

where G₁ to G_M represent frequency responses to speech components included in channels 1 to M, and complex spectra, respectively. Instead of equation (2), an output signal X₀ of a noise decorrelator 301 is given by:X₀=G₀S+N_i0   (17)

where G₀ represents a frequency response to a speech component, and a complex spectrum. The correctors 722₁ to 722_M perform correction using correction coefficients Q₁ to Q_M so that the speech components in equation (16-1) to (16-M) become identical to the speech component indicated by equation (17). The correction coefficients Q₁ to Q_M are given by:

$\begin{array}{cc}{Q}_1=\frac{G_0}{G_1}& \left(18\text{-}1\right)\\ Q_M=\frac{G_0}{G_M}& \left(18\text{-}M\right)\end{array}$

That is, the input signals X₁ to X_M are multiplied by the correction coefficients Q₁ to Q_M, respectively, given by:Q₁X₁=G₀S+Q₁N_C1+Q₁N_i1   (19-1)Q_MX_M=G₀S+Q_MN_CM+Q_MN_iM   (19-M)

Assume thatG₀S=Ś  (20)Q₁X₁={acute over (X)}₁   (21-1)Q_MX_M={acute over (X)}_M   (21-M)Q₁N_C1=Ń_C1   (22-1)Q_MN_CM=Ń_CM   (22-M)Q₁N_i1=Ń_i1   (23-1)Q_MN_iM=Ń_iM   (23-M)

In this case, equations (19-1) to (19-M) and (17) are written by:{acute over (X)}₁=Ś+Ń_C1+Ń_i1   (24-1){acute over (X)}_M=Ś+Ń_CM+Ń_iM   (24-M)X₀=Ś+N_i0   (25)

By receiving signals X′₁ to X′_M of multiple channels indicated by equations (24-1) to (24-M) and the signal X₀ indicated by equation (25), the phase difference-based noise remover 421 can remove residual noise included in X₀.

The correction coefficients Q₁ to Q_M indicated by equations (18-1) to (18-M) can be predetermined depending on, for example, the arrangement of microphones for acquiring the multi-channel input signals X₁ to X_M, the positions of speakers who speak, and processing contents in the noise decorrelator 301. The correction coefficients Q₁ to Q_M can be calculated using X₀, the signals X₁ to X_M of the multiple channels before correction, and the signals X′₁ to X′_M of the multiple channels after correction. Operations for channels 1 to M are the same, and thus FIG. 8 exemplifies only the case of channel 1. FIG. 8 shows a correction coefficient calculator 801 and a corrector 802 for channel 1. The corrector 802 is the same as the corrector 722₁ except that it exchanges the correction coefficient Q₁ with the correction coefficient calculator 801.

(Correction Coefficient Calculator)

The correction coefficient calculator 801 updates the correction coefficient Q₁ so as to minimize the error between X₀ and X′₁. X₀ and X′₁ have high correlation with respect to only speech components included in both the signals. The LMS (Least Mean Square) method, normalization LMS method, or the like used to update an adaptive filter is used for the update processing.Q₁(f,t+1)=Q₁(f,t)+μX*₁(f,t){X₀(f,t)−{acute over (X)}₁(f,t)}  (26)

where μ represents a step size parameter for adjusting the degree of update.

In this embodiment, even if there are differences between the frequency response G₀ to the speech component included in X₀ indicated by equation (17) and the frequency responses G₁ to G_M to the speech components included in the multi-channel input signals X₁ to X_M indicated by equations (16-1) to (16-M), the correctors 722₁ to 722_M correct the multi-channel input signals X₁ to X_M, respectively. This allows the residual noise remover 702 to remove a residual noise component included in X₀. That is, the signal processing apparatus according to this embodiment can remove only noise components without removing desired speech components.

[Fourth Embodiment]

A signal processing apparatus according to the fourth embodiment of the present invention will be described with reference to FIGS. 9 and 10. The signal processing apparatus according to this embodiment is the same as that according to the second embodiment except that the residual noise remover 302 shown in FIG. 3 is replaced by a residual noise remover 902 shown in FIG. 9. Therefore, only the residual noise remover 902 will be described.

FIG. 9 shows the arrangement of the residual noise remover 902. The residual noise remover 902 includes correctors 922₁ to 922_M, a phase difference-based noise remover 421, and a noise re-remover 923. The operations of the correctors 922₁ to 922_M are the same as those of the corrector 722₁ to 722_M shown in FIG. 7, and the phase difference-based noise remover 421 performs the same operation as that of the phase difference-based noise remover 421 shown in FIG. 4. Thus, a description of the correctors 922₁ to 922_M and phase difference-based noise remover 421 will be omitted.

(Noise Re-remover)

The noise re-remover 923 receives an output signal X₀ of a noise decorrelator, and an output signal S₁ of the phase difference-based noise remover, which is obtained by removing residual noise included in X₀, and re-removes the residual noise included in X₀. FIG. 10 shows the arrangement of the noise re-remover 923. The noise re-remover 923 includes power calculators 1001 and 1002, a residual noise estimator 1003, a re-suppression coefficient calculator 1004, and a suppressor 1005.

(Power Calculator)

The power calculators 1001 and 1002 calculate the power of X₀ and the power of S₁, and output them, respectively. That is, the power calculators 1001 and 1002 respectively output X_0P and S_1P given by:X_0P=|X₀|²=X₁X*₁   (27)S_1P=|S₁|²=S₁S*₁   (28)

(Residual Noise Estimator)

The residual noise estimator 1003 estimates the power of the residual noise using X_0P and S_1P, and outputs it as an estimated noise power. That is, the residual noise estimator 1003 outputs N_0P given by:N_0P=max[0,X_0P−S_1P]  (29)

where max[] represents an operator for acquiring a maximum value.

(Re-Suppression Coefficient Calculator)

The re-suppression coefficient calculator 1004 calculates a re-suppression coefficient W_S0 using X_0P, S_1P, and N_0P, and outputs it. For example,

$\begin{array}{cc}{W}_{S0}\left(f,t\right)=\frac{{\eta }_{\mathrm{DD}}\left(f,t\right)}{1+{\eta }_{\mathrm{DD}}\left(f,t\right)}& \left(30\right)\end{array}$ where η_DD represents a pre-SNR given by:

$\begin{array}{cc}{\eta }_{\mathrm{DD}}\left(f,t\right)=\alpha \frac{W_{S0}\left(f,t-1\right)X_{0P}\left(f,t-1\right)}{N_{0P}\left(f,t-1\right)}+\left(1-\alpha \right)\frac{S_{1P}\left(f,t\right)}{N_{0P}\left(f,t\right)}& \left(31\right)\end{array}$ where α represents a constant, and is predetermined, for example, α=0.98. By combination with a past signal, the estimation accuracy of η_DD is improved.

Furthermore, η_DD may be calculated by:

$\begin{array}{cc}{\eta }_{\mathrm{DD}}\left(f,t\right)=\frac{S_{1\mathrm{PDD}}\left(f,t\right)}{N_{0\mathrm{PDD}}\left(f,t\right)}& \left(32\right)\end{array}$ whereS_1PDD(f,t)=αW_S0(f,t−1)X_0P(f,t−1)+(1−α)S_1P(f,t)   (33)N_0PDD(f,t)=α{1−W_S0(f,t−1)}X_0P(f,t−1)+(1−α)N_0P(f,t)   (34)By separately calculating the denominator and numerator of equation (32) using the past signal, as indicated by equations (33) and (34), the value of η_DD becomes more stable.

Furthermore, S_1P and S_1PDD of equations (31) to (34) can be corrected by the pattern (model) of a desired signal (for example, speech) using a method described in patent literature 3: Japanese Patent No. 4765461.

Instead of using equation (30), the re-suppression coefficient W_S0 may be calculated by:

$\begin{array}{cc}{W}_{S0}\left(f,t\right)=\frac{{\eta }_{\mathrm{DD}}\left(f,t\right)\gamma \left(f,t\right)}{1+{\eta }_{\mathrm{DD}}\left(f,t\right)+{\eta }_{\mathrm{DD}}\left(f,t\right)\gamma \left(f,t\right)}& \left(35\right)\end{array}$ here γ represents a post-SNR given by:

$\begin{array}{cc}\gamma \left(f,t\right)=\frac{X_{0P}\left(f,t\right)}{N_{0P}\left(f,t\right)}& \left(36\right)\end{array}$

By using the current signal X_0P for calculation of the re-suppression coefficient, the suppression accuracy is improved at the rising of a speech signal. N_0PDD of equation (34) may be used as N_0P of the denominator on the right-hand side of equation (36), as a matter of course. A method such as the MMSE STSA (Minimum Mean Square Error Short Time Spectral Amplitude) method or MMSE LSA (Minimum Mean Square Error Log Spectral Amplitude) method, which is different from equations (30) and (35), may be used, as a matter of course.

(Suppressor)

The suppressor 1005 receives the signal X₀ from a noise decorrelator 301 and the re-suppression coefficient W_S0, and removes residual noise included in X₀.S₀=√{square root over (W_S0)}X₀   (37)The suppressor 1005 outputs a signal S₀.

In this embodiment, as indicated by equations (31), (33), and (34), a re-suppression coefficient is calculated by combination with a past signal, or calculated by performing correction by the pattern (model) of a desired signal. As indicated by equation (36), the current signal X_0P is used for calculation of a re-suppression coefficient. This makes it possible to more accurately remove only noise components without removing desired speech components.

[Fifth Embodiment]

A signal processing apparatus according to the fifth embodiment of the present invention will be described with reference to FIGS. 11 and 12. The signal processing apparatus according to this embodiment is the same as that according to the second embodiment except that the residual noise remover 302 shown in FIG. 3 is replaced by a residual noise remover 1102 shown in FIG. 11. Therefore, only the residual noise remover 1102 will be described.

FIG. 11 shows the arrangement of the residual noise remover 1102. The residual noise remover 1102 includes correctors 722₁ to 722_M, a phase difference-based noise remover 421, a noise re-remover 923, and an amplitude-based noise remover 1121. The correctors 722₁ to 722_M perform the same operations as those of the correctors described with reference to FIG. 7, and are denoted by the same reference numerals, and a description thereof will be omitted. The phase difference-based noise remover 421 performs the same operation as that of the phase difference-based noise remover shown in FIG. 4, and is denoted by the same reference numeral, and a description thereof will be omitted. The noise re-remover 923 performs the same operation as that of the noise re-remover shown in FIG. 9, and is denoted by the same reference, and a description thereof will be omitted.

(Amplitude-Based Noise Remover)

The amplitude-based noise remover 1121 receives at least an output signal S₁ of the phase difference-based noise remover 421, removes residual noise included in S₁, and outputs S₂. FIG. 12 shows the arrangement of the amplitude-based noise remover 1121. The amplitude-based noise remover 1121 includes a power calculator 1201, an amplitude-based noise estimator 1202, an amplitude-based suppression coefficient calculator 1203, and a suppressor 1204.

(Power Calculator)

The power calculator 1201 calculates the power of S₁, and outputs it. That is, the power calculator 1201 outputs S_1P given by:S_1P=|S₁|²=S₁S*₁   (38)

(Amplitude-Based Noise Estimator)

The amplitude-based noise estimator 1202 estimates the power of residual noise included in S_1P using at least S_1P, and outputs it. That is, the amplitude-based noise estimator 1202 outputs N_1P given by:N_1P=NE[S_1P]  (39)Note that NE[] represents a noise power estimation operator which can use various noise power estimation methods such as the minimum statistics method and a weighted noise estimation method described in patent literature 4: Japanese Patent No. 4282227.

(Amplitude-Based Suppression Coefficient Calculator)

The amplitude-based suppression coefficient calculator 1203 calculates an amplitude-based suppression coefficient W_S2 using S_1P and N_1P, and outputs it. For example,

$\begin{array}{cc}{W}_{S2}\left(f,t\right)=\frac{{\eta }_{\mathrm{DD}}\left(f,t\right)}{1+{\eta }_{\mathrm{DD}}\left(f,t\right)}& \left(40\right)\end{array}$ where η_DD represents a pre-SNR given by:

$\begin{array}{cc}{\eta }_{\mathrm{DD}}\left(f,t\right)=\alpha \frac{W_{S2}\left(f,t-1\right)S_{1P}\left(f,t-1\right)}{N_{1P}\left(f,t-1\right)}+\left(1-\alpha \right)\max \left[0,\frac{S_{1P}\left(f,t\right)}{N_{1P}\left(f,t\right)}-1\right]& \left(41\right)\end{array}$ where α is a constant, and is predetermined, for example, α=0.98.

Furthermore, η_DD may be calculated by:

$\begin{array}{cc}{\eta }_{\mathrm{DD}}\left(f,t\right)=\frac{S_{1\mathrm{PDD}}\left(f,t\right)}{N_{1\mathrm{PDD}}\left(f,t\right)}& \left(42\right)\end{array}$ whereS_1PDD(f,t)=αW_S2(f,t−1)S_1P(f,t−1)+(1−α)max[0,S_1P(f,t)−N_1P(f,t)]  (43)N_1PDD(f,t)=α{1−W_S2(f,t−1)}S_1P(f,t−1)+(1α)N_1P(f,t)   (44)By separately calculating the denominator and numerator of equation (42) using a past signal, as indicated by equations (43) and (44), the value of η_DD becomes more stable.

Instead of using equation (40), the amplitude-based suppression coefficient W_S2 may be calculated by:

$\begin{array}{cc}{W}_{S2}\left(f,t\right)=\frac{{\eta }_{\mathrm{DD}}\left(f,t\right)\gamma \left(f,t\right)}{1+{\eta }_{\mathrm{DD}}\left(f,t\right)+{\eta }_{\mathrm{DD}}\left(f,t\right)\gamma \left(f,t\right)}& \left(45\right)\end{array}$ where γ represents a post-SNR given by:

$\begin{array}{cc}\gamma \left(f,t\right)=\frac{S_{1P}\left(f,t\right)}{N_{1P}\left(f,t\right)}& \left(46\right)\end{array}$

By using the current signal S_1P for calculation of the amplitude-based suppression coefficient, the suppression accuracy is improved at the rising of a speech signal. N_1PDD of equation (44) may be used as N_1P of the denominator on the right-hand side of equation (46), as a matter of course.

(Suppressor)

The suppressor 1204 receives the signal S₁ from the phase difference-based noise remover 421 and the amplitude-based suppression coefficient W_S2, and removes residual noise included in S₁.S₂=√{square root over (W_S2)}S₁   (47)The suppressor 1204 outputs a signal S₂.

In this embodiment, the amplitude-based noise remover 1121 is used at not the succeeding stage but the preceding stage of the noise re-remover 923. This allows the phase difference-based noise remover 421 to more accurately remove only noise components without removing desired speech components even if E_S1 and E_N1 indicated by equations (9) and (10) are not zero.

[Other Embodiments]

While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions. For example, a microphone unit including the signal processing apparatus according to the above embodiments is incorporated in the present invention.

The present invention is applicable to a system including a plurality of devices or a single apparatus. The present invention is also applicable even when a multi-channel noise removal program for implementing the functions of the embodiments is supplied to the system or apparatus directly or from a remote site. Hence, the present invention also incorporates the program installed in a computer to implement the functions of the present invention by the computer, a medium storing the program, and a WWW (World Wide Web) server that causes a user to download the program. Especially, the present invention incorporates at least a non-transitory computer readable medium storing a program that causes a computer to execute processing steps included in the above-described embodiments.

[Other Expressions of Embodiments]

Some or all of the above-described embodiments can also be described as in the following supplementary notes but are not limited to the followings.

(Supplementary Note 1)

There is provided a signal processing apparatus comprising:

a noise decorrelator that removes noise signals having correlation between at least two input signals, in each of which a desired signal and a noise signal coexist, by receiving the at least two input signals from at least two channels; and

a residual noise remover that removes residual noise included in an output signal of the noise decorrelator based on a phase difference between the output signal of the noise decorrelator and at least one input signal included in the at least two input signals.

(Supplementary Note 2)

There is provided the signal processing apparatus according to supplementary note 1, wherein the residual noise remover includes a phase difference-based noise remover.

(Supplementary Note 3)

There is provided the signal processing apparatus according to supplementary note 2, wherein the phase difference-based noise remover includes

a suppression coefficient calculator that calculates a suppression coefficient based on the phase difference between the output signal of the noise decorrelator and the at least one input signal,

a suppression coefficient integrator that receives the suppression coefficient from the at least one suppression coefficient calculator, and outputs an integrated suppression coefficient, and

a suppressor that suppresses the residual noise included in the output signal of the noise decorrelator using the integrated suppression coefficient from the suppression coefficient integrator.

(Supplementary Note 4)

There is provided the signal processing apparatus according to supplementary note 2 or 3, wherein the residual noise remover includes a corrector that corrects the input signal of each channel at a preceding stage of the phase difference-based noise remover.

(Supplementary Note 5)

There is provided the signal processing apparatus according to any one of supplementary notes 2 to 4, wherein the residual noise remover includes a noise re-remover at a succeeding stage of the phase difference-based noise remover.

(Supplementary Note 6)

There is provided the signal processing apparatus according to supplementary note 5, wherein the noise re-remover includes

a residual noise estimator that estimates a power of the residual noise from a power of the output signal of the noise decorrelator and a power of an output signal of the phase difference-based noise remover,

a re-suppression coefficient calculator that calculates a re-suppression coefficient using the power of the output signal of the noise decorrelator, the power of the output signal of the phase difference-based noise remover, and the estimated power of the residual noise, and

a suppressor that suppresses the residual noise included in the output signal of the noise decorrelator using the re-suppression coefficient from the re-suppression coefficient calculator.

(Supplementary Note 7)

There is provided the signal processing apparatus according to supplementary note 5, wherein the residual noise remover includes an amplitude-based noise remover at the succeeding stage of the phase difference-based noise remover and at a preceding stage of the noise re-remover.

(Supplementary Note 8)

There is provided the signal processing apparatus according to supplementary note 7, wherein the amplitude-based noise remover includes

an amplitude-based noise estimator that estimates a power of noise included in an output signal of the phase difference-based noise remover,

an amplitude-based suppression coefficient calculator that calculates an amplitude-based suppression coefficient using a power of the output signal of the phase difference-based noise remover and the estimated noise power from the amplitude-based noise estimator, and

a suppressor that suppresses noise included in the output signal of the phase difference-based noise remover using the amplitude-based suppression coefficient from the amplitude-based suppression coefficient calculator.

(Supplementary Note 9)

There is a signal processing method comprising:

removing noise signals having correlation between at least two input signals, in each of which a desired signal and a noise signal coexist, by receiving the at least two input signals from at least two channels; and

removing residual noise included in an output signal in the removing the noise signals, based on a phase difference between the output signal in the removing the noise signals and at least one input signal included in the at least two input signals.

(Supplementary Note 10)

There is provided a signal processing program for causing a computer to execute a method, comprising:

removing noise signals having correlation between at least two input signals, in each of which a desired signal and a noise signal coexist, by receiving the at least two input signals from at least two channels; and

removing residual noise included in an output signal in the removing the noise signals, based on a phase difference between the output signal in the removing the noise signals and at least one input signal included in the at least two input signals.

This application claims the benefit of Japanese Patent Application No. 2014-054239, filed on Mar. 17, 2014, which is hereby incorporated by reference in its entirety.

Claims

1. A signal processing apparatus comprising: a processor that includes:a noise decorrelator that removes noise signals having correlation between at least two input signals, in each of which a desired signal and a noise signal coexist, by receiving the at least two input signals from at least two channels; anda residual noise remover that removes residual noise included in an output signal of said noise decorrelator based on a phase difference between the output signal of said noise decorrelator and at least one input signal included in the at least two input signals,wherein said residual noise remover includes a phase difference-based noise remover having:a suppression coefficient calculator that calculates a suppression coefficient based on the phase difference between the output signal of said noise decorrelator and the at least one input signal,a suppression coefficient integrator that receives the suppression coefficient from said at least one suppression coefficient calculator, and outputs an integrated suppression coefficient, anda suppressor that suppresses the residual noise included in the output signal of said noise decorrelator using the integrated suppression coefficient from the suppression coefficient integrator.

2. The signal processing apparatus according to claim 1, wherein said residual noise remover includes a corrector that corrects the input signal of each channel at a preceding stage of said phase difference-based noise remover.

3. The signal processing apparatus according to claim 1, wherein said residual noise remover includes a noise re-remover at a succeeding stage of said phase difference-based noise remover.

4. The signal processing apparatus according to claim 3, wherein said noise re-remover includes a residual noise estimator that estimates a power of the residual noise from a power of the output signal of said noise decorrelator and a power of an output signal of said phase difference-based noise remover,a re-suppression coefficient calculator that calculates a re-suppression coefficient using the power of the output signal of said noise decorrelator, the power of the output signal of said phase difference-based noise remover, and the estimated power of the residual noise, anda suppressor that suppresses the residual noise included in the output signal of said noise decorrelator using the re-suppression coefficient from said re-suppression coefficient calculator.

5. The signal processing apparatus according to claim 3, wherein said residual noise remover includes an amplitude-based noise remover at the succeeding stage of said phase difference-based noise remover and at a preceding stage of said noise re-remover.

6. The signal processing apparatus according to claim 5, wherein said amplitude-based noise remover includes an amplitude-based noise estimator that estimates a power of noise included in an output signal of said phase difference-based noise remover,an amplitude-based suppression coefficient calculator that calculates an amplitude-based suppression coefficient using a power of the output signal of said phase difference-based noise remover and the estimated noise power from said amplitude-based noise estimator, anda suppressor that suppresses noise included in the output signal of said phase difference-based noise remover using the amplitude-based suppression coefficient from the amplitude-based suppression coefficient calculator.

7. A signal processing method comprising: removing noise signals having correlation between at least two input signals, in each of which a desired signal and a noise signal coexist, by receiving the at least two input signals from at least two channels; andremoving residual noise included in an output signal in the removing the noise signals, based on a phase difference between the output signal in the removing the noise signals and at least one input signal included in the at least two input signals,wherein said removing residual noise includes using a phase difference-based noise remover having:a suppression coefficient calculator that calculates a suppression coefficient based on the phase difference between the output signal and the at least one input signal,a suppression coefficient integrator that receives the suppression coefficient from said at least one suppression coefficient calculator, and outputs an integrated suppression coefficient, anda suppressor that suppresses the residual noise included in the output signal using the integrated suppression coefficient from the suppression coefficient integrator.

8. A non-transitory computer readable medium storing a signal processing program for causing a computer to execute a method, comprising: removing noise signals having correlation between at least two input signals, in each of which a desired signal and a noise signal coexist, by receiving the at least two input signals from at least two channels; andremoving residual noise included in an output signal in the removing the noise signals, based on a phase difference between the output signal in the removing the noise signals and at least one input signal included in the at least two input signals,wherein said removing residual noise includes using a phase difference-based noise remover having:a suppression coefficient calculator that calculates a suppression coefficient based on the phase difference between the output signal and the at least one input signal,a suppression coefficient integrator that receives the suppression coefficient from said at least one suppression coefficient calculator, and outputs an integrated suppression coefficient, anda suppressor that suppresses the residual noise included in the output signal using the integrated suppression coefficient from the suppression coefficient integrator.