Patent Grant
9648435
The present disclosure relates to sound-source separation method, apparatus, and program that form a directivity toward a sound source located in an arbitrary direction based on a sound wave signal.
In order to precisely separate sound waves of a target sound source, and to suppress target external sound like noises, in general, it is necessary to apply a directional microphone, and to dispose the plural directional microphones side by side at equal to or wider than a certain pitch. However, in the case of a compact sound collector device like an IC recorder, it is difficult to apply the sound collecting technology of employing a directional microphone and of utilizing the plural microphones with a wide pitch. In addition, a precise sound-source separation by an application of such a sound collecting technologies to recorded sound from plural sound sources and having undergone an artificial down-mix process is also difficult.
Hence, a large number of technologies of analyzing an amplitude difference and a phase difference between signals output by respective microphones after recording of sound wave, and performing a signal processing in accordance with an analysis result, thereby separating and extracting a target sound source have been proposed. In recent years, a statistical analysis, a frequency analysis, a complex analysis, etc., are applied to detect a difference in waveform structure of input signals, and the detection result is utilized for a sound-source separation process.
For example, a signal processing such that a conversion from a time axis to a frequency axis is performed on an input signal, a phase difference for each frequency is calculated, a frequency band of an input sound wave from a target sound source is specified based on the calculated difference, and the sound wave within that frequency band is emphasized is performed (see Patent Document 1).
In addition, in the signal processing, it is determined whether or not an input sound wave is in a target direction based on input signals from two microphones closely disposed to each other, a phase difference between the two input signals is corrected, thereby emphasizing sound present in the target direction (see Patent Document 2). The two input signals are referred to each other, and a filter is sequentially updated based on an obtained signal (see Patent Document 3).
Downsizing of sound collector devices or devices equipped with the sound collector device involves a further narrowing of the disposing pitch of microphones, and thus an amplitude difference and a phase difference between signals are quite small. Hence, a large amount of efforts to clearly specify such amplitude difference and phase difference is necessary. This is particularly remarkable in a low-frequency range that has a longer wavelength of several ten times or more than the pitch of two microphones, and in a high-frequency range where the phase difference of sound wave reaching the two microphones becomes equal to or longer than a cycle.
In recent years, as disclosed in Patent Documents 1 to 3, the frequency analysis, the complex analysis, or the statistical analysis to a waveform structure is becoming highly sophisticated, thereby coping with the narrow disposing pitch of microphones. However, the sophistication of the analysis results in an elongation of a frame length, a large number of delay devices, a long filter length, and a long filter coefficient in the case of a conversion to a frequency range. Hence, because of the capacity of the arithmetic processing performance, it becomes difficult to form a real-time directivity. In order to reduce the arithmetic processing load, the number of microphones can be increased, but due to the limited dimension of a device, the pitch between microphones becomes further narrow.
The present disclosure has been made in order to address the above-explained technical problems of conventional technologies, and it is an objective of the present disclosure to provide sound-source separation method, apparatus, and program which can emphasize or suppress and output sound coming from an arbitrary direction with a little amount of calculation using microphones closely disposed to each other and without a highly sophisticated analysis.
To accomplish the above objective, a sound-source separation method according to an embodiment is to form a directivity in a specific direction relative to a pair of input signals, and the method includes:
The sound-source separation method may further include a delaying step of causing, to the other one of the pair of input signals, a delay time that is equal to or longer than a necessary time for sound wave to travel a distance between the pair of microphones,
In the filtering step, filtering may be performed on the one of the pair of input signals by a transfer function T1 that delays the input signal by a specific time,
In the generating and updating steps:
To accomplish the above objective, a sound-source separation apparatus according to an embodiment forms a directivity in a specific direction relative to a pair of input signals, and the apparatus includes:
The filter may perform filtering on the one of the pair of input signals by a transfer function T1 that delays the input signal by the specific time, and
The sound-source separation apparatus may further include a delay that causes, to the other one of the pair of input signals, a delay time that is equal to or longer than a necessary time for sound wave to travel a distance between the pair of microphones,
The filter may perform filtering on the one of the pair of input signals by a transfer function T1 that delays the input signal by a specific time,
The error signal generator and the recurrence formula calculator may:
To accomplish the above objective, a sound-source separation program according to an embodiment causes a computer to form a directivity in a specific direction relative to a pair of input signals, and the program causes the computer to function as:
The filter may perform filtering on the one of the pair of input signals by a transfer function T1 that delays the input signal by the specific time, and
The sound-source separation program may further cause the computer to function as a delay that causes, to the other one of the pair of input signals, a delay time that is equal to or longer than a necessary time for sound wave to travel a distance between the pair of microphones,
The filter may perform filtering on the one of the pair of input signals by a transfer function T1 that delays the input signal by a specific time,
The error signal generator and the recurrence formula calculator may:
According to the present disclosure, by focusing on the time difference caused due to a physical disposing position of the target sound source and those of the microphones, a simple method that emphasizes sound with no dime difference can be applied to relatively emphasize sound from the target sound source present in an arbitrary direction. Hence, the number of calculations can be remarkably reduced, while at the same time, sound wave signals coming from the arbitrary direction can be precisely emphasized without a complex analyze that specifies an amplitude and a difference between signals.
Embodiments of sound-source separation method, apparatus, and program according to the present disclosure will be explained in detail with reference to figures.
(First Embodiment)
(Structure)
This sound-source separation apparatus includes a filter 1a at the subsequent stage to the microphone R located near the target sound source S1. The filter 1a delays the time waveform of the input signal InR(k) by a specific time represented by a transfer function T1. This filter 1a is, for example, an FIR filter or an IIR filter.
The transfer function T1 of the filter 1a can be expressed by the following formula (1). In this formula, C11 is a transfer function of a path from the target sound source S1 in the specific direction to the microphone R. C12 is a transfer function of a pass from the target sound source S1 in the specific direction to the microphone L.
C11×T1≅C12 (1)
By the transfer function T1 that satisfies this formula (1), the filter 1a adjusts the input signal InL(k) and the input signal InR(k) obtained by recording sound wave from the target sound source S1 in the specific direction to have the same amplitude and the same phase, and adds a time difference to an input signal In(L) and an input signal In(R) obtained by recording sound wave coming from a direction out of the specific direction. In this case, the more the direction out of the specific direction is, the larger the time difference to be added becomes.
That is, the transfer function T1 is adjusted in such a way that a delay time represented by the transfer function T1 becomes equivalent to a time difference of the same sound wave reaching the microphones L and R from the target sound source S1.
The input signal InR(k) input from the microphone R and having passed through the filter 1a, and the input signal InL(k) input from the microphone L are distributed to a route where a characteristic correcting circuit 2a, an interchanging circuit 2, and a coefficient updating circuit 3 are connected in series, and a route directed to a synthesizing circuit 4. In addition, this sound-source separation apparatus performs a process of adding, to the input signal InL(k) input from the microphone Land the input signal InR(k) output by the filter 1a, a gain based on a time difference between the input signal InL(k) and the input signal InR(k) using those interchanging circuit 2, coefficient updating circuit 3, and synthesizing circuit 4.
The characteristic correcting circuit 2a includes a frequency-characteristic correcting filter, and a phase-characteristic correcting circuit. The frequency-characteristic correcting filter extracts a sound wave signal in a desired frequency band. The phase-characteristic correcting circuit decreases an effect of the acoustic characteristics of the microphones L and R to the input signal InL(k) and the input signal InR(k).
The interchanging circuit 2 alternately interchanges and outputs the input signal InL(k) and the input signal InR(k) for each sampling. That is, the data sequence of an interchanged signal InA(k) and that of an interchanged signal InB(k) become as follow where k=1, 2, 3, 4 and the like.
The interchanged signal InA(k) and the interchanged signal InB(k) are input to the coefficient updating circuit 3. This coefficient updating circuit 3 calculates an error between the interchanged signal InA(k) and the interchanged signal InB(k), and sets a coefficient m(k) in accordance with the error. In addition, the coefficient updating circuit 3 sequentially updates the coefficient m(k) with reference to a past coefficient m(k−1).
An error signal e(k) between the interchanged signal InA (k) and the interchanged signal InB(k) reaching at the same time is defined as the following formula (2).
e(k)=InB(k)−m(k−1)×InA(k) (2)
This coefficient updating circuit 3 searches the coefficient m(k) that minimizes the error signal e(k) by calculating a recurrence formula between adjacent two terms of the coefficient m(k) containing the error signal e(k) with the error signal e(k) being as the function of the coefficient m(k−1). The larger the time difference between the input signal InL(k) and the input signal InR(k) is, the more the coefficient updating circuit 3 updating the coefficient m(k) decreases such a coefficient, and approximates the coefficient m(k) to 1 when there is no time difference through the arithmetic processing.
The coefficient m(k) is input to the synthesizing circuit 4 together with the input signal InL(k) and the input signal InR(k). The synthesizing circuit 4 multiplies the input signal InL(k) and the input signal InR(k) by the coefficient m(k) at an arbitrary ratio, and adds together at an arbitrary ratio, thereby outputting resultant output signal OutL(k) and output signal OutR(k).
An example coefficient updating circuit 3 will be further explained.
In this coefficient updating circuit 3, the error signal e(k) is generated using the interchanged signal InB(k) as a reference signal. That is, the interchanged signal InA(k) is input to an integrator 5. The integrator 5 multiplies the interchanged signal InA(k) by −1 time of the coefficient m(k−1) one sampling before. An adder 6 is connected to the output side of the integrator 5. The signal output by the integrator 5 and the interchanged signal InB(k) are input to this adder 6, and those signals are added together to obtain a momentary error signal e(k). The error signal e(k) through this arithmetic processing can be expressed as the following formula (3).
e(k)=−m(k−1)×InA(k)+InB(k) (3)
The error signal e(k) is input to an integrator 7 that multiplies the input signal by μ times. The coefficient) μ is a step-size parameter that is smaller than 1. An integrator 8 is connected to the output side of the integrator 7. The interchanged signal InA(k) and a signal μe(k) that has passed through the former integrator are input to the integrator 8. This integrator 8 multiplies the interchanged signal InA(k) by the signal μe(k), and obtains a differential signal ∂E(m)2/∂m of momentary square error that is expressed by the following formula (4).
∂E(m)2/∂m=μ×e(k)×InA(k) (4)
An adder 9 is connected with the integrator 8. The adder 9 completes the coefficient m(k) by calculating the following formula (5), and sets the coefficient m(k) to the synthesizing circuit 4 that generates output signals OutL(k) and OutInR(k) from the input signal InL(k) and InR(k).
m(k)=m(k−1)×β+∂E(m)2/∂m (5)
That is, the adder 9 adds a signal β·m(k−1) to the differential signal ∂E(m)2/∂m, thereby completing the coefficient m(k).
A delay device 10 that delays the signal by what corresponds to a sampling, and an integrator 11 that integrates a constant β are connected to the output side of the adder 9, and the integrator 11 multiplies the coefficient m(k−1) updated through the signal processing one sampling before by the constant β, and thus the signal β·m(k−1) is generated.
Hence, according to the coefficient updating circuit 3, the arithmetic processing of the following recurrence formula (6) is realized, the coefficient m(k) is generated, and is sequentially updated for each sampling.
m(k)=m(k−1)×β+(−m(k−1)×InA(k)+InB(k))×μ×InA(k) (6)
(Action)
It is presumed that a sound velocity is 340 m/s, and a transfer time to the microphone L from each sound source is Y1. In addition, a transfer time to the microphone R from each sound source is presumed as Y2. In this case, a time difference calculated by (Y1−Y2), i.e., a delay time of sound wave which has reached the microphone R and which then reaches the microphone L can be expressed by a graph of
As illustrated in
In this case, the filter 1a delays the input signal InR(k) that has reached the microphone R. It is presumed that the transfer function T1 applies a delay of 0.1159 ms that is a time difference of the same sound wave which reaches the microphones L and R from 80 deg. In this case, as illustrated in
That is, the input signal InL(k) and the input signal InR(k) that have come from 80 deg and output by the microphones L and R have the same amplitude and the same phase in a time waveform, thus emphasized relative to each other
First, as illustrated in
Hence, a gain that relatively emphasizes the output signal OutL(k) and the signal OutInR(k) by the coefficient m(k) can be obtained, and the closer the location of the sound source to the 80-deg direction is, the closer to 1 the coefficient m(k) becomes. Conversely, a gain that relatively suppresses by the coefficient m(k) can be obtained, and the more the location is apart from the 80-deg direction, the smaller the coefficient m (k) becomes which is smaller than 1.
Next, the purpose of the interchanging circuit will be explained. Through the interchanging circuit, the coefficient updating circuit alternatively calculates the following formulae (7).
When k is an odd number
m(k)=m(k−1)×β+(−m(k−1)×InL(k)2+InL(k)×InR(k))×μ
When k is an even number
m(k)=m(k−1)×β+(−m(k−1)×InR(k)2+InR(k)×InL(k))×μ (7)
In the formulae (7), the square term of a signal acts so as to decrease the uncorrelated components like white noises as time advances. Conversely, the adjacent term is equivalent to the numerator part of the following formula (8) that sequentially calculates the correlation coefficient, and the effect of the correlation component is reflected on the coefficient m.
That is, when the coefficient updating circuit 3 attempts to approximate the input signal InR(k) to the input signal InL(k), the uncorrelated components of the input signal InL(k) tend to be amplified, and the uncorrelated components of the input signal InR(k) tend to be suppressed. In addition, when it is attempted to approximate the input signal InL(k) to the input signal InR(k), the uncorrelated components of the input signal InR(k) tend to be amplified, and the uncorrelated components of the input signal InL(k) tend to be suppressed.
Hence, when the interchanging circuit 2 is placed at the forward stage of the coefficient updating circuit 3, an operation of approximating the input signal InR(k) to the input signal InL(k) and performing a synchronous addition, and an operation of approximating the input signal InL(k) to the input signal InR(k) and performing a synchronous addition are alternately repeated. Hence, the operations of amplifying and suppressing the uncorrelated components are canceled with each other, and the effect of the correlation component is well reflected on the coefficient m(k).
(Effects)
As explained above, according to the sound-source separation device of this embodiment, a filtering containing a delay by a specific time is performed on either the one of the pair of input signals input from the microphones L and R. Next, after the filtering, the pair of input signals InL(k) and InR(k) input from the microphones L and R is alternately interchanged by the interchanging circuit 2 for each sampling, and thus the pair of interchanged signals InA(k) and InB(k) is generated. Subsequently, the error signal between the interchanged signals InA(k) and InB(k) is generated by multiplying either one of the interchanged signals InA(k) and InB(k) by the coefficient m. Still further, the recurrence formula of the coefficient m containing the error signal is calculated, and the coefficient m is updated for each sampling. Eventually, the pair of input signals is multiplied by the sequentially updated coefficient m, and output.
For example, by causing the input signal to pass through the filter 1a that has the transfer function T1 which gives a delay of the specific time, the filtering is performed on either one of the pair of input signals InL(k) and InR(k). This transfer function T1 substantially satisfies a condition T1×C11=C12 where C11 is the transfer function of sound wave from the sound source S1 to the microphone that outputs the input signal subjected to the filtering, and C12 is the transfer function of the sound wave to the other microphone.
Either one interchanged signal is caused to pass through the integrator 5 set with −1 time of the past coefficient m calculated one sampling before, and after through the integrator 5, the pair of interchanged signals is caused to pass through the adder 6 that adds both interchanged signals. After through the adder 6, the addition signal is caused to pass through the integrator 7 set with the constant μ, and after through the integrator, the resultant signal is caused to pass through the integrator 8 set with the one interchanged signal prior to the multiplication by the past coefficient m. After through the integrator 8, the resultant signal is caused to pass through the adder 9 set with the past coefficient m calculated one sampling before. Accordingly, the coefficient m is updated for each sampling.
Hence, the sound-source separation apparatus of this embodiment focuses on the time difference caused due to the physical disposing position of the target sound source 1 and those of the microphones L and R, avoids a complex calculation. In addition, even if a phase difference and an amplitude difference between the input signal In(L) and the input signal In(R) are quite small, or conversely, the time difference is equal to or larger than a cycle, the directivity can be easily formed to the target sound S1 in the specific direction out of the center position of the microphones L and R without an analysis.
Still further, the directivity formation can be realized by the interchanging circuit and the one coefficient updating circuit that calculates the recurrence formula without depending on a filter, etc., with a large tap number. Hence, the number of calculations can be remarkably reduced, and the final delay can be set within several ten micro-seconds to several mili-seconds.
The specific direction in which the directivity is formed in this embodiment is merely an example. Needless to say, the specific direction can be set freely in accordance with the adjustment of the transfer function T1 and the selection of the microphone L or R to be equipped with the filter 1a.
(Second Embodiment)
(Structure)
A sound-source separation apparatus according to a second embodiment includes, as illustrated in
The delay time by the delay 1b is set to be equal to or longer than a necessary time for sound wave to travel the distance between the microphones L and R. When the target sound source S1 is present in the 270-deg direction, the difference in reaching time of the sound wave to the microphones L and R becomes the maximum, and the microphone L receives the sound wave before the microphone R. The delay 1b delays the input signal InL(k) by equal to or loner than this maximum time. That is, the input signal InR(k) is always advanced in time waveform more than that of the input signal InL(k).
A transfer function D1 of the delay 1b and the transfer function T1 of the filter 1a are adjusted so as to satisfy the following formula (9). That is, the transfer function T1 is adjusted so as to eliminate a time difference in sound wave coming from the specific direction in consideration of the delay of the input signal InL(k) by the delay 1b.
C11×T1=D1×C12 (9)
(Action)
The positional relationship model in
In this case, as illustrated in
Hence, the time waveform of the input signal InR(k) output by the microphone R is shifted so as to be delayed. The shifting amount is set to be the time difference of the sound wave that reaches the microphones L and R from 280 deg based on a presumption that the target sound source S1 is present in 280 deg. In this case, as illustrated in
Likewise, when the transfer function T1 of the filter 1a satisfies the above-explained formula (9), InR(k) and InL(k) which have come from 280 deg and which are output by the microphones L and R are adjusted so as to have the same amplitude and the same phase in time waveform, and the time difference is eliminated. Hence, those signals are relatively emphasized.
(Effects)
As explained above, according to this sound-source separation apparatus, the one of the pairs of input signals is caused to pass through the filter 1a, while the other one of the pair of input signals is caused to pass through the delay 1b. Hence, a delay time that is equal to or longer than the necessary time for the sound wave to travel the distance between the microphones L and R is caused in the other one of the pair of input signals. Next, the filter 1a performs filtering that considers the time delay obtained by adding the delay time by the delay 1b and the time difference of the sound wave which comes from the target sound source S. More specifically, it is appropriate if the transfer function T1 of the filter 1a and the transfer function D1 of the delay 1b substantially satisfy a condition T1×C11=D1×C12. Hence, no matter where the target sound source S1 is located, the sound wave from this target sound source S can be relatively emphasized.
The specific direction in which the directivity is formed in this embodiment is merely an example. Needless to say, the specific direction can be set freely in accordance with the adjustment of the transfer function T1, that of the transfer function D, and the selection of the microphones L and R to be equipped with the filter 1a.
(Third Embodiment)
A sound-source separation apparatus of a third embodiment generates, in addition to the action of the first embodiment or the second embodiment, a synthesized signal InC(k) obtained by adjusting the time difference and amplitude difference of sound wave coming from a noise source N1 to be zero, and subtracting from the one of the pair of input signals, and a gain process is performed on the synthesized signal InC(k) by the synthesizing circuit 4, thereby relatively enhancing the sensitivity to the target sound source S1 in the specific direction, and further emphasizing sound wave from this target sound source S1.
As illustrated in
A transfer function H1 of the filter 1c satisfies the following formula (10). A transfer function from the noise source N1 to the microphone R is C21, and a transfer function from the noise source N1 to the microphone L is C22.
H1≅−C21/C22 (10)
As is indicated by the formula (10), when the input signal InL(k) passes through the filter 1c, the input signal InL(k) that comes from the noise source N1 and an input signal (R) satisfy a relationship in which the phase is the same but the positive and negative signs of the amplitude are inverted. Hence, through the adder 1d, the smaller the time difference is, the more those input signal InL(k) and the input signal (R) are canceled with each other, and thus the synthesized signal InC(k) that has suppressed sound wave in the 270-deg direction can be generated.
The synthesized signal InC(k) is an output with a directivity that has a low sensitivity in the set direction, and by multiplying the synthesized signal InC(k) by m(k) at an arbitrary ratio, an output Out that has a further intensive directivity can be obtained in comparison with the first embodiment and the second embodiment.
(Other Embodiments)
As explained above, several embodiments of the present disclosure were shown, but those embodiments are merely presented as examples, and are not intended to limit the scope of the present disclosure. Those novel embodiments can be carried out in various other forms, and various omissions, replacements, and modifications can be made thereto without departing from the scope of the present disclosure. Those embodiments and modifications thereof are within the scope and spirit of the present disclosure, and are also within the aspects of the claimed invention and the equivalent range thereto.
For example, in the above-explained embodiment, the explanation was given based on the presumption that the sound-source separation apparatus is provided in a device, such as an IC recorder or a mobile terminal that has a recording function, but can be provided in all other acoustic devices, and instead of the microphones, the input signals In(L) and In(R) may be provided from a memory that stores sound wave data. That is, the expression “a directivity in a specific direction is formed relative to a pair of input signal input from a pair of microphones” means to form a directivity in a specific direction relative to, in addition to the input signals input from the microphones in real time, input signals obtained by recording in advance using a pair of microphones connected with the sound-source separation apparatus, input signals obtained by recording in advance using a pair of completely different microphones, and simulated input signals generated as resembling sound wave recorded by a pair of microphones using a computer, etc.
In addition, as illustrated in
Still further, this sound-source separation apparatus may be realized as the software process by a CPU and a DSP, or may be constructed by a dedicated digital circuit. When realized as the software process, in a computer that includes a CPU, an external memory, and a RAM, a program described with the same process details as those of the filter 1a, the delay 1b, the filter 1c, the adder 1e, the interchanging circuit 2, the coefficient updating circuit 3, and the synthesizing circuit 4 may be stored in a ROM or an external memory, such as a hard disk or a flash memory, extracted in the RAM as needed, and the CPU may perform arithmetic processing in accordance with this program.
This application is a Continuation of PCT Application No. PCT/JP2013/051558, filed on Jan. 25, 2013, the entire content of which is incorporated herein by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 8135142 | Fischer | Mar 2012 | B2 |
| 20090175466 | Elko | Jul 2009 | A1 |
| 20120140948 | Terada | Jun 2012 | A1 |
| 20150213811 | Elko | Jul 2015 | A1 |
| Number | Date | Country |
|---|---|---|
| 2-188050 | Jul 1990 | JP |
| 2005-236407 | Sep 2005 | JP |
| 2009-135593 | Jun 2006 | JP |
| 2007-318528 | Dec 2007 | JP |
| 2009-027388 | Feb 2009 | JP |
| Number | Date | Country | |
|---|---|---|---|
| 20150296318 A1 | Oct 2015 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/JP2013/051558 | Jan 2013 | US |
| Child | 14749699 | US |
