Patent Grant
8954323
This application is a National Stage of International Application No. PCT/JP2010/051750 filed Feb. 8, 2010, claiming priority based on Japanese Patent Application No. 2009-031109 filed Feb. 13, 2009, the contents of all of which are incorporated herein by reference in their entirety.
The present invention relates to a multichannel acoustic signal processing method, a system therefor, and a program.
One example of the related multichannel acoustic signal processing system is described in Patent literature 1. This system is a system for extracting objective voices by removing out-of-object voices and background noise from mixed acoustic signals of voices and noise of a plurality of talkers observed by a plurality of microphones arbitrarily arranged. Further, the above system is a system capable of detecting the objective voices from the above-mentioned mixed acoustic signals.
While the noise removal system described in the Patent literature 1 explained above aims for detecting and extracting the objective voices from the mixed acoustic signals of voices and noise of a plurality of the talkers observed by a plurality of the microphones arbitrarily arranged, it includes the following problem.
The above problem is that the objective voices cannot be efficiently detected and extracted from the mixed acoustic signals in some cases. The reason thereof is that the signal separation is required in some cases and is not required in some cases, dependent upon microphone signals when it is supposed that a plurality of the microphones are arbitrarily arranged, and for example, the objective voices are detected by employing the signals coming from a plurality of the microphones (microphone signals, namely, input time series signals in
Further, another reason is that the system of the Patent Literature 1 has a configuration of detecting the noise section and the voice section by employing an output of the signal separator 101 for extracting the objective voices. For example, now think about the case of supposing an arrangement of talkers A and B, and microphones A and B as shown in
However, the voice of the talker A mixedly entering the microphone B is few as compared with the voice of the talker B entering the microphone B because a distance between the microphone B and the talker A is far away as compared with a distance between the microphone B and the talker B (see
Thereupon, the present invention has been accomplished in consideration of the above-mentioned problems, and an object thereof lies in providing a multichannel acoustic signal processing system capable of efficiently detecting the objective voices from the input signals of the multichannel.
The present invention for solving the above-mentioned problems is a multichannel acoustic signal processing method of processing input signals of a plurality of channels including voices of a plurality of talkers, comprising: calculating a first feature for each channel from the input signals of a multichannel; calculating an inter-channel similarity of said by-channel first feature; selecting a plurality of the channels of which said similarity is high; separating the signals by employing the input signals of a plurality of the selected channels; and detecting said by-talker voice section or said by-channel voice section with the input signals of a plurality of the channels of which said similarity is low and the signals subjected to said signal separation taken as an input, respectively.
The present invention for solving the above-mentioned problems is a multichannel acoustic signal processing system for processing input signals of a plurality of channels including voices of a plurality of talkers, comprising: a first feature calculator that calculates a first feature for each channel from the input signals of a multichannel; a similarity calculator that calculates an inter-channel similarity of said by-channel first feature; a channel selector that selects a plurality of the channels of which said similarity is high; a signal separator that separates the signals by employing the input signals of a plurality of the selected channels; and a voice detector that detects said by-talker voice section or said by-channel voice section with the input signals of a plurality of the channels of which said similarity is low and the signals subjected to said signal separation taken as an input, respectively.
The present invention for solving the above-mentioned problems is a program for processing input signals of a plurality of channels including voices of a plurality of talkers, said program causing an information processing device to execute: a first feature calculating process of calculating a first feature for each channel from the input signals of a multichannel; a similarity calculating process of calculating an inter-channel similarity of said by-channel first feature; a channel selecting process of selecting a plurality of the channels of which said similarity is high; a signal separating process of separating the signals by employing the input signals of a plurality of the selected channels; and a voice detecting process of detecting said by-talker voice section or said by-channel voice section with the input signals of a plurality of the channels of which said similarity is low and the signals subjected to said signal separation taken as an input, respectively.
The present invention makes it possible to omit the unnecessary calculation, and to efficiently detect the objective voices.
The first exemplary embodiment of the present invention will be explained.
It is assumed that input signals 1 to M are x1(t) to xM(t), respectively. Where, t is an index of time. The first feature calculators 1-1 to 1-M calculate the first features 1 to M from the input signals 1 to M, respectively (step S1).
Where, F1(T) to FM(T) are the features 1 to M calculated from the input signals 1 to M, respectively. T is an index of time, and it is assumed that a plurality of t is one section, and T may be used as an index in its time section. As shown in numerical equations (1-1) to (1-M), each of the first features F1(T) to FM(T) is configured as a vector having an element of an L-dimensional feature (L is a value equal to or more than 1). As the element of the first feature, for example, a time waveform (input signal), a statistics quantity such as an averaged power, a frequency spectrum, a logarithmic spectrum of frequency, a cepstrum, a melcepstrum, a likelihood for a acoustic model, a confidence measure (including entropy) for the acoustic model, a phoneme/syllable recognition result, a voice section length, and the like are thinkable.
It can be assumed that not only the features to be directly obtained from the input signals 1 to M, as described above, but also the by-channel value for a certain criteria, being the acoustic model, are the first feature, respectively. Additionally, the above-mentioned features are only one example, and needless to say, the other features are also acceptable.
Next, the similarity calculator 2 receives the first features 1 to M, and calculates the inter-channel similarity (step S2).
The method of calculating the similarity differs dependent upon the element of the feature. A correlation value, as a rule, is suitable as an index expressive of the similarity. Further, a distance (difference) value becomes an index expressive of the fact that smaller the value, the higher the similarity. Further, with the case that the first feature is the phoneme/syllable recognition result, the method of calculating the similarity is a method of comparing character strings, and a DP matching etc. is utilized for calculating the above similarity in some cases. Additionally, the above-mentioned correlation value and distance value and the like are only one example, and needless to say, the similarity may be calculated with the indexes other than them. Further, the similarities of all combinations of all channels do not need to be calculated, and with a certain channel, out of M channels, taken as a reference, only the similarity for the above channel may be calculated. Further, with a plurality of times T taken as one section, the similarity in the above time section may be calculated. With the case that the voice section length is included in the feature, it is also possible to omit the processing subsequent it for the channel in which no voice section is detected.
The channel selector 3 receives the inter-channel similarity coming from the similarity calculator 2, and selects and groups the channels of which the similarity is high (step S3).
As a selection method, the method of clustering, for example, the method of grouping the channels of which the similarity is higher than a threshold as a result of comparing the similarity with the threshold, and the method of grouping the channels of which the similarity is relatively high are employed. At that moment, the channel that is selected for a plurality of the groups may exist.
Further, the channel that is not selected for any group may exist. The input signals of the channels having a low similarity are not grouped into the input signals of any channel in such a manner, and are outputted to the multichannel voice detector 5.
Additionally, the similarity calculator 2 and the channel selector 3 may perform the processing in such a manner that the channels to be selected are narrowed by repeating the processing for the different features such as the calculation of the similarity and the selection of the channel.
The signal separators 4-1 to 4-N perform the signal separation for each group selected by the channel selector 3 (step S4).
The technique founded upon an independent component analysis, the technique founded upon a mean square error minimization, and the like are employed for the signal separation. While it is expected that the output of each signal separator is low in the similarity, there is a possibility that the outputs of the different signal separators include the output having a high similarity. In that case, some of the outputs resembling each other may be discarded, namely, for example, when three outputs resembling each other exist, two of three outputs may be discarded.
The multichannel voice detector 5 detects the voice of each of a plurality of the talkers in the signals of a plurality of the channels by use of anyone of the channels with the output signals of the signal separators 4-1 to 4-N, and the signals, which have been determined to be low in the similarity by the channel selector 3 and have not been grouped, taken as the input, respectively (step S5).
Herein, it is assumed that the output signals of the signal separators 4-1 to 4-N, and the signals that have been determined to be low in the similarity by the channel selector 3, and have not been grouped (the signals that are not inputted into the signal separators 4-1 to 4-N, and are directly inputted into the multichannel voice detector 5 from the channel selector 3) are y1(t) to yK(t). The multichannel voice detector 5 detects the voices of a plurality of the talkers in the signals of a plurality of the channels from the signals y1(t) to yK(t) with anyone of the channels, respectively. For example, on the assumption that the different voices have been detected in the channels 1 to P, respectively, the signals of the above voice sections are expressed as follows.
Where, ts1, ts2, ts3, . . . , and tsP are start times of the voice section detected in the channel 1 to P, respectively, and te1, te2, te3, . . . , and teP are end times of the voice section detected in the channel 1 to P, respectively (see
The first exemplary embodiment performs the signal separation in a small-scale unit based upon the inter-channel similarity without performing the signal separation for all channels, and further, does not input the channel requiring no signal separation into the signal separators 4-1 to 4-N. For this reason, the signal separation can be efficiently performed as compared with the case of performing the signal separation for all channels. And, performing the multichannel voice detection with the input signals of the channels having a low similarity (the signals that are not inputted into the signal separators 4-1 to 4-N, and are directly inputted into the multichannel voice detector 5 from the channel selector 3), and the signals subjected to the signal separation taken as the input makes it possible to efficiently detect the objective voice.
The second exemplary embodiment of the present invention will be explained.
Additionally, operations of the first feature calculators 1-1 to 1-M, the similarity calculator 2, the channel selector 3, the signal separators 4-1 to 4-N, and the multichannel voice detector 5 of the second exemplary embodiment are similar to those of the first exemplary embodiment, so only the overlapped section detector 6, the second feature calculators 7-1 to 7-P, the crosstalk quantity estimator 8, and the crosstalk remover 9 are explained in the following explanation.
The overlapped section detector 6 receives time information of the start edges and the end edges of the voice sections detected in the channels 1 to P, and detects the overlapped sections (step S6).
The overlapped section, which is a section in which the detected voice sections are overlapped among the channels 1 to P, can be detected from a magnitude relation of ts1, ts2, ts3, . . . , tsP, and te1, te2, te3, . . . , teP as shown in
Next, the second feature calculators 7-1 to 7-P calculate the second features 1 to P from signals y1(t) to yP(t), respectively (step S7).
Where, G1(T) to GP(T) are the second features 1 to P calculated from signals y1(t) to yP(t), respectively. As shown in numerical equations (2-1) to (2-P), each of the second features G1(T) to GP(T) is configured as a vector having an element of an H-dimensional feature (H is a value equal to or more than 1). As the element of the second feature, for example, a time waveform (input signal), a statistics quantity such as an averaged power, a frequency spectrum, a logarithmic spectrum of frequency, a cepstrum, a melcepstrum, a likelihood for a acoustic model, a confidence measure (including entropy) for the acoustic model, a phoneme/syllable recognition result, and the like are thinkable.
It can be assumed that not only the features to be directly obtained from the input signals 1 to P, as described above, but also the by-channel value for a certain criteria, being the acoustic model, are the second feature, respectively. Additionally, the above-mentioned features are only one example, and needless to say, the other features are also acceptable. Further, while all of the voice sections of a plurality of the channels in which at least the voice has been detected may be employed as the section in which the second feature is calculated, the feature can be desirably calculated in the following sections so as to reduce the calculation amount for calculating the second feature.
When the feature is calculated with the first channel, it is desirable to employ the following section of (1)+(2)−(3).
(1) The first voice section detected in the first channel.
(2) The n-th voice section of the n-th channel having the overlapped section common to the above first voice section.
(3) The overlapped section with the m-th voice section of the m-th channel other than the first voice section, out of the n-th voice section.
The above-mentioned sections in which the second feature is calculated will be explained by making a reference to
<When the Channel 1 is the First Channel>
(1) The voice section of the channel 1=(ts1 to te1).
(2) The voice section of the channel P having the overlapped section common to the voice section of the channel 1=(tsP to teP).
(3) The overlapped section with the voice section of the channel 2 other than the voice section of the channel 1, out of the voice section of the channel P,=(ts2 to teP)
The second feature of the section of (1)+(2)−(3)=(ts1 to ts2) is calculated.
<When the Channel 2 is the First Channel>
(1) The voice section of the channel 2=(ts2 to te2).
(2) The voice section of the channel 3 and the voice section of the channel P having the overlapped section common to the voice section of the channel 2=(ts3 to te3 and tsP to teP).
(3) The overlapped section with the voice section of the channel 1 other than the voice section of the channel 2, out of the voice section of the channel 3 and the voice section of the channel P,=(tsP to te1)
The second feature of the section of (1)+(2)−(3)=(te1 to te2) is calculated.
<When the Channel 3 is the First Channel>
(1) The voice section of the channel 3=(ts3 to te3).
(2) The voice section of the channel 2 having the overlapped section common to the voice section of the channel 3=(ts2 to te2).
(3) The overlapped section with the voice section of the channel P other than the voice section of the channel 3, out of the voice section of the channel 2,=(ts2 to teP)
The second feature of the section of (1)+(2)−(3)=(teP to te2) is calculated.
<When the Channel P is the First Channel>
(1) The voice section of the channel P=(tsP to teP).
(2) The voice section of the channel 1 and the voice section of the channel 2 having the overlapped section common to the voice section of the channel P=(ts1 to te1 and ts2 to te2).
(3) The overlapped section with the voice section of the channel 3 other than the voice section of the channel P, out of the voice section of the channel 1 and the voice section of the channel 2,=(ts3 to te3)
The second feature of the section of (1)+(2)−(3)=(ts1 to ts3 and te3 to te2) is calculated.
Additionally, when the calculation of the first feature and that of the second feature are overlapped, needless to say, the latter can be omitted.
Next, the crosstalk quantity estimator 8 estimates magnitude of an influence upon the first voice of the first channel that is exerted by the crosstalk due to the n-th voice of the n-th channel having the overlapped section common to the first voice of the first channel (step S8). The explanation is made with
<Estimation Method 1>
The estimation method 1 compares the feature of the channel 1 with that of the channel P in the section te1 to ts2, being the voice section that does not include the overlapped section. And, it estimates that an influence upon the channel 1 that is exerted by the voice of the channel P is large when the former is close to the latter.
For example, the estimation method 1 compares a power of the channel 1 with that of the channel P in the section te1 to ts2. And, it estimates that an influence upon the channel 1 that is exerted by the voice of the channel P is large when the former is close to the latter. Further, it estimates that an influence upon the channel 1 that is exerted by the voice of the channel P is small when the former is sufficiently larger than the latter.
<Estimation Method 2>
At first, the estimation method 2 calculates a difference of the feature between the channel 1 and the channel P in the section tsP to te1. Next, it calculates a difference of the feature between the channel 1 and the channel P in the section te1 to ts2, being the voice section that does not include the overlapped section. And, it compares the above-mentioned two differences, and estimates that an influence upon the channel 1 that is exerted by the voice of the channel P is large when a difference between the two differences of the features is small.
<Estimation Method 3>
The estimation method 3 calculates a power ratio of the channel 1 and the channel P in the section ts1 to tsP, being the voice section that does not include the overlapped section. Next, it calculates a power ratio of the channel 1 and the channel P in the section te1 to ts2, being the voice section that does not include the overlapped section. And, it employs the above-mentioned two power ratios, and the power of the channel 1 and the power of the channel P in the section tsP to te1, and calculates a power of the crosstalk due to the voice of the channel 1 and the voice of the channel P in the overlapped section tsP to te1 by solving a simultaneous equation. It estimates that an influence upon the channel 1 that is exerted by the voice of the channel P is large when the power of the voice of the channel 1 and the power of the crosstalk are close to each other.
As described above, the estimation method 3 employs at least the voice section that does not include the overlapped section, and estimates an influence of the crosstalk by use of a ratio based upon the inter-channel features, the correlation value, and the distance value.
Needless to say, the crosstalk quantity estimator 8 may estimate an influence of the crosstalk by employing the other methods. Additionally, it is difficult to estimate magnitude of an influence upon the channel 2 that is exerted by the crosstalk due to the voice of the channel 3 because the voice section of the channel 3 of
The crosstalk remover 9 receives the input signals of a plurality of the channels each estimated as the channel that is largely influenced by the crosstalk, and the channel that exerts a large influence as the crosstalk in the crosstalk quantity estimator 8, and removes the crosstalk (step S9).
The technique founded upon an independent component analysis, the technique founded upon a mean square error minimization, and the like are appropriately employed for the removal of the crosstalk. Additionally, in some cases, the crosstalk remover 9 can appropriate a value of a signal separation filter used in the signal separators 4-1 to 4-N to an initial value of the filter for removing the crosstalk.
Further, with the section in which the crosstalk is removed, it is at least the overlapped section. For example, when the power of the channel 1 and that of the channel P in the section te1 to ts2 are compared with each other, and an influence upon the channel 1 that is exerted by the voice of the channel P is estimated to be large, it is assumed that the overlapped section (tsP to te1), out of the voice section (ts1 to te1) of the channel 1, is the section, being a target of the crosstalk processing due to the channel P, and the other sections are not the section, being a target of the crosstalk processing, and only the voice is removed. Doing so makes it possible to reduce the target of the crosstalk processing, and to alleviate a burden of the processing of the crosstalk.
The second exemplary embodiment of the present invention, in addition to the function of the first exemplary embodiment, detects the overlapped section of the voice sections of a plurality of the talkers, and decides the channel, being a target of the crosstalk removal processing, and the section thereof by employing at least the voice section that does not include the detected overlapped section. In particularly, the second exemplary embodiment estimates magnitude of an influence of the crosstalk by employing at least the features of a plurality of the channels in the aforementioned voice section that does not include the overlapped section, and removes the crosstalk of which an influence is large. This makes it possible to omit the calculation for removing the crosstalk of which an influence is small, and to efficiently remove the crosstalk.
Additionally, while in the above-mentioned exemplary embodiments, the explanation was made in such a manner that the section was a section for time, it may be assumed that the section is a section for frequency in some cases, and it may be assumed that the section is a section for time/frequency in some cases. For example, the so-called overlapped section in the case where the section is a section for time/frequency becomes the section in which the voice is overlapped at the identical time and frequency.
Further, while in the above-described exemplary embodiments, the first feature calculators 1-1 to 1-M, the similarity calculator 2, the channel selector 3, the signal separators 4-1 to 4-N, the multichannel voice detector 5, the overlapped section detector 6, the second feature calculators 7-1 to 7-P, the crosstalk quantity estimator 8, and the crosstalk remover 9 were configured with hardware, one part or an entirety thereof can be also configured with an information processing device that operates under a program.
Further, the content of the above-mentioned exemplary embodiments can be expressed as follows.
(Supplementary note 1) A multichannel acoustic signal processing method of processing input signals of a plurality of channels including voices of a plurality of talkers, comprising:
calculating a first feature for each channel from the input signals of a multichannel;
calculating an inter-channel similarity of said by-channel first feature;
selecting a plurality of the channels of which said similarity is high;
separating the signals by employing the input signals of a plurality of the selected channels; and
detecting said by-talker voice section or said by-channel voice section with the input signals of a plurality of the channels of which said similarity is low and the signals subjected to said signal separation taken as an input, respectively.
(Supplementary note 2) A multichannel acoustic signal processing method according to Supplementary note 1, wherein said first feature to be calculated for each channel includes at least one of a time waveform, a statistics quantity, a frequency spectrum, a logarithmic spectrum of frequency, a cepstrum, a melcepstrum, a likelihood for an acoustic model, a confidence measure for an acoustic model, a phoneme recognition result, a syllable recognition result, and a voice section length.
(Supplementary note 3) A multichannel acoustic signal processing method according to Supplementary note 1 or Supplementary note 2, wherein an index expressive of said similarity includes at least one of a correlation value and a distance value.
(Supplementary note 4) A multichannel acoustic signal processing method according to one of Supplementary note 1 to Supplementary note 3, comprising repeating calculation of said by-channel similarity and selection of a plurality of the channels of which the similarity is high a plurality of number of times by employing the different features, and narrowing the channels that are selected.
(Supplementary note 5) A multichannel acoustic signal processing method according to one of Supplementary note 1 to Supplementary note 4, comprising detecting said by-talker voice section correspondingly to anyone of a plurality of the channels.
(Supplementary note 6) A multichannel acoustic signal processing method according to one of Supplementary note 1 to Supplementary note 5, comprising:
detecting an overlapped section, being a section in which said detected voice sections are overlapped between the channels;
deciding the channel, being a target of crosstalk removal processing, and the section thereof by employing at least the voice section that does not include said detected overlapped section; and
removing crosstalk of the section of said channel decided as a target of the crosstalk removal processing.
(Supplementary note 7) A multichannel acoustic signal processing method according to Supplementary note 6, comprising:
estimating an influence of the crosstalk by employing at least the voice section that does not include said detected overlapped section; and
assuming the channel of which an influence of the crosstalk is large, and the section thereof to be a target of the crosstalk removal processing, respectively.
(Supplementary note 8) A multichannel acoustic signal processing method according to Supplementary note 7, comprising determining an influence of the crosstalk by employing at least the input signal of each channel in the voice section that does not include said overlapped section, or a second feature that is calculated from the above input signal.
(Supplementary note 9) A multichannel acoustic signal processing method according to Supplementary note 8, comprising deciding the section in which said second feature is calculated by employing the voice section detected in an m-th channel, the voice section of an n-th channel having the overlapped section common to said voice section of the m-th channel, and the overlapped section with the voice sections of the channels other than the voice section of the m-th channel, out of said voice section of the n-th channel.
(Supplementary note 10) A multichannel acoustic signal processing method according to Supplementary note 8 or Supplementary note 9, wherein said second feature includes at least one of the statistics quantity, the time waveform, the frequency spectrum, the logarithmic spectrum of frequency, the cepstrum, the melcepstrum, the likelihood for the acoustic model, the confidence measure for the acoustic model, the phoneme recognition result, and the syllable recognition result.
(Supplementary note 11) A multichannel acoustic signal processing method according to one of Supplementary note 7 to Supplementary note 10, wherein an index expressive of said influence of the crosstalk includes at least one of a ratio, the correlation value and the distance value.
(Supplementary note 12) A multichannel acoustic signal processing system for processing input signals of a plurality of channels including voices of a plurality of talkers, comprising:
a first feature calculator that calculates a first feature for each channel from the input signals of a multichannel;
a similarity calculator that calculates an inter-channel similarity of said by-channel first feature;
a channel selector that selects a plurality of the channels of which said similarity is high;
a signal separator that separates the signals by employing the input signals of a plurality of the selected channels; and
a voice detector that detects said by-talker voice section or said by-channel voice section with the input signals of a plurality of the channels of which said similarity is low and the signals subjected to said signal separation taken as an input, respectively.
(Supplementary note 13) A multichannel acoustic signal processing system according to Supplementary note 12, wherein said first feature calculator calculates at least one of a time waveform, a statistics quantity, a frequency spectrum, a logarithmic spectrum of frequency, a cepstrum, a melcepstrum, a likelihood for an acoustic model, a confidence measure for an acoustic model, a phoneme recognition result, a syllable recognition result, and a voice section length as the feature.
(Supplementary note 14) A multichannel acoustic signal processing system according to Supplementary note 12 or Supplementary note 13, wherein said similarity calculator calculates at least one of a correlation value and a distance value as an index expressive of said similarity.
(Supplementary note 15) A multichannel acoustic signal processing system according to one of Supplementary note 12 to Supplementary note 14:
wherein said first feature calculator calculates the by-channel different first features by use of different kinds of the features; and
wherein said similarity calculator selects the channels a plurality number of times by employing the different first features, and narrows the channels that are selected.
(Supplementary note 16) A multichannel acoustic signal processing system according to one of Supplementary note 12 to Supplementary note 15, wherein said voice detector detects said by-talker voice section correspondingly to anyone of a plurality of the channels.
(Supplementary note 17) A multichannel acoustic signal processing system according to one of Supplementary note 12 to Supplementary note 16, comprising:
an overlapped section detector that detects an overlapped section, being a section in which said detected voice sections are overlapped between the channels;
a crosstalk processing target decider that decides the channel, being a target of crosstalk removal processing, and the section thereof by employing at least the voice section that does not include said detected overlapped section; and
a crosstalk remover that removes crosstalk of the section of said channel decided as a target of the crosstalk removal processing.
(Supplementary note 18) A multichannel acoustic signal processing system according to Supplementary note 17, wherein said crosstalk processing target decider estimates an influence of the crosstalk by employing at least the voice section that does not include said detected overlapped section, and assumes the channel of which an influence of the crosstalk is large, and the section thereof to be a target of the crosstalk removal processing, respectively.
(Supplementary note 19) A multichannel acoustic signal processing system according to Supplementary note 18, wherein said crosstalk processing target decider determines an influence of the crosstalk by employing at least the input signal of each channel in the voice section that does not include said overlapped section, or a second feature that is calculated from the above input signal.
(Supplementary note 20) A multichannel acoustic signal processing system according to Supplementary note 19, wherein said crosstalk processing target decider decides the section in which said second feature is calculated for each said channel by employing the voice section detected in an m-th channel, the voice section of an n-th channel having the overlapped section common to said voice section of the m-th channel, and the overlapped section with the voice sections of the channels other than the voice section of the m-th channel, out of said voice section of the n-th channel.
(Supplementary note 21) A multichannel acoustic signal processing system according to Supplementary note 19 or Supplementary note 20, wherein said second feature includes at least one of the statistics quantity, the time waveform, the frequency spectrum, the logarithmic spectrum of frequency, the cepstrum, the melcepstrum, the likelihood for the acoustic model, the confidence measure for the acoustic model, the phoneme recognition result, and the syllable recognition result.
(Supplementary note 22) A multichannel acoustic signal processing system according to one of Supplementary note 18 to Supplementary note 21, wherein an index expressive of said influence of the crosstalk includes at least one of a ratio, the correlation value and the distance value.
(Supplementary note 23) A program for processing input signals of a plurality of channels including voices of a plurality of talkers, said program causing an information processing device to execute:
a first feature calculating process of calculating a first feature for each channel from the input signals of a multichannel;
a similarity calculating process of calculating an inter-channel similarity of said by-channel first feature;
a channel selecting process of selecting a plurality of the channels of which said similarity is high;
a signal separating process of separating the signals by employing the input signals of a plurality of the selected channels; and
a voice detecting process of detecting said by-talker voice section or said by-channel voice section with the input signals of a plurality of the channels of which said similarity is low and the signals subjected to said signal separation taken as an input, respectively.
(Supplementary note 24) A program according to Supplementary note 23, wherein said first feature calculating process calculates at least one of a time waveform, a statistics quantity, a frequency spectrum, a logarithmic spectrum of frequency, a cepstrum, a melcepstrum, a likelihood for an acoustic model, a confidence measure for an acoustic model, a phoneme recognition result, a syllable recognition result, and a voice section length as the feature.
(Supplementary note 25) A program according to Supplementary note 23 or Supplementary note 24, wherein said similarity calculating process calculates at least one of a correlation value and a distance value as an index expressive of said similarity.
(Supplementary note 26) A program according to one of Supplementary note 23 to Supplementary note 25:
wherein said first feature calculating process calculates the by-channel different first features by use of different kinds of the features; and
wherein said similarity calculating process selects the channels a plurality number of times by employing the different first features, and narrows the channels that are selected.
(Supplementary note 27) A program according to one of Supplementary note 23 to Supplementary note 26, wherein said voice detecting process detects said by-talker voice section correspondingly to anyone of a plurality of the channels.
(Supplementary note 28) A program according to one of Supplementary note 23 to Supplementary note 27, comprising:
an overlapped section detecting process of detecting an overlapped section, being a section in which said detected voice sections are overlapped between the channels;
a crosstalk processing target deciding process of deciding the channel, being a target of crosstalk removal processing, and the section thereof by employing at least the voice section that does not include said detected overlapped section; and
a crosstalk removing process of removing crosstalk of the section of said channel decided as a target of the crosstalk removal processing.
(Supplementary note 29) A program according to Supplementary note 28, wherein said crosstalk processing target deciding process estimates an influence of the crosstalk by employing at least the voice section that does not include said detected overlapped section, and assumes the channel of which an influence of the crosstalk is large, and the section thereof to be a target of the crosstalk removal processing, respectively.
(Supplementary note 30) A program according to Supplementary note 29, wherein said crosstalk processing target deciding process determines an influence of the crosstalk by employing at least the input signal of each channel in the voice section that does not include said overlapped section, or a second feature that is calculated from the above input signal.
(Supplementary note 31) A program according to Supplementary note 30, wherein said crosstalk processing target deciding process decides the section in which said second feature is calculated for each said channel by employing the voice section detected in an m-th channel, the voice section of an n-th channel having the overlapped section common to said voice section of the m-th channel, and the overlapped section with the voice sections of the channels other than the voice section of the m-th channel, out of said voice section of the n-th channel.
(Supplementary note 32) A program according to Supplementary note 30 or Supplementary note 31, wherein said second feature includes at least one of the statistics quantity, the time waveform, the frequency spectrum, the logarithmic spectrum of frequency, the cepstrum, the melcepstrum, the likelihood for the acoustic model, the confidence measure for the acoustic model, the phoneme recognition result, and the syllable recognition result.
(Supplementary note 33) A program according to one of Supplementary note 29 to Supplementary note 32, wherein an index expressive of said influence of the crosstalk includes at least one of a ratio, the correlation value and the distance value.
Above, although the present invention has been particularly described with reference to the preferred embodiments, it should be readily apparent to those of ordinary skill in the art that the present invention is not always limited to the above-mentioned embodiment, and changes and modifications in the form and details may be made without departing from the spirit and scope of the invention.
This application is based upon and claims the benefit of priority from Japanese patent application No. 2009-031109, filed on Feb. 13, 2009, the disclosure of which is incorporated herein in its entirety by reference.
The present invention may be applied to applications such as a multichannel acoustic signal processing apparatus for separating the mixed acoustic signals of voices and noise of a plurality of talkers observed by a plurality of microphones arbitrarily arranged, and a program for causing a computer to realize a multichannel acoustic signal processing apparatus.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2009-031109 | Feb 2009 | JP | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/JP2010/051750 | 2/8/2010 | WO | 00 | 10/5/2011 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2010/092913 | 8/19/2010 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 7403609 | Hirai et al. | Jul 2008 | B2 |
| 7664643 | Gopinath et al. | Feb 2010 | B2 |
| 20030061185 | Lee et al. | Mar 2003 | A1 |
| 20030120485 | Murase et al. | Jun 2003 | A1 |
| 20050060142 | Visser et al. | Mar 2005 | A1 |
| 20060058983 | Araki et al. | Mar 2006 | A1 |
| 20070021958 | Visser et al. | Jan 2007 | A1 |
| 20070135952 | Chubarev | Jun 2007 | A1 |
| 20080052074 | Gopinath et al. | Feb 2008 | A1 |
| 20080215651 | Sawada et al. | Sep 2008 | A1 |
| 20080228470 | Hiroe | Sep 2008 | A1 |
| 20080262834 | Obata et al. | Oct 2008 | A1 |
| 20090048824 | Amada | Feb 2009 | A1 |
| 20090164212 | Chan et al. | Jun 2009 | A1 |
| 20100092007 | Sun | Apr 2010 | A1 |
| 20100142327 | Kepesi et al. | Jun 2010 | A1 |
| 20100232621 | Aichner et al. | Sep 2010 | A1 |
| 20120197637 | Gratke et al. | Aug 2012 | A1 |
| Number | Date | Country |
|---|---|---|
| 2005-308771 | Nov 2005 | JP |
| 2006-5100869 | Mar 2006 | JP |
| 2008-0892363 | Apr 2008 | JP |
| 2005024788 | Mar 2005 | WO |
| Entry |
|---|
| Wrigley, Brown, Wan and Renals, Speech and Crosstalk Detection in Multichannel Audio, IEEE Transactions on Speech and Audio Processing, pp. 84-91, vol. 13, No. 1, Jan. 2005. |
| Pfau, Ellis, and Stolcke, Multispeaker Speech Activity Detection for the ICSI Meeting Recorder, Proceedings IEEE Automatic Speech Recognition and Understanding Workshop, Madonna di Campiglio, 2001. |
| Jin, Laskowski, Schultz, and Waibel, Speaker Segmentation and Clustering in Meetings, Proceedings of the 8th International Conference on Spoken Language Processing, Jeju Island, Korea, 2004. |
| Huang and Yang, A New Approach of LPC Analysis Based on the Normalization of Vocal-Tract Length, 9th International Conference on Pattern Recognition, pp. 634-636, Nov. 1988. |
| Wolfel, Channel Selection by Class Separability Measures for Automatic Transcriptions on Distant Microphones, Interspeech 2007, Aug. 27-31, Antwerp, Belgium. |
| Obuchi, Yasunari. “Multiple-microphone robust speech recognition using decoder-based channel selection.” ISCA Tutorial and Research Workshop (ITRW) on Statistical and Perceptual Audio Processing. 2004. |
| Wölfel, Matthias, et al. “Multi-source far-distance microphone selection and combination for automatic transcription of lectures.” Interspeech. 2006. |
| Anguera, Xavier, Chuck Wooters, and Javier Hernando. “Acoustic beamforming for speaker diarization of meetings.” Audio, Speech, and Language Processing, IEEE Transactions on 15.7 (2007): 2011-2022. |
| Aarabi, Parham, and Sam Mavandadi. “Robust speech separation using two-stage independent component analysis.” Information Fusion, 2003. Proceedings of the Sixth International Conference of. vol. 2. IEEE, 2003. |
| Asano, Futoshi, et al. “Combined approach of array processing and independent component analysis for blind separation of acoustic signals.” Speech and Audio Processing, IEEE Transactions on 11.3 (2003): 204-215. |
| Winter, Stefan, Hiroshi Sawada, and Shoji Makino. “Geometrical understanding of the PCA subspace method for overdetermined blind source separation.” Acoustics, Speech, and Signal Processing, 2003. Proceedings.(ICASSP'03). 2003 IEEE International Conference on. vol. 2. IEEE, 2003. |
| Number | Date | Country | |
|---|---|---|---|
| 20120046940 A1 | Feb 2012 | US |
