Patent Application
20210327435
The present disclosure relates to a voice processing device, a voice processing method, and a program recording medium.
A voice processing device that calculates speaker characteristics indicating individuality to specify a speaker who utters a voice from a voice signal has been known. A speaker recognition device that estimates the speaker who utters a voice signal using this speaker characteristics has been known.
This type of speaker recognition device using the voice processing device evaluates a similarity between a first speaker characteristic extracted from a first voice signal and a second speaker characteristic extracted from a second voice signal in order to specify the speaker. Then, the speaker recognition device determines whether speakers of the two voice signals are the same on the basis of the evaluation result regarding the similarity.
NPL 1 describes a technique for extracting speaker characteristics from a voice signal. The speaker characteristics extraction technique described in NPL 1 calculates voice statistics using a voice model. Then, the speaker characteristics extraction technique described in NPL 1 processes the voice statistics on the basis of the factor analysis technique and calculates the amount as a vector expressed by the predetermined number of elements. That is, in NPL 1, a speaker characteristics vector is used as the speaker characteristics that indicate the individuality of the speaker.
However, the technique described in NPL 1 has had a problem in that accuracy of speaker recognition using the extracted speaker characteristics is not sufficient.
The technique described in NPL 1 executes predetermined statistical processing on a voice signal input to the speaker characteristics extraction device and calculates a speaker characteristics vector. More specifically, the technique described in NPL 1 calculates individuality characteristics of a speaker who pronounces each sound by executing acoustic analysis processing in partial section units on the voice signal input to the speaker characteristics extraction device and calculates the speaker characteristics vector of the entire voice signal by executing statistical processing on the individuality characteristics. Therefore, according to the technique described in NPL 1, it is not possible to capture the individuality of the speaker that appears in a range wider than the partial section of the voice signal. Therefore, there is a possibility that accuracy of the speaker recognition is deteriorated.
The present disclosure has been made in consideration of the above problems, and an example of an object is to provide a voice processing device, a voice processing method, and a program recording medium that enhance accuracy of speaker recognition.
A voice processing device according to one aspect of the present disclosure includes voice statistics calculation means for calculating voice statistics that indicates an appearance of each type of sound included in a voice signal that indicates a voice and
second feature calculation means for calculating a second feature to recognize specific attribute information on the basis of a temporal change of the voice statistics.
A voice processing method according to one aspect of the present disclosure includes calculating voice statistics that indicates an appearance of each type of sound included in a voice signal that indicates a voice and calculating a second feature to recognize specific attribute information on the basis of a temporal change of the voice statistics.
A program recording medium according to one aspect of the present disclosure records a program for causing a computer to execute processing including processing for calculating voice statistics that indicates an appearance of each type of sound included in a voice signal that indicates a voice and processing for calculating a second feature to recognize specific attribute information on the basis of a temporal change of the voice statistics.
According to the present disclosure, it is possible to provide a voice processing device, a voice processing method, and a program recording medium that enhance accuracy of speaker recognition.
Example embodiments are described below with reference to the drawings. Note that, because components denoted with the same reference numerals in the example embodiments perform similar operations, there is a case where overlapped description is omitted. A direction of an arrow in the drawings indicates an example, and does not limit directions of signals between blocks.
Hardware included in a voice processing device according a first example embodiment and another example embodiment is described.
As illustrated in
The storage device 14 stores a program 18. The processor 11 executes the program 18 related to the voice processing device or the voice processing method using the RAM 12. The program 18 may be stored in the ROM 13. In addition, the program 18 may be recorded in a recording medium 20 and read by a drive device 17 or may be transmitted from an external device via a network.
The input/output interface 15 exchanges data with peripheral devices (keyboard, mouse, display device, or the like) 19. The input/output interface 15 can function as means for acquiring or outputting data. The bus 16 connects the components.
There are various modifications of the method for achieving the voice processing device. For example, each unit of the voice processing device can be achieved as hardware (dedicated circuit). The voice processing device can be achieved by a combination of a plurality of devices.
A processing method for causing a recording medium to record a program (more specifically, program that causes computer to execute processing illustrated in
As the recording medium, for example, a floppy (registered trademark) disk, a hard disk, an optical disk, a magneto-optical disk, a Compact Disc (CD)-ROM, a magnetic tape, a nonvolatile memory card, and a ROM can be used. Not only a single program recorded in the recording medium that executes processing but also a program that operates on an Operating System (OS) and executes processing in cooperation with another software and a function of an expansion board are also included in the scope of each example embodiment.
The voice section detection unit 110 receives a voice signal from outside. The voice signal is a signal representing a voice based on an utterance of a speaker. The voice section detection unit 110 detects and segments a voice section included in the received voice signal. At this time, the voice section detection unit 110 may segment the voice signal into sections having a certain length or into sections having different lengths. For example, the voice section detection unit 110 may determine a section in the voice signal in which a volume is smaller than a predetermined value continuously for a certain period of time as a section having no sound and may determine and segment sections before and after the section having no sound as different voice sections. Then, the voice section detection unit 110 outputs a segmented voice signal that is a segmented result (processing result of voice section detection unit 110) to the voice statistics calculation unit 120. Here, reception of the voice signal means, for example, reception of a voice signal from an external device or another processing device or transfer of a processing result of voice signal processing from another program. An output means, for example, transmission to an external device or another processing device or transfer of the processing result of the voice section detection unit 110 to another program.
The voice statistics calculation unit 120 receives the segmented voice signal from the voice section detection unit 110. The voice statistics calculation unit 120 calculates acoustic characteristics on the basis of the received segmented voice signal and calculates voice statistics regarding a type of sound included in the segmented voice signal using the calculated acoustic characteristics and one or more voice models (to be described later in detail). Here, the type of the sound is, for example, a group defined based on linguistic knowledge such as phonemes. The type of the sound may be a group of sound obtained by clustering the voice signal on the basis of a similarity. Then, the voice statistics calculation unit 120 outputs the calculated voice statistics (processing result of voice statistics calculation unit 120). Hereinafter, voice statistics calculated for a certain voice signal is referred to as voice statistics of the voice signal. The voice statistics calculation unit 120 serves as voice statistics calculation means for calculating voice statistics that indicates an appearance of each type of sound included in a voice signal that indicates a voice.
An example of a method for calculating the voice statistics by the voice statistics calculation unit 120 is described. The voice statistics calculation unit 120 first calculates acoustic characteristics by executing frequency analysis processing on the received voice signal. A procedure for calculating the acoustic characteristics by the voice statistics calculation unit 120 is described.
The voice statistics calculation unit 120 converts, for example, the segmented voice signal received from the voice section detection unit 110 into a short-time frame time series by segmenting the segmented voice signal as a frame each short time and arranging the frames. Then, the voice statistics calculation unit 120 analyzes a frequency of each frame in the short-time frame time series and calculates the acoustic characteristics as a processing result. The voice statistics calculation unit 120 generates, for example, a frame of 25 milliseconds section for each 10 milliseconds as the short-time frame time series.
The voice statistics calculation unit 120 executes, for example, Fast Fourier Transform (FFT) and filter bank processing as the frequency analysis processing in such a way as to calculate frequency filter bank characteristics that are the acoustic characteristics. Alternatively, the voice statistics calculation unit 120 calculates Mel-Frequency Cepstrum Coefficients (MFCC) that are acoustic characteristics, or the like by executing discrete cosine transformation in addition to the FFT and the filter bank processing.
Next, a procedure for calculating the voice statistics using the calculated acoustic characteristics and one or more voice models stored in the voice model storage unit 150 by the voice statistics calculation unit 120 is described.
The voice model storage unit 150 stores one or more voice models. The voice model is configured to identify a type of sound indicated by a voice signal. The voice model stores an association relationship between the acoustic characteristics and the type of the sound. The voice statistics calculation unit 120 calculates a time series of numerical value information that indicates the type of the sound using the time series of the acoustic characteristics and the voice model. The voice model is a model that is trained in advance according to a general optimization reference using a voice signal prepared for training (voice signal for training). The voice model storage unit 150 may store, for example, two or more voice models trained for each of the plurality voice signals for training such as for each gender (male or female) of a speaker, for each recording environment (indoor or outdoor), or the like. In the example in
For example, when the voice model to be used is a Gaussian Mixture Model (GMM), plural element distributions of the GMM are respectively related to different types of the sound. Then, the voice statistics calculation unit 120 extracts a parameter (average, variance) of each of the plurality of element distributions and a mixing coefficient of each element distribution from the voice model (GMM) and calculates a posterior probability of each element distribution on the basis of the calculated acoustic characteristics and the extracted parameter (average, variance) of the element distribution and the extracted mixing coefficient of each element distribution. Here, the posterior probability of each element distribution is an appearance of each type of sound included in a voice signal. A posterior probability Pi (x) of an i-th element distribution of the Gaussian mixture model for a voice signal x can be calculated by the following formula (1).
Here, a function N ( ) represents a probability density function of the Gaussian distribution, the reference θi represents a parameter (mean and variance) of the i-th element distribution of the GMM, and the reference wi represents a mixing coefficient of the i-th element distribution of the GMM.
For example, when the voice model to be used is a neural network, elements of an output layer included in the neural network are respectively related to different sound types. Therefore, the voice statistics calculation unit 120 extracts a parameter (weighting coefficient, bias coefficient) of each element from the voice model (neural network) and calculates an appearance of each type of sound included in the voice signal on the basis of the calculated acoustic characteristics and the extracted parameter (weighting coefficient, bias coefficient) of the element.
The appearance of each type of sound included in the voice signal calculated as described above is voice statistics. The first feature calculation unit 130 receives the voice statistics output from the voice statistics calculation unit 120. The first feature calculation unit 130 calculates a first feature using the voice statistics. The first feature is information to recognize specific attribute information from a voice signal. The first feature calculation unit 130 serves as first feature calculation means for calculating the first feature to recognize the specific attribute information, indicating speaker characteristics, on the basis of the voice statistics.
An example of a procedure for calculating the first feature by the first feature calculation unit 130 is described. Here, an example is described in which the first feature calculation unit 130 calculates a feature vector F (x) based on i-vector as the first feature of the voice signal x. The first feature F (x) calculated by the first feature calculation unit 130 may be a vector that can be calculated by performing predetermined calculation on the voice signal x, and it is sufficient that the first feature F (x) be characteristics of the speaker, and i-vector is an example thereof.
The first feature calculation unit 130 receives, for example, a posterior probability (hereinafter, also referred to as “acoustic posterior probability”) Pt (x) calculated for each short-time frame and acoustic characteristics At (x) (t is natural number equal to or more than one and equal to or less than L, L is natural number equal to or more than one) from the voice statistics calculation unit 120 as the voice statistics of the voice signal x. Pt (x) is a vector of which the number of elements is C. The first feature calculation unit 130 calculates a zero-order statistics S0 (x) of the voice signal x on the basis of the following formula (2) using the acoustic posterior probability Pt (x) and the acoustic characteristics At (x). The, the first feature calculation unit 130 calculates a first-order statistics S1 (x) on the basis of the formula (3).
Subsequently, the first feature calculation unit 130 calculates F (x) that is the i-vector of the voice signal x on the basis of the following formula (4).
F(x)=(1+TTΣ−1S0(x)T)−1TTΣ−1S1(x) (4)
In the above formulas (2) to (4), the reference Pt, c (x) indicates a value of a c-th element of Pt (x), the reference L indicates the number of frames obtained from the voice signal x, the reference S0, c indicates a value of a c-th element of the statistics S0 (x), the reference C indicates the number of elements of the statistics S0 (x) and the number of elements of statistics S1 (x), the reference D indicates the number of elements (the number of dimensions) of the acoustic characteristics At (x), the reference mc indicates a mean vector of acoustic characteristics of a c-th region in an acoustic characteristics space, the reference ID indicates an identity matrix (the number of elements is D×D), and the reference 0 indicates a zero matrix (the number of elements is D×D). The superscript T represents a transpose, and the character T which is not a superscript is a parameter for i-vector calculation. The reference Σ indicates a covariance matrix of the acoustic characteristics in the acoustic characteristics space.
As described above, the first feature calculation unit 130 calculates the feature vector F (x) based on the i-vector as the first feature F (x).
Next, a procedure for calculating a second feature to recognize specific attribute information from a voice signal by the second feature calculation unit 140 is described. The second feature calculation unit 140 serves as second feature calculation means for calculating the second feature to recognize the specific attribute information on the basis of a temporal change of the voice statistics.
First, an example of a method for calculating F2 (x) as the second feature of the voice signal x by the second feature calculation unit 140 is described. The second feature calculation unit 140 receives, for example, an acoustic posterior probability Pt (x) (t is natural number equal to or more than one and equal to or less than T, T is natural number equal to or more than one) calculated for each short-time frame as the voice statistics of the voice signal x from the voice statistics calculation unit 120. The second feature calculation unit 140 calculates an acoustic posterior probability difference ΔPt (x) using the acoustic posterior probability Pt (x). The second feature calculation unit 140 calculates the acoustic posterior probability difference ΔPt (x), for example, by the following formula (5).
ΔPt(x)=Pt(x)−Pt(x) (5)
That is, the second feature calculation unit 140 calculates a difference between acoustic posterior probabilities of which indexes are adjacent to each other (at least two time points) as the acoustic posterior probability difference ΔPt (x). Then, the second feature calculation unit 140 calculates a speaker characteristics vector calculated by replacing At (x) in the formulas (2) to (4) with ΔPt (x) as the second feature F2 (x). Here, the second feature calculation unit 140 may use some indexes, for example, even numbers or odd numbers, instead of using all the indexes t of the acoustic characteristics.
In this way, in the voice processing device 100, the second feature calculation unit 140 calculates the feature vector F2 (x) using the acoustic posterior probability difference ΔPt (x) as information (statistics) that indicates the temporal change of the appearance (voice statistics) of each type of sound included in the voice signal, for the voice signal x. The information that indicates the temporal change of the voice statistics indicates individuality of speaking style of a speaker. That is, the voice processing device 100 can output a feature that indicates the individuality of the speaking style of the speaker.
Next, another example of the method for calculating F2 (x) as the second feature of the voice signal x by the second feature calculation unit 140 is described. The second feature calculation unit 140 receives text information Ln (x) (n is natural number equal to or more than one and equal to or less than N, N is natural number equal to or more than one) that is a symbol string that represents pronunciation (utterance content) of the voice signal x. The text information is, for example, a phoneme string.
The second feature calculation unit 140 associates each element of the text information Ln (x) with each element of the acoustic posterior probability Pt (x). For example, it is assumed that the element of the text information Ln (x) be a phoneme, and the type of sound related to the element of the acoustic posterior probability Pt (x) be a phoneme. At this time, the second feature calculation unit 140 associates each element of the text information Ln (x) with each element of the acoustic posterior probability Pt (x) by using a matching algorithm based on dynamic programming, for example, using the appearance probability value of each phoneme in each index t of the acoustic posterior probability Pt (x) as a score.
A specific description will be made with reference to
The second feature calculation unit 140 associates the acoustic posterior probability Pt(x) with the phoneme using the matching algorithm based on the dynamic programming using the appearance probability value as a score. For example, a “similarity” between the acoustic posterior probability P1 (x) at the time t=1 and the text information “/a/” of an order n=1 is set to “0.7”. Similarly, similarities between all the elements of the acoustic posterior probability Pt (x) and all the elements of the text information are set. Then, on the basis of arrangement restriction of the text information “/a//k//a/”, each frame is associated with the phoneme in such a way as to maximize the similarity.
In
The second feature calculation unit 140 calculates the number of indexes On of the acoustic posterior probability Pt (x) that can be associated with each element of the text information Ln (x).
As illustrated in
The second feature calculation unit 140 calculates a vector having the number of indexes On of the acoustic posterior probability Pt (x) that can be associated with each element of the text information Ln (x) as an element as a second feature F2(x). Each value of the numbers of indexes On represents an utterance time length of each phoneme (symbol) of the text information Ln (x).
In this way, the second feature calculation unit 140 of the voice processing device 100 calculates the feature vector F2(x), for the voice signal x, using the utterance time length of each element of the text information by further using text information that indicates a pronunciation of the voice signal x. With this calculation, the voice processing device 100 can output a feature that indicates the individuality of the speaking style of the speaker.
As described above, in the voice processing device 100 according to the present example embodiment, the first feature calculation unit 130 can calculate the feature vector that indicates the voice characteristics of the speaker. The second feature calculation unit 140 can calculate the feature vector that indicates the individuality of the speaking style of the speaker. As a result, the feature vector considering both the voice characteristics and the speaking style of the speaker can be output for the voice signal. That is, because the voice processing device 100 according to the present example embodiment can calculate the feature vector that indicates at least the individuality of the speaking style of the speaker, speaker characteristics suitable for enhancing accuracy of speaker recognition can be calculated.
Operation of First Example Embodiment
Next, an operation of the voice processing device 100 according to the first example embodiment is described with reference to the flowchart in
The voice processing device 100 receives one or more voice signals from outside and provides the signals to the voice section detection unit 110. The voice section detection unit 110 segments the received voice signal and outputs segmented voice signals to the voice statistics calculation unit 120 (step S101).
The voice statistics calculation unit 120 executes short-time frame analysis processing on each of the one or more received segmented voice signals and calculates time-series acoustic characteristics and time-series voice statistics (step S102).
The first feature calculation unit 130 calculates and outputs a first feature on the basis of the one or more time-series acoustic characteristics and time-series voice statistics that have been received. (step S103).
The second feature calculation unit 140 calculates and outputs a second feature on the basis of the one or more time-series acoustic characteristics and time-series voice statistics that have been received. (step S104). When the reception of the voice signals from outside is completed, the voice processing device 100 terminates the series of processing.
Effects of First Example Embodiment
As described above, according to the voice processing device 100 according to the present example embodiment, it is possible to enhance the speaker recognition using the speaker characteristics calculated by the voice processing device 100. This is because the voice processing device 100 calculates the first feature that indicates the voice characteristics of the speaker by the first feature calculation unit 130 and calculates the second feature that indicates the speaking style of the speaker by the second feature calculation unit 140 in such a way as to output the feature vector considering both of the voice characteristics and the speaking style of the speaker as the feature.
In this way, according to the voice processing device 100 according to the present example embodiment, the feature vector considering the voice characteristics and the speaking style of the speaker is calculated for the voice signal. As a result, even in a case where voice qualities of speakers are similar to each other, a feature suitable for the speaker recognition can be obtained on the basis of a difference in the speeches, for example, a difference in a speed of speaking a word, a timing of switching sounds in a word, or the like.
By using the second feature calculated by the second feature calculation unit 140 described in the first example embodiment, the attribute recognition unit 160 can perform speaker recognition for estimating a speaker of a voice signal.
For example, the attribute recognition unit 160 calculates a cosine similarity as an index that represents a similarity between two second features from a second feature calculated from a first voice signal and a second feature calculated from a second voice signal. For example, for speaker verification, verification determination information based on the similarity may be output.
Furthermore, for speaker identification, the plurality of second voice signals is prepared for the first voice signal, for example, a similarity between the second feature calculated from the first voice signal and each second feature calculated from each of the plurality of second voice signals is obtained, and a pair having the largest similarity may be output.
As described above, according to the second example embodiment, the voice processing device 200 obtains an effect that the attribute recognition unit 160 can perform the speaker recognition for estimating the speaker on the basis of the similarity between the features respectively calculated from the plural voice signals.
The attribute recognition unit 160 may perform the speaker recognition for estimating the speaker of the voice signal using the second feature calculated by the second feature calculation unit 140 and the first feature calculated by the first feature calculation unit 130. As a result, the attribute recognition unit 160 can enhance accuracy of the speaker recognition.
A minimum configuration of an example embodiment of the present disclosure is described.
The voice statistics calculation unit 120 calculates voice statistics that indicates an appearance of each type of sound included in a voice signal that indicates a voice. The second feature calculation unit 140 calculates a second feature to recognize specific attribute information on the basis of a temporal change of the voice statistics.
By adopting the above configuration, according to the third example embodiment, a feature vector that indicates individuality of speaking style of a speaker can be calculated. Therefore, an effect is obtained that accuracy of speaker recognition can be enhanced.
The voice processing device 100 is an example of a feature calculation device that calculates a feature to recognize specific attribute information from a voice signal. When a specific attribute indicates a speaker who utters a voice signal, the voice processing device 100 can be used as a speaker characteristics extraction device. The voice processing device 100 can be used, for example, as a part of a voice recognition device that includes a mechanism for adapting to characteristics of speaking style of a speaker on the basis of speaker information estimated by using the speaker characteristics with respect to a voice signal of a sentence utterance. Here, information indicating the speaker may be information indicating a gender of the speaker or may be information indicating an age or an age group of the speaker.
The voice processing device 100 can be used as a language characteristics calculation device when the specific attribute is assumed as information indicating a language (language configuring voice signal) transferred by the voice signal. The voice processing device 100 can be used, for example, as a part of a voice translation device that includes a mechanism for selecting a language to be translated on the basis of language information estimated by using the language characteristics with respect to a voice signal of a sentence utterance.
The voice processing device 100 can be used as an emotion characteristics calculation device when the specific attribute is assumed as information indicating an emotion of the speaker at the time of speech. The voice processing device 100 can be used, for example, as a part of a voice search device or a voice display device that includes a mechanism for specifying a voice signal related to a specific emotion on the basis of the emotion information estimated by using the emotion characteristics with respect to voice signals of a large number of accumulated utterances. The emotion information includes, for example, information indicating an emotional expression, information indicating personality of a speaker, or the like.
As described above, the specific attribute information according to the present example embodiment is information indicating at least one of the speaker who utters the voice signal, the language configuring the voice signal, the emotional expression included in the voice signal, and the personality of the speaker estimated from the voice signal.
While the present disclosure has been described above with reference to exemplary embodiments, the present disclosure is not limited to these embodiments. It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the claims. That is, it goes without saying that the present disclosure can be variously modified without being limited to the above example embodiments and these modifications are included in the scope of the present disclosure.
As described above, the voice processing device or the like according to one aspect of the present disclosure has an effect that the voice processing device can extract the feature vector considering how to pronounce words in addition to the voice characteristics of the speaker and enhance the accuracy of the speaker recognition and is useful as the voice processing device or the like and the speaker recognition device.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2018/033027 | 9/6/2018 | WO | 00 |
