Patent Application
20090150157
This application is based upon and claims the benefit of priority from the prior Japanese Patent Application No. 2007-316637, filed on Dec. 7, 2007; the entire contents of which are incorporated herein by reference.
The present invention relates to a speech synthesizing apparatus configured for synthesizing speech from a given text and, more specifically, to a speech processing apparatus for entertainment application such as video and game, and a program of the same.
In the related art, a technology of text-speech synthesis for creating speech signals artificially from a given sentence (text) has been proposed. A speech synthesizing apparatus which realizes the text-speech synthesis as such generally includes three units of a language processing unit, prosody processing unit and a speech synthesizing unit.
The speech synthesizing apparatus is operated as follows.
First of all, morpheme analysis or syntax analysis of an entered text is carried out in the language processing unit to divide the text into the unit, for example, of morpheme, word or accent phrase, and generate a phoneme sequence or a part of speech sequence for each unit.
Subsequently, processing of accent or intonation is carried out in a prosody processing unit to calculate information such as a basic frequency and a phonetic sound duration.
Lastly, in a speech synthesizing unit, characteristic parameters or speech waveforms referred to as speech unit data stored for each unit of synthesis, which is a unit of connection of the speech when generating a synthesized speech in advance (for example, phoneme, syllable, etc.), are connected on the basis of the basic frequency or the phonetic sound duration calculated in the prosody processing unit.
The technology of text-speech synthesis as described above is used for speech message outputs of characters in video games (see JP-A-2001-34282 (Kokai)). In the speech message output by reproduction of the recorded speech in the related art, only pre-recorded terms can be reproduced as a speech. However, with the employment of the text-speech synthesis, production of terms which cannot be recorded in advance, such as names entered by players, as a speech is enabled.
As described above, the text-speech synthesis may be used in speech messages of characters in video games, in particular, of humans or human-type robots.
However, there are characters which are not suitable to speak the same language as the human (for example, Japanese language). For example, in the case of a character such as “Intellectually gifted Alien”, it is not unnatural when it speaks language. However, if it speaks Japanese or other existing language, a problem of lack of authenticity arises.
In this case, it is possible to use meaningless effect sounds instead of speech. However, in this case, it does not sound like a language, and hence a problem of lack of authenticity also arises.
Accordingly, it is an object of the present invention to provide a speech processing apparatus for generating a phoneme sequence which is able to be used for speech synthesis to generate a synthesized speech which is meaningless but sounds like a language and has a ring of truth, and a program of the same.
According to embodiments of the present invention, there is provided a speech processing apparatus including an input unit configured to enter a text; a dictionary including sets of a character string which constitutes a word, a phoneme sequence which constitutes pronunciation of the word and a part of speech of the word; a generating unit configured to divide the text into one or more subtexts on the basis of the dictionary and generate speech information including a phoneme sequence for each divided subtext; a determining unit configured to cross-check the speech information of the subtext and a list of speech information stored in advance and determine whether or not to carry out conversion of phonetic sounds which belong to the phoneme sequence of the subtext; and a processing unit configured to (1) convert each phonetic sound in the phoneme sequence of the subtext, which is determined to be carried out the conversion of phonetic sounds, into a different phonetic sound according to a conversion rules stored in advance and output the same, and (2) output the phoneme sequence of the subtext, which is determined not to be carried out the conversion of phonetic sound, without carrying out the conversion.
There is also provided a speech processing apparatus including an input unit configured to enter a text and determination information which indicates portions to be converted and portions not to be converted into different phonetic sounds in the text and, a dictionary including sets of a character string which constitutes a word, a phoneme sequence which constitutes pronunciation of the word and a part of speech of the word; a generating unit configured to divide the text into one or more subtexts on the basis of the dictionary and the determination information and generate information including the phoneme sequence with an attribute indicating whether the conversion is necessary or not for each divided subtext; and a processing unit configured to (1) convert each phonetic sound in the phoneme sequence of the subtext, whose attribute indicates that the conversion is necessary, into a different phonetic sound according to conversion rules stored in advance and output the same, and (2) output the phoneme sequence of the subtext, whose attribute indicates that the conversion is not necessary, without carrying out the conversion.
There is provided a speech processing apparatus including: an input unit configured to enter a text; a first dictionary including sets of a character string which constitutes the word whose phonetic sounds are to be converted, a converted phoneme sequence in which a combination of phonetic sounds which constitutes pronunciation of the word is converted into a combination of different phonetic sounds on the basis of given conversion rules and a part of speech of the word; a second dictionary including sets of a character string which constitutes the word whose phonetic sounds are not to be converted, a no-conversion phoneme sequence which constitutes pronunciation of the word as it is, and a part of speech of the word; and a processing unit configured to (1) divide the text into one or more subtexts on the basis of the first dictionary and the second dictionary, (2) generate the converted phoneme sequence of the subtext included in the first dictionary on the basis of the first dictionary and output the same, and (3) generate the no-conversion phoneme sequence of the subtext included in the second dictionary on the basis of the second dictionary and output the same.
According to an aspect of the invention, a synthesized speech which is meaningless while maintaining its languageness grammatically, phonetically and prosodically.
A speech synthesizing apparatus according to embodiments of the invention will be described below.
Referring now to
Referring now to
The speech synthesizing apparatus includes a text input unit 101 configured to enter a text, a phoneme sequence generating unit 109 configured to generate a phoneme sequence or a part of speech of each word from the text entered from the text input unit 101, a prosody processing unit 103 configured to generates prosody information such as pitch and the duration of each phonetic sound from information described above, a speech synthesizing unit 104 configured to generate a synthesized speech from the phoneme sequence and the prosody information, and a synthesized speech output unit 105 configured to output the synthesized speech generated in the speech synthesizing unit 104.
The speech synthesizing apparatus may be realized also by using, for example, a multipurpose computer apparatus as a basic hardware. In other words, the phoneme sequence generating unit 109, the prosody processing unit 103, and the speech synthesizing unit 104 may be realized by causing a processor mounted to the computer apparatus to execute a program. At this time, the speech synthesizing apparatus may be realized by installing the program in the computer apparatus in advance, or by installing the program to the computer apparatus as needed by storing the program in a storage device such as a CD-ROM or the like and distributing the program via a network. The text input unit 101 may be realized by using a keyboard integrated in the computer apparatus or connected thereto as needed. The synthesized speech output unit 105 may be realized by using a speaker integrated in the computer apparatus or connected thereto and a head phone as needed.
The prosody processing unit 103 and the speech synthesizing unit 104 may be realized by using a prosody processing method and a speech synthesizing method known in the related art, respectively.
For example, in order to generate the pitch in the prosody processing, there is a method of generating a variation pattern of the pitch in one sentence by selecting and connecting the variation pattern of the pitch in each typical accent phrase, and in order to generate the duration of the phonetic sound, there is a method of using an estimation model on the basis of Quantification type 1.
As the speech synthesizing method, there is a method of selecting a speech waveform (speech unit) for each phoneme and syllable according to the phoneme sequence, deforming the prosody according to the prosody information, and connecting the same.
Subsequently, the phoneme sequence generating unit 109 will be described on the basis of
The phoneme sequence generating unit 109 includes a language processing unit 102, a language dictionary storage unit 107, a phonetic sound converting unit 106, a no-conversion list storage unit 108 and a conversion rule storage unit 110 as shown in
The language dictionary storage unit 107 stores information of a number of Japanese words, and the information of each word includes a notation (character string) having Kanji (Chinese characters) and Hiragana (Japanese phonetic sign) mixed therein, a phoneme sequence which constitutes pronunciation, a part of speech, conjugation, an accent position.
The language processing unit 102 analyzes the entered text by referencing the word information stored in the language dictionary storage unit 107, delimiting the entered text into words, and outputs the speech information on each word, such as the phoneme sequence, the part of speech, and the accent position.
The phonetic sound converting unit 106 determines whether to convert the phoneme sequence of the word or not by referencing the list of the speech information stored in the no-conversion list storage unit 108 and, when it is determined to convert, converts the phoneme sequence of the word according to the conversion rules stored in the conversion rule storage unit 110 and outputs the converted phoneme sequence.
Referring now to
The language processing unit 102 analyzes the morpheme of the text entered in the text input unit 101 (Step S101). As an example, an analysis of a text “Taro-san-ohayo (Good morning, Taro) will be described.
The word information in the language dictionary storage unit 107 is referenced and the input text is expressed by a word sequence. The word sequence is not necessarily determined to be one type and, for example, it is expressed in a network as shown in
Subsequently, referring to the rules about easiness of connection between words using the part of speech of the word or the like, scores are given to candidates (network passes) of the analytical result.
Lastly, the scores of the candidates are compared, and the highest likelihood pass is selected, and the character string, the phoneme sequence, and the part of speech of each word are outputted as the analytical result. In this example, since the unique noun is often connected to the suffix, the result shown in
Subsequently, the phonetic sound converting unit 106 references the result of morpheme analysis, and determines whether or not conversion of the phonetic sounds of each word is carried out (Step S102).
The determination is carried out on the basis of the speech information list stored in the no-conversion list storage unit 108. The speech information list is a list having the speech information as elements. The speech information is information obtained for each word as a result of delimiting the entered text into words, and analyzing while referencing the word information, and includes, for example, the phoneme sequence, the character string, the part of speech, and the accent position. The list may include any one of those (for example, the character string), or may include various types mixed therein (for example, the character string and the part of speech). Alternatively, the list may includes a plurality of combinations such as “the character string is ‘Chiba’ and the part of speech is ‘personal name’ ” as elements. An example of the case in which the speech information list is the character string list will be shown in
The character string of each word in the entered word sequence is collated with the character string list. When there is a match, it is determined that the phonetic sound conversion of the word is not to be carried out. When nothing matches, it is determined that the phonetic sound conversion is to be carried out. In this example, since the word “Taro” exists in the character string list, it is determined not to be converted, and since the words “san” and “ohayo” do not exit, they are determined to be converted.
Then, the phonetic sound of the word which is determined to be converted according to the conversion rules stored in the conversion rule storage unit 110 (Step S103).
The conversion of the phonetic sound is an operation to output a different phonetic sound from the entered phonetic sound on the basis of at least the entered phonetic sound and the conversion rules. The conversion rules is used at least when converting the entered phonetic sound to a phonetic sound different from the entered phonetic sound, and is rules to follow when a certain entered phonetic sound is converted into a different phonetic sound.
The conversion of the phonetic sound in the first embodiment is realized by replacing the positions of the phonetic sounds in the word. An example of the conversion rules is shown in
With the speech synthesizing apparatus in the first embodiment, when the text “Taro-san-ohayo.” is entered, a speech “taro-nsa-hayooo” is synthesized.
In this manner, since the phonetic sound and the intonation have the same characteristics as Japanese language, it is possible to synthesize a speech which is meaningless but is provided with “languageness”, so that the speech can be used as the speech of the character in the game.
Since the personal name is pronounced in the same manner even when the language is different, by adapting the apparatus not to convert specific words such as the name entered by the player, reality is effectively increased.
Depending on the method of conversion to use, the text before conversion can be analogized. Therefore, an entertainment property such as to analogize the meaning of the speech spoken by the character in the game is provided.
In the phonetic sound converting unit 106 in the first embodiment, whether to convert or not is determined by referencing the character string list. However, the method of determination is not limited thereto, and may be determined by referencing the phoneme sequence list or the part of speech list.
For example, when the phoneme sequence list includes a registration of the word “Hiroshi”, the words which are pronounced as “Hiroshi” as the personal name are not converted and synthesized with its original phonetic sound irrespective of the Kanji used.
When the part of speech list includes a registration of “Unique Noun”, the unique nouns such as the personal name are not converted. When the input interface of the game cannot be entered with Kanji and only accept Kana input, collation with the phoneme sequence makes mounting easier.
The ratio of the converting portion may be controlled easily by controlling the determination of conversion by the part of speech, and, by increasing the number of parts of speech in the no-conversion list, the portions to be converted are decreased, so that a representation such as “the character is gradually learning Japanese” is created.
Referring now to
The speech synthesizing apparatus according to the second embodiment includes a text synthesizing unit 201, a converted sentence storage unit 203 and a no-conversion sentence storage unit 204 added thereto.
The converted sentence storage unit 203 stores texts whose phonetic sounds are to be converted. The no-conversion sentence storage unit 204 stores texts whose phonetic sounds are not to be converted. For example, texts in established portions of speeches of the game characters are stored in the converted sentence storage unit 203 in advance, and the names entered by the player or the like are registered in the no-conversion sentence storage unit 204.
Referring now to
The text synthesizing unit 201 generates an input text by combining specified texts in the converted sentence storage unit 203 and the no-conversion sentence storage unit 204 (Step S201).
Then, the text synthesizing unit 201 generates determination information which indicates portions whose phonetic sounds are to be converted and portions whose phonetic sounds are not to be converted in the input text (Step S202).
The determination information may be realized by inserting into the input text as a tag or outputting data indicating the boundary positions between the conversion and the non-conversion and discrimination whether to convert or not to convert of each section separately from the input text.
For example, a case in which a list of texts as shown in
An input text is generated by inserting the text specified in
It is also possible to output information “the section of a length from the first character to the second character is the section not to be converted” as a portion-to-be-converted information instead of the tag.
Subsequently, a language processing unit 202 divides an input text into words and generates a character string, phoneme sequence and a part of speech of each word as in the case of the morpheme analysis in the first embodiment (Step S102).
Then, the attribute indicating conversion or no-conversion is given to each word referencing portion-to-be-converted information. An example of output from the language processing unit 202 is show in
Subsequently, a phonetic sound converting unit 206 references the attribute indicating conversion or no-conversion of the output from the language processing unit 202, and determines the word whose phonetic sound is to be converted (Step S204).
Then, the conversion of the phonetic sounds is carried out for the words whose phonetic sounds are determined to be converted according to the conversion rules stored in the conversion rule storage unit 110 (Step S205).
The conversion of the phonetic sound is realized by replacing the positions of the phonetic sounds in the word as in the case of the first embodiment. When the input text is “<no-conversion> Taro </no-conversion>-san-ohayo”, the generated phoneme sequence will be “taro-nsa-hayooo”.
Then, the prosody information is generated in the prosody processing unit 103 on the basis of the phoneme sequence, the speech “taro-nsa-hayooo” is synthesized by the speech synthesizing unit 104 and outputted from the synthesized speech output unit 105.
With the speech synthesizing apparatus according to the second embodiment as well, the speech “taro-nsa-hayooo” is synthesized from the text “Taro-san-ohayo”, and the same advantages as the first embodiment are achieved.
Referring now to
Referring to
A phoneme sequence generating unit 309 in the third embodiment includes a language processing unit 302, a converting language storage unit 307, a no-conversion language storage unit 308, a phonetic sound converting unit 306, the conversion rule storage unit 110, and the language dictionary storage unit 107.
The language processing unit 302 operates by referencing the two language dictionaries; the converting language storage unit 307 and the no-conversion language storage unit 308. Information of the words stored in the converting language storage unit 307 is the same as that stored in the language dictionary storage unit 107. However, the phoneme sequence information is converted in advance on the basis of the conversion rules.
In other words, the phonetic sound converting unit 306 converts the phoneme sequence information of all the words in the language dictionary storage unit 107 on the basis of the conversion rules stored in the conversion rule storage unit 110, and stores the conversion phoneme sequence and other information (such as the character string, the part of speech, conjugation, and the accent position) in the converting language storage unit 307.
Subsequently, the operation of the speech synthesizing apparatus according to the third embodiment will be described.
An example of the word information stored in the language dictionary storage unit 107 is shown in
The phonetic sound converting unit 306 converts the phoneme sequence in the language dictionary storage unit 107 on the basis of the phonetic sound replacement table to generate the word information shown in
It is assumed that the non-conversion language storage unit 308 stores the word information shown in
Assuming that the text “Taro-san-ohayo” is entered from the text input unit 101, the language processing unit 302 carries out the morpheme analysis in the same manner as the language processing unit 102 in the first embodiment, and outputs the character string, the phoneme sequence and the part of speech sequence of each word as the analytic result. However, the language processing unit 302 in the third embodiment references the two language dictionaries; the converting language storage unit 307 and the no-conversion language storage unit 308.
When the word having the same character string exists in the both two dictionaries, the registration in the no-conversion language storage unit 308 is given with a priority to be used for the analysis.
As a consequence, the analytic result shown in
Then, the prosody processing unit 103 generates prosody information on the basis of the phoneme sequence, and the speech synthesizing unit 104 generates the synthesized speech as “taro-nsa-hayooo”, which is outputted from the synthesized speech output unit 105.
With the speech synthesizing apparatus according to the third embodiment as well, when the text “Taro-san-ohayo.” is entered, a speech “taro-nsa-hayooo” is synthesized, and the same effects as the first embodiment are achieved.
The invention is not limited to the above-described embodiments, and may be modified in various manners without departing from the scope of the invention.
In the description in the embodiments shown above, the conversion of the phonetic sound is achieved by the replacement of the positions of the phonetic sounds in the word. However, other conversion rules may be used.
For example, a phonetic sound conversion table as shown in
In any cases of the replacement and the conversion of the phonetic sound, the conversion table does not necessarily have to be fixed and, for example, a plurality of tables may be switched for use.
These tables do not necessarily have to be such that the output is uniquely determined with respect to the input and, for example as the table shown in
It does not necessarily have to be changed periodically, and a configuration in which output probability is provided to a plurality of output phonetic sounds which correspond to one input phonetic sound so that the output is determined on the basis of the probability as shown in the table in
In this manner, the degree of possibility of analogy of the original text from the converted synthesized speech depending on the method of conversion of the phonetic sound, the setting of the game character or conversion suitable to the state of advancement are advantageously possible.
In the description in the embodiments shown above, the word sequence is outputted as a result of processing in the language processing unit 102. However, the invention is not limited thereto, and it may be outputted in the unit of morpheme or accent phrase.
An example in which the accent phrase is employed as the unit in the first embodiment is shown in
The registration of the no-conversion list is “Taro” and it does not match the character string of the accent phrase “Taro-san” completely. However, in this case, it is determined that the conversion is not carried out when the registered word in the non-conversion list is included, the accent phrase “Taro-san” is not converted as a whole.
In the case of the accent phrase including a plurality of words, there is a case in which a plurality of part of speech are allocated to one accent phrase. Therefore, when determination is carried out by the non-conversion list of the part of speech, determination whether there is a match with the part of speech sequence of the accent phrase may be carried out by registering the part of speech sequence (for example, “unique noun+suffix”) to the list, or by registering one part of speech to the list and determining depending on whether it is included in the part of speech sequence of the accent phrase in the same manner as the character string.
The description in the embodiments shown above, the phonetic sound is a syllable. However, the invention is not limited thereto, and may use the unit of mora or phoneme as the phonetic sound.
When the unit of phoneme is employed, consonants which cannot be continued in Japanese language may be continued as a result of conversion, so that the atmosphere as if it is a foreign language is created.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2007-316637 | Dec 2007 | JP | national |
