Patent Application
20050283365
(1) Field of the Invention
The present invention relates to a portable dialogue supporting apparatus which converts a speech-inputted source language sentence into a target language and outputs the converted sentence in speech or text form.
(2) Description of the Related Art
Dialogue supporting systems which translate speech input have been developed as software for workstations and personal computers. In a laboratory environment, performance has reached a practical level for users familiar with their usage, in the case where the scope of conversation is limited to applications for travel conversation, and the like. However, in terms of usability, performance has not yet reached a level that allows usage by the average overseas tourist in an actual trip. In order to improve usability, packaging in readily-portable size hardware and implementation in a user interface that is easy to master and operate, is necessary.
Conventionally, usability is improved by limiting the functions and performance of speech translation software developed for workstations and personal computers, and transplanting these into a Personal Digital Assistant (PDA) (see “AN EXPERIMENTAL MULTILINGUAL SPEECH TRANSLATION SYSTEM”, Kenji Matsui et al., Workshops on Perceptual/Perceptive User Interfaces 2001, ACM Digital Library, ISBN 1-58113-448-7, for example).
In the sample sentence-based translation method using speech input, many thousands of sample sentences need to be prepared in order to cover the average travel conversation, and depending on the result of speech recognition, there are cases where many sample sentences appear as candidates. For example, in the case where words having a high frequency of appearance, in terms of the sample sentences as a whole (for example, ┌ (aru)┘ (be, have), ┌
(en)┘ (Yen), and so on), are accurately recognized in the speech recognition but the rest of the sample sentences cannot be recognized satisfactorily, the number of sample sentences selected as candidates and presented to the user consequently increases. In such cases, it is not easy for the user to select a desired sample sentence from among the candidate-sample sentences. In particular, when the display device for viewing the list of candidate-sample sentences is small and the number of sample sentences that can be viewed simultaneously is considerably small, selecting a desired sample sentence from among the candidates presented becomes even more difficult. Furthermore, when a sample sentence that is similar to the desired sample sentence is found while looking through the candidate-sample sentences, there are many instances where there is hesitation in deciding as to whether to select such sample sentence or to search for a closer matching sample sentence. As such, a function for aiding selection for large amounts of candidate-sample sentences is necessary.
For this reason, the applicant of the present application proposes a speech conversion apparatus that enables a flexible search by searching out a sample sentence based on the dependence relationship of words within the sample sentence, and can improve the usability of displayed details (see Japanese Laid-Open Patent Application No. 2003-288339 Publication, for example).
However, the improvement of sample sentence search accuracy has limitations. In particular, in the case where the display device for viewing the list of candidate-sample sentences is small and the number of sample sentences that can be viewed simultaneously is considerably small, selecting a desired sample sentence from among the insufficiently presented candidates is difficult.
Thus, the present invention is conceived in view of the above-mentioned circumstances, and has as an objective to provide a dialogue supporting apparatus that can easily select a desired sample sentence from among candidate-sample sentences corresponding to inputted speech.
In order to achieve the aforementioned objective, the dialogue supporting apparatus in the present invention is a dialogue supporting apparatus which recognizes speech inputted in a source language, and presents a corresponding translation in a target language, depending on a result of the speech recognition, said dialogue supporting apparatus including a sample sentence search unit operable to search out a sample sentence in the source language based on a keyword included in the speech recognition result, a sample sentence comparison unit operable to compare the speech recognition result and the searched-out sample sentence, and a sample sentence display control unit operable to display the searched-out sample sentence, and to highlight the keyword within the sample sentence based on the comparison of the speech recognition result and the sample sentence.
As is clear from the previous explanation, according to the dialogue supporting apparatus in the present invention, it is possible to highlight-display a keyword included in sample sentences expected to be desired by the user, within a large number of candidate-sample sentences. As such, even in the case where the result of speech recognition is insufficient and a large number of candidates are obtained, it is possible for the user to easily and promptly select a desired sample sentence from the highlight-displayed or sorted candidate-sample sentences. Accordingly, the user is able to search for the desired sample sentence smoothly through speech input, and as the other party in the dialogue is not made to wait, it is possible to carry out a dialogue smoothly via the dialogue supporting apparatus. As the occasions in which use of languages other than the source language increases, the practical value of the dialogue supporting apparatus, which is gaining popularity nowadays, is very high.
The disclosure of Japanese Patent Application No. 2004-117184 filed on Apr. 12, 2004 including specification, drawings and claims is incorporated herein by reference in its entirety.
These and other objects, advantages and features of the invention will become apparent from the following description thereof taken in conjunction with the accompanying drawings that illustrate a specific embodiment of the invention. In the Drawings:
The dialogue supporting apparatus in the embodiments of the present invention is a dialogue supporting apparatus which recognizes speech inputted in a source language, and presents a corresponding translation in a target language, depending on a result of the speech recognition, said dialogue supporting apparatus including a sample sentence search unit operable to search out a sample sentence in the source language based on a keyword included in the speech recognition result, a sample sentence comparison unit operable to compare the speech recognition result and the searched-out sample sentence, and a sample sentence display control unit operable to display the searched-out sample sentence, and to highlight the keyword within the sample sentence based on the comparison of the speech recognition result and the sample sentence.
Accordingly, it is possible to highlight-display a keyword that is included in the sample sentence that is probably desired by the user, among many candidate sample sentences. As such, the user can easily and promptly select a desired sample sentence from the among many candidate sample sentences corresponding to the inputted speech.
Here, it is preferable that said sample sentence comparison unit is operable to derive a keyword correspondence degree of the keyword by comparing a location within the sample sentence, of the keyword included in the speech recognition result and a location of the keyword within the speech recognition result, and said sample sentence display control unit is operable to highlight the keyword within the sample sentence based on the keyword correspondence degree.
Furthermore, it is preferable that said sample sentence comparison unit is operable to derive, based on the keyword correspondence degree, a display parameter for highlighting the keyword, and said sample sentence display control unit is operable to highlight the keyword within the sample sentence based on the display parameter.
Furthermore, it is preferable that said sample sentence comparison unit is operable to derive the location within the sample sentence, of the keyword included in the speech recognition result and the location of the keyword within the speech recognition result, using an appearance location in a character string of one of Japanese phonetic inscription and pronunciation symbols.
Accordingly, deriving the location of keywords can be performed in a short period of time compared to, for example, deriving the location of keywords using the elapsed times in pronunciations according to speech recognition.
Furthermore, it is preferable that said sample sentence comparison unit is operable to change the number of characters in one of the Japanese phonetic inscription and pronunciation symbols depending on a character type, and to count the changed number of characters, in deriving the location within the sample sentence, of the keyword included in the speech recognition result and the location of the keyword within the speech recognition result, using the appearance location in a character string of one of Japanese phonetic inscriptions and pronunciation symbols.
Furthermore, it is preferable that said sample sentence comparison unit is operable to change the number of characters depending on whether the character type is any of a short sound, a prolonged sound, and a choked sound, of the Japanese phonetic inscriptions, and to count the changed number of characters, in the case where the source language is Japanese.
Furthermore, it is preferable that said sample sentence comparison unit is operable to change the number of characters depending on whether the character type is any of a vowel, a long vowel, and a consonant, of the pronunciation symbols, and to count the changed number of characters, in the case where the source language is English.
Furthermore, it is preferable that said sample sentence comparison unit is operable to adopt as the keyword correspondence degree, the difference between the location within the sample sentence, of the keyword included in the speech recognition result and the location of the keyword within the speech recognition result.
Furthermore, it is preferable that said sample sentence comparison unit is operable to adopt as the keyword correspondence degree, a normalized value of the difference between the location within the sample sentence, of the keyword included in the speech recognition result and the location of the keyword within the speech recognition result.
Furthermore, it is preferable that said sample sentence comparison unit is operable to derive a sentence correspondence degree for the sample sentence based on a keyword correspondence degree of each keyword included in the sample sentence.
Here, it is preferable that said sample sentence comparison unit is operable to derive as the sentence correspondence degree, the number of the keywords in the sample sentence, having a keyword correspondence degree that is not lower than a predetermined value.
Furthermore, it is preferable that said sample sentence display control unit is operable to determine, when displaying a plurality of sample sentences, a display order of the sample sentences based on the sentence correspondence degree.
Accordingly, it is possible to preferentially display the sample sentence that is probably desired by the user, from among the many candidate sample sentences. As such, the user can easily and promptly select a desired sample sentence from among the many candidate sample sentences.
Furthermore, the sample sentence searching apparatus in the present invention is a sample sentence searching apparatus which searches out a sample sentence corresponding to an inputted sentence, said sample sentence searching apparatus including a sample sentence search unit operable to search out the sample sentence based on a keyword included in the sentence, a sample sentence comparison unit operable to compare the sentence and the searched-out sample sentence, and a sample sentence display control unit operable to display the searched-out sample sentence, and to highlight the keyword within the sample sentence based on the comparison of the sentence and the sample sentence.
Accordingly, it is possible to highlight-display a keyword that is included in the sample sentence that is probably desired by the user, among many candidate sample sentences. As such, the user can easily and promptly select a desired sample sentence from the among many candidate sample sentences.
Moreover, the present invention can be realized, not only as such a dialogue supporting apparatus, but also as a dialogue support method having the characteristic units included in such dialogue supporting apparatus as steps, and as a program which causes a computer to execute such steps. In addition, it goes without saying that such a program can be distributed via a recording medium such as a CD-ROM, a transmission medium such as the Internet, and so on.
Hereinafter, the embodiments of the present invention shall be explained with reference to the diagrams.
A dialogue supporting apparatus 100 is a dialogue supporting apparatus which performs speech recognition on, and presents translation language (target language) corresponding to, a source language, according to the result of the speech recognition. As shown in
The control unit 101 instructs each component, and controls the flow of information among the respective components. The GUI unit 102 receives and sends, to the control unit 101, the input from a user, and displays information on sample sentences, and so on, from the control unit 101, based on a display parameter. The speech input unit 103 collects the sound of the user's speech. The speech recognition unit 104 performs sequential speech recognition of the user's speech sent from the speech input unit 103. The sample sentence database 105 holds the correspondence of sample sentences in the source language and the target language. The class-word information 106 holds information regarding words in the sample sentence database 105 that have been classified. The word dictionary 107 holds information regarding all words (keywords) used in the sample sentence database 105 and the class-word information 106. The sample sentence selection unit 108 selects one or more sample sentences from within the sample sentence database 105 according to the speech recognition result sent by the control unit 101 or the operation of the GUI unit 102.
The sample sentence comparison unit 112 compares the one or more sample sentences selected by the sample sentence selection unit 108 and the result of the speech recognition by the speech recognition unit 104, and calculates the appearance location of words. Furthermore, the sample sentence comparison unit 112 calculates a word score (keyword correspondence degree) from the appearance locations, and derives a display parameter for each word in each sample sentence, based on such word score. The word selection unit 109, following instructions from the control unit 101, selects a classified word in one of the sample sentences within the sample sentences selected by the sample sentence selection unit 108. The alternative word selection unit 110 refers to the class-word information 106 and selects an alternative word for the classified word identified by the control unit 101. The language conversion unit 111 converts the sample sentence specified by the control unit 101 into the target language by referring to the sample sentence database 105 and the class-word information 106. The speech synthesizer unit 113 converts the sample sentence in the target language, identified by the control unit 101, into synthesized speech. The speech output unit 114 provides the output of the speech synthesizer unit 113 to the user in the form of speech.
In the “Source language:” field, slash marks “/” indicate the separation between words managed in the word dictionary 107. Furthermore, encircled numbers in the “Source language:” field are pointers indicating words recorded in the “Source language components:” field. For example, encircled number 2 in the “Source language:” field in sample sentence 203 indicates encircled number 2 (kakari )┘ (take) in the “Source language components:” field. Accordingly, the “Source language:” field in sample sentence 203 is interpreted in the dialogue supporting apparatus 100 as, “
(nissuu)> (number of days)
(kakarimasu) (take)
(ka) (?) (Does it take <number of days>?)”.
In the dialogue supporting apparatus 100, being classified means “being associated with words that are of the same type or same kind in terms of meaning”. For example, encircled number 1 in the “Source language components:” of sample sentence 203 and encircled number 2 in the “Source language components:” of sample sentence 204 are each classified words. A classified word can be replaced with a word in the same class, defined in the class-word information 106. In the present embodiment, classified words are shown enclosed in inequality marks, for convenience.
(kusuri)┘ (medicine) and
(kudamono)┘ (fruit) are assigned as class names. In the specific example 301, the first lines in the pairs of words in the “Source language:” and the “Target language” belonging to the same class IDs are class representative words. For example, in the class name
kusuri)>(medicine),
(kusuri)┘ (medicine) and “medicine” are class representatives words for c1. The rest of the lines are member words representing specific entities in the class. For example, in c1
(asupirin)┘ (aspirin) and “aspirin”,
(torôchi)┘ (troche) and “troche”, and so on, are member words of c1. Moreover, the class-word information 106 may be structured with hierarchized classes. The word dictionary 107 holds and manages all the words included the class-word information 106 and the words appearing in the “Source language components:” field of the sample sentence database 105. It is structured by an identifier (word ID) carried by each word, which is unique within the present apparatus, their inscription, Japanese phonetic inscription and pointer for the class-word information 106. For example, a word having the word ID “w2” in the specific example 302 has
(inu)┘ (dog) as it's inscription, a Japanese phonetic inscription of
(INU), and class information of “c3” which indicates a class name
(petto)> (pet) in the specific example 301. Moreover, although the inscription and class name of the word is used for convenience in the “Source language:” field and “Source language components:” field in the specific example for the sample sentence database 105 shown in
Next, the operation of the dialogue supporting apparatus 100 structured in the above-mentioned manner shall, for the sake of simplicity, be initially explained without the operation of the sample sentence comparison unit 112.
First, when the button 408 is pressed by the user, as shown in (ano, nanikakusuriwaarimasenka)┘ (Wmm, don't you have any medicine?). The speech input unit 103 transmits the inputted speech to the speech recognition unit 104 (step S501). The speech recognition unit 104 performs speech recognition and outputs the recognition result to the control unit 101 (step S502).
From here on, explanation shall be made for the case where a misrecognition-inclusive recognition result, (nanokakusuriwaarimasuka)┘ (Do you have seven days medicine?) is outputted by the speech recognition unit 104 for the input
(ano, nanikakusuriwaarimasenka┘ (Wmm, don't you have any medicine?.
The control unit 101 commands the sample sentence selection unit 108 to search put sample sentences basing from z,23 (nanokakusuriwaarimasuka)┘ (Do you have seven days medicine?). The sample sentence selection unit 108 searches out sample sentences basing from
(nanokakusuriwaarimasuka)┘ (Do you have seven days medicine?) (step S503). In other words, the sample sentence selection unit 108 extracts, basing from the speech recognition result
(nanokakusuriwaarimasuka)┘ (Do you have seven days medicine?), words appearing in the “Source language components:” field of the sample sentences defined in the sample sentence database 105, in other words, the collection of important words
(nanoka)┘ (seven days),
(kusuri┘ (medicine) and
(ari)┘ (have). Moreover, words belonging to the same class as a class which appears in the “Source language components:” field shall be considered as appearing in the “Source language components:”. For example, as shown in
(nanoka)┘ (seven days) is a member-word of the class-word <
(nissu)> (number of days), and
(kusuri)┘ (medicine) is a member-word of the class-word <
(kusuri)> (medicine). Therefore, both are works appearing in the “Source language components:” field.
The sample sentence selection unit 108 scans a “Source language dependence relationship:” field, and sequentially verifies the dependence relationships for each of the sample sentences in the sample sentence database 105. Subsequently, sample sentences are sequentially selected from among sample sentences having one or more dependence relationships, in the order of most dependence relationships established. In the sample sentences shown in (kakari)┘ (take) is not present among the collection of important words, the number of dependence relationships established is “0”. With regard to sample sentence 204,
(nanika)┘ (any) is not present among the collection of important words. Therefore, among the dependence relationships of components, dependency is not established between the encircled numbers (1→2) but established for the encircled numbers (2→3). Accordingly, the number of established dependence relationships is “1”.
When the sample sentence selection unit 108 is designed to select, from within the sample sentence database 105, a sample sentence having one or more dependency relationships established, sample sentence 204 is selected without sample sentence 203 being selected. In the succeeding explanation, explanation shall be made under the assumption that as other sample sentences within the sample sentence database 105, (kusuridesuka)┘ (Is this medicine?) and
(kusuridesu)┘ (This is medicine) are selected in the same manner.
The control unit 101 transmits the sample sentences transmitted from the sample sentence selection unit 108 to the GUI unit 102. The GUI unit 102 displays the selected sample sentence on the candidate-sample sentence selection area 405, as shown in
When the user selects one of the sample sentences, (nanikakusuriwaarimasuka)┘ (Do you have any medicine?), displayed in the candidate-sample sentence selection area 405 as shown in
(kusuri)┘ (medicine) is extracted. The control unit 101 transmits the word to the GUI unit 102, and the GUI unit 102 underlines
(kusuri)┘ (medicine), displayed in the sample sentence selection result display area 406, to indicate that it is a user-changeable word.
Next, the control unit 101 decides whether to change the word within the selected sample sentence or to perform translation (step S506). In other words, as shown in
On the other hand, when the user clicks on the underlined word area of the sentence in the sample sentence selection result display area 406, as shown in (kusuri)┘ (medicine) selected by the user. Referring to the class-word information 106, the alternative word selection unit 110 extracts and transmits to the control unit 101,
(asupirin)┘ (aspirin),
(kazegusuri)┘ (cold medicine),
(torôchi)┘ (troche), and
(ichouyaku)┘ (digestive medicine), which are member-words of the same class as the word
(kusuri)┘ (medicine) selected by the user (step S507).
Next, the control unit 101 transmits the list of alternative words to the GUI unit 102, and as shown in
When the user clicks on and selects an alternative word 1401, which is the desired word, from within the alternative word list shown in the list window 1301 shown in (asupirin)┘ (aspirin) and transmits this to the control unit 101 (step S509).
Using the alternative word (asupirin)┘ (aspirin) selected by the user, the control unit 101 converts the sample sentence to
(nanikaasupirinwaarimasuka)┘ (Do you have any aspirin?). Subsequently, as shown in
(nanikaasupirinwaarimasuka)┘ (Do you have any aspirin?) and displays the conversion result (step S510). Then, the process returns to the decision process of whether to change the word within the sample sentence or to perform translation (step S506).
Subsequently, the same process (step S507 to S510) is repeated as necessary. As shown in (nanikaasupirinwaarimasuka)┘ (Do you have any aspirin?) is converted to the target language “Do you have any aspirin?” and synthesized speech is outputted.
Next, the operation of the dialogue supporting apparatus 100 when added with the sample sentence comparison unit 112 shall be explained.
Hereinafter, explanation shall be carried out using another specific example of the sample sentence database 105, shown in (
) (imushitsudeichouyakuwamoraemasenka)┘ (Can'I have medicine in the medical room?) after pressing the button 408, as shown in
(u hoshii ichouyaku mora ari)┘ (buy want digestive medicine have is) (
(UHOSHIICHOUYAKUMORAARI) is obtained as the result of speech recognition. Moreover, in the case where the sample sentence comparison unit 112 is not added, one dependence relationship is established for each of sample sentence 1701 to 1704. Accordingly, the sample sentence selection unit 108 outputs all the four sample sentences as candidates, and the user must decide on a desired sample sentence while viewing the many sample sentences displayed on the candidate-sample sentence selection area 405.
The sample sentence comparison unit 112 first calculates the appearance location of words in all of the sample sentences outputted by the sample sentence selection unit 108 as candidates, appearing in the speech recognition result (step S1901). A sample sentence is converted to a Japanese phonetic inscription character string with the use of the word dictionary 107, and the location (number of characters) in the character string is used as the appearance location. For example, sample sentence 1701 is converted into a katakana character string, as shown in (ichouyaku)┘ (digestive medicine) appearing in the speech recognition result, appears in location “0” and
(hoshii)┘ (want) appears in location “6”, as shown in sample sentence 2001. In the same manner, the sample sentence comparison unit 112 also calculates the appearance location of each word in the speech recognition result. For example, as shown in
(ichouyaku)┘ (digestive medicine) included in the speech recognition result has a location of “4”.
Next, the sample sentence comparison unit 112 normalizes the appearance location of the words using the length of the Japanese phonetic inscription character string (step S1902). For example, the length of the Japanese phonetic inscription for sample sentence 1701 is 13 words and the appearance location of (ichouyaku)┘ (digestive medicine) and
(hoshii)┘ (want) are normalized as 0/13=0 and 6/13=0.46, respectively. In the same manner, the sample sentence comparison unit 112 also normalizes the appearance location of each word in the speech recognition result. For example, as shown in
(ichouyaku)┘ (digestive medicine) included in the speech recognition result, is normalized as 4/13=0.31.
Next, the sample sentence comparison unit 112 calculates, from the location of a word within a sample sentence and the location of a word in the speech recognition result, the word score for every word in each sample sentence, that is included in the speech recognition result (step S1903). The word score is calculated using the following formula:
(Word score)=(location in sample sentence)−(location in speech recognition result)
For example, with regard to sample sentence 1703, the score for (ichouyaku)┘ (digestive medicine) is 0.29−0.31=0.2, and the score for
(mora)┘ (have) is 0.65−0.69=−0.04, as shown in sample sentence 2101 in
Next, the sample sentence comparison unit 112 derives a display parameter for each word of each sample sentence, based on the calculated word scores (step S1904). For example, when a parameter is previously set in which an underline is placed when the absolute value for a word score is 0.1 or lower, the GUI unit 102 places an underline under (ichouyaku)┘ (digestive medicine) and
(mora)┘ (have) of sample sentence 1703 and
(ichouyaku)┘ (digestive medicine) of sample sentence 1704, which fall under such condition, and displays these candidate sample sentences on the candidate-sample sentence selection area 405, as shown in
(imushitsudeichouyakuwamoraemasuka)┘ (Can I have medicine in the medical room?), from among the many candidate-sample sentences.
Furthermore, although the sample sentence comparison unit 112 calculates the display parameter based on the word score in the explanation above, in addition, it also is possible to change the display order of the sample sentences by deriving a sentence score (sentence correspondence degree) on a per-sample sentence basis.
The sample sentence comparison unit 112 calculates the number of words having a word score absolute value of, for example, 0.1 or lower, as the sentence score for each sample sentence (step S2301). With respect to the example shown in (imushitsudeichouyakuwamoraemasuka)┘ (Can I have medicine in the medical room?), from among the many candidate-sample sentences.
Here, although rearrangement of sample sentences is performed based on the sentence score, it is not limited to such, and it is possible to display a sample sentence having, for example, a sentence score equaling or exceeding a predetermined value, by highlighting the entire sentence.
Next, the operation of the present embodiment shall be explained in the case where the sample sentence comparison unit 112 processes English as the source language. Moreover, although explanation shall be carried out using the flowchart in
Hereinafter, explanation shall be made using another specific example of the sample sentence database 105, shown in
In the case where the source language is English, the sample sentence comparison unit 112 first calculates the appearance location of the words in all the sample sentences outputted by the sample sentence selection unit 108 as candidates, appearing in the speech recognition result, (step S1901). A sample sentence is converted to a pronunciation symbol character string with the use of the word dictionary 107, and the location (number of characters) in the character string is used as the appearance location. For example, sample sentence 2601 is converted into a pronunciation symbol character string, as shown in
Here, a rule is applied in which consonants inscribed in italics, in other words consonants uttered lightly, are deleted in the conversion to pronunciation symbols. Furthermore, with regard to the count of the number of words for the converted pronunciation symbols, counting is varied, depending on the type of the letter, in accordance to the following rules: (1) spaces between words are not counted; (2) consonants are counted as 0.5 letters; (3) vowels (a, i, and so on) are counted as 1 character; (4) long vowels (a:, and so on) are counted as 2 characters.
By counting according to such a rule, with regard to sample sentence 2801, “want” appearing in the speech recognition result, appears in location “2” and “medicine” appears in location “4.5”, as shown in sample sentence 2801. In the same manner, the sample sentence comparison unit 112 also calculates the appearance location of each word in the speech recognition result. For example, the word “medicine” included in the speech recognition result has a location of “6.5”, as shown in
Next, the sample sentence comparison unit 112 normalizes the appearance location of the words using the length of the pronunciation symbol character string (step S1902). For example, the length of the pronunciation symbols for sample sentence 2801 is 7.5 words, and the appearance location of “want” and “medicine” are normalized as 2/7.5=0.27 and 4.5/7.5=0.60, respectively. In the same manner, the sample sentence comparison unit 112 also normalizes the appearance location of each word in the speech recognition result. For example, as shown in
Next, the sample sentence comparison unit 112 calculates, from the location of a word within a sample sentence and the location of a word in the speech recognition result, the word score for every word in each sample sentence, that is included in the speech recognition result (step S1903). The word score is calculated using the following formula:
(Word score)=(location in sample sentence)−(location in speech recognition result)
For example, as shown in sample sentence 2901 in
Next, the sample sentence comparison unit 112 derives a display parameter for each word of each sample sentence, based on the calculated word scores (step S1904). For example, when a parameter is previously set in which an underline is placed when the absolute value for a word score is 0.10 or lower, the GUI unit 102 places an underline under “have” and “medicine” of sample sentence 2603, which fall under such condition, and displays the candidate sample sentences on the candidate-sample sentence selection area 405, as shown in
Furthermore, in the same manner as in the conversion from Japanese to English, the sample sentence comparison unit 112 can change the display order of the sample sentences by deriving a sentence score (sentence correspondence degree) for each of the sentences. For example, when the sentence score is calculated as the number of words having a word score absolute value of 0.10 or lower, a sentence score 3101 shown in
Moreover, although explanation was carried out with the user's input to the GUI unit 102 being limited to the respective touch-panel inputs and button inputs in the aforementioned explanation, it is possible for words and sample sentences to be selected and decided on through speech, using the speech recognition process. Furthermore, operation through a combination of the respective input modalities of touch-panels, buttons, and speech is also possible. In addition, although Japanese and English are given as an examples, the present invention is not dependent on the language and can be implemented in the same manner even for other languages such as Chinese, and so on.
Furthermore, the language model used within the speech recognition unit 104 is constructed centering on the sentences in the “Source language:” field of the sample sentences held by the sample sentence database 105. In general, in order to construct a language model, it is necessary to break down a sentence into the smallest units such as a morpheme, and the output of the speech recognition unit 104 is the grouping of such smallest units. The information in the word dictionary 107 can be used in the break down to the smallest units. Furthermore, a language model can be constructed by performing a break down which is more detailed than in the word dictionary 107, and this can be outputted as the output of the speech recognition unit 104, after being formed into a word that is registered in the word dictionary 107.
Furthermore, aside from placing an underline under a coinciding word, the control of the display parameter by the sample sentence comparison unit 112 can also use various display methods such as the shading of the color of characters and the blinking of the characters. Furthermore, with respect to the rearrangement of candidate-sample sentences by sentence score performed by the sample sentence comparison unit 112, the display parameter can be controlled, on a per sample sentence basis in accordance with the sentence score, such as by thinning-out the display color of a sample sentence having a low score, and so on.
Furthermore, although explanation is made in the present embodiment regarding the dialogue supporting apparatus, the present invention is not limited to such, and can also be applied as a sample sentence searching apparatus which searches out a sample sentence corresponding to an inputted sentence.
Although only some exemplary embodiments of this invention have been described in detail above, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiments without materially departing from the novel teachings and advantages of this invention. Accordingly, all such modifications are intended to be included within the scope of this invention.
As in the aforementioned manner, the dialogue supporting apparatus in the present invention includes a function for easily searching out a desired sample sentence with the use of speech, and is useful for providing on, for example, a mobile phone, portable terminal, and the like, translation language corresponding to inputted source language. Furthermore, the present invention can also be applied to applications in public street-side terminal devices and guidance terminal devices.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2004-117184 | Apr 2004 | JP | national |
This is a continuation of PCT application No. PCT/JP2005/006596, filed on Apr. 4, 2005.
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/JP05/06596 | Apr 2005 | US |
| Child | 11166239 | Jun 2005 | US |
