BACKGROUND OF THE INVENTION
The present invention relates to a voice recognition system installed and used in a mobile unit such as a vehicle, and particularly, to technology concerning a dictionary structure for voice recognition capable of shortening recognition time and improving recognition accuracy.
A voice recognition system needs dictionaries for a voiced language. The dictionaries proposed for voice recognition include a network grammar language dictionary that employs a network structure to express the connected states or connection grammar of words and morphemes and a statistical language dictionary to statistically express connections among words. Reference 1 (“Voice Recognition System,” Ohm-sha) points out that the network grammar language dictionary demonstrates high recognition ability but is limited in the number of words or sentences to handle and the statistical language dictionary may handle a larger number of words or languages but demonstrates an insufficiently low recognition rate for voice recognition.
To solve the problems, Reference 2 (“Speech Recognition Algorithm Combining Word N-gram with Network Grammar” by Tsurumi, Lee, Saruwatari, and Shikano, Acoustical Society of Japan, 2002 Autumn Meeting, Sep. 26, 2002) has proposed another technique. This technique adds words, which form connected words in a network grammar language dictionary, to an n-gram statistical language dictionary to uniformly increase the transition probabilities of the words.
SUMMARY OF THE INVENTION
A voice recognition application such as a car navigation system used in a mobile environment is only required to receive voices for limited tasks such as an address inputting voice and an operation commanding voice. For this purpose, the network grammar language dictionary is appropriate. On the other hand, the n-gram statistical language dictionary has a high degree of freedom in the range of acceptable sentences but lacks voice recognition accuracy compared with the network grammar language dictionary. The n-gram statistical language dictionary, therefore, is not efficient to handle task-limited voices.
An object of the present invention is to utilize the characteristics of the two types of language dictionaries, perform a simple prediction of a next speech, change the probabilities of connected words in an n-gram statistical language dictionary at each turn of speech or according to output information, and efficiently conduct voice recognition in, for example, a car navigation system.
An aspect of the present invention provides a voice recognition system includes that a memory unit configured to store a statistical language dictionary which statistically registers connections among words, a voice recognition unit configured to recognize an input voice based on the statistical language dictionary, a prediction unit configured to predict, according to the recognition result provided by the voice recognition unit, connected words possibly voiced after the input voice, and a probability changing unit configured to change the probabilities of connected words in the statistical language dictionary according to the prediction result provided by the prediction unit, wherein the voice recognition unit recognizes next input voice based on the statistical language dictionary changed by the probability changing unit and wherein the memory unit, the voice recognition unit, the prediction unit and the probability changing unit are configured to be installed in the mobile unit
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a block diagram showing a voice recognition system according to a first embodiment of the present invention.
FIG. 2 is a block diagram showing a voice recognition system according to a second embodiment of the present invention.
FIG. 3 shows an example of hardware for a voice recognition system according to an embodiment of the present invention.
FIG. 4 is a flowchart showing a voice recognition method according to an embodiment of the present invention.
FIG. 5 is a view explaining the probability changing method carried out in step S150 of FIG. 4.
FIG. 6 shows an example to improve, in the statistical language dictionary, the connection probabilities of words contained in the network grammar language dictionary and of words or morphemes connectable to the words contained in the network grammar language dictionary.
FIG. 7 shows a method of switching a plurality of small-scale network grammar language dictionaries from one to another.
FIG. 8 shows an example storing a large-scale network grammar language dictionary 802 in a memory unit 801 and dynamically activating only a node of the dictionary 802 related to a predicted next speech.
FIG. 9 shows an example of a network grammar language dictionary displayed on a display unit according to the present invention.
FIG. 10 shows connected words displayed in the right window of the display 901 of FIG. 9 after the right window is scrolled.
FIG. 11 shows an example that forms connected words from a displayed word and lower-level words and predicts the connected words to be voiced in the next speech.
FIG. 12 shows an example of a screen displaying four groups of words each group containing a plurality of connected words.
FIG. 13 shows an example of a screen displaying four words displayed on a screen.
FIG. 14 shows an example of the network grammar language dictionary.
FIG. 15 shows an example of a bi-gram statistical language dictionary.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
Various embodiments of the present invention will be described with reference to the accompanying drawings. It is to be noted that the same or similar reference numerals are applied to the same or similar parts and elements throughout the drawings, and the description of the same or similar parts and elements will be omitted or simplified. The drawings are merely representative examples and do not limit the invention.
General maters about voice recognition will be explained in connection with the present invention. Voice recognition converts an analog input into a digital output, provides a discrete series x, and predicts a language expression ω most suitable for the discrete series x. To predict the language expression ω, a dictionary of language expressions (hereinafter referred to as “language dictionary”) must be prepared in advance. Dictionaries proposed so far include a network grammar language dictionary employing a network structure to express the grammar of word connections and a statistical language dictionary to statistically express the connection probabilities of words.
FIG. 14 shows an example of the network grammar language dictionary that allows a voice input of, for example, “address is (prefecture name) prefecture.” This system presumes that the speech “address is” is always followed by a voicing of (prefecture name) and then a voicing of “prefecture.” For the part (prefecture name), the dictionary must store prefecture names in advance. In addition to the prefecture names, the dictionary may contain city names, ward names, town names, and the like. This technique limits words and connections of the words, to improve recognition performance.
On the other hand, the statistical language dictionary statistically processes a large amount of sample data to estimate the transition probabilities of words and morphemes. For this, a widely-used simple technique is an n-gram model. This technique receives a word string ω1ω2 . . . ωn and estimates an appearing probability P(ω1ω2 . . . ωn) according to the following approximation model:
The case of n=1 is called uni-gram, n=2 bi-gram (2-gram), and n=3 tri-gram (3-gram).
FIG. 15 shows an example of a bi-gram statistical language dictionary. For an input word “Nara” as ωn−1, the transition probability of each word string formed with ωn is calculated as follows:
P( . . . Nara, Prefecture, . . . )=P(Nara| . . . )×P(Prefecture|Nara)×P( . . . |Prefecture) (2)
According to this expression, the probability is dependent on only the preceding word. If there are many data words, the n-gram statistical language dictionary can automatically include connection patterns among the words. Therefore, unlike the network grammar dictionary, the n-gram statistical language dictionary can accept a speech whose grammar is out of the scope of design. Although the statistical language dictionary has a high degree of freedom, its recognition rate is low when conducting voice recognition for limited tasks.
To solve the problem, the Reference 2 proposes a GA method. Employing this method will improve recognition accuracy by five points or more compared with employing only the n-gram statistical language dictionary.
A voice recognition application such as a car navigation system used in a mobile environment is only required to receive voices for limited tasks such as an address inputting voice and an operation commanding voice. Accordingly, this type of applications generally employs the network grammar language dictionary. Voice recognition based on the network grammar language dictionary needs predetermined input grammar, and therefore, is subjected to the following conditions:
- (1) a user must memorize grammar acceptable by a voice recognition system, or
- (2) a designer must install every grammar used by a user in a voice recognition system.
On the other hand, the n-gram statistical language dictionary has a high degree of freedom in the range of acceptable grammar but is low in voice recognition accuracy compared with the network grammar language dictionary. Due to this, the n-gram statistical language dictionary is generally not used to handle task-limited speeches. The above-mentioned condition (2) required for the network grammar language dictionary is hardly achievable due to the problem of designing cost Consequently, there is a requirement for a voice recognition system having a high degree of freedom in the range of acceptable speeches like the n-gram statistical language dictionary and capable of dynamically demonstrating recognition performance like the network grammar language dictionary under specific conditions.
The GA method described in the Reference 2 predetermines a network grammar language dictionary, and based on it, multiplies a log likelihood of each connected word that is in an n-gram statistical language dictionary and falls in a category of the network grammar language dictionary by a coefficient, to thereby adjust a final recognition score of the connected word. The larger the number of words in the network grammar language dictionary, the higher the number of connected words adjusted for output Namely, an output result approaches the one obtainable only with the network grammar language dictionary. In this case, simply applying the GA method to car navigation tasks provides little effect compared with applying only the network grammar language dictionary to the same.
An embodiment of the present invention conducts a simple prediction of a next speech and changes the probabilities of connected words in an n-gram statistical language dictionary at every speech turn (including a speech input and a system response to the speech input), or according to the contents of output information. This results in realizing the effect of the GA method even in voice recognition tasks such as car navigation tasks. The words “connected words” include not only compound words, conjoined words and a set of words but also words linked in a context.”
FIG. 1 is a block diagram showing a voice recognition system according to a first embodiment of the present invention. The voice recognition system includes a sound receiver 110 to receive sound, a memory unit 140 to store dictionaries, a voice recognition unit 120 to recognize, according to the dictionaries in the memory unit 140, voice from the sound received by the sound receiver 110 and output a recognition resultant signal R100, a prediction unit 130 to predict a next speech according to the recognition resultant signal R100 and output the prediction, and a probability changing unit 150 to update the dictionaries in the memory unit 140 according to the prediction. The sound receiver 110 converts a sound voiced by a user into a sound signal. The voice recognition unit 120 recognizes the sound signal and outputs a recognition resultant signal R100. The recognition resultant signal R100 is in the form of, for example, text According to recognized words in the text, the prediction unit 130 predicts the contents of a speech to be voiced by the user and transfers the prediction to the probability changing unit 150. According to the transferred prediction, the probability changing unit 150 increases the correct answer probabilities of grammar among words in a statistical language dictionary stored in the memory unit 140. The memory unit 140 stores phonemic and word dictionaries needed for voice recognition and at least one language dictionary describing word connections. The memory unit 140 is referred to for operations such as a recognition operation.
FIG. 2 is a block diagram showing a voice recognition system according to a second embodiment of the present invention. The voice recognition system includes a sound receiver 210 to receive sound, a memory unit 270 to store dictionaries, a voice recognition unit 220 to recognize, according to the dictionaries in the memory unit 270, voice from the sound received by the sound receiver 210 and output a recognition resultant signal, an information controller 230 to receive the recognition resultant signal, control information, and output information to an information providing unit 240 and a prediction unit 250, the prediction unit 250 to predict a next speech according to the information from the information controller 230 and output the prediction, and a probability changing unit 260 to update the dictionaries in the memory unit 270 according to the prediction. The sound receiver 210 may be equivalent to the sound receiver 110 of the first embodiment The voice recognition unit 220 may be equivalent to the voice recognition unit 120 of the first embodiment The information controller 230 determines information and outputs the information to the information providing unit 240 and prediction unit 250. The prediction unit 250 refers to the information provided by the information controller 230, predicts the contents of a next speech to be voiced by a user, and transfers the prediction to the probability changing unit 260. The prediction units 130 and 250 of FIGS. 1 and 2 receive different input information and may output equivalent information. The memory unit 270 may contain, in addition to the dictionaries of the first embodiment, a thesaurus and history data. The information providing unit 240 provides the user with the information output from the information controller 230. This information relates to the five senses such as image information, sound information, tactile information, and the like. The information providing unit 240 may be realized with a visual display, sound speaker, tactile display, force feedback switch, and the like.
FIG. 3 shows an example of hardware for a voice recognition system according to an embodiment of the present invention. The voice recognition system includes a sound signal receiving device 310 to receive an analog sound signal, an A/D converter 320 to convert the input analog signal into a digital signal, a memory unit 350 to store various dictionaries, a processor 330 to process the digital signal from the A/D converter 320 and conduct voice recognition according to the dictionaries in the memory unit 350, and an information providing device 340 to provide a result of the voice recognition. The sound receivers 110 and 210 of FIGS. 1 and 2 correspond to the sound signal receiving device 310 and A/D converter 320 of FIG. 3 and may employ a microphone. The A/D converter 320 may employ a real-time signal discretizing device. The A/D converter 320 collects a sound signal and converts it into a discrete voice signal. The voice recognition unit 120 of FIG. 1 may be realized with the processor 330 of FIG. 3 and the memory unit 270 of FIG. 2. The processor 330 may be a combination of CPU, MPU, and DSP used to form an operational system such as a standard personal computer, microcomputer, or signal processor. The processor 330 may have a real-time operating ability. The memory unit may be realized with cache memories, main memories, disk memories, flash memories, ROMs, and the like used as data storage for standard information processing units.
FIG. 4 is a flowchart showing a voice recognition method according to an embodiment of the present invention. A first step S110 initializes a voice recognition system. The first step loads dictionaries for voice recognition to a memory (RAM). It is not necessary to load all dictionaries possessed. Step S120 determines whether or not an input signal is a voice signal. If it is a voice signal (Yes in step S120), step S130 is carried out, and if not (No in step S120), a voice signal input is waited for. Step S130 recognizes an “n−1”th voice signal and converts information contained in the voice signal into, for example, text data. According to the data provided by step S130, step S140 detects a change in the speech. For example, step S130 outputs text data of “an address is going to be input” and step S140 provides “Yes” to indicate that the user will voice a particular address in the next speech. In this way, step S140 detects a state change and predicts the contents of the next speech. If step S140 is “No” to indicate no state change, the flow returns to step S120 to wait for a voice input.
According to the state change and next speech detected and predicted in step S140, step S150 changes the probabilities of grammar related to words that are in the predicted next speech and are stored in the statistical language dictionary. The details of this will be explained later. Step S160 detects the next speech. Step S170 detects an “n”th voice. Namely, if step S160 is “Yes” to indicate that there is a voice signal, step S170 recognizes the voice signal and converts information contained in the voice signal into, for example, text data. If step S160 is “No” to indicate no voice signal, a next voice signal is waited for. At this moment, step S150 has already corrected the probabilities of grammar related to words in the statistical language dictionary. Accordingly, the “n”th voice signal is properly recognizable. This improves a recognition rate compared with that involving no step S150. Step S180 detects a state change and predicts a next speech. If step S180 detects a state change, step S190 changes the probabilities of grammar concerning words that are in the predicted next speech and are stored in the statistical language dictionary. If step S180 is “No” to detect no state change, a state change is waited for.
FIG. 5 is a view explaining the probability changing method carried out in step S150 of FIG. 4. This example is based on an assumption that step S140 of FIG. 4 predicts an address input to be made in the next speech and step S150 updates the statistical language dictionary accordingly The network grammar language dictionary shown on the right side of FIG. 5 includes “Kanagawa,” “Nara,” and “Saitama” followed by “Prefecture.” From the network grammar language dictionary, connected words “Kanagawa Prefecture,” “Nara Prefecture,” and “Saitama Prefecture” are picked up, and probabilities assigned to them are increased in the statistical language dictionary shown on the left side of FIG. 5. For example, the following is calculated for “Nara Prefecture”:
PNew(Prefecture|Nara)=Pold(Prefecture|Nara)1/,, (3)
where ,, >1, and ,, is predetermined.
FIG. 6 shows an example to improve, in the statistical language dictionary, the connection probabilities of words contained in the network grammar language dictionary and of words or morphemes connectable to the words contained in the network grammar language dictionary. This example changes, in the statistical language dictionary, the probabilities of connections of words found in the network grammar language dictionary as well as the connection probabilities of morphemes and words connectable to the words in the network grammar language dictionary. The network grammar language dictionary of FIG. 6 contains “Kanagawa,” “Nara,” and “Saitama” followed by “Prefecture.” Then, the connection probability of each word that is in the statistical language dictionary and is connectable to “Kanagawa,” “Nara,” and “Saitama” is changed. For example, words connectable to “Kanagawa” and contained in the statistical language dictionary are “Prefecture” and “Ku (means “Ward” in Japanese) and therefore, the probabilities P(Prefecture|Kanagawa) and P(Ku|Kanagawa) are changed according to the expression (3). Such probability changing calculations can be conducted on the statistical language dictionary before voice recognition. Alternatively, the calculations may be conducted in voice recognition by comparing connected words contained in a recognized voice candidate with words in the network grammar language dictionary and if the connected words in the candidate are found in the network grammar language dictionary.
A method of using network grammar language dictionaries will be explained.
FIG. 7 explains a method of switching a plurality of small-scale network grammar language dictionaries from one to another. This is suitable when contents to be displayed are difficult to predict, like an Internet webpage. In FIG. 7, information displayed on a screen is fetched in a memory unit 701, and at the same time, a dictionary is registered. In this case, it is required to recognize contents presently displayed and those previously displayed. For this, a plurality of small-scale network grammar language dictionaries 702 and 703 are read in the memory unit 701, or unnecessary dictionaries are deleted from the memory unit 701. Each language dictionary once registered in the memory unit 701 is connectable to a voice recognition unit 704 through a switch controller 705. Any dictionary that is unnecessary for the time being is disconnected from the voice recognition unit 704 through the switch controller 705. Any dictionary that is not used for a long time is deleted from the memory unit 701.
FIG. 8 explains an example storing a large-scale network grammar language dictionary 802 in a memory unit 801 and dynamically activating only a node of the dictionary 802 related to a predicted next speech. This example is appropriate for, for example, a car navigation system that stores, in the memory unit 801, a large amount of essential information including addresses, facilities, and the like required when setting a destination on the car navigation system.
According to this example, the memory unit 801 stores a statistical language dictionary 803 and at least one network grammar language dictionary 802 containing words to be voiced. A probability changing unit 804 selects a node in the network grammar language dictionary 802 suitable for a next speech predicted by a prediction unit 805 so that the transition probabilities of connected words that are contained in the statistical language dictionary 803 and are in the selected node of the network grammar language dictionary 802 are increased.
The network grammar language dictionary has a tree structure involving a plurality of hierarchical levels and a plurality of nodes. The tree structure is a structure resembling a tree with a thick trunk successively branched into thinner branches. In the tree structure, higher hierarchical levels are divided into lower hierarchical levels.
A prediction method conducted with any one of the systems of FIGS. 1 and 2 will be explained in detail. As an example, an address inputting task carried out according to the present invention with a car navigation system having a display unit will be explained The address inputting task is achieved by arranging data including prefecture names, city, ward, town, and village names, town area names, and addresses in a hierarchical structure, storing the data in the car navigation system, and prompting a user to voice-input information for the top of the hierarchy. At this time, there are a technique of prompting the user to enter information level by level and a technique of prompting the user to continuously enter information along the hierarchical levels. Information on input commands is stored in a network grammar language dictionary.
FIG. 9 shows an example of a network grammar language dictionary displayed on a display unit according to the present invention. The displayed information helps the user grasp words in the network grammar language dictionary, informs the user of commands to be input into the system, and prompts the user to enter one of the displayed words as a command Any one of the displayed connected words is to be included in a command voiced by the user. Accordingly, a prediction unit of the system predicts that the displayed connected words have a possibility of being pronounced in the next speech. In the example of FIG. 9, the display 901 displays four prefecture names and four city names (Saitama City, Kawaguchi City, Shiki City, and Kawagoe City) corresponding to the prefecture name “Saitama Prefecture” on which a cursor (underbar) is set Accordingly, the probabilities of the displayed connected words are changed in a statistical language dictionary.
FIG. 10 shows connected words displayed in the right window of the display 901 of FIG. 9 after the right window is scrolled. In FIG. 9, the city-town-village-name window (i.e., the right window) of the display 901 shows Saitama City, Kawaguchi City, Shiki City, and Kawagoe City. In FIG. 10, the same window shows Kawagoe City, Kounosu City, Fukiage City, and Kasukabe City. Accordingly, the connected words displayed in the display 901 of FIG. 10 are predicted to be voiced in the next speech and their probabilities are changed in the statistical language dictionary.
In addition to the displayed connected words, other connected words made by connecting the displayed words with grammatically connectable morphemes may be predicted to be voiced in the next speech. In this case, the memory unit of the system may store a connection list of parts of speech for an objective language and processes for specific words, to improve efficiency.
FIG. 11 shows an example that forms connected words from a displayed word and lower-level words and predicts the connected words to be voiced in the next speech. For example, a displayed word “Saitama City” is followed by a plurality of lower-level ward names. Accordingly, city-ward-name connected words such as “Saitama City,” “City Urawa Ku,” and “City Ohmiya Ku” are formed and predicted to be voiced in the next speech.
Next, groups of words and words in sentences that are frequently used in displaying Internet webpage will be explained in connection with voice recognition according to the present invention.
FIG. 12 shows an example of a screen displaying four groups of words each group containing a plurality of connected words. In this case, each connected word is predicted as a word to be voiced next time. In addition, connected words made of the individual words and words connectable thereto are predicted as words to be voiced next time. In FIG. 12, a word “Skyline” forms a displayed connected word “Skyline Coupe” which is predicted to be voiced next time. In addition, if stored as commodity lineup information in the memory unit, a connected word of “Skyline Sedan” is also predicted to be voiced in the next speech.
Information made of a group of words or a sentence may be provided as voice guidance. Information provided with voice guidance is effective to reduce the number of words to be predicted as words to be voiced next time. In the example of FIG. 12, if the first group of words “New Skyline Coupe Now Available” among the four word groups in the displayed menu is voiced, connected words whose probabilities are changed include:
- Connected words group 1: New Skyline, New Cube . . .
- Connected words group 2: Skyline Coupe, Skyline Sedan, . . .
- Connected words group 3: Coupe Now,
If the second group of words “Try! Compact Car Campaign” is presented by voice, connected words whose probabilities are changed include:
- Connected words group 4: Try Compact, . . .
- Connected words group 5: Compact Car, . . .
- Connected words group 6: Car Campaign, Car Dealer, . . .
In this case, the probabilities of the connected words are changed in order of the voiced sentences, and after a predetermined time period, the probabilities are gradually returned to original probabilities. In this way, the present invention can effectively be combined with voice guidance, to narrow the range of connected words to be predicted as words to be pronounced next time.
Synonyms of displayed or voiced connected words may also be predicted as words to be voiced next time. The simplest way to achieve this is to store a thesaurus in the memory unit, retrieves synonyms of an input word, prepares connected words made by replacing the input word with the synonyms, and predicts the prepared connected words to be voiced in the next speech.
FIG. 13 shows four words displayed on a screen. Among the words, “Air Conditioner” is selected to find synonyms thereof, and the synonyms are added to a statistical language dictionary. For the synonyms, connected words are prepared and added as predicted connected words. The added words include “Cooler” and “Heater”, and connected words are prepared for them as follows:
- Cooler ON, Cooler OFF
- Heater ON, Heater OFF
These connected words are predicted to be voiced in the next speech. The predicted words may be limited to those that can serve as subjects or predicates to improve processing efficiency.
Finally, a method of predicting words to be voiced in the next speech according to the history of voice inputs.
As explained with the voice guidance example, the history of presented information can be used to gradually change the probabilities of connected words as the history of information is accumulated in a statistical language dictionary. This method is effective and can be improved into the following alternatives:
- 1. Continuously changing for a predetermined period of time the probabilities of connected words belonging to a hierarchical level once displayed by a user.
- 2. Continuously changing for a predetermined period of time the probabilities of connected words input several turns before.
- 3. Continuously changing the probabilities of connected words related to a user's habit appeared in the history of presented information.
Examples of the user's habit mentioned in the above item 3 are:
- always setting the radio at the start of the system; and
- turning on the radio at a specific hour.
If such a habitual behavior is found in the history of presented information, the probabilities of connected words related to the behavior are increased to make the system more convenient for the user.
If a predicted connected word is absent in the statistical language dictionary in any one of the above-mentioned examples, the word and the connection probability thereof can be added to the statistical language dictionary at once.
The embodiments and examples mentioned above have been provided only for clear understanding of the present invention and are not intended to limit the scope of the present invention.
As mentioned above, the present invention realizes a voice recognition system for a mobile unit capable of maintaining recognition accuracy without increasing grammatical restrictions on input voices, the volume of storage, or the scale of the system.
The present invention can reduce voice recognition computation time and realize real-time voice recognition in a mobile unit These effects are provided by adopting recognition algorithms employing a tree structure and by managing the contents of network grammar language dictionaries. In addition, the present invention links the dictionaries with information provided for a user, to improve the accuracy of prediction of the next speech.
The present invention can correctly predict words to be voiced in the next speech according to information provided in the form of word groups or sentences. This results in increasing the degree of freedom of speeches made by a user without increasing the number of words stored in a statistical language dictionary. Even if a word not contained in the statistical language dictionary is predicted for the next speech, the present invention can handle the word.
The entire content of Japanese Patent Application No. 2003-129740 filed on May 8th, 2003 is hereby incorporated by reference.
Although the invention has been described above by reference to certain embodiments of the invention, the invention is not limited to the embodiments described above. Modifications and variations of the embodiments described above will occur to those skilled in the art, in light of the teachings. The scope of the invention is defined with reference to the following claims.
Patent Application
20050004798
