Patent Grant
8498857
The present invention relates to the field of speech recognition.
The following are definitions of terms known in the art and used in this specification:
Today, interactive technologies play a key role for improving customer service. Interactive technologies like IVR (Interactive Voice Response) accept verbal user input and/or request and provide pre-recorded or dynamically generated output in response to user's request.
Typically, IVR applications use speech recognition systems to recognize and convert either a spoken word or a sequence of spoken words to machine readable form for further processing and/or answering a user query. Typically, these speech recognition systems are deployed for a particular language, thus, when the same system has to be deployed for a different language, one has to port the existing system to enable it to understand the new language, which is equivalent to building a fresh application. Most of the existing systems are deployed in English due to:
However, with increasing acceptability of speech based solutions in various countries, where the native language is other than English, there is an urgent need to convert existing speech recognition based applications in a source language, for instance English, to a target language for instance, Hindi.
Typically, an existing speech recognition based solution requires the following components:
The first three components work in tandem to convert the spoken speech to text, while the fourth component helps the existing speech recognition based solution to communicate with users. Typically, converting the existing speech recognition based solution from a source language to a target language needs these four components to be ported to the target language.
Although, acoustic models are tuned for a particular language, source acoustic models can be used to recognize speech in another language with decent accuracy if the other two components, namely, the pronunciation lexicon and the speech grammar are addressed adequately in the target language.
Essentially, converting the speech recognition based solution from one language to another necessitates creation of a new pronunciation lexicon for the target language that contains all words to be recognized by the speech recognition based solution and also a speech grammar model in the target language. Additionally, prompts in the source language have to be converted into prompts in the target language.
These modifications for porting the existing speech recognition based solution in the source language into the target language requires efforts equivalent to building an entirely new speech recognition based solution.
There have been various attempts in the prior art to develop systems which will enable the easy portability of applications from one language to another.
Particularly, U.S. Pat. No. 7,406,417 discloses a method for conditioning a database for automatic speech processing. The document discloses a neural network that can be trained for synthesizing or recognizing speech with the aid of a database produced by automatically matching graphemes and phonemes. First, graphemes and phonemes are matched for words which have the same number of graphemes and phonemes. Next, graphemes and phonemes are matched for words that have more graphemes than phonemes in a series of steps that combine graphemes with preceding phonemes. Then, graphemes and phonemes are matched for words that have fewer graphemes than phonemes. After each step, infrequent and unsuccessful matches made in the preceding step are erased. After this process is completed, the database can be used to train the neural network and graphemes, or letters of a text can be converted into the corresponding phonemes with the aid of a trained artificial neural network.
Further, United States Patent Application 2005197835 discloses method and apparatus for generating acoustic models for speaker independent speech recognition of foreign words uttered by non-native speakers. The document discloses acoustic models for speech recognition which are automatically generated and utilize trained acoustic models from a native language and a foreign language. A phoneme-to-phoneme mapping is utilized to enable the description of foreign language words with native language phonemes. The phoneme-to-phoneme mapping is used for training foreign language words, described by native language phonemes on foreign language speech material. A new phonetic lexicon is created containing foreign language words and native language words transcribed by native language phonemes. Robust native language acoustic models can be derived utilizing foreign language and native language training material. The mapping may be used for training a grapheme to phoneme transducer (i.e., foreign language to native language) to generate native language pronunciations for new foreign language words.
Furthermore, United States Patent Application 2009150153 discloses grapheme-to-phoneme conversion using acoustic data. The document discloses the use of acoustic data to improve grapheme-to-phoneme conversion for speech recognition, such as to more accurately recognize spoken names in a voice-dialing system. A joint model of acoustics and graphonemes (acoustic data, phonemes sequences, grapheme sequences and an alignment between phoneme sequences and grapheme sequences) is described, as is retraining by maximum likelihood training and discriminative training in adapting grapheme model parameters using acoustic data. Also described is the unsupervised collection of grapheme labels for received acoustic data, thereby automatically obtaining a substantial number of actual samples that may be used in retraining. Speech input that does not meet a confidence threshold may be filtered out so as to not be used by the retrained model.
Additionally, World Intellectual Property Organisation document number 2009/150591 discloses a method and device for the generation of a topic-specific vocabulary and computer program product. The document discloses a method for the computer-aided generation of a topic-specific vocabulary from public text. The steps followed as disclosed in this document are: automatic selection of a language and topic-specific text; automatic generation of vocabulary entries each comprising a word together with a phonetic transcription on the basis of the selected text; automatic generation of the vocabulary entries is done employing a grapheme structure-based classification of the vocabulary entries, to classify the vocabulary entries according to a number of predetermined types; vocabulary entry type-specific grapheme-to-phoneme conversion; and to obtain phonetic transcriptions for words.
However, the aforementioned documents are not suitable for porting existing speech recognition solutions to plurality of target languages with minimum changes in the existing deployment. Therefore, there is a need for a system which will enable the existing applications to be quickly ported and/or modified to work in multiple target languages by reusing the speech recognition engine of the existing application.
It is an object of the present invention to provide a system for enabling an existing speech recognition solution to be quickly ported to work in another target language.
It is another object of the present invention to provide a system for accurate source to target language lexicon and speech grammar transliterations and translations.
It is yet another object of the present invention to provide a system which automatically generates source language phonemic pronunciations of target language words.
A system for porting a speech recognition solution in a source language to recognize a target language, the speech recognition solution consisting of a speech recognition engine, a pronunciation lexicon in the source language, a speech grammar file for the source language, prompts in the source language, the system comprising:
In accordance with the present invention, there is provided a method for porting a speech recognition solution in a source language to work for target language, the method comprises the following steps:
The solution in target language then:
Typically, the step of modifying a pronunciation lexicon of the source language to provide a pronunciation lexicon in the target language includes the following steps:
Preferably, the step of modifying the speech grammar file of the source language to provide a speech grammar file in the target language includes the steps of translating speech grammar file of the source language to the target language and transliteration said translated speech grammar file from the target language to the source language.
Further, the step of converting voice prompts in the source language to the target language includes the following steps:
The invention will now be described with reference to the accompanying drawings, in which:
The conventional speech recognition solutions are typically built for a particular source language, typically English, however with increasing acceptability of speech based solutions in various countries, where the native language is different from the source language there is a need to quickly convert an existing speech solution working in the source language to a target language with minimum development efforts. In view to overcome these shortcomings of the existing speech recognition solutions, the present invention envisages building a speech recognition system in the target language from the existing speech recognition based solution in the source language.
Particularly, the system envisaged by the present invention enables porting of any existing speech recognition solution in the source language to the target language, thus minimising the time and effort involved in the development process and enabling reuse of existing speech recognition solution components.
Referring to the accompanying drawings,
The conventional speech recognition applications are built of one or more call flow units, represented generally by reference numeral 10 of FIG. 1. Each of the conventional call flow units 10 include modules for performing the following functions:
Typically, the step of processing the recognized text involves processing of two types of data:
1. Speech (acoustic) data; and
2. Textual data.
The speech data is used at the point of interaction with users, while the textual data is processed internally for processing information extracted from the speech data.
To port such an existing speech recognition solution to the target language, the present invention proposes porting the existing solution by adopting the following steps:
In accordance with the present invention, referring to
The present invention proposes to modify the phoneme lexicon, speech grammar and the voice prompts for porting any existing speech recognition solution to a target language efficiently.
The system 100 comprises the following components for porting the existing speech recognition solution from the source language to the target language:
The aforementioned components of the present invention work in conjunction with the components of the existing speech recognition solution to port the existing solution to recognize a target language. The existing speech recognition solution components are as follows:
The Lexicon Conversion Means 102 takes each word from the source language lexicon and determines its translation using the translation means 202. The translation means 202 checks if the word is present in the first database 200, if the word is present, the corresponding translated word in target language is taken from the first database 200. If the word is not present in the first database 200 then the word is transliterated into the target language graphemes using the transliteration means 204. The transliteration to the target language is performed based on the assumption that the word is a proper noun. Thus, the Lexicon Conversion Means 102 can handle both common nouns and proper nouns.
Further, the translated/transliterated word is transliterated into the source language graphemes by the transliteration means 204. The transliterated word in the source language is given to the grapheme to phoneme conversion means 206 which receives the transliterated word and generates a source language phoneme sequence to obtain the phonetic pronunciation of the target language word in the source language.
The porting of the source language lexicon to the target language will now be described with the help of the following example. For instance, if we have to port a word “gold” from the source language English to target language Hindi, this can be achieved by the following steps: firstly, the system envisaged by the present invention checks if an analogous word for “gold” is present in the first database 200, if yes then from the first database 200 the translated target language word is selected by the translation means 202. Then a transliteration is done to convert the target language translated word
to “sona” by the transliteration means 204. Next, the pronunciation is determined from sona as “s/ow/n/aa” using the grapheme to phoneme conversion means 206 in the source language as seen in TABLE 1.
To avoid the overhead of processing the target language word each time and obtaining its pronunciation in the source language, the Lexicon Conversion Means 102 prepares a lookup table using the lookup table creation means 208. The lookup table creation means 208 receives the transliterated target language word represented in the source language along with its phonetic pronunciation in the source language and creates a lookup table mapping the two.
Thus, next time the speech recognition solution needs to recognize a word in the target language, the step of grapheme to phoneme conversion can be skipped. The solution can directly obtain the phoneme sequence in the source language for any transliterated word which was a part of the pronunciation lexicon of the source language. This process speeds up the process of lexicon creation in the target language.
Referring to
In accordance with the present invention, the Grammar Conversion Means 104 is generally not required for an existing menu driven speech recognition solution because the solution expects only a word or a small sequence of words as input from users. Speech grammar modification (source-to-target) is required in cases where the speech recognition solution is expected to handle free speech user queries. The speech grammar creation for the target language is achieved by Grammar Conversion Means 104 by employing a translation means 300 which receives the speech grammar file of the source language and translates the grammar file to the target language. This translated file in the target language is transliterated to provide a transliterated grammar file for the target language in the source language by the transliteration means 302.
Referring to
In accordance with the present invention, if recorded prompts are used in the existing solution, then a similar database of prompts in the target language is created and the existing speech recognition solution points to this database for prompts and responses to users, else the textual prompts in the source language are translated into the target language by translation means 500. The translated prompts are given to identification means 502 which uses the words in the lookup table of the lexicon conversion means 102 to identify the phonetic pronunciation corresponding to the words containable in the prompts in the target language and provides a phoneme sequence of words containable in the prompts in the source language. This phoneme sequence of words is aligned to form a string of words containable in the prompts in the source language by the text to speech conversion means 504. The aligned string is then passed on to the grammar adjustment means 506 which converts the string of words containable in the prompts in the source language and arranges the words in accordance with the transliterated grammar file for the target language and provide grammatically modified text based prompts which are converted to speech in real-time by the speech generation means 508.
Referring to
Thus, with the addition of the modifications proposed by the present invention a working speech recognition solution in the source language can be ported into a working speech recognition solution in the target language.
In accordance with the present invention, there is provided a method for porting a speech recognition solution in a source language to recognize a target language, the method comprises the following steps as shown in
The technical advancements of the present invention include:
While considerable emphasis has been placed herein on the components and component parts of the preferred embodiments, it will be appreciated that many embodiments can be made and that many changes can be made in the preferred embodiments without departing from the principles of the invention. These and other changes in the preferred embodiment as well as other embodiments of the invention will be apparent to those skilled in the art from the disclosure herein, whereby it is to be distinctly understood that the foregoing descriptive matter is to be interpreted merely as illustrative of the invention and not as a limitation.
| Number | Date | Country | Kind |
|---|---|---|---|
| 1263/MUM/2009 | May 2009 | IN | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 7406417 | Hain | Jul 2008 | B1 |
| 20040186732 | Okura et al. | Sep 2004 | A1 |
| 20050197835 | Reinhard et al. | Sep 2005 | A1 |
| 20070255567 | Bangalore et al. | Nov 2007 | A1 |
| 20090006097 | Etezadi et al. | Jan 2009 | A1 |
| 20090150153 | Li et al. | Jun 2009 | A1 |
| 20090248394 | Sarikaya et al. | Oct 2009 | A1 |
| Number | Date | Country |
|---|---|---|
| 2009-150591 | Dec 2009 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20100299133 A1 | Nov 2010 | US |
