Patent Application
20150348543
This disclosure relates generally to the fields of speech recognition and, more particularly, to systems and methods for recognizing partial proper names using natural language processing.
In the field of speech recognition, the recognition of dynamic slot values, which include proper names such as personal names, street names, restaurant names, etc., is a key problem, as the recognizers rely on the accurate recognition of proper names to understand the intentions for the users. In some domains, such as the media or audio domain, the proper names could be very long, for example, a song name, an audio book name or a movie name. In such cases, people do not always use the full proper names. Partial names may be used to refer to their corresponding entities when the names are long. In spoken dialog applications, partial names tend to be used instead of the full names. At least one empirical user study shows that at least 25% of proper names are partial names. For example, a user might say “play the song ‘one last dance’” instead of using the full song name “play the song ‘Just One Last Dance’”. While in the user's private song database (e.g. the USB or MP3 from the user) or the public song database, only full names are stored. In such situation, only the transcriptions of the full names are available and incorporated into the speech recognizer.
If the user says some partial name as in the example, the recognizer cannot recognize it correctly because it does not have the information of partial proper names. On the other hand, if the user is required to speak only the full name, it is less natural and will have negative impact on the user experience. Therefore, how to correctly recognize these partial names is an important topic. Compared to the recognition of full names, little attention has been put on the recognition of partial names. Consequently, systems and methods for improving the recognition of partial names in speech recognition systems would be beneficial.
A speech recognition framework enables recognition of partial proper names in spoken dialog applications. The framework uses NLP technologies to analyze the internal structure of the proper names and generate the partial candidates for the names, especially those in some specific domains. By incorporating NLP technologies, the framework generates natural candidates which are more consistent with the user behavior, so that the system can achieve higher coverage and better performance on speech recognition.
In one embodiment, the framework performs a method for speech recognition of partial proper names. The method includes receiving with a processor a corpus including a plurality of complex proper names, performing with the processor natural language processing to generate candidate partial names from the plurality of complex names in the corpus, incorporating with the processor the plurality of candidate partial names into a speech recognizer for recognizing the partial names in a speech recognition process, and storing the plurality of candidate names in association with the complex proper names for identification of the complex proper names in a speech recognition process after recognizing the partial names.
In another embodiment, a method for producing a model for speech recognition of complex proper names in response to speech input including partial proper names has been developed. The method includes receiving with a processor a corpus including a plurality of complex proper names, performing with the processor natural language processing to generate syntactic structure corresponding to a plurality of words in one complex proper name in the corpus, generating with the processor a plurality of candidate partial names corresponding to the one complex proper name using a machine learning process with reference to the syntactic structure corresponding to the one complex proper name and the plurality of words in the one complex proper name, incorporating with the processor the plurality of candidate partial names into a model for a speech recognizer to recognize partial names in a speech recognition process, and storing with the processor the model for the speech recognizer in a memory for identification of the complex proper names in a speech recognition process after recognizing the partial names.
In another embodiment, a speech recognition system that identifies complex proper names from speech inputs including partial proper names has been developed. The system includes a memory and a processor operatively connected to the memory. The memory is configured to store a corpus including a plurality of complex proper names and a model for a speech recognizer. The processor is configured to perform natural language processing to generate syntactic structure corresponding to a plurality of words in one complex proper name in the corpus, generate a plurality of candidate partial names corresponding to the one complex proper name using a machine learning process with reference to the syntactic structure corresponding to the one complex proper name and the plurality of words in the one complex proper name, incorporate the plurality of candidate partial names into the model for the speech recognizer to recognize partial names in a speech recognition process, and store the model for the speech recognizer in the memory for identification of the complex proper names in a speech recognition process after recognizing the partial names.
For the purposes of promoting an understanding of the principles of the embodiments disclosed herein, reference is now be made to the drawings and descriptions in the following written specification. No limitation to the scope of the subject matter is intended by the references. The present disclosure also includes any alterations and modifications to the illustrated embodiments and includes further applications of the principles of the disclosed embodiments as would normally occur to one skilled in the art to which this disclosure pertains.
The term “complex proper name” refers to a proper name that includes multiple words in one or more phrases. For example, “Dr. Strangelove or: How I Learned to Stop Worrying and Love the Bomb” is an example of a complex proper name referring to the title of a movie. The term “partial proper name” refers to a simplification of a complex proper name that is commonly associated with a longer complex proper name and is often used in conversation by humans.
The system 100 depicted in
The system 100 of
The NLP processing can also include categories of syntactic units, such as verb phrase, adjective phrase and the like. The syntactic units represent the syntactic structure of the names, and are helpful in generating the meaningful and intuitive candidates of partial proper names which are likely to be spoken by the users.
For example, some users are more likely to keep the original noun phrases in the proper names, while omitting some adverbial phrase or prepositional phrase. So such information can be used to generate the candidates of partial names. The purpose of the presented framework is to improve the speech recognition on the partial proper names by incorporating the NLP technologies. This framework can be combined with any speech recognizer and used in any spoken dialog applications.
The system 100 includes four main components, the NLP module 104, the module for candidate generation of partial proper names 108, the speech recognition module 112 and the post processing module 116. The description of each module is as follows.
The NLP module 104 performs natural language processing on a corpus of full, complex proper names. When the full proper names are obtained from the public database or personal database (e.g. the music player that a user plugs into the system), these full names will first be processed by the NLP module. Compared to the short proper names such as a person name or city name, the complex proper names are usually long and have complicated internal syntactic structures, such as a song name or a movie name. Consequently, the names are suitable for NLP techniques. The processing includes sub modules of Part-of-Speech (POS) tagging, shallow chunking, syntactic parsing or other necessary processing steps.
In the NLP, the specific tags in the results represent the different syntactic role of each word in the complex proper name. The NLP identifies the syntactic structure of the complex proper name and assigns syntactic type information to individual words or groups of words in the proper name. For example the NLP produces the following results, given the full song name: “Somewhere Over The Rainbow”
The output from the NLP module 104 includes the syntactic structural results of one or more NLP techniques to form the input to the candidate generation module 108. The candidate generation module 108 performs one or more machine learning techniques to identify candidate partial proper names that correspond to full proper names using the both structured syntactical elements from the NLP module 104 and the words in the complex proper name as input. The candidate generation module 108 uses an existing machine learning techniques to process the output results from the NLP module 104. Examples of machine learning techniques include Conditional Maximum Entropy (CME), Conditional Random Field (CRF), Hidden Markov Model (HMM), Neural Network, or their variants. The candidate generation module 108 uses the syntactic structural identifiers for the phrases (noun phrase, verb phrase, prepositional phrase, etc.), sub-clauses or parse tree from the NLP module in addition to using the actual words from the full proper name to generate the partial proper name candidates. For example, the term “rainbow” is a noun and “the rainbow” is a noun phrase in the example above from the NLP module 104, and the candidate generation module 108 applies the machine learning techniques to both the words “rainbow” and “the rainbow” and the syntactical structural types for these terms.
The information about the syntactic structure of a complex proper name is used to generate the partial candidates in the partial proper name candidate generation module 108.
Once again using “Somewhere Over The Rainbow” as an example of complex proper name, if only the noun phrase (NP) is kept, then results include one candidate of partial name: NP: The Rainbow
If all types of phrases are retained, the results include three candidates of partial names:
In some embodiments, the system 100 performs a training process to obtain more robust prediction of the partial names based on a set of speech input data of partial names that are collected from users of the system 100. Such data can be used as training data. Then the machine learning based approaches can be applied to learn the model of partial proper name generation, depending on the word features or syntactic features such as the types or occurrence of the syntactic constituents. The learning procedure can be regarded as a sequence labeling process, where each word in the original name is labeled to be kept in the partial name candidate or to be omitted from the partial name candidate.
The NLP processing and candidate generation can be executed offline using one or more network-connected servers for the existing public database of proper names. The servers optionally include additional computational processing power that exceeds the capabilities of mobile electronic devices, in-vehicle driver information systems, and other electronic devices that commonly serve as front-ends for speech interaction with a user. For those dynamic name lists from an individual user, the procedure can be executed online by the device for a more limited range of complex proper names.
After the candidate list of partial names is generated, the candidate list is sent to the transcription service in module 112. The transcription service in the speech recognizer 112 produces a corresponding phonetic transcription of each partial name and incorporates the partial name into the model for the speech recognizer 112 that is stored in the memory. The updated model for the speech recognizer 112 is used in subsequent speech recognition operations that receive speech input data including partial proper names to enable the speech recognizer 112 to identify complex proper names from the partial proper names. If a partial name candidate occurs in the user speech, the partial candidate name is recognized more accurately when the transcription has been integrated into the speech recognizer. The NLP processing module 104 in the system 100 improves the robustness of the results of the partial candidate name generation module 108 and the corresponding results of the speech recognizer 112.
As depicted in
After post-processing the system 100 provides the single or n-best results from the post-processing module 116 as input to another software process. For example, a search facility in a media playback program receives the recognized result of “Somewhere Over the Rainbow” in response to the speech input that only includes the partial name “Over the Rainbow”. The media playback program then identifies an audio or video file or online feed that corresponds to the full proper name for the song “Somewhere Over the Rainbow,” and begins playback of the selected media file. In other configurations, the computing system performs another action in response to the identified proper name, such as searching for a proper name of a point of interest (POI) in an in-vehicle navigation application using the identified full proper name, and generating navigational route or other suitable output for the POI.
It will be appreciated that variants of the above-disclosed and other features and functions, or alternatives thereof, may be desirably combined into many other different systems, applications or methods. For example, while the foregoing embodiments are configured to use standard tokens corresponding to English words, various other languages are also suitable for use with the embodiments described herein. Various presently unforeseen or unanticipated alternatives, modifications, variations or improvements may be subsequently made by those skilled in the art that are also intended to be encompassed by the following claims.
This application claims priority to U.S. Provisional Patent Application No. 62/006,496, which is entitled “Speech Recognition of Partial Proper Names by Natural Language Processing,” and was filed on Jun. 2, 2014, the entire contents of which are hereby expressly incorporated by reference herein.
| Number | Date | Country | |
|---|---|---|---|
| 62006496 | Jun 2014 | US |
