Patent Application
20070118372
The invention relates generally to generating closed captions and more particularly to a system and method for automatically generating closed captions using speech recognition.
Closed captioning is the process by which an audio signal is translated into visible textual data. The visible textual data may then be made available for use by a hearing-impaired audience in place of the audio signal. A caption decoder embedded in televisions or video recorders generally separates the closed caption text from the audio signal and displays the closed caption text as part of the video signal.
Speech recognition is the process of analyzing an acoustic signal to produce a string of words. Speech recognition is generally used in hands-busy or eyes-busy situations such as when driving a car or when using small devices like personal digital assistants. Some common applications that use speech recognition include human-computer interactions, multi-modal interfaces, telephony, dictation, and multimedia indexing and retrieval. The speech recognition requirements for the above applications, in general, vary, and have differing quality requirements. For example, a dictation application may require near real-time processing and a low word error rate text transcription of the speech, whereas a multimedia indexing and retrieval application may require speaker independence and much larger vocabularies, but can accept higher word error rates.
Embodiments of the invention provide a system for generating closed captions. The system includes a speech recognition engine configured to generate one or more text transcripts corresponding to one or more speech segments from an audio signal. The system further includes a processing engine, one or more context-based models and an encoder. The processing engine is configured to process the text transcripts. The context-based models are configured to identify an appropriate context associated with the text transcripts. The encoder is configured to broadcast the text transcripts corresponding to the speech segments as closed captions.
In another embodiment, a method for automatically generating closed captioning text is provided. The method includes obtaining one or more speech segments from an audio signal. Then, the method includes generating one or more text transcripts corresponding to the one or more speech segments and identifying an appropriate context associated with the text transcripts. The method then includes processing the one or more text transcripts and broadcasting the text transcripts corresponding to the speech segments as closed captioning text.
These and other features, aspects, and advantages of the present invention will become better understood when the following detailed description is read with reference to the accompanying drawings in which like characters represent like parts throughout the drawings, wherein:
The context-based models 16 are configured to identify an appropriate context 17 associated with the text transcripts 22 generated by the speech recognition engine 12. In a particular embodiment, and as will be described in greater detail below, the context-based models 16 include one or more topic-specific databases to identify an appropriate context 17 associated with the text transcripts. In a particular embodiment, a voice identification engine 30 may be coupled to the context-based models 16 to identify an appropriate context of speech and facilitate selection of text for output as captioning. As used herein, the “context” refers to the speaker as well as the topic being discussed. Knowing who is speaking may help determine the set of possible topics (e.g., if the weather anchor is speaking, topics will be most likely limited to weather forecasts, storms, etc.). In addition to identifying speakers, the voice identification engine 30 may also be augmented with non-speech models to help identify sounds from the environment or setting (explosion, music, etc.). This information can also be utilized to help identify topics. For example, if an explosion sound is identified, then the topic may be associated with war or crime.
The voice identification engine 30 may further analyze the acoustic feature of each speech segment and identify the specific speaker associated with that segment by comparing the acoustic feature to one or more statistical models corresponding to a set of possible speakers and determining the closest match based upon the comparison. The speaker models may be trained offline and loaded by the voice identification engine 30 for real-time speaker identification. For purposes of accuracy, a smoothing/filtering step may be performed before presenting the identified speakers to avoid instability (generally caused due to unrealistic high frequency of changing speakers) in the system.
The processing engine 14 processes the text transcripts 22 generated by the speech recognition engine 12. The processing engine 14 includes a natural language module 15 to analyze the text transcripts 22 from the speech recognition engine 12 for word errors. In particular, the natural language module 15 performs word error correction, named-entity extraction, and output formatting on the text transcripts 22. A word error correction of the text transcripts is generally performed by determining a word error rate corresponding to the text transcripts. The word error rate is defined as a measure of the difference between the transcript generated by the speech recognizer and the correct reference transcript. In some embodiments, the word error rate is determined by calculating the minimum edit distance in words between the recognized and the correct strings. Named entity extraction processes the text transcripts 22 for names, companies, and places in the text transcripts 22. The names and entities extracted may be used to associate metadata with the text transcripts 22, which can subsequently be used during indexing and retrieval. Output formatting of the text transcripts 22 may include, but is not limited to, capitalization, punctuation, word replacements, insertions and deletions, and insertions of speaker names.
Referring to
In some embodiments, the context-based models 16 analyze the text transcripts 22 based on a topic specific word probability count in the text transcripts. As used herein, the “topic specific word probability count” refers to the likelihood of occurrence of specific words in a particular topic wherein higher probabilities are assigned to particular words associated with a topic than with other words. For example, as will be appreciated by those skilled in the art, words like “stock price” and “DOW industrials” are generally common in a report on the stock market but not as common during a report on the Asian tsunami of December 2004, where words like “casualties,” and “earthquake” are more likely to occur. Similarly, a report on the stock market may mention “Wall Street” or “Alan Greenspan” while a report on the Asian tsunami may mention “Indonesia” or “Southeast Asia”. The use of the context-based models 16 in conjunction with the topic-specific database 34 improves the accuracy of the speech recognition engine 12. In addition, the context-based models 16 and the topic-specific databases 34 enable the selection of more likely word candidates by the speech recognition engine 12 by assigning higher probabilities to words associated with a particular topic than other words.
Referring to
An encoder 44 broadcasts the text transcripts 22 corresponding to the speech segments as closed caption text 46. The encoder 44 accepts an input video signal, which may be analog or digital. The encoder 44 further receives the corrected and formatted transcripts 23 from the processing engine 14 and encodes the corrected and formatted transcripts 23 as closed captioning text 46. The encoding may be performed using a standard method such as, for example, using line 21 of a television signal. The encoded, output video signal may be subsequently sent to a television, which decodes the closed captioning text 46 via a closed caption decoder. Once decoded, the closed captioning text 46 may be overlaid and displayed on the television display.
While the invention has been described in detail in connection with only a limited number of embodiments, it should be readily understood that the invention is not limited to such disclosed embodiments. Rather, the invention can be modified to incorporate any number of variations, alterations, substitutions or equivalent arrangements not heretofore described, but which are commensurate with the spirit and scope of the invention. Additionally, while various embodiments of the invention have been described, it is to be understood that aspects of the invention may include only some of the described embodiments. Accordingly, the invention is not to be seen as limited by the foregoing description, but is only limited by the scope of the appended claims.
