Patent Application
20080077859
This application refers to material comprising a portion of a computer program listing presented as an appendix on CD. The file on the accompanying CD entitled “Appendix for Spelling and Grammar Checking System.doc”, created May 8, 2001, size 80,384 bytes, on the CD is incorporated herein by reference. The file includes three appendices, entitled “Appendix A”, “Appendix B”, and “Appendix C”.
1. Field of the Invention
The present invention relates generally to a spelling and grammar checking system, and more particularly to a spelling and grammar checking system which corrects misspelled words, incorrectly-used words, and contextual and grammatical errors. The invention has particular utility in connection with machine translation systems, word processing systems, and text indexing and retrieval systems such as World Wide Web search engines.
2. Description of the Related Art
Conventional spelling correction systems, such as those found in most common word processing applications, check whether each word in a document is found in a dictionary database. When a word is not found in the dictionary, the word is flagged as being incorrectly spelled. Suggestions for replacing the incorrectly-spelled word with its correctly-spelled counterpart are then determined by inserting, deleting and/or transposing characters in the misspelled word. For example, in a sentence like My son thre a ball at me, the word thre is not correctly-spelled. Conventional spelling correction systems, such as those described in U.S. Pat. No. 4,580,241 (Kucera) and U.S. Pat. No. 4,730,269 (Kucera), suggest words such as threw, three, there and the, as possible alternatives for the misspelled word by adding and deleting characters at different locations in the misspelled word. These alternative words are then displayed to a user, who must then select one of the alternatives.
One of the drawbacks of conventional systems is that they lack the ability to suggest alternative words based on the context in which the misspelled word appears. For example, in the following three sentences, the word thre appears in different contexts and, therefore, should be corrected differently in each sentence.
More specifically, in the first sentence, the incorrectly-spelled word thre should be replaced by threw. In the second sentence, the word thre should be replaced by the. In the third sentence, the word thre should be replaced by three. In spite of these differences in context, conventional spelling correction systems suggest the same list of alternative words, ranked in the same order, for all three of the foregoing sentences. For example, the spelling correction program provided in Microsoft™ Word '97 suggests the following words, in the following order, for all three of the foregoing sentences: three, there, the, throe, threw.
Since conventional spelling correction systems do not rank alternative words according to context, such systems are not able to correct spelling mistakes automatically, since to do so often leads to an inordinate number of incorrectly corrected words. Rather, such systems typically use an interactive approach to correcting misspelled words. While such an approach can be effective, it is inefficient, and oftentimes very slow, particularly when large documents are involved. Accordingly, there exists a need for a spell checking system which is capable of ranking alternative words according to context, and which is also capable of automatically correcting misspelled words without significant user intervention.
Conventional spelling correction systems are also unable to correct grammatical errors in a document or other input text, particularly if those words are spelled correctly but are misused in context. By way of example, although the word too is misused in the sentence He would like too go home, conventional spelling correction systems would not change too to, since too is correctly spelled. In this regard, grammar checking systems are available which correct improperly used words (see, e.g., U.S. Pat. No. 4,674,065 (Lange), U.S. Pat. No. 5,258,909 (Damerau), U.S. Pat. No. 5,537,317 (Schabes), U.S. Pat. No. 4,672,571 (Bass), and U.S. Pat. No. 4,847,766 (McRae)). Such systems, however, are of limited use, since they are only capable of correcting relatively short lists of predefined words. More importantly, such systems are not capable of performing grammar corrections on words that have been misspelled.
Accordingly, there exists a need for a spelling and grammar checking system which is capable of correcting words that have misused in a given context in cases where the words have been spelled incorrectly and in cases where the words have been spelled correctly.
The present invention addresses the foregoing needs by providing a system which corrects both the spelling and grammar of words using finite state machines, such as finite state transducers and finite state automata. For each word in a text sequence, the present invention provides a list of alternative words ranked according to a context of the text sequence, and then uses this list to correct words in the text (either interactively or automatically). The invention has a variety of uses, and is of particular use in the fields of word processing, machine translation, text indexing and retrieval, and optical character recognition, to name a few.
In brief, the present invention determines alternatives for misspelled words, and ranks these alternatives based on a context in which the misspelled word occurs. For example, for the sentence My son thre a ball through the window, the present invention suggests the word threw as the best correction for the word thre, whereas for the sentence He broke thre window, the present invention suggests the word the as the best correction for the word thre. In its interactive mode, the invention displays alternative word suggestions to a user and then corrects misspelled words in response to a user's selection of an alternative word. In contrast, in its automatic mode, the present invention determines, on its own, which of the alternatives should be used, and then implements any necessary corrections automatically (i.e., without user input).
Advantageously, the invention also addresses incorrect word usage in the same manner that it addresses misspelled words. Thus, the invention can be used to correct improper use of commonly-confused words such as who and whom, homophones such as then and than, and other such words that are spelled correctly, but that are improper in context. For example, the invention will correct the sentence He thre the ball to the sentence He threw the ball (and not three, the, . . . ); the sentence fragment flight simulator to flight simulator (and not stimulator); the sentence fragment air baze to air base (and not baize, bass, babe, or bade); the phrase Thre Miles Island to Three Miles Island (and not The or Threw); and the phrase ar traffic controller to air traffic controller (and not are, arc, . . . ). The invention also can be used to restore accents (such as á, à, é, . . . ) or diacritic marks (such as ñ, ç, . . . ) in languages such as French and Spanish. For example, the current invention corrects the sentence il l'a releve to il l'a relevé (and not relève, relèvent, . . . ).
According to one aspect, the present invention is a system (i.e., an apparatus, a method and/or computer-executable process steps) for correcting misspelled words in input text. The system detects a misspelled word in the input text, and determines a list of alternative words for the misspelled word. The list of alternative words is then ranked based on a context of the input text.
According to another aspect, the present invention is a word processing system for creating and editing text documents. The word processing system inputs text into a text document, spell-checks the text so as to replace misspelled words in the text with correctly-spelled words, and outputs the document. The spell-checking performed by the system comprises detecting misspelled words in the text, and, for each misspelled word, determining a list of alternative words for the misspelled word, ranking the list of alternative words based on a context in the text, selecting one of the alternative words from the list, and replacing the misspelled word in the text with the selected one of the alternative words.
According to another aspect, the present invention is a machine translation system for translating text from a first language into a second language. The machine translation system inputs text in the first language, spell-checks the text in the first language so as to replace misspelled words in the text with correctly-spelled words, translates the text from the first language into the second language, and outputs translated text. The spell-checking performed by the system comprises detecting misspelled words in the text, and, for each misspelled word, determining a list of alternative words for the misspelled word, ranking the list of alternative words based on a context in the text, selecting one of the alternative words from the list, and replacing the misspelled word in the document with the selected one of the alternative words.
According to another aspect, the present invention is a machine translation system for translating text from a first language into a second language. The machine translation system inputs text in the first language, translates the text from the first language into the second language, spell-checks the text in the second language so as to replace misspelled words in the text with correctly-spelled words, and outputs the text. The spell-checking performed by the system comprises detecting misspelled words in the text, and, for each misspelled word, determining a list of alternative words for the misspelled word, ranking the list of alternative words based on a context in the text, selecting one of the alternative words from the list, and replacing the misspelled word in the document with the selected one of the alternative words.
According to another aspect, the present invention is an optical character recognition system for recognizing input character images. The optical character recognition system inputs a document image, parses character images from the document image, performs recognition processing on parsed character images so as to produce document text, spell-checks the document text so as to replace misspelled words in the document text with correctly-spelled words, and outputs the document text. The spell-checking performed by the system comprises detecting misspelled words in the document text, and, for each misspelled word, determining a list of alternative words for the misspelled word, ranking the list of alternative words based on a context in the text, selecting one of the alternative words from the list, and replacing the misspelled word in the document text with the selected one of the alternative words.
According to another aspect, the present invention is a system for retrieving text from a source. The system inputs a search word, corrects a spelling of the search word to produce a corrected search word, and retrieves text from the source that includes the corrected search word.
According to another aspect, the present invention is a system for retrieving text from a source. The system inputs a search phrase comprised of a plurality of words, at least one of the plurality of words being an incorrect word, and replaces the incorrect word in the search phrase with a corrected word in order to produce a corrected search phrase. Text is then retrieved from the source based on the corrected search phrase.
According to another aspect, the present invention is a system for correcting misspelled words in input text sequences received from a plurality of different clients. The system stores, in a memory on a server, a lexicon comprised of a plurality of reference words, and receives the input text sequences from the plurality of different clients. The system then spell-checks the input text sequences using the reference words in the lexicon, and outputs spell-checked text sequences to the plurality of different clients.
According to another aspect, the present invention is a system for selecting a replacement word for an input word in a phrase. The system determines alternative words for the input word, the alternative words including at least one compound word which is comprised of two or more separate words, each alternative word having a rank associated therewith. The system then selects, as the replacement word, an alternative word having a highest rank.
According to another aspect, the present invention is a system for correcting grammatical errors in input text. The system generates a first finite state machine (“FSM”) for the input text, the first finite state machine including alternative words for at least one word in the input text and a rank associated with each alternative word, and adjusts the ranks in the first FSM in accordance with one or more of a plurality of predetermined grammatical rules. The system then determines which of the alternative words is grammatically correct based on the ranks associated with the alternative words, and replaces the at least one word in the input text with a grammatically-correct alternative word determined in the determining step.
According to another aspect, the present invention is a word processing system for creating and editing text documents. The word processing system inputs text into a text document, checks the document for grammatically-incorrect words, replaces grammatically-incorrect words in the document with grammatically-correct words, and outputs the document. The checking performed by the system comprises (i) generating a finite state machine (“FSM”) for text in the text document, the finite state machine including alternative words for at least one word in the text and a rank associated with each alternative word, (ii) adjusting the ranks in the FSM in accordance with one or more of a plurality of predetermined grammatical rules, and (iii) determining which of the alternative words is grammatically correct based on ranks for the alternative words.
According to another aspect, the present invention is a machine translation system for translating text from a first language into a second language. The machine translation system inputs text in the first language, checks the text in the first language for grammatically-incorrect words, and replaces grammatically-incorrect words in the text with grammatically-correct words. The machine translation system then translates the text with the grammatically-correct words from the first language into the second language, and outputs the text in the second language. The checking performed by the machine translation system comprises (i) generating a finite state machine (“FSM”) for the text in the first language, the finite state machine including alternative words for at least one word in the text and a rank associated with each alternative word, (ii) adjusting the ranks in the FSM in accordance with one or more of a plurality of predetermined grammatical rules, and (iii) determining which of the alternative words is grammatically correct based on ranks for the alternative words.
According to another aspect, the present invention is a machine translation system for translating text from a first language into a second language. The machine translation system inputs text in the first language, translates the text from the first language into the second language, checks the text in the second language for grammatically-incorrect words, replaces grammatically-incorrect words in the text with grammatically-correct words, and outputs the text with the grammatically-correct words. The checking performed by the system comprises (i) generating a finite state machine (“FSM”) for the text in the second language, the finite state machine including alternative words for at least one word in the text and a rank associated with each alternative word, (ii) adjusting the ranks in the FSM in accordance with one or more of a plurality of predetermined grammatical rules, and (iii) determining which of the alternative words is grammatically correct based on ranks for the alternative words.
According to another aspect, the present invention is an optical character recognition system for recognizing input character images. The optical character recognition system inputs a document image, parses character images from the document image, performs recognition processing on parsed character images so as to produce document text, checks the document text for grammatically-incorrect words, replaces grammatically-incorrect words in the document text with grammatically correct words, and outputs the document text. The checking performed by the system comprises (i) generating a finite state machine (“FSM”) for the document text, the finite state machine including alternative words for at least one word in the text and a rank associated with each alternative word, (ii) adjusting the ranks in the FSM in accordance with one or more of a plurality of predetermined grammatical rules, and (iii) determining which of the alternative words is grammatically correct based on ranks for the alternative words.
According to another aspect, the present invention is a system for retrieving text from a source. The system inputs a search phrase comprised of a plurality of words, at least one of the plurality of words being a grammatically-incorrect word, replaces the grammatically-incorrect word in the search phrase with a grammatically-correct word in order to produce a corrected search phrase, and retrieves text from the source based on the corrected search phrase.
According to another aspect, the present invention is a system of spell-checking input text. The system detects a misspelled word in the input text, stores one or more lexicon finite state machines (“FSM”) in a memory, each of the lexicon FSMs including plural reference words, generates an input FSM for the misspelled word, selects one or more reference words from the lexicon FSMs based on the input FSM, the one or more reference words substantially corresponding to a spelling of the misspelled word, and outputs selected ones of the one or more reference words.
This brief summary has been provided so that the nature of the invention may be understood quickly. A more complete understanding of the invention can be obtained by reference to the following detailed description of the preferred embodiments thereof in connection with the attached drawings.
Also included in PC 4 are display interface 29, keyboard interface 30, mouse interface 31, disk drive interface 32, CD-ROM drive interface 34, computer bus 36, RAM 37, processor 38, and printer interface 40. Processor 38 preferably comprises a microprocessor or the like for executing applications, such those noted above, out of RAM 37. Such applications, including spelling and grammar checking code 49 of the present invention, may be stored in memory 20 (as noted above) or, alternatively, on a floppy disk in disk drive 16 or a CD-ROM in CD-ROM drive 17. In this regard, processor 38 accesses applications (or other data) stored on a floppy disk via disk drive interface 32 and accesses applications (or other data) stored on a CD-ROM via CD-ROM drive interface 34.
Application execution and other tasks of PC 4 may be initiated using keyboard 12 or mouse 14, commands from which are transmitted to processor 38 via keyboard interface 30 and mouse interface 31, respectively. Output results from applications running on PC 4 may be processed by display interface 29 and then displayed to a user on display 11. To this end, display interface 29 preferably comprises a display processor for forming images based on data provided by processor 38 over computer bus 36, and for outputting those images to display 11. Output results from applications, such spelling and grammar checking code 49, running on PC 4 may also be provided to printer 19 via printer interface 40. In this case, processor 38 also executes print driver 24 so as to perform appropriate formatting of the output results prior to their transmission to printer 19.
Turning to spelling and grammar checking code 49, this code is comprised of computer-executable process steps for, among other things, detecting a misspelled word in input text, determining a list of alternative words for the misspelled word, ranking the list of alternative words based on a context of the input text, selecting one of the alternative words from the list, and replacing the misspelled word in the text with the selected one of the alternative words. In preferred embodiments, the present invention is operable in an interactive mode, in which the selecting step is performed manually (i.e., a user selects an alternative word from the list), or in an automatic mode, in which the selecting step is performed automatically (i.e., without user intervention) based on predetermined criteria. These modes are described in more detail below.
Interactive Mode
Following step 51, the misspelled word is passed to spelling suggestion module 52. Spelling suggestion module 52 suggests “out-of-context” corrections for the misspelled word. That is, spelling suggestion module 52 determines a list of correctly-spelled alternative (or “replacement”) words for the misspelled word without regard to the context in which the misspelled word appears in input text 50. A detailed description of the operation of spelling suggestion module 52 is provided below. For now, suffice it to say that spelling suggestion module 52 determines this list of alternative words by inserting, deleting, replacing, and/or transposing characters in the misspelled word until correctly-spelled alternative words are obtained. Spelling suggestion module 52 also identifies portions (e.g., characters) of the misspelled word which sound substantially similar to portions of correctly-spelled alternative words in order to obtain additional correctly-spelled alternatives words. Once all alternative words have been determined, spelling suggestion module 52 ranks these words in a list based, e.g., on a number of typographical and/or phonetic modifications that must be made to the misspelled word in order to arrive at each alternative word.
List 54 of alternative words, which was output by spelling suggestion module 52, is then passed to automaton conversion module 55, along with original text 50. A detailed description of the operation of automaton conversion module 55 is provided below. For now, suffice it to say that automaton conversion module 55 converts text 50 and list 54 into an input finite state machine (hereinafter “FSM”), such as a finite state transducer (hereinafter “FST”) or a finite state automaton (hereinafter “FSM”), having a plurality of arcs. Each arc in the input FSM includes an alternative word and a rank (e.g., a weight, a probability, etc.) associated with each alternative word. This rank corresponds to a likelihood that the alternative word, taken out of context, comprises a correctly-spelled version of the original misspelled word.
In this regard, the concept of FSTs is described in Roche, Emmanuel, “Text Disambiguation by Finite-State Automata: An Algorithm and Experiments on Corpora”, Proceedings of the Conference, Nantes (1992), Roche, Emmanuel and Schabes, Yves, “Introduction to Finite-State Language Processing”, Finite-State Language Processing. (1997), Koskenniemi, Kimmo, “Finite-State Parsing and Disambiguation”, Proceedings of the Thirteenth International Conference on Computational Linguistics, Helsinki, Finland (1990), and Koskenniemi et al. “Compiling and using Finite-State Syntactic Rules”, Proceedings of the Fifteenth International Conference on Computational Linguistics. (1992). The contents of these articles are hereby incorporated by reference into the subject application as if set forth herein in full. To summarize, FSTs are FSMs have a finite number of states with arcs between the states. These arcs have one input and one or more outputs. Generally speaking, an FST functions as a particular method for mapping inputs to outputs. The present invention uses FSTs with weights, such as the those described in Pereira et al. “Weighted Rational Transductions and Their Application to Human Language Processing”, ARPA Workshop on Human Language Technology (1994). The contents of this article is hereby incorporated by reference into the subject application as if set forth herein in full.
Returning to
Automaton conversion module 55 then transmits input FSM 56 (which in preferred embodiments is an FST) to contextual ranking module 57. Contextual ranking module 57 ranks alternative words in input FSM 56 by taking into account the context (e.g., grammar, parts-of-speech, etc.) of text 50. In brief, contextual ranking module 57 generates a second FSM for text 50 and the alternative words in accordance with one or more of a plurality of predetermined grammatical rules. This second FSM is comprised of a plurality of arcs which include the alternative words and ranks (e.g., weights) associated therewith, where a rank of each alternative word corresponds to a likelihood that the alternative word, taken in grammatical context, comprises a correctly-spelled version of the misspelled word. Contextual ranking module 57 then combines corresponding ranks of input FSM 56 and the second FSM (e.g., contextual ranking module 57 adds weights from respective FSMs) so as to generate an “in-context” ranking of the alternative words. Then, contextual ranking module 57 outputs a list 59 of alternative words for the misspelled word, which are ranked according to context. A more detailed description of the operation of contextual ranking module 57 is provided below.
Ranked list 59 of alternative words, which was generated by contextual ranking module 57, is then displayed to a user, e.g., on display screen 11. In step 60, the user can then manually select (using, e.g., mouse 14, keyboard 12, etc.) one of the alternative words from ranked list 59. Thereafter, the selected one of the alternative words (i.e., selected alternative 61) is provided to replacement module 62, along with original text 50. Replacement module 62 replaces the misspelled word in text 50 with user-selected alternative word 61, and then outputs corrected text 63.
Automatic Mode
To begin, text 50 is input to spell checking module 64. Spell checking module 64 is identical to that described above in the interactive mode, except that, in this mode, spell checking module 64 searches through all of text 50 in order to detect all misspelled words. Which mode (i.e., interactive or automatic) spell checking module 64 operates in is set beforehand, e.g., in response to a user input. Once all misspelled words have been detected, spell checking module 64 outputs text 66 with the incorrectly-spelled words appropriately identified.
Next, text 66, i.e., the text with the incorrectly spelled words identified, is provided to spelling suggestion module 52. Spelling suggestion module 52 is identical to that described above, except that, in this mode, spelling suggestion module 52 determines and outputs a list of correctly-spelled alternative (or “replacement”) words for every misspelled word in text 50, rather than for just one misspelled word. Which mode (i.e., interactive or automatic) spelling suggestion module 52 operates in is set beforehand, e.g., in response to a user input.
As before, spelling suggestion module 52 outputs a list of “out-of-context” alternative words to automaton conversion module 55. Automaton conversion module 55 is identical to that described above, except that, in this mode, automaton conversion module 55 generates an FSM 56 (see above) for each misspelled word in input text 50. These FSMs are then transmitted to contextual ranking module 57. Contextual ranking module 57 is identical to that described above, in that it generates a second FSM for input text 50 based on a plurality of predetermined grammatical rules and combines this second FSM with FSM 56 generated by automaton conversion module 55 in order to provide a contextually-ranked list 59 of the alternatives for each misspelled word in text 50.
Thereafter, list 59 is provided from contextual ranking module 57 to best suggestion selection module 60. Best suggestion selection module 60 selects the “best” alternative for each misspelled word, replaces each misspelled word in the text with its corresponding best alternative, and outputs corrected text 61, which includes these best alternatives in place of the misspelled words. In preferred embodiments of the invention, best suggestion selection module 60 selects each best alternative based on list 59 without any user intervention. For example, best suggestion module 60 may select the first, or highest, ranked alternative word in list 59, and then use that word to correct the input text.
Spelling Suggestion Module
In brief, spelling suggestion module 52 determines and outputs alternative words for a misspelled word in input text 50. In preferred embodiments of the invention, these alternative words are not ranked according to context, but rather are ranked based on the number of typographical changes that must be made to the misspelled word to arrive at an alternative word. To this end, spelling suggestion module 52 is comprised of computer-executable process steps to store one or more lexicon FSTs (in general, FSMs), where each of the lexicon FSTs includes plural reference words and a phonetic representation of each reference word, and to generate an input FST (in general, an FSM) for a misspelled word, where the input FST includes the misspelled word and a phonetic representation of the misspelled word. Spelling suggestion module 52 also includes computer-executable process steps to select one or more reference words from the lexicon FSTs based on the input FST, where the one or more reference words substantially corresponds to either a spelling of the misspelled word or to the phonetic representation of the misspelled word.
In more detail,
Input FST 71 represents the spelling and pronunciation of input word 70. More specifically, each arc of input FST 71 includes a pair of characters c/p, where c is a character in input word 70 and p is a phonetic symbol representing the pronunciation of character c.
Lexicon FST 74 is preferably stored in a single memory, and comprises one or more lexicon FSTs (or FSMs, in general) which have been generated by the process steps of the present invention. Each of these lexicon FSTs includes plural reference words (e.g., English-language words, French-language words, German-language words, etc.) and a phonetic representation of each reference word. An example of a lexicon FST is shown in
Spelling FSA 76 comprises an additional FSM which has been generated by the process steps of the present invention. Specifically, spelling FSA 78 includes a plurality of states, the states comprising at least states of lexicon FST 74 and states of input FST 71. Spelling FSA 76 is used to select one or more reference words from lexicon FST 74. These selected reference words comprise the alternative words for output by spelling suggestion module 52.
In more detail, each state of spelling FSA 76 is identified by a quadruple (i,l,t,cost), in which the first element i is a state in input FST 71 and records which portion of input word 70 has been already processed; the second element l is a state in lexicon FST 74 which records words that are potential alternatives for the input word; the third element t indicates whether a character transposition has occurred in the input word (e.g., rluer to ruler, in which the l and u have been transposed) and thus whether characters preceding the transposed characters must be re-examined; and the fourth element cost is the cost associated with a current suggested alternative to input word 70, meaning an indication of the likelihood that the current suggested alternative is actually the correct spelling of input word 70. In this regard, in preferred embodiments of the invention, the lower the cost of a state in spelling FSA 76, the more likely that state represents a path to the correct spelling of input word 70.
Once state selection module 77 has selected a state (i,l,t,cost) to be processed, this state is provided as input to each of following modules: character identity module 80, phonetic identity module 81, character insertion module 82, character deletion module 83, character replacement module 84, character transposition module 85, and character transposition completion module 86. Each of these modules process the current state (i,l,t,cost) 78 of spelling FSA 76 (as set by state selection unit 77), and may also add new states to spelling FSA 76.
In brief, character identity module 80 determines whether characters of a reference word in lexicon FST 74 match characters of word 70 in input FST 71. Phonetic identity module 81 determines whether characters of the reference word are pronounced the same as characters of the input word. Character insertion module 82 determines whether a character inserted in the input word causes at least part of the input word to match at least part of the reference word. Character deletion module 83 determines whether a character deleted from the input word causes at least part of the input word to match at least part of the reference word. Character replacement module 84 replaces characters in the input word with characters in the reference word in order to determine whether at least part of the input word matches at least part of the reference word. Character transposition module 85 changes the order of two or more characters in the input word and compares a changed character in the input word to a corresponding character in the reference word. Finally, character transposition completion module 86 compares characters in the input word which were not compared by character transposition module 85 in order to determine if at least part of the input word matches at least part of the reference word.
In more detail, character identity module 80 checks whether there is a word in lexicon FST 74 which starts at state l and which has a next character that is the same as the next character in input FST 71 at state i. Given a current spelling FSA state of (i,l,t,cost), for all outgoing arcs from state l in lexicon FST 74 going to a state l′ and labeled with pair c/p (where c is a character and p is a pronunciation of that character), and for all outgoing arcs from state i in input FST 71 going to state i′ and labeled with the pair c/p′ (where c is a character and p′ is a pronunciation of the character), character identity module 80 creates an arc in spelling FSA 76 from state (i,l,t,cost) to a newly-added state (i′,l′,0,cost), and labels that arc with character c.
Phonetic identity module 81 checks whether there is a word in lexicon FST 74 starting at state l whose next character is pronounced the same as the next character in input FST 71 at state i. For this processing, the phonetic representations of characters are processed. That is, given a current spelling FSA state of (i,l,t,cost), for all outgoing arcs from state l in lexicon FST 74 going to a state I′ and labeled with the pair c/p (where c is a character and p is a pronunciation of that character), and for all outgoing arcs from state in input FST 71 going to state i′ and labeled with the pair c′/p (where c′ is a character and p is a pronunciation of the character), phonetic identify module 81 creates an arc in spelling FSA 76 from state (i,l,t,cost) to a newly-added state (i′,l′,0,cost+phonetic_identity_cost), and labels that arc with character c. This newly-added state has its cost increased by a predetermined cost, called phonetic_identity_cost, which has a pre-set value that is associated with the fact that the pronunciation of a current character in input FST 71 is identical to the pronunciation of the current character in lexicon FST 74 even though the characters are different.
Character insertion module 82 inserts a character from lexicon FST 74 into input word 70 in input FST 71. More specifically, given a current spelling FSA state of (i,l,t,cost), for all outgoing arcs from state l in lexicon FST 74 going to a state l′ and labeled with the pair c/p (where c is a character and p is a pronunciation of that character), character insertion module 82 creates an arc in spelling FSA 76 from state (i,l,t,cost) to state (i,l′,0 insertion_cost), and labels that arc with character c. This newly-added state has its cost increased by a predetermined cost, called insertion_cost, which has a pre-set value that is associated with the fact that a character has been inserted into word 70 in input FST 71.
Character deletion module 83 deletes a character from input word 70 in input FST 71. More specifically, given a current spelling FSA state of (i,l,t,cost), for all outgoing arcs from state i in input FST 71 going to a state i′ and labeled with the pair c/p (where c is a character and p is a pronunciation of that character), character deletion module 83 creates an arc in spelling FSA 76, which is labeled with “empty character” _ from state (i,l,t,cost) to a newly added state (i′,l,0,cost+deletion_cost). This newly added state has a cost that is increased by a predetermined cost, called deletion_cost, which has a pre-set value that is associated with the fact that a character has been deleted from input word 70 in input FST 71.
Character replacement module 84 replaces a next character in input word 70 with a next character in lexicon FST 74. More specifically, given a current spelling FSA state of (i,l,t,cost), for all outgoing arcs from state l in lexicon FST 74 going to a state l′ and labeled with the pair c/p (where c is a character and p is a pronunciation of that character), and for all outgoing arcs from state i in input FST 71 going to a state l′ and labeled with the pair c′/p′ (where c′ is a character and p′ is a pronunciation of that character), character replacement module 84 creates an arc in spelling FSA 76 to a newly added state (i′,l′,0,cost+replacement_cost), and labels that arc with character c from state (i,l,t,cost). This newly-added state has its cost increased by a predetermined cost, called replacement_cost, that has a pre-set value and that is associated with the fact that a character has been replaced by another character in input word 70.
Character transposition module 85 interchanges the order of two consecutive characters in input word 70, and checks the validity of the next character while remembering the original order of the characters. More specifically, given a current spelling FSA state of (i,l,t,cost), for all outgoing arcs from state i in input FST 71 going to a state i1 and labeled with the pair c1/p1 (where c1 is a character and p1 is a pronunciation of that character), for all outgoing arcs from state i1 in input FST 71 going to a state 12 and labeled with the pair c2/p2 (where c2 is a character and p2 is a pronunciation of that character), and for all outgoing arcs in lexicon FST 74 going from state l to state l′ labeled with the pair c2/p′ (where c2 is a character and p′ is a pronunciation of that character), character transposition module 85 creates an arc in spelling FSA 76 from state (i,l,t,cost) to a newly-added state (i2,l′,c1,cost+transposition_cost), and labels that arc with character c2. This newly-added state has its cost increased by a predetermined cost, called transposition_cost, which has a value that is pre-set and that is associated with the fact that two characters have been transposed in input word 70.
Character transposition completion module 86 completes the transposition of two characters that was started by character transposition module 85. More specifically, given a current spelling FSA state (i,l,t,cost), where t is not zero (indicating that character transposition has occurred), for all outgoing arcs in lexicon FST 74 going from state l to state l′ labeled with the pair t/p′ (where t is a character and p is a pronunciation of that character), character transposition completion module 86 creates an arc in spelling FSA 76 from the state (i,l,t,cost) to a newly-added state (i,l′,0,cost+transposition_completion cost), and labels that arc with the character t. This newly-added state has its cost increased by a predetermined cost, called transposition_completion_cost, which has a value that is pre-set and that is associated with the fact that the second of the transposed characters has been read.
The following describes operation of some of the foregoing modules in an actual example. More specifically, with reference to
Similar processing is also performed for the other states shown in lexicon FST 74 to create additional states 97 to 101 with character deletion module 83 being used between states 97 and 99, and with an_in arcs between those states indicating that a character has been deleted from the word in input FST 71. Once this processing is finished, as shown in
At this point, it is noted that although spelling suggestion module 52, and the rest of the invention for that matter, is described with respect to a word in an input text sequence comprised of plural words, the spell-checking aspect of the invention can be used equally well with a single-word input. Of course, the grammar checking aspects of the invention would not apply in this instance. Accordingly, those modules shown in
Once all states of input FST 71 and lexicon FST 74 have been processed in the foregoing manner, as determined in block 103 of
Pronunciation Conversion Module
As noted above, pronunciation conversion module 73 converts input word 70 into input FST 71. In general, pronunciation conversion module 73 converts any word, whether correctly spelled or misspelled, into an input FST which includes a phonetic representation of the input word, together with the input word. As noted above,
Pronunciation conversion module 73 utilizes a pre-stored phonetic dictionary of words, in which a pronunciation of each character of a word is associated with a phonetic symbol which represents the pronunciation of that character in the context of a word. In order to associate to each character of an input word with a pronunciation, pronunciation conversion module 73 reads the input word from left to right and finds the longest context in the phonetic dictionary which matches the input word. Pronunciation conversion module 73 then transcribes that longest match with phonetic characters until no characters in the input word are left unpronounced. The output is represented as an FST (see, e.g.,
Automaton Conversion Module
Returning to
In more detail, in preferred embodiments of the invention, automaton conversion module 55 generates an FST; although an FSM may be used in the present invention as well. For the sake of brevity, however, the invention will be described with respect to an FST. In this regard, such an FST comprises a finite-number of states, with arcs between the states. Each arc is labeled with a pair of symbols. The first symbol in each pair is an alternative word to the misspelled word found in text 50. The second symbol of each pair is a number representing a rank for that word. As noted above, these rankings are determined based on the number of character transpositions, deletions, additions, etc. that must be performed on the misspelled word in order to arrive at each alternative word.
If step S1203 determines that the ith input word is misspelled, step S1204 generates a new state labeled i+1 for each of the alternative words provided by spelling suggestion module 52. Step S1204 also adds a transition from state i to state i+1. This transition is labeled with an alternative word and with a ranking (e.g., a negative weight). If, on the other hand, step S1203 determines that the ith input word is not misspelled, step S1205 creates a new state i+1 and a transition from state i to state i+1. This transition is labeled with the ith word and has a weight of zero. Thereafter, in step S1206, current state i is increased by one, and processing proceeds to step S1207. If step S1207 determines that a current state i is less than the number of words n, meaning that there are words in the input text still to be processed, flow returns to step S1203. If i equals n, processing ends, and the FST generated by steps S1201 to S1207 is output in step S1208.
As noted above, in preferred embodiments of the invention, automaton conversion module 55 may characterize words which are correctly spelled, but which are commonly confused, as misspelled words. This is done in order to flag these words as possible candidates for the grammar correction process which is described in more detail below. Examples of such words include who and whom. That is, these words are often misused, such as in the sentence I need an assistant who I can trust. Similarly, homophones, such as principal and principle are often confused. Appendix B shows a short lists of such words. Of course, this list is merely representative, and, in the actual invention, the list is much more extensive. This list is preferably stored in a database, e.g., in memory 20, and can be updated or modified via, e.g., fax/modem line 10. Alternatively, this list may be accessed from a remote location via network connection 9. Thus, automaton conversion module 55 identifies words which are often misused or confused based on such a list, and treats these words in the same manner as misspelled words provided by spelling suggestion module 52. That is, such words are included in arcs in the FST generated by automaton conversion module 55.
Contextual Ranking Module
Returning to
In more detail,
Compound Words and Lexical Phrases Module
Compound word and lexical phrases module 110 identifies words which may comprise part of a predetermined list of compound words (i.e., a word comprised of two separate words), and also adds these words as arcs in FST 56. By way of example, in the sentence Pilots practice with flight stimulators, the word stimulators is not necessarily misspelled, but is incorrect in context. That is, the typist meant to type flight simulators, but accidentally included an extra t in simulators. Compound words and lexical phrases module 110 compares the word stimulators to a pre-stored database of compound words. In a case that an input word, in this case stimulators, is similar to a word in a compound word (as measured, e.g., by a number of typographical changes between the input word and a word in a compound word, in this case simulators), compound words and lexical phrases module 110 includes the compound word as an alternative word in an arc of FST 56, together with a single rank associated with the compound word.
In the present invention, a database of compound words is preferably pre-stored, e.g., in memory 20. In preferred embodiments of the invention, each of the compounds words in the database is associated with a part-of-speech that defines a syntactic behavior of the compound word in a sentence. For example, a noun-noun compound, such as air base may be stored in the database and defined therein as a noun (“N”). Another example of a compound word is commercial passenger flight, which is defined in the database as a noun (“N”). Similarly, the phrase according to will be defined in the database as a preposition (“Prep”).
As borne out in the examples provided above, in the database, each compound word or phrase has a single part-of-speech (e.g., part-of-speech tag “N”, “Adv”, etc.) associated therewith. Moreover, these words and phrases exhibit very little morphological or syntactic variation. For example, according to exhibits no morphological or syntactic variation. Similarly air base can be pluralized (air bases), but little else. Appendix C shows a list of representative compound words and phrases, together with their associated parts-of-speech, that are included in the database that is used by compound words and lexical phrase module 110.
In preferred embodiments of the invention, compound words and lexical phrases module 110 also adds, to FST 56, a part-of-speech tag for each compound word or phrase. In addition, compound words and lexical phrases module 110 also adds a relatively large weight to arcs containing potential compound words, reflecting the fact a word may, more likely than not, be a compound word. For the example FST shown in
Morphology Module
Morphology module 112 adds all possible morphological analyses of each word to FST 111. This morphological analysis is performed using a pre-stored morphological dictionary of words. In preferred embodiments of the invention, this morphological dictionary is represented as a collection of small FSTs, each representing a possible morphological analysis of each word. Weights in such FSTs correspond to a relative likelihood that a word is a particular part-of-speech.
For example, for the word left, FST 114 shown in
In the present invention, a weight of a particular path through an FST is computed as the sum of the weights of each of the arcs in the FST. For example, in the FST shown in
Grammar Application Module
Returning to
In more detail,
Weights application module 119 combines (e.g., adds) weights of input FST 116 and grammar FST 120 in order to produce a combined FST 121 in which weights therein are adjusted according to grammatical rules. More specifically, for each path from an initial state to a final state of grammar FST 120, weights application module finds a corresponding path in input FST 116. Thereafter, weights application module 119 replaces weights of input FST 116 with the combined weights of input FST 116 and grammar FST 120. By doing this, weights application module 119 reinforces paths in input FST 116 which are also found in grammar FST 120. For example, grammar FST 120 might include a path which indicates that a singular noun precedes a verb in the third person. Such a path can be used to reinforce portions of input FST 116 where a noun precedes a verb in third person.
Construction of Grammar FST
Grammar FST 120 (see
A “*” before an item in a rule, such as “*NP/0”, indicates that the item is defined elsewhere by a definition rule. When there is no “*” before an item, the item refers to a word which can be specified with the word itself, its root form, and its part of speech. For example, the application rule
indicates that the word there should be matched as an adverb, followed immediately by the word be in the third person singular, i.e., is. If a match is found, meaning that words in input FST 116 comply with this rule, weights 10 and 20 are added to weights of the matching words in input FST 116.
Items to be matched by application rules can have any of the following formats:
Examples of some of the foregoing items are shown in the FST of
Definition rules include an “equal” sign. The left side of the equal sign includes an item of the form “*SYMBOL”; and the right side of the equal sign includes any sequence of items. For example,
is a definition rule. In this example, *NP3S indicates that a noun phrase in the third person singular is formed by an adjective (*ADJP/0) and a noun (:N/10). In a case that words in input FST comply with this rule, a noun in such words is incremented by 10 (from the 10 in “:N/10”) and the adjective is not incremented (from the 0 in “*ADJP/0”).
In the present invention, the grammatical rules are non-recursive, meaning that at no point does a symbol refer to itself. As a result, the rules can be combined into a grammar FST for comparison with input FST 116. Specifically, to generate grammar FST 120, items with a “*” preceding them are recursively replaced by their definitions. Next, the grammatical rules are converted into an FST by concatenating an FST of each obtained item. Application rules are then used to define paths from an initial state to a final state in the constructed FST.
In addition to general grammatical rules (such as subject-verb agreement rules), the present invention also includes specific grammatical constructions in the grammar FST. For example the application rule
corresponds to the construction “too ADJECTIVE to”, as in the sentence “He is too young to drive”. Another example of such a construction is:
which is used for sentences such as “There is a car in his parking space”. Grammar FST 120 also includes auxiliary verb groups (“*VG”), examples of which are also shown in Appendix A.
Post-Grammar Application Module Processing
Returning to
The spelling and grammar checking system of the present invention may be used in conjunction with a variety of different types of applications. Examples of such uses of the invention are provided in more detail below.
Word Processing
Spelling and grammar checking code 49 of the present invention may be used in the context of a word processing application, such as those described above.
More specifically, step S2001 inputs text into a text document. Next, step S2002 spell-checks the text so as to replace misspelled words in the text with correctly-spelled words. In preferred embodiments of the invention, step S2002 is performed in accordance with
Machine Translation
Spelling and grammar checking code 49 of the present invention may be used in the context of a machine translation system which translates documents from one language to another language, such as those described above.
More specifically, step S2101 inputs text in a first language, and step S2102 spell-checks the text in the first language so as to replace misspelled words in the text with correctly-spelled words. In preferred embodiments of the invention, this spell-checking step is performed in accordance with
Following step S2104, step S2105 translates the text from the first language into the second language, and step S2106 spell-checks the text in the second language so as to replace misspelled words in the text with correctly-spelled words. In preferred embodiments of the invention, step S2106 spell checks the text in the same manner as did step S2102. Accordingly, a detailed description of this process is not repeated. Thereafter, step S2107 checks the text in the second language for grammatically-incorrect words in the same manner that step S2103 checked the text in the first language. Accordingly, a detailed description of this process is not repeated. Step S2108 then replaces grammatically-incorrect words in the text with grammatically-correct words, and step S2109 outputs the text with little or no grammatical and/or spelling errors.
Optical Character Recognition
Spelling and grammar checking code 49 of the present invention may be used in the context of an optical character recognition system which recognizes input character images.
More specifically, step S2201 inputs a document image, e.g., via scanner 13, and step S2202 parses character images from the document image. Thereafter, step S2203 performs character recognition processing on parsed character images so as to produce document text. Step S2204 then spell-checks the document text so as to replace misspelled words in the document text with correctly-spelled words. This spell checking is performed in accordance with
Text Indexing and Retrieval
Spelling and grammar checking code 49 of the present invention may be used in the context of a text indexing and retrieval system for retrieving text from a source based on an input search word. Examples of such text indexing and retrieving systems in which the present invention may be used include, but are not limited to, Internet search engines, document retrieval software, etc.
More specifically, step S2301 comprises inputting a search word or a search phrase comprised of plural search words, and step S2302 comprises correcting a spelling of each search word to produce corrected search word(s). Next, in a case that a search phrase is input, step S2303 replaces grammatically-incorrect words in the search phrase with a grammatically-correct word in order to produce a corrected search phrase. In the invention, steps S2302 and S2303 are preferably performed by spelling and grammar checking code 49 shown in
Client-Server Configuration
The spelling and grammar checking system of the present invention may also be utilized in a plurality of different hardware contexts. For example, the invention may be utilized in a client-server context. In this aspect of the invention, a single computer, such as PC 4, can service multiple requests for spelling correction at the same time by executing multiple threads of the same program, such as spelling suggestion module 52. To perform this function, in this embodiment of the invention, processor 38 is multi-tasking.
In brief, this aspect of the invention comprises computer-executable process steps to correct misspelled words in input text sequences received from a plurality of different clients. The process steps include code to store in a memory on a server (e.g., PC 4 shown in
Each of program threads 151 to 153 contains a corresponding memory (i.e., memories 154, 155 and 156) that is used by processor 38 to execute that thread, as well as to perform other processing in relation thereto. Each spelling memory also stores an FSA generated by spelling suggestion module 52 (see
Client-Server Information Retrieval System
More specifically, in
The present invention has been described with respect to particular illustrative embodiments. It is to be understood that the invention is not limited to the above-described embodiments and modifications thereto, and that various changes and modifications may be made by those of ordinary skill in the art without departing from the spirit and scope of the appended claims.
This application is a continuation of and claims priority to U.S. patent application Ser. No. 10/153,460 filed May 22, 2002 which is a continuation of and claims priority to U.S. patent application Ser. No. 09/084,535, filed May 26, 1998, the contents both of which are hereby incorporated by reference in their entirety.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 10153460 | May 2002 | US |
| Child | 11765631 | Jun 2007 | US |
| Parent | 09084535 | May 1998 | US |
| Child | 10153460 | May 2002 | US |
