User interfaces are provided to enable users to enter text or other content elements into application documents, such as a word processing document. In touch-interface smart phones and other touch interface devices, for example, a traditional “QWERTY” or other keyboard may be displayed via a touch-sensitive display, such as a capacitive display. User touches are processed initially to determine which key was most likely intended. The key to which a touch is mapped may then be displayed, for example in a text entry field in which the user is entering text, a document or other application object, etc.
In addition to mapping touches to keys, sequences of touches must be mapped to words and in some system auto-correction and/or auto-completion suggestions are identified, evaluated, and if appropriate suggested to the user as inline corrections/completions.
Various embodiments of the invention are disclosed in the following detailed description and the accompanying drawings.
The invention can be implemented in numerous ways, including as a process; an apparatus; a system; a composition of matter; a computer program product embodied on a computer readable storage medium; and/or a processor, such as a processor configured to execute instructions stored on and/or provided by a memory coupled to the processor. In this specification, these implementations, or any other form that the invention may take, may be referred to as techniques. In general, the order of the steps of disclosed processes may be altered within the scope of the invention. Unless stated otherwise, a component such as a processor or a memory described as being configured to perform a task may be implemented as a general component that is temporarily configured to perform the task at a given time or a specific component that is manufactured to perform the task. As used herein, the term ‘processor’ refers to one or more devices, circuits, and/or processing cores configured to process data, such as computer program instructions.
A detailed description of one or more embodiments of the invention is provided below along with accompanying figures that illustrate the principles of the invention. The invention is described in connection with such embodiments, but the invention is not limited to any embodiment. The scope of the invention is limited only by the claims and the invention encompasses numerous alternatives, modifications and equivalents. Numerous specific details are set forth in the following description in order to provide a thorough understanding of the invention. These details are provided for the purpose of example and the invention may be practiced according to the claims without some or all of these specific details. For the purpose of clarity, technical material that is known in the technical fields related to the invention has not been described in detail so that the invention is not unnecessarily obscured.
Processing of user input, such as user touches on a keyboard interface displayed in a touch-sensitive display device, is disclosed. As touches to the interface are received, each touch is mapped to one or more candidate keys associated with the touch, for example, one or more keys displayed nearest the touch. Each touch is mapped to a key selected to be displayed in the document or other text entry area being displayed to the user. In addition, sequences of touches are evaluated to determine auto-correction and/or auto-completion suggestions, if any, to be displayed to the user as “inline” corrections, shown for example in a bubble or otherwise adjacent and/or near the sequence currently displayed, i.e., to sequence of keys to which a current sequence of touches have been mapped. In various embodiments, keyboard geometry and/or language models are used to map touches to keys and/or to generate auto-correction and/or auto-completion candidates.
In some embodiments, a set of candidate key sequences are updated as subsequent touches are received. An initial score is assigned to each of a plurality of candidate sequences each of which incorporates a candidate key that has been identified as a candidate with which a touch or other received user input is associated. In some embodiments, the initial score is based at least in part on a keyboard geometry-based value associated with the received user input with respect to the candidate key, for example, an error vector from a touch to the location at which the key was displayed. At least initially, only a subset of candidate sequences so derived are further evaluated to determine for each candidate sequence in the subset a refined score. The initial score of each candidate sequence is configured to comprise an upper bound of a refined score for that sequence, such that if the refined score of a candidate sequence in the subset is higher than the initial score of one or more candidate sequences not in the subset, the candidate sequence with the refined score may be included in a starting set of candidates for a next iteration (for example, processing a subsequent touch) without first determining a refined score for such candidate sequences not in the subset. In this way, further processing to determine refined scores for candidates not in the subset may be delayed and possibly avoided, for example if the starting set is filled with a prescribed number of members before such further processing is performed.
In some embodiments, unigram (e.g., one word at a time) analysis is extended to detect and correct automatically errors by which a user who intended to select a “space” key to insert a space between words enters a touch or other input that instead gets mapped at least initially to another key, such as a key adjacent to the space key as displayed. In various embodiments, a unigram model provides the probability of a given key (character) given a preceding sequence of zero or more characters. In some embodiments, a candidate sequence is evaluated based at least in part on the respective probability of each key in the candidate sequence occurring after the sequence that precedes that key in the sequence. In some embodiments, the probabilities are expressed as values between zero and one and the respective probabilities are multiplied together to determine a probability for the sequence. In various embodiments, the unigram approach is extended to include the possibility that a “space” key, as opposed to a letter, was intended to be entered. In some embodiments, the probability associated with the space key is equal to and/or determined at least in part based on a probability of a key immediately following the space in the candidate sequence occurring at the beginning of a word.
“Lazy” Evaluation of Candidate Key Sequences
In some embodiments, initial scores are scaled to fall within a range from zero to one. The respective initial score for each candidate key is considered to comprise an upper bound score for that key. Refined scores are determined by multiplying the initial score by further probabilities that likewise have been scaled to values between zero and one, such as probabilities determined by considering information other than the touch or other input currently being evaluated, such as preceding touches and/or words entered and/or determined to have been entered by the user, including without limitation language model-derived probabilities as described above. As a result of such an approach, if the refine score of candidate A is greater than the initial, unrefined score of candidate B, it can be concluded that the refined score of candidate A will (or would) be greater than the refined score of candidate B, if it were computed, since the refine score of candidate B would always be equal to or less than the initial score of candidate B. In various embodiments, this observation and approach is used to perform further processing of key candidates and/or associated candidate key sequences only “lazily”, and to avoid performing such further processing with respect to candidates that can with confidence be excluded without performing such further processing to determine for such candidates a refined score.
In the example shown in
In various embodiments, the respective initial scores shown in
In some embodiments, once one or more auto-correction and/or auto-completion candidate words have been determined, further processing is performed to refine auto-correction candidate scores based on contextual information, such as one or more words preceding a current sequence being evaluated. For example, as words are identified as having been entered by the user in some embodiments part-of-speech tagging and/or other language model based processing is performed to generate contextual information that is used to evaluate one or more auto-correction and/or completion candidates associated with a sequence currently being evaluated. For example, if a definite article followed by a noun has been typed and tagged, in some embodiments an auto-correction candidate that is a verb may be considered more likely than a second candidate that is not a verb. The term “n-gram” is used to refer to text processing in which decisions are made based at least in part on a context comprising a set of n words that include and/or otherwise provide context for the text being processed.
Extending Unigram Analysis to Incorporate the “Space” Key
In various embodiments, unigram analysis is extended to incorporate consideration of the “space” bar or other key, for example, in order to detect and suggest auto-correction of errors by which a user who intended to enter a space instead made an input (e.g., soft keyboard touch) that was mapped to a key adjacent to the space key, such as a “v”, “b”, or “n” in a keyboard using the familiar QWERTY layout.
In prior approaches, a unigram model and analysis typically would be used to evaluate a sequence believed to comprise a single word or portion thereof. For example, on detection of the end of a word, for example detecting entry of a space and/or end of sentence punctuation, in prior approaches a unigram model such as the one represented in
Extending a unigram model and analysis to detect errors resulting in a key other than the “space” key being mapped to a touch or other user input when the user in fact intended to type a space is disclosed.
Although the foregoing embodiments have been described in some detail for purposes of clarity of understanding, the invention is not limited to the details provided. There are many alternative ways of implementing the invention. The disclosed embodiments are illustrative and not restrictive.
Number | Name | Date | Kind |
---|---|---|---|
5253325 | Clark | Oct 1993 | A |
5305205 | Weber et al. | Apr 1994 | A |
5581484 | Prince | Dec 1996 | A |
5615378 | Nishino et al. | Mar 1997 | A |
5736974 | Selker | Apr 1998 | A |
5748512 | Vargas | May 1998 | A |
5758314 | McKenna | May 1998 | A |
5765168 | Burrows | Jun 1998 | A |
5774834 | Visser | Jun 1998 | A |
5778405 | Ogawa | Jul 1998 | A |
5797008 | Burrows | Aug 1998 | A |
5818451 | Bertram et al. | Oct 1998 | A |
5896321 | Miller et al. | Apr 1999 | A |
5943443 | Itonori et al. | Aug 1999 | A |
5953541 | King et al. | Sep 1999 | A |
6023536 | Visser | Feb 2000 | A |
6040824 | Maekawa et al. | Mar 2000 | A |
6073036 | Heikkinen et al. | Jun 2000 | A |
6169538 | Nowlan et al. | Jan 2001 | B1 |
6259436 | Moon et al. | Jul 2001 | B1 |
6271835 | Hoeksma | Aug 2001 | B1 |
6298321 | Karlov et al. | Oct 2001 | B1 |
6307548 | Flinchem et al. | Oct 2001 | B1 |
6469722 | Kinoe et al. | Oct 2002 | B1 |
6470347 | Gillam | Oct 2002 | B1 |
6597345 | Hirshberg | Jul 2003 | B2 |
6671856 | Gillam | Dec 2003 | B1 |
6675169 | Bennett et al. | Jan 2004 | B1 |
6714221 | Christie et al. | Mar 2004 | B1 |
6795059 | Endo | Sep 2004 | B2 |
6803905 | Capps et al. | Oct 2004 | B1 |
6804677 | Shadmon et al. | Oct 2004 | B2 |
6857800 | Zhang et al. | Feb 2005 | B2 |
7038659 | Rajkowski | May 2006 | B2 |
7057607 | Mayoraz et al. | Jun 2006 | B2 |
7194699 | Thomson et al. | Mar 2007 | B2 |
7277088 | Robinson et al. | Oct 2007 | B2 |
7283072 | Plachta et al. | Oct 2007 | B1 |
7319957 | Robinson et al. | Jan 2008 | B2 |
7443316 | Lim | Oct 2008 | B2 |
7475063 | Datta et al. | Jan 2009 | B2 |
7477240 | Yanagisawa | Jan 2009 | B2 |
7490034 | Finnigan et al. | Feb 2009 | B2 |
7508324 | Suraqui | Mar 2009 | B2 |
7526738 | Ording et al. | Apr 2009 | B2 |
7565380 | Venkatachary | Jul 2009 | B1 |
7584093 | Potter et al. | Sep 2009 | B2 |
7609179 | Diaz-Gutierrez et al. | Oct 2009 | B2 |
7679534 | Kay et al. | Mar 2010 | B2 |
7683886 | Willey | Mar 2010 | B2 |
7707026 | Liu | Apr 2010 | B2 |
7712053 | Bradford et al. | May 2010 | B2 |
7725838 | Williams | May 2010 | B2 |
7797269 | Rieman et al. | Sep 2010 | B2 |
7809744 | Nevidomski et al. | Oct 2010 | B2 |
7880730 | Robinson et al. | Feb 2011 | B2 |
8037034 | Plachta et al. | Oct 2011 | B2 |
8041557 | Liu | Oct 2011 | B2 |
8090571 | Elshishiny et al. | Jan 2012 | B2 |
20020010726 | Rogson | Jan 2002 | A1 |
20020015064 | Robotham et al. | Feb 2002 | A1 |
20020085037 | Leavitt et al. | Jul 2002 | A1 |
20020126097 | Savolainen | Sep 2002 | A1 |
20020140679 | Wen | Oct 2002 | A1 |
20020191029 | Gillespie et al. | Dec 2002 | A1 |
20030090467 | Hohl et al. | May 2003 | A1 |
20030149978 | Plotnick | Aug 2003 | A1 |
20030193481 | Sokolsky | Oct 2003 | A1 |
20030197736 | Murphy | Oct 2003 | A1 |
20030204392 | Finnigan et al. | Oct 2003 | A1 |
20040070567 | Longe et al. | Apr 2004 | A1 |
20040135774 | La Monica | Jul 2004 | A1 |
20040160419 | Padgitt | Aug 2004 | A1 |
20040178994 | Kairls, Jr. | Sep 2004 | A1 |
20040183833 | Chua | Sep 2004 | A1 |
20040196256 | Wobbrock et al. | Oct 2004 | A1 |
20050024341 | Gillespie et al. | Feb 2005 | A1 |
20050099398 | Garside et al. | May 2005 | A1 |
20050162395 | Unruh | Jul 2005 | A1 |
20050190970 | Griffin | Sep 2005 | A1 |
20050216331 | Ahrens et al. | Sep 2005 | A1 |
20050246365 | Lowles et al. | Nov 2005 | A1 |
20050278647 | Leavitt et al. | Dec 2005 | A1 |
20050283726 | Lunati | Dec 2005 | A1 |
20060004744 | Nevidomski et al. | Jan 2006 | A1 |
20060152496 | Knaven | Jul 2006 | A1 |
20060206454 | Forstall et al. | Sep 2006 | A1 |
20060246955 | Nirhamo et al. | Nov 2006 | A1 |
20060265208 | Assadollahi | Nov 2006 | A1 |
20060265648 | Rainisto et al. | Nov 2006 | A1 |
20060274051 | Longe et al. | Dec 2006 | A1 |
20060293880 | Elshishiny et al. | Dec 2006 | A1 |
20070040813 | Kushler et al. | Feb 2007 | A1 |
20070046641 | Lim | Mar 2007 | A1 |
20070061754 | Ardhanari et al. | Mar 2007 | A1 |
20070067272 | Flynt et al. | Mar 2007 | A1 |
20070130128 | Garg et al. | Jun 2007 | A1 |
20070152978 | Kocienda et al. | Jul 2007 | A1 |
20070198566 | Sustik | Aug 2007 | A1 |
20070229323 | Plachta et al. | Oct 2007 | A1 |
20070260595 | Beatty et al. | Nov 2007 | A1 |
20070285958 | Platchta et al. | Dec 2007 | A1 |
20070288449 | Datta et al. | Dec 2007 | A1 |
20080059876 | Hantler et al. | Mar 2008 | A1 |
20080167858 | Christie et al. | Jul 2008 | A1 |
20080168366 | Kocienda et al. | Jul 2008 | A1 |
20080259022 | Mansfield et al. | Oct 2008 | A1 |
20090174667 | Kocienda et al. | Jul 2009 | A1 |
20090249198 | Davis et al. | Oct 2009 | A1 |
20090327977 | Bachfischer et al. | Dec 2009 | A1 |
20100023318 | Lemoine | Jan 2010 | A1 |
20100036655 | Cecil et al. | Feb 2010 | A1 |
20100325588 | Reddy et al. | Dec 2010 | A1 |
Number | Date | Country |
---|---|---|
WO 9833111 | Jul 1998 | WO |
WO 0038041 | Jun 2000 | WO |
WO 2005008899 | Jan 2005 | WO |
Entry |
---|
CALL Centre 1999, “Word Prediction,” Copyright (c) The CALL Centre & Scottish Executive Education Dept., pp. 63-73. |
Dyslexic.com, “AiphaSmart 3000 with CoWriter SmartApplet: Don Johnston Special Needs,” http://www.dyslexic.com/procuts.php?catid=2&pid=465&PHPSESSID=2511b800000f7da . . . , downloaded Dec. 6, 2005, 13 pages. |
Glossary of Adaptive Technologies: Word Prediction, http://www/utoronto.ca/atrc/reference/techwordpred.html, printed Dec. 6, 2005, 5 pages. |
MacTech, “Keystrokes 3.5 for Mac OS X Boosts Word Prediction,” http://www.mactech.com/news/?p=1007129, printed Jan. 7, 2008, 3 pages. |
Masui, T., “PO Box: An Efficient Text Input Method for Handheld and Ubiquitous Computers,” Proceedings of the International Symposium on Handheld and Ubiquitous Computers (HUC'99), Sep. 1999, 12 pages. |
Mobile Tech News, “T9 Text Input Software Updated,” htpp://www.mobiletechnews.com/info/2004/11/23/122155.html, Nov. 23, 2004, 4 pages. |
NCIP: Word Prediction Collection, “What is Word Prediction?” http;//www2.edc.org/NCIP/library/wp/What—is.htm, printed Jan. 7, 2008, 2 pages. |
NCIP: Word Prediction Collection, NCIP Library: Word Prediction Collection, http://www2.edc.org/NCIP/library/wp/toc.htm, printed Jan. 7, 2008, 4 pages. |
International Search Report and Written Opinion dated May 8, 2008, received in International Application PCT/US2007/088872, which corresponds to U.S. Appl. No. 11/620,641, 12 pages (Christie). |
International Search Report and Written Opinion dated May 8, 2008, received in International Application PCT/US2007/088873, which corresponds to U.S. Appl. No. 11/620,642, 11 pages (Kocienda). |
Notice of Allowance dated Mar. 18, 2011, received in U.S. Appl. No. 11/640,641, 14 pages (Christie). |
Office Action dated Mar. 30, 2010, received in U.S. Appl. No. 11/620,642, 13 pages (Kocienda). |
Final Office Action dated Nov. 29, 2010, received in U S. Appl. No. 11/620,642, 16 pages (Kocienda). |
Office Action dated Feb. 18. 2011, received in U.S. Appl. No. 11/620,642, 16 pages (Kocienda). |
Notice of Allowance dated Oct. 24, 2011, received in U.S. Appl. No. 11/620,642, 9 pages (Kocienda). |
Office Action dated Apr. 17, 2012, received in ROC (Taiwan) Patent Application No. 097100079, which corresponds to U.S. Appl. No. 11/620,642, 12 pages (Kocienda). |
Office Action dated Nov. 21, 2011, received in U.S. Appl. No. 12/165,554, 18 pages (Kocienda). |
Notice of Allowance dated Apr. 2, 2012, received in U.S. Appl. No. 12/165,554, 9 pages (Kocienda). |
Office Action dated Jan. 5, 2012, received in U.S. Appl. No. 12/505,382, 36 pages (Westerman). |
Final Office Action dated May 3, 2012, received in U.S. Appl. No. 12/505,382, 27 pages (Westerman). |
Final Office Action dated Jul. 9, 2012, received in U.S. Appl. No. 12/505,382, 35 pages (Westerman). |
Office Action dated Feb. 17, 2012, received in U.S. Appl. No. 12/727,219, 16 pages (Kocienda). |
Office Action dated Dec. 7, 2012, received in U.S. Appl. No. 13/559,495, 10 pages (Kocienda). |
Notice of Allowance dated Jun. 25, 2013, received in U.S. Appl. No. 13/559,495, 6 pages (Kocienda). |
Notice of Allowance dated August 15, 2013, received in U.S. Appl. No. 13/559,495, 10 pages (Kocienda). |
Office Action dated Nov. 20, 2009, received in U.S. Appl. No. 11/620,641, 22 pages (Christie). |
Final Office Action dated Jun. 25, 2010, received in U.S. Appl. No. 11/620,641, 32 pages (Christie). |
Office Action dated Apr. 1, 2010, received in Australian Patent Application No. 2007342164, which corresponds to U.S. Appl. No. 11/620,641. |
Office Action dated Nov. 25, 2010, received in Chinese Patent Appilcation No. 200780052020.1, which corresponds to U.S. Appl. No. 11/620,641, 14 pages (Christie). |
Office Action dated May 26, 2010, received in European Application No. 07 869 922.0, which corresponds to U.S. Appl. No. 11/620,641, 5 pages (Christie). |
Office Action dated Dec. 7, 2010, received in European Patent Application No. 07 869 922.0, which corresponds to U.S. Appl. No. 11/620,641, 5 pages (Christie). |
Office Action dated May 26, 2010, received in European Application No. 07 869 923.8, which corresponds to U.S. Appl. No. 11/620,642 4 pages (Kocienda). |
Office Action dated Apr. 10, 2009, received in U.S. Appl. No. 11/549,624, 7 pages (Mansfield). |
Office Action dated Jul. 22, 2009, received in U.S. Appl. No. 11/549,624, 9 pages (Mansfield). |
Final Office Action dated Feb. 1, 2010, received in U.S. Appl. No. 11/549,624, 12 pages (Mansfield). |
Notice of Allowance dated Jun. 3, 2010, received in U.S. Appl. No. 11/549,624, 6 pages (Mansfield). |
Number | Date | Country | |
---|---|---|---|
20130050089 A1 | Feb 2013 | US |