Patent Application
20180067645
The present invention relates to the field of electronics and, in particular, to applications for electronic products. More specifically, the invention is directed to a system and method for efficient text entry with a touch screen.
Nowadays, portable electronic devices such as smart phones and tablet PCs have been increasingly popular. As basic tools for human-computer interaction, text entry methods have a direct impact on users' experience on the use of such electronic devices.
During use of a text entry application, typing operations are made intensively by a user at an average interval ranging from tens of microseconds to one or two hundred microseconds. Upon the receipt of each input signal, a processor acquires a corresponding character matched with the signal and obtains a result from an action made on the basis of the acquired character, such as a search performed in a dictionary in accordance with an algorithm or a prediction. A variety of entry methods additionally provide responses for improving the user's experience, such as highlighting a pressed key or a swipe path, displaying a character just entered by the user in a particular color in candidate words, or sorting candidate words updated based on the character just entered by the user.
However, reference is now made to
In order to overcome the contradiction between fast response and accurate entry, there have been proposed a number of solutions in the art intended to achieve a balance therebetween. For instance, in one solution, a concept is introduced in which a large dictionary is broken down into a multitude of smaller ones, called cell dictionaries. This solution requires the user to choose in advance dictionaries to be used, and only when the user has chosen a limited number of appropriate cell dictionaries, could response be accelerated without compromising the input accuracy. However, in most cases, since it is impossible for the user to know which dictionaries the words or phrases intended to be entered belong to, the preliminary choice of appropriate dictionaries may fail. On the other hand, choosing all available cell dictionaries is equivalent to choosing the parent dictionary, which could not result in faster response, but rather, would further retard the response due to the overhead on construction of the cell dictionaries from the parent.
Therefore, there is a need for a method and device which can mitigate the contradiction between fast response and accurate entry.
It is an object of the present invention to overcome the above drawbacks of the conventional techniques by presenting a system and method for efficient text entry with a touch screen, which allow the user to enter the text faster with guaranteed high accuracy.
To this end, a system and method for efficient text entry with a touch screen proposed in the present invention are as follows.
According to part of the present invention, the system comprises:
a touch detection module, configured to detect whether presses on the touch screen and lifts therefrom occur;
a thread management module, configured to, in the event that the touch detection module detects a press on the touch screen, initiate an auxiliary thread therefor,
a search module, configured to, in the auxiliary thread, based on an area where the press occurs, perform a search in a dictionary for a character entered by a user and/or candidate words; and
an output module, configured to, output the entered character and/or candidate words as a result of the search performed by the search module in the event that the touch detection module detects a lift from the touch screen, and discard the result of the search performed by the search module if the touch detection module fails to detect the lift.
Further, upon detecting the lift, the touch detection module further determines whether the press and the lift constitute a tap.
According to part of the present invention, the present invention also relates to a method for efficient text entry with a touch screen, characterized essentially in comprising a main-thread process and an auxiliary-thread process.
The main-thread process comprises:
detecting whether a press on the touch screen occurs;
upon detecting the press, initiating at least one auxiliary thread therefor;
detecting whether a lift from the touch screen occurs; and
upon detecting the lift, obtaining an entered character and/or candidate words as a result of a search performed for the press in an auxiliary thread and outputting the result of search; otherwise discarding the result.
The auxiliary-thread process comprises:
performing the search in a dictionary for the entered character and/or candidate words corresponding to an area where the press occurs and delivering the result of the search to the main-thread process upon detecting the lift.
Further, detecting whether a lift from the touch screen occurs further comprises: when a lift is detected, determining whether the press and the lift constitute a tap.
Compared to the prior art, the proposed system and method offer the benefits as follows.
The present invention will be more apparent from the following detailed description of specific embodiments which is to be read in conjunction with the accompanying drawings. Before describing the embodiments in detail, it should be noted that these embodiments are primarily directed to combinations of method steps and device components for inputting text to a terminal device. These device components and method steps are shown at proper positions of the accompanying drawings and indicated at conventional numerals in such a manner that only their details related to understanding the embodiments of the invention are presented in order to avoid obscuring the present disclosure by the details apparent to those of ordinary skill in the art who benefit from the present invention.
As used herein, the terms “left”, “right”, “upper”, “lower”, “front”, “rear”, “first”, “second” and so on are intended only to distinguish one entity or action from another without necessarily requiring or implying that these entities actually have such a relationship or that these actions are actually carried out in such an order. In addition, the terms “comprise”, “include” or any other variation thereof are intended to cover a non-exclusive inclusion. As such, a process, method, article, or apparatus that comprises a list of features is not necessarily limited only to those features but may include other features not expressly listed or inherent to such process, method, article, or apparatus.
Reference is now made to
The touch detection module 1 is configured to detect whether a press on the touch screen or a lift therefrom corresponding to the press occurs, wherein the press and lift constitute a tap gesture. Upon the touch detection module 1 detecting the press, the thread management module 2 will initiate at least one auxiliary thread therefor. The search module 4 then performs a search based on the press, and when the touch detection module 1 detects the lift, a result of the search in the auxiliary thread is output by the output module 3.
In some embodiments, when the touch detection module 1 detects another press before the result of the search for the first press is generated in the auxiliary thread, the thread management module 2 may initiate another auxiliary thread for searching a character corresponding to the second press.
In this way, there may be multiple auxiliary threads in the proposed system in order to improve the entry efficiency with the touch screen. The auxiliary threads are referred to with respect to a main thread that is adapted to allocate events to components. These events may include detecting whether the press on or lift from the touch screen occurs, initiating the at least on auxiliary thread upon detection of the press, and outputting the entered character resulting from the search carried out in the auxiliary thread upon detecting of the lift. The events may also include rendering events including changing the brightness or color of a specific screen area. The auxiliary thread is adapted to, based on the press on the touch screen, perform the search for the entered character corresponding to the press. The result of the search is output by the output module 3.
In a preferred embodiment, referring to
In the conventional techniques, for a character entered by the user, a single thread is utilized to handle the corresponding press, lift and search. That is, in such techniques, only when a press and a lift corresponding thereto are detected, a search will be performed in a single thread for an entered character based on a tap constituted by the press and the lift. However, with the dictionary size increasingly expanding, longer time is required for a search carried out in the dictionary, making the response to each tap slower. In contrast, the thread management module 2 is employed in the present invention which can initiate multiple auxiliary threads each for carrying out a search for an entered character corresponding to a respective press. Additionally, entry methods according to different embodiments employ differently structured dictionaries, in which different numbers of auxiliary threads may be initiated for the same press. For example, in a preferred embodiment, a dictionary is divided into multiple sub-dictionaries such that for a single press of the user, multiple auxiliary threads may be initiated according to the pressed location in order to simultaneously search different sub-dictionaries. This results in a higher search speed and hence better human-computer interaction by allowing the search module 4 to obtain the entered character before the lift occurs.
In another embodiment, the thread management module 2 is further configured to manage multiple auxiliary threads. For example, during simultaneous operation of the multiple auxiliary threads, if the time elapsed since a search is commenced in one of the auxiliary threads exceeds a first threshold, the thread management module 2 may merge this thread with one of the threads initiated subsequently, in order to achieve higher efficiency. In one embodiment, the first threshold may be 100 ms or defined by the user.
In a preferred embodiment, with reference to
After an auxiliary thread is initiated, the search module 4 performs a search for a character entered by the user based on the currently pressed area. Here, it is to be noted that the search module 4 is adapted to search a local dictionary or a dictionary deployed in a cloud server by means of a communication module (not shown). Herein, the term “touch screen” is intended in a broad sense to refer to any touch screen capable of displaying virtual keyboards with various layouts for various languages and contains characters arranged at predetermined locations with coordinates serving as a basis for the search module 4 to perform a search in the dictionary to obtain a corresponding entered character or acquire corresponding candidate words based on a previous user input.
In one embodiment, the output module 3 is configured to output the entered character or candidate words resulting from the search performed by the search module 4 when the touch detection module 1 detects a lift from the touch screen, or discard the entered character or candidate words if no lift is detected by the touch detection module 1. In another embodiment, if the press and lift detected by the touch detection module 1 are not of the same tap gesture or if a time interval between them is longer than a predetermined threshold such as, for example, 100 ms, the touch detection module 1 generates a simulated lift signal and sends it to the output module 3 so that the output module 3 obtains the search result from the search module 4 and outputs it.
Referring to
in step S11, detecting, by the touch detection module 1, whether a press on the touch screen occurs;
in step S12, if a press on the touch screen occurs, initiating an auxiliary thread by the thread management module 2;
in step S13, detecting, by the touch detection module 1, whether a lift from the touch screen occurs, and if so, notifying the auxiliary thread and proceeding to step S14; otherwise, proceeding to step S15;
in step S14, if a lift from the touch screen is detected, outputting, by the output module 3, an entered character and/or candidate words resulting from a search carried out in the auxiliary thread; and
in step S15, if no lift from the touch screen is detected, discarding, by the output module 3, the entered character and/or candidate words resulting from the search carried out in the auxiliary thread.
The auxiliary-thread process includes:
in step S21, searching, by the search module 4, the entered character and/or corresponding candidate words based on a pressed area; and
in step S22, upon detection of a lift, sending the entered character and/or candidate words to the output module 3;
In order to describe the proposed method in further detail, preferred embodiments of the steps involved therein will be presented below. It is to be noted that the preferred embodiments set forth below are not intended to limit the present invention in any sense.
In step S11, the touch detection module 1 detects whether there is a press on the touch screen. The touch screen may be a four-wire resistive screen, an acoustic screen, a five-wire resistive screen, an infrared screen or a capacitive screen equipped in a terminal device which may be a portable device such as a mobile phone, a tablet PC or a mobile PC or a terminal unsuitable to be carried such as a TV set, a desktop PC or a set-top box.
In step S12, when a press is detected on the touch screen, an auxiliary thread is initiated. In one embodiment, step S12 may specifically include: creating a Thread object, wherein Looper is employed for message queue processing and an object with a built-in Runnable interface is used as parameters for the creation of the Thread object; during initiation of the auxiliary thread, a start method for the Thread class is invoked to initiate the thread and a Run( )method for the Runnable is executed to accomplish relevant necessary tasks.
Further, step S12 may also include changing a state of the touched area. With reference to
In another embodiment, with reference to
Additionally, referring to
In some embodiments, auxiliary thread(s) may be ended based on the user's subsequent input(s). For example, based on one or more subsequent inputs of the user, a determination may be made, optionally with the aid of the system correction process, that there is no word corresponding to the inputs. At this point, if the user is keeping typing, the system will terminate the corresponding auxiliary thread(s) while maintaining the remainder. Furthermore, for a set of consecutive taps made by the user, i.e., multiple presses in a series, the system may initiate the same number of auxiliary threads each for one of the presses, in order to expedite the search process and improve the user's experience.
In the aforesaid auxiliary-thread process, when a press by the user is detected, the search module 4 may search the dictionary based on the pressed area in order to obtain the character entered by the user and candidate words corresponding to the entered character. The obtained character and the corresponding candidate words are then sent to the output module 3. Here, the dictionary may include a local dictionary and/or a remote dictionary deployed on a cloud server. In one embodiment, step S21 may include: searching the local dictionary and, if there is no result obtained, then the remote dictionary; or vice versa. In another embodiment, step S21 may include detecting a network state, and if the network state meets a predefined condition, for example, if the network is a WiFi network, searching the remote dictionary. Otherwise, only the local dictionary is searched.
As the search process is generally time-consuming, in some embodiments, step S22 may further include: upon receiving a signal from the main thread that indicates the detection of a lift, waiting for a first time interval and then sending the search result to the output module 3 by the search module 4. In order to avoid making the user feel a delay, the first time interval may be set to a value that is equal to or less than 60 ms.
In some embodiments, step S21 may further include: obtaining the entered character based on the pressed area and obtaining the candidate words by searching the dictionary based on the entered character.
The entered character corresponding to the pressed area may include the character located just in the pressed area or a character determined by the system correction process performed based on the pressed area. In the system correction process, the system may make a reasonable prediction on the currently entered character based on the keyboard layout used as well as on the text that has been entered by the user. For example, when in the Chinese Quanpin mode using the QWERTY keyboard layout as shown in
Each word node may correspond to a word object which, however, has a data structure independent of that of the dictionary. According to some embodiments, a word object may carry the following information: a statistical frequency, related words, context rules, alternative forms and the like of the word. The statistical frequency may result from statistics of commonly entered words or of the user's input preference and may be represented by a numerical value such as a number in the range from 1 to 8, where 8 indicates the highest frequency and 1 denotes the lowest. Statistical frequencies may serve as a critical criterion for prioritization of candidate words. With other criteria being precluded, the more frequently a word is used, the higher a priority it will have.
The related words may include, for example, plural form(s) if the word is a noun, different tense forms if the word is a verb, parts of speech of the word, and so on. For instance, related words of the English word “jump” may include “jumps”, “jumping” and “jumped”. In particular, according to some embodiments, a list of the related words may be accomplished by a pointer. That is, a word object may point to other word objects related thereto. Obtaining related words as a result of the search performed in the dictionary facilitates the user to quickly select a related word for a given word. For example, when “jump” appears as a candidate word, the user may make a predefined motion (e.g., a downward swipe from the location of the word) to cause the system to display all its related words and then select one therefrom. According to some embodiments, obtaining the candidate words based on the entered character may further comprise, based on the candidate words, obtaining words related thereto.
Further, the context rules may include context-related information about the word such as commonly-used phrases containing the word, as well as grammatical rules. For example, context rules for the word “look” may include the commonly-used phrases “look at”, “look forward to”, “look for”, etc., and those for the word “am” may include “I am” and so on. As another example, context rules for the word “of” may include the grammatical rule that the word following it should be a noun or a gerund. With such information, the system may smartly determine priorities for the candidate words based on the context. According to some embodiments, in step S21, obtaining the candidate words based on the entered character may further comprise obtaining the candidate words corresponding to the entered character based on the context.
Further, the context rules may also be applied to the related words. For example, when the context rules contain “look forward to”, even if the user enters “looking”, “look forward to” may be obtained as “looking” is a related word of “look”. The alternative forms are certain related forms of the word. For example, “asap” is an abbreviation of “as soon as possible”. So, if the user enters “asap”, the system can automatically correlate it to “as soon as possible”. That is to say, “as soon as possible” is an alternative form of the word object “asap”. In another example, “don't” may be configured as an alternative form of “dont”, and if the user enters “dont”, it will be automatically corrected to “don't”. In fact, the word object “dont” here functions as an index. If a word has an alternative form, the candidate words module will output the alternative form with higher priority.
Based on the above-defined dictionary structure and the data structure of each word object (i.e., the statistical frequency, related words, context rules, alternative forms, etc.), in step S21, searching the dictionary based on the entered character and thereby obtaining the candidate words may further comprises: during the search for the character entered by the user corresponding to the press by the search module 4, predicting a subsequent press. For example, the search performed for the first press may result in “s”, and the search module 4 may predict the next (second) press that does not occur yet. For example, based on the result of the search performed in the dictionary, the search module 4 may predict “save”, “surprise”, “see” and the like as the words most possibly intended by the user. Based on this prediction, the search module 4 may obtain the predicted characters corresponding to the next pressed area, i.e., “a”, “u”, “e”, etc. As such, after the touch detection module 1 detects the next press, the search module 4 may first search the prediction results for the previous press. This can expedite the search process, shorten the time required for displaying candidate words and enhance the human-computer interaction.
In step S13, the touch detection module 1 detects whether there is a lift from the touch screen. A complete tap gesture consists of a press and a lift. Conventionally, only when a lift is detected, will a search be started for a character or candidate words corresponding to the pressed area. In contrast, according to the present invention, upon detection of a press, an auxiliary thread will be immediately initiated to perform a search, and it is determined in the main thread whether there occurs a lift. If the determination is positive, a result of the search is output. In this way, the user's entry speed is greatly enhanced and better human-computer interaction experience is achieved.
When the lift corresponding to the press is detected, the UI control module 5 changes the state of the area that has been changed due to the detection of the press back to the original, indicting the completion of the tap gesture consisting of the press and lift. For example, the area that has been highlighted is not highlighted any more, or the color of the area that has been change is restored as before the change. In addition, in order to give the user a feeling of smoothness and hence better experience, the output module 3 outputs the result of the search at the time when the highlighted area is not highlighted any more or when the area that have experienced a change in color is restored to the original color. In another embodiment in which a search process is merged with a subsequent search process, a string resulting from the search process is not output upon the cancellation of the highlighting or restoration to the original color for this press but is output together with a string resulting from the subsequent search process upon the cancellation or restoration for the next press. In this case, while there is a delay in the displaying of the entered string, the displaying of the candidate words and hence the user's entry speed are not affected.
In one embodiment, step S13 may further include determining whether the press and the lift constitute a tap. Referring to
in step S131, detecting by the touch detection module 1 whether there is a lift from the touch screen; and
in step S132, if there is a lift from the touch screen, determining by the touch detection module 1 whether the press and the lift constitute a tap.
Step S132 may include detecting whether there is a difference between the locations where the press and lift occur or whether a time interval between the press and lift is longer than a predetermined value. It is possible for a swipe to occur during the press. So, even though the system has detected the press and lift, if the locations where they occur are different, for example, corresponding to different keys, or if the time interval between them is not within a predetermined range that justifies them as constituting a tap, even when they occur at the location of the same key, they are not considered to form a tap.
When the press and lift constitute a tap, step S14 is performed to return an indication of the occurrence of the lift from the touch screen. With reference to
If the press and lift does not constitute a tap, step S15 is performed to return an indication of the absence of the lift from the touch screen. With reference to
If the lift is not detected from the touch screen, step S15 or S35 is performed to return an indication of the absence of the lift.
In some embodiments, although the touch detection module 1 has detected the press, as the touch detection module 1 fails to detect the lift, for example, because the user presses the key such as “w” for a long time, an indication of the absence of the lift is returned. Additionally, with reference to
In general terms, when the time interval between the press and lift is about 100 ms, it is considered that the press and lift constitute a tap gesture. However, it is a matter of course that the time interval may vary with the touch screen used and can be configured by the manufacturer of the terminal device before factory or by the user according to his/her entry preference or typing speed. Nevertheless, the configured time interval must be equal to or greater than a minimum that allows the press and lift to constitute a tap gesture. In addition, in order to distinguish a tap from a long press, the time interval between the press and lift should not be longer than a certain value. Further, the time interval between the press and lift may also be configured according to a sensitivity of the touch screen.
Referring to
In S141, the output module 3 determines whether the search module 4 has obtained the entered character and candidate words from the search performed in the auxiliary thread;
In S142, if the search module 4 has obtained the entered character and candidate words, the output module 3 outputs the entered character and candidate words.
In S143, if the search module 4 does not obtain the entered character and candidate words, the output module 3 determines whether the search performed by the search module 4 has taken a time that is longer than a first threshold.
In S144, if the time taken by the search is longer than the first threshold, the thread management module 2 determines whether there is another ongoing auxiliary thread;
In S145, if there is another ongoing auxiliary thread, the search module 4 merges the auxiliary thread and the other ongoing auxiliary thread into a new search;
In S146, if there is no any other ongoing auxiliary thread, the search module 4 commences a new search;
In S147, if the time taken by the search does not exceed the first threshold, the output module 3 does nothing and the search module 4 continues the search for the entered character and candidate words.
According to the present invention, if the search in the auxiliary thread is not completed after the lift has been detected, the output module 3 will determine whether the search has taken a time exceeding the first threshold, i.e., a maximum search time allowed by the system. If the determination is positive, it is considered that the search is not successful, and if there are other ongoing auxiliary threads, the search in the specific auxiliary thread may be merged with a search in an immediately subsequent one of the other ongoing auxiliary threads so that the search module 4 can perform a new search based on the two characters of the pre-merged searches in order to accelerate the search process. The first threshold may be set to 50-80 ms.
It is to be noted that the systems and methods described herein are applicable to languages other than English as well as to other keyboards. The term “keyboard” is defined in a broad sense to include any input means with defined areas, including, but not limited to, physical, mechanical keyboards, physical, inductive keyboards and virtual keyboards on touchscreen displays. While the embodiments of the invention have been described in the context of virtual keyboards on touchscreen displays, one of ordinary skill in the art will appreciate that the methods and devices mentioned herein may also be used in physical keyboards.
The system and method according to the present invention offer the following benefits over the prior art:
First, with reference to
Second, from the view of point of memory usage, a large number of caches are used in the prior art in order to speed up the response to each tap, but no significant acceleration is observed. With the present invention, the number of required caches and hence the required memory space can be reduced, resulting in enhancements in the overall system operation.
In this specification, the invention has been described with reference to specific embodiments thereof. However, it is apparent that various modifications and changes may be made without departing from the spirit and scope of the invention. Accordingly, the specification and drawings are to be regarded as illustrative rather than restrictive.
| Number | Date | Country | Kind |
|---|---|---|---|
| 201510095074.2 | Mar 2015 | CN | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/CN2016/075184 | 3/1/2016 | WO | 00 |
