Patent Grant
9921665
A conventional input method editor (IME) is a computer application that assists a user to input text at a computing device. An IME may provide several input candidates based on a received input from the user. The input and the provided text candidates may be the same language or different languages.
For example, the user may input one or more initial English characters of a word or phrase and an IME, based on the initial characters, provides one or more complete words or phrases for the user to select a proper one. For another example, an IME may also assist the user to input non-Latin characters such as Chinese. The user may input Latin characters through a keyboard. The Latin characters may be a spelling of the Chinese characters. The IME returns one or more Chinese characters based on the spelling to the user to select a proper one. As the current keyboard usually only supports inputting Latin characters, the IME is useful for the user to input non-Latin characters.
The input candidate selected by the user can be inserted into various other computer applications, such as a chatting application, a document editing application, a gaming application, etc.
Typically, the conventional IME only provides input candidates in the form of text, and a single source is typically used to identify the input candidates, regardless of the type of application with which the IME is being used.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. The term “techniques,” for instance, may refer to device(s), system(s), method(s) and/or computer-readable instructions as permitted by the context above and throughout the present disclosure.
An input method editor (IME) as described herein provides candidates from which a user may select one or more to input into a host application at least partly based on one or more parameters. The one or more parameters, for example, may include a query input by the user into the IME, a scenario of the host application, a context of the user input, and/or a language mode. The provided candidates may include not only text candidates but also rich candidates and informative candidates. The user may select one or more of the text candidates and rich candidates as the user input. The one or more informative candidates are available to the user to illustrate or provide information associated with another candidate.
In addition, one or more applications may be associated with the IME to provide extended functionality to the IME, such as providing additional candidates. The one or more applications may be regarded as extensions of the IME. Some applications may have components residing at a local device and a remote server and are capable to call components at the server or other web services to provide the candidates. The user may select the one or more applications from an application store. Alternatively, one or more applications may be pre-installed or embedded within the IME. The IME may select one or more applications from the associated multiple applications to provide candidates at least partly based on the one or more parameters. In addition, the user may directly select and launch one or more applications from the IME to provide candidates.
For example, the IME may calculate a score or ranking of the applications based on the parameters and select one or more applications with rankings or scores higher than a threshold. Machine learning techniques may be used to adjust some attributes including scores or rankings for a particular application. The user may also modify the attributes of the particular application based on his/her preference.
Multiple applications may run concurrently and independently to provide the candidates. The applications' user interfaces and functionalities are decoupled. For example, the applications' appearance styles, such as skins of the user interfaces, may be different from each other and/or different from that of the IME.
The detailed description is described with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the drawings to reference like features and components.
The disclosed techniques describe an input method editor (IME) that provides candidates to a user for input into another computer application, i.e., a host application. The IME provides candidates at least partly based on one or more parameters. The IME may collect the one or more parameters that may include, for example, a query input by the user into the IME, a scenario of the host application (e.g., chatting or emailing), a context of the user input (e.g., a prior use history of the computing application such as chatting history or contents displayed at the user interface of the computer application such as the user's prior inputs), and/or a language mode (e.g., English or Chinese for the desired candidates). Candidates and their rankings for display positions among multiple candidates may also be generated or identified based on the combination of parameters for a current user input. For example, a same query input by the user into the IME at different scenarios may have different candidates and/or candidate rankings.
The IME may have or call one or more engines to search candidates based on the one or more parameters. The engines search a local database and/or a network to provide candidates based on different settings and configurations. In addition, one or more applications may be associated with the IME to provide extended functionalities to the IME. Each application may have or call its one or more engines to provide additional candidates. A graphical user interface of an application store may be available to the user to select and associate the applications in which the user is interested with the IME. For example, the selected application may be installed with the IME. Alternatively, one or more applications may be pre-installed or embedded into the IME. There are various installation configurations of the applications at the IME. In one example, some applications are installed at the IME in an executable file format. In another example, a file in a form of declarative markup language may be used to describe one or more characteristics of the one or more applications associated with the IME and to enable behaviors of the associated applications. The file, which may be called a manifest file herein, is human-readable and editable. Characteristics of the one or more applications associated with the IME and represented in the manifest file may include, for example, a condition when a particular application is qualified to run or be selected by the IME and a preset score or ranking of the particular application under a particular combination of one or more parameters relating to the user input.
Based on the collected one or more parameters, the IME may calculate a score and/or a ranking of some or all associated applications and/or the engines of the IME. The IME may select its own engine or one or more applications whose scores or rankings are higher than a preset threshold to provide the candidates. An application configuration interface may be available to the user to manage the associated applications including editing their characteristics described in the manifest file.
In addition, the user may directly select and launch one or more applications from the IME to request such applications to provide additional candidate based on the one or more parameters. Such launched applications may have their own stand-alone user interfaces, different from the interface of the IME, such that the user can input queries and receive candidates from the launched applications.
Multiple selected and/or launched applications may run concurrently and independently to provide the candidates. In one example, the functionality to provide candidates is decoupled from the functionality of appearance styles of the user interfaces such as skins of the user interfaces. For example, the skins of the user interface may define the appearance, such as color and size, of a display window of the user interface. In other words, the appearance styles of the multiple applications may be different from each other and/or different from that of the IME.
The IME and/or the selected application may provide candidates based on the one or more parameters. The candidates may include one or more text candidates, rich candidates, and/or informative candidates. The text candidates are expressions represented by or associated with the query in the form of text and may include non-Lain characters such as Chinese and Latin characters such as English. The rich candidates are the expressions represented by or associated with the query in forms other than text and may include images or multimedia to provide supplemental information to a user to enhance the user experience. The user may select one or more text candidates and rich candidates as input into the user interface of the host application. The informative candidate provides information, such as background or detailed illustration, of another candidate to the user. For example, the user may intend to learn information of a specific text or rich candidate. After the user indicates his interest, a respective informative candidate is provided to the user for reference.
After the IME receives the generated or identified candidates from its own engine and/or the selected applications, the IME may display them through the user interface of the IME. The IME may have one or more display windows to display the candidates. For example, the IME may have a text candidate window to display the text candidates, a rich candidate window to display the rich candidates, and an informative candidate window to display the informative candidates. Unless contradicted with the descriptions herein, details of text candidates and rich candidates are described in U.S. patent application Ser. No. 13/331,023, Scott, et al. “Scenario-adaptive input method editor,” which is incorporated herein.
The one or more display windows may present concurrently with the IME or be available when the IME determines that there are corresponding types of candidates to be displayed. In some examples, the informative candidate window may appear and display the informative candidates only when the user indicates he/she is interested in learning more information associated with one particular candidate and the IME determines that there exists such an informative candidate.
After the IME receives a selection from the user for one or more displayed candidates such as text or rich candidates, the IME inserts the selected candidate into the user interface of the host application as the user input. The IME may insert the selected candidate into the host application in an optimal format supported by the host application.
There may be various configurations for the deployment of the IME, the application store, and/or the applications. In one embodiment, all components of the IME, the applications, and/or the application store are located at a client device. In another embodiment, each of the IME, the applications, and/or the application store may be implemented in a client-server architecture in which some components are located at the client device and some components are located at the remote server relative to the client device.
Example Scenario for User Input by Using IME
In the example of
The input window 116 includes one or more input objects 118 that are to be input into the chatting window 114 as the objects 108. In the example of
The user 102 may use the IME 106 to provide the input objects 118. For example, the user 102 inputs a query 122 into the user interface 104 of the IME 106. The IME 106 presents one or more candidates at the user interface 104 to be selected by the user 102 as the input objects 118 into the input window 116 of the host application 110. In addition, one or more applications 124 may be associated with the IME 106 to provide extended functionality such as additional candidates to the IME 106. The one or more associated applications 124 may be selected by the user 102 from an application store (not shown in
In the example of
The user 102 inputs the query 122 into the user interface 104 of the IME 106 by using any inputting technique. The query 122 may be any kind of inputs, such as text, multimedia, or audio input by the user 102. For example, the query 122 may represent a series of initial characters, an abbreviation, a spelling, and/or a translation of the text candidates 136. In the example of ”
The IME 106 collects one or more parameters relating to the input from the user 102 and selects at least one of the associated applications 124 to provide one or more candidates to the user 102 based on the one or more parameters.
For example, the one or more parameters may include the query 122 and a scenario of the user input. The scenario of the user inputs may include a scenario of the host application 110, i.e. chatting in
The one or more parameters may also include a language mode, e.g., English or Chinese, to indicate the language of the user input. For example, if the language mode is simplified Chinese, the selection and ranking priority has a preference to simplified Chinese characters, a preference of expression style under simplified Chinese, or a preference to a data source from a webpage having simplified Chinese characters. In contrast, if the language mode is traditional Chinese, the selection and ranking priority has a preference to traditional Chinese characters, a preference of expression style under traditional Chinese, or a preference to a data source from webpages having traditional Chinese characters.
The one or more parameters may further include a previous use history of the host application, such as a chatting history when the host application 110 is a chatting application.
The IME 106 may calculate scores for some or all of the applications 124 and rank them based on at least one of the one or more parameters. The IME 106 may select one or more applications from the applications 124 that have scores higher than a score threshold and/or rankings higher than a ranking threshold to provide one or more candidates to the user 102.
The selected application may further identify or generate the one or more candidates to the user 102. In other words, the candidates may be provided by one or more engines called by the IME 106, one or more selected applications from the applications 124, or both. The selection and ranking of the candidates by the IME 106 and/or the selected application may also be based on the relevancies of the candidates to the one or more collected parameters. For example, a text candidate or rich candidate with higher relativity with the scenario of the host application 110 may be ranked with a higher priority than other candidates in the text candidates 136 and/or the rich candidates 138 to be presented to the user 102
The candidates presented at the user interface 104 of the IME 106 may include one or more text candidates 136, one or more rich candidates 138, and/or one or more informative candidates 140. The user 102 may select one or more candidates from the text candidates 136 and/or the rich candidates 138 as the input objects 118 to be input into the input window 116 of the host application 110.
The text candidates 136 may be any kind of text such as Latin characters (e.g. English), non-Latin characters (e.g. Chinese), symbols, numbers, or a mix of different kinds of texts. The query 122 and the text candidates 136 may be the same language or different languages.
The rich candidates 138 may include pictures, maps, videos, and/or other forms of representations in addition to text that provide supplemental information to the user 102 in addition to the text candidates 136.
The informative candidates 140 are used to illustrate or provide information of the query 122, a text candidate 136, or a rich candidate 138. For example, the user 102 may use the input technique such as the mouse to point to a respective text candidate 136 or rich candidate 138 to show that the user 102 wants to know more information associated with such candidate. Then the IME 106 and/or a selected application may present the informative candidate 140 to the user. In some examples, if the user 102 is interested in learning information associated with some contents such as words or multimedia within the informative candidate 140, the user 102 may continue to point to such contents to obtain another informative candidate (not shown in
As the informative candidates 140 are for illustration purpose, they usually cannot be directly selected by the user 102 as the input objects 118. However, in some examples, the user 102 may still select and insert them as the input objects. For example, the user 102 may copy contents of the informative candidates 140 and paste them into the user interface of the host application 110 as the input objects 118.
In the example of ” as the query or even one or more rich candidates 138 (not shown) as the search term.
In the example of ,” a second text candidate “
village,” and a third text candidate “
,” which means “sanlitun bar” in English. The text candidates 136, for example, may be provided by the engine of the IME 106 based on the query 122 and the language mode of the IME for the query 122 and/or some other parameters. Alternatively, the text candidates 136 may be provided by a text candidate application (not shown in
In the example of ” The map application 128, based on the query 122 and/or other parameters, generates or identifies a map of “
.” Based on some other parameters such as the context of the user input, the map application 128 may further generate another map showing a route from a current location of the user 102 and/or “Jack,” the other person who is communicating with the user 102 through the host application 110, to “
” (not shown in
After the candidates are generated, identified, and ranked by the IME 106 and/or the selected applications, the IME 106 displays them through the user interface 104 to the user 102 in a format suitable to the type of the candidate.
In the example of
The user interface 104 of the IME 106 may further include an informative candidate window 148 to display the informative candidate 140. In the example of ,” the IME 106 may display the informative candidate 140 associated with the point of interest 150 such as an address and operation hours of the restaurant, i.e., “
. . .
: 10:00 am-10:00 pm” which means “No. 1, Gongti Road . . . Business hours: 10:00 am-10:00 pm” in English in the informative candidate window 148.
In addition to the approach that the IME 106 automatically selects one or more applications from the applications 124, the user 102 may directly select and launch one or more applications from the applications 124. For example, the IME 106 may provide an interface (not shown in
In the example of ,” represents the Chinese character of “sanlitun.” The display window 154 displays one or more candidates 158 provided by the weather application 130 based on the query 156 and/or some other collected parameters, such as the scenario of the host application 110, the context of the user input, etc.
In the example of ,” e.g. “
23 C . . .
,” which means “current temperature 23 Celsius . . . tonight cloudy” in English. The weather application 130 takes into account the parameter of a language mode, for example, simplified Chinese, and presents the candidates 158 not only in simplified Chinese characters but also in expression styles preferred in simplified Chinese. For instance, Celsius instead of Fahrenheit is the preferred expression for temperature under simplified Chinese. The weather application 130 also takes into account the parameter of context of the user input, e.g., based on the input objects “meet me at,” the weather application 130 predicts that the user 102 would like to know the weather forecast and provides “tonight cloudy” information in the candidates 158. The user 102 may view the candidates 158 as an additional information source or may select one or more candidates 158 as the input objects 118 to be input into the user interface of the host application 110.
When the user 102 points to the query 152 or the candidates 158, another informative candidate (not shown in
Example methods and computing systems for performing techniques described herein are discussed in detail below. The IME 106 in
The example methods are sometimes illustrated as a collection of blocks in a logical flow graph representing a sequence of operations that can be implemented in hardware, software, firmware, or a combination thereof. The order in which the methods are described is not intended to be construed as a limitation, and any number of the described method blocks can be combined in any order to implement the methods, or alternate methods. Additionally, individual operations may be omitted from the methods without departing from the spirit and scope of the subject matter described herein. In the context of software, the blocks represent computer executable instructions that, when executed by one or more processors, perform the recited operations.
Example Techniques for Providing Candidates
In another example, in the query state, the IME collects the multiple parameters without pre-selecting an application and chooses the one or more applications from a plurality of applications in the service state.
In the example of
The IME 106 may also capture current contents displayed at the user interface of the host application 110, such as the objects 108 in the chatting window 114 and the input objects 118 at the input window 116. For example, the IME 106 may use screen pixel information based on visual understanding techniques such as OCR to identify the displayed contents. In the example of
The IME 106 may also collect a previous user history of using the host application 110. Such user history may be stored locally or remotely, and may be helpful to analyze the context of the recent inputs. For example, when the host application 110 is Microsoft MSN®, the user history may be a chatting history of the user.
The contents displayed at the user interface of the host application 110 and the previous user history may also be used to identify the scenario of the host application 110. For example, based on the context of the contents displayed at the user interface of the host application 110, the IME may determine that it is a scenario of chatting based on a pattern of chatting identified from the objects 108 in the chat window 114.
At 204, the IME selects one or more applications from a plurality of applications to provide candidates based on the collected parameters. Operations at 204 may correspond to a service state of the IME during which the computing system of the IME receives the enhanced query including multiple parameters relating to the user input, processes the enhanced query, selects one or more engines from the applications and/or the engines of the IME to provide candidates, processes candidate responses from the selected one or more engines, packages the candidate responses from the selected engines, and returns the responses into a proper form to be processed at a display state which is described in detail below.
In the example of
Some of the selected applications may be already associated with the IME, such as the associated applications 124 in
At 206, the IME displays the one or more candidates using the user interface of the IME. Operations at 206 correspond to a display state of the IME during which the computing system of the IME receives the resulting response from the service state, preprocesses, displays the resulting response, and listens and responds to a user interaction at a user interface of the IME. There are various techniques to display the candidates.
For example, the IME may have a text candidate window to display text candidates, a rich candidate window to display rich candidates, and an informative candidate window to display informative candidates. The IME may present the text candidate window and/or the rich candidate window when there are corresponding text candidates and/or rich candidates. In another example, the IME may present the text candidate window and the rich candidate window even when there are no corresponding text candidates and/or rich candidates.
The IME may present the informative candidate window to display the informative candidate associated with a respective candidate when receiving an indication that the user is interested in learning more about the respective candidate. Alternatively, such informative candidate window may automatically present with the respective candidate when there is a corresponding informative candidate associated with the respective candidate, regardless of whether the user points to the respective candidate. For example, the informative candidate window and/or the respective candidate may be translucent and the informative candidate window may be displayed in close proximity to the respective candidate.
In one example, the user interface of the IME provides one text candidate window, one rich candidate window, and/or one informative candidate window. The IME receives the text candidates from different selected applications and/or its own engine, ranks them based on the one or more collected parameters, and displays them in one text candidate window. Similarly, the IME receives the rich candidates from different selected applications and/or its own engine, ranks them based on the one or more collected parameters, and displays them in one rich candidate window. In another example, each of the applications may have its own text candidate window, rich candidate window, and/or informative candidate window. The candidates provided by a particular application are also ranked by the particular application and displayed in the corresponding candidate window.
There are various presentation styles, such as skins, applicable to the display of candidates described herein. In one example, the IME may provide a uniform presentation style to the applications. In another example, each of the selected applications and/or the IME may have separate and independent presentation styles, such as skins, to display their generated or identified candidates in the display windows respectively. For example, the IME and the applications may have separate display windows such as text candidate windows, rich candidate windows, or informative candidate windows to display their corresponding candidates. For instance, the rich candidate window 146 displays the rich candidates 138 provided by the map application 128. The presentation style, such as a skin, of the rich candidate window 146 may be different from the text candidate window 144 that displays the text candidates 136 provided by another application or the IME 106. For another instance, the presentation style of the candidate window 154 of the weather application 130 launched by the user 102 in a stand-alone window may be different from that of the text candidate window 144 and the rich candidate window 146 at the user interface 104 of the IME 106. The IME may further provide an interface to the user to select different presentation styles such as different skins for different applications. The user may choose different available skins for the IME and the applications respectively.
At 208, the IME receives an indication that the user has selected one or more candidates from the displayed candidates, and inserts the selected candidates into the user interface of the host application. Operations at 208 correspond to an insertion state of the IME during which the computing system of the IME makes an insertion request into the host application after receiving a user interaction at the display state. For example, the IME may check the forms or formats supported by the host application, and format the selected candidates in an optimal format supported by the host application.
Example Application Store
In the example of
The GUI of the application store 302 includes one or more icons and each of the icons represents a respective application that is available to the user for selection. Some of the applications may be already available at the client device 304. For example, one or more applications are already pre-installed at the IME 306 and do not require a user selection. For another example, some applications may reside at a remote server. The user selects an icon representing his/her desired application to download the application or its client component from a server. In the example of
In the example of
After the user selects his/her desired application icons, such as the map application icon 312, the weather application icon 314, and the calculator application icon 316 in
In the example of
In one example, as shown in
Some of the applications may be domestic or online service, such as Bing® map, that do not require a registration of the user to use. Some of the applications may be domestic or online services that require the registrations of the user to use, or that require the registration of the user to provide more functionality based on prior saved user preferences, historic data, etc. The IME 306 or the application may pre-store the authentication credentials of the user to log onto and use such services.
Example Manifest File Describing Application Characteristics
Various application configurations may be installed at the IME. In one example, some applications are installed at the IME in an executable file format. In another example, the IME may use a file in a form of declarative markup language, such as the Extensible Markup Language (XML), to describe one or more characteristics of the one or more applications associated with the IME. The markup language file is human-readable and editable, and is called a manifest or manifest file herein. The manifest file is a self-describing document. It is the starting point when loading a respective application, and is interpreted by the computing system of the IME at the client device and/or the server in a declarative way to enable application behaviors. It is also used to automatically expose configuration information about the respective application. The manifest file may reside at both the client and the server components of the computing system of the IME.
The manifest file may include a manifest schema and a manifest. The manifest schema, which is in the form of declarative markup language, describes the manifest features supported by the computing system of the IME. The manifest describes a particular application. The features in the manifest for the particular application shall comply with the specification in the manifest schema. The manifest for the particular application may be an independent file from the manifest schema. Alternatively, the manifests for different applications and the manifest schema may be combined in one manifest file. The manifest schema may expose enumerations for supported scenarios. An application developer of the applications may refer to the manifest schema as a baseline reference to the current capability of the computing system of the IME.
In the example of
The version 402 describes a current version of the manifest schema 400. For example, the manifest schema 400 may reside at a server of the computing system of the IME. A client device of the computing system of the IME may have a local manifest schema previously downloaded from the server.
As the manifest schema defines the specifications of the applications supported by the computing system of the IME, the computing system of the IME checks whether the version of the local manifest matches the version of the manifest schema 400 stored at the server. If the two versions of the manifest schema at the client device and server do not match, the computing system of the IME updates the local manifest schema to the latest version and/or checks whether the two versions are compatible. If the versions of the manifest schema at the client device and the server do not match and are incompatible, the computing system of the IME may disable the IME or request the user to install the IME with the latest version of manifest schema.
For example, if the versions of the manifest schema at the client device and the server do not match, the computing system of the IME checks whether the client component of the IME has been updated but the local manifest schema is not updated. If it is determined that the client component of the IME has been updated but the local manifest schema is not updated, the computing system of the IME updates the local manifest schema to the latest version of the manifest schema 400. For example, the client component of the IME checks an update element in the local manifest schema. If the update exists, the IME may prompt the user to download the latest version of the manifest schema 400 or the manifest schema 400 may be automatically downloaded from the server to the client depending on attributes in the update element. Optionally, the manifest schema 400 may be downloaded or installed in the background.
If the latest version of the manifest schema supported by the client component of the IME is downloaded and installed and it still does not match the server version of the manifest schema 400, it is possible that an updated version of the client component of the IME exists and the computing system of the IME will check for the latest version of the client component of the IME, and if such version exists, it will prompt the user to update the client component of the IME to support the manifest schema 400.
The various elements 404 as shown in
The selection probability element 414 is a required element that provides a probability under what combination of parameters such as input and context conditions should the respective application be considered a contender for servicing. Particularly, it may declare or calculate “scores” for different types of input and context. In short, it has a language for leveraging semantic labeling and also regular expressions.
The engine element 416 is a required element that declares whether the engine of the application is located at the client device, the server, or both. If the engine is at the local device, the engine element 416 may further declare the “hooks” needed and capabilities required, for example, for storage and network access. The engine of a respective application provides the candidates based on the collected one or more parameters by the IME and/or the respective application.
The display element 418 is a required element that declares how the computing system of the IME will display this particular application. It takes into consideration scenarios of different host applications and describes how to transform the presentation according to the scenarios. In this way, the host application may have influence on how candidates are displayed.
The insertion element 420 is a required element that declares how to transform the data from the chosen candidate into a form the host application can accept and declares whether the candidate can be transformed “optimally” into the host application according to the scenario. For example, an email candidate might be transformed into a special email link in an email program, or plain text in a simple note taking program.
The advanced query element 422 is an optional element that declares certain kinds of additions needed to enhance the query that would be formed if the particular application is a contender for servicing. For example, the local text candidates generated by the IME could be sent for servicing, as well as some of the context, e.g. the left n-grams or the previously written M characters of text in the queries of the IME or the host application. The n-gram is a contiguous sequence of n items from a given sequence of text, where n and M can be any integer and be stipulated in the advanced query element 422.
The informative candidate element 424 is an optional display element related to the informative candidate which is also referred to as tooltip herein. It declares how to render a special tooltip for the candidates generated by the particular application. For example, it provides a way for the user to obtain information about a candidate on mouse hover or keyboard shortcut indicated by the user.
The IME access control element 426 is an optional element that declares whether the application requires lower level scripting capability. For example, the IME access control element 426 may point to a JScript file within an application package and a callback function so that it can be activated. JScript is Microsoft®'s implementation of the script language that is used in Microsoft®'s Internet Explorer®. JScrpt is just an example here. Any other script language file such as JavaScript ActionScript may also be used.
Because the IME access control element 426 may allow modification of any part of the IME, only one application with enabled characteristics under the IME access control element 426 may be loaded at any time, and the IME, based on the IME access control element 426. will alert the user that the particular application requires special permission. The usefulness of this element is typically seen in enterprises which need additional augmentation of the IME for IT needs, such as universal deployment, overwriting, or prohibiting an individual configuration of the particular application or the IME for security reasons.
An example manifest schema is shown below in Table 1 Example Manifest Schema Implementation. In Table 1, the identity element 410 is named “Identity,” the running qualification element 412 is named “When Run,” the selection probability element 414 is named “LikelihoodQuickCheck,” the engine element 416 is named “Engine,” the display element 418 is named “Display,” the insertion element 420 is named “Insertion,” the advanced query element 422 is named “QueryAugmentation,” the informative candidate element 424 is named “Tooltip,” and the IME access control element 426 is named “CustomScript.” Table 1 shows that the example manifest schema provides the templates for different elements and their corresponding sub-elements and attributes that the respective application shall have or support.
Based on the manifest schema, a manifest for an example map application is shown below in Table 2. The example map application is capable to provide and present a map as a rich candidate. Table 2 shows that the manifest follows the template and specification in the manifest schema and provides detailed attribute data of different elements for the particular map application.
The IME may also provide a mechanism through which the associated applications may fully leverage functionality provided by the IME and may even modify the IME. The IME access control element 426, as shown in
The CustomScript element in Table 1 and Table 2 is a specific instance of the IME access control element 426. For example, the CustomScript element in the manifest declares lower level scripting capability for a respective application. For example, because the CustomScript element can be called to modify any part of the IME, only one application with enabled CustomScript functionalities may be loaded at any time, as two applications both with enabled CustomScript functions running may conflict with each other and compete for resources.
The CustomScript element provides a powerful technique for leveraging the functionalities of the IME but also raises a security concern. Thus, when the respective application is installed, the computing system may also warn and alert the user that the application requires special permission. In addition, the user may have to be identified as having privilege to install the application whose manifest has the CustomScript element. For example, the user may be prompted to agree to the installation in an administrator mode instead of as a guest under the Microsoft Windows® operating system.
To define a CustomScript, the elements required are Identity and CustomScript in the manifest. For example, the CustomScript may point to a Jscript file in the CustomScript folder of the application package and a callback string to point to the main function of the script. Table 3 illustrates an example manifest for IME access control including the CustomScript element.
In an example implementation, the script-enabled IME based on the IME access control element 426, may be built on Object Linking and Embedding (OLE) Automation (“Automation”) technology when used in the Microsoft Windows® operating system. Automation is based on Component Object Model (“COM”) and is an inter-process communication technology that enables one application to gain access to the exposed functionality of another. In the scenario of the example implementation, one application could have a CustomScript and the other application is the IME with core components, which the script needs access to. To enable the communication between IME components that, for example, may be written in C++ and JScript language, the IME components may implement an IDispatch interface. Considering the interoperation between C++ and JScript language involves argument checking, error handling etc., the computing system may implement a dependent class to act as a proxy between the IME core component and the script language. An example flow is as follows: (1) an end-user starts the IME; (2) the IME initializes the script host; (3) the script host loads and parses the Jscript file; and (4) the programming logic of the Jscript file does not directly access the IME core components, but rather a proxy layer. A user uses IME and the IME core initializes the script host, which loads and parses the JS file, and the JS file manipulates the IME core through a script host by ImeRoot, which acts as the proxy. Through this proxy, major functions of the IME are exposed. The script is an interpreted language so “script host” is needed to parse and run the script and the script operates the IME through the script host.
The application package format 500 may be based on open standards and can therefore be examined on any operating system and without special interpretation by any particular product. The application package format 500 may use zip packaging (zlib) and XML formats. The additional parts 506 map semantically to the runtime states described above, and may also contain open format files, such as HTML, CSS, Jscript, Png, XSLT, etc.
For example, the sub-folders of the parts 506 may be optional as they are only required if they are referenced in the manifest 504. Binary files (excluding .png, .gif, .jpg) such as .dll and lexicon binaries are typically only allowed in the engine sub-folder 514. In one example, only files may be added to the application package 502, except for the parts 506. The intention of restricting what can be added is to make the package format 500 more efficient, reliable, and secure. The package format 500 is efficient as the format is as flat as possible and so sub-paths do not need to be enumerated or searched. The format is reliable as the format strives to avoid as much as possible unnecessary complexity which often carries bugs. The format is secure as potentially malicious or otherwise hidden payload cannot be buried within a folder hierarchy.
Both the client component and the server component of a respective application may have the same or similar packaging formats. For example, an example naming convention is [publisher].[appname].v[version].app, such as Contoso.maps.v1.0.0.6.client.app that is used for the name of the client component of the respective application and Contoso.maps.v1.0.0.6.server.app. Contoso is the name of a publisher. Maps is the name of the application. 1.0.0.6 is the version number.
As shown above, the manifest file in the form of declarative markup language to describe the application behavior improves efficiency, maintainability, reliability, extensibility, and security. For efficiency purposes, the manifest file is loaded into memory by de-serializing into an efficient data structure using native code objects. Alternatively, a script using scripting language may be used to describe the behaviors of the application. As the script needs to run with high frequency, potentially on every keystroke, and there could be N scripts if there are N applications installed, where N may be any integer. In short, the manifest file enables scaling of multiple applications.
Maintainability is achieved as the declarative language, such as XML, is human readable and the behavior is implied by the self-description. Reliability is achieved by leveraging schema enforcement of format. Extensibility may be achieved by leveraging the built-in versioning functionality via the XML namespace, for example. Security is in part achieved because markup is itself not executed, it is in clear text and easily scannable. Conversely, a binary or scripting solution would be executing code itself. Additionally, since the manifest file is a declaration, the user can see what system capability or unique behavior is “requested” by the application. For example, an application that requires local storage capability would be clearly presentable to the user at install time when the user selects to install the application, thereby giving the user the opportunity to reject it if the user feels the application may cause security concerns.
Each application managing interface, such as the map application interface 610, includes one or more basic function buttons to help the user configure the functionalities of the particular application. In the example of
The application configuration interface 600 may also provide more freedom to the user to adjust one or more characters of the particular application. In one example, some characteristics or attributes may be set unchangeable by the computing system or the user. In another example, all characteristics or attributes may be open to edit through machine learning or by the user. In the example of
In the example of
The user may edit these properties and values, such as adding or deleting one or more scenarios in which the map application 604 is qualified to run. For example, the user may add another scenario such as “document” in the values of the scenario type property. Based on this change, when the host application is a document application, such as Microsoft Word®, the map application 604 is also qualified to run.
The score window 620 lists different scores of the map application 604 for different parameter combinations. In the example of
The display window 622 may list various properties and values of the map application 604 at the display state, such as the type of candidate the map application 604 supports and whether to present the candidate at an interface separate from that of the IME.
The insertion window 624 may list various properties and values of the map application 604 at the insertion state, such as a format of the candidate to be inserted into the host application. The application configuration interface 600 may be part of the IME and can be launched from an IME bar window 626 of the IME. The IME bar window 626 may include a plurality of buttons for different functions of the IME such as a language mode button 628, a manage application button 630, a launch application button 632, and a help button 634. The user can select the language mode button 628 to set the language mode, e.g., English or simplified Chinese, for the candidates. The user can select the manage application button 630 to launch the application configuration interface 600. If the user selects the launch application button 632, a list of associated applications will be presented to the user to select to launch, which is described in detail below. The help button 634, when selected by the user, provides some helpful information such as questions and answers to the user. The IME bar window 626 in
Example Method for Selection of Application
At 702, the IME screens multiple applications associated with the IME to pre-select one or more applications that are qualified to run based on one or more collected parameters. For example, the IME collects the query input by the user, the scenario of the host application, such as the application name, and the context, such as previously written characters retained by the host application in memory which may be cleared frequently.
For example, the IME may load manifests of all associated applications into the memory of the computing system of the client device and check their running qualification elements in their manifests. If the scenario of the host application is listed as a value in the qualification element of the manifest of a particular application, such particular application is qualified to run and is pre-selected.
At 704, the IME calculates a score for each pre-selected application based on the one or more collected parameters. The one or more parameters, for example, may include the query input by the user, the scenario of the user input such as the scenario of the host application, a context of the user input, and/or a language mode. In one example, the IME 106 may calculate the scores in real-time based on a plurality of preset rules. In another example as shown in
At 706, the IME 106 ranks the pre-selected applications based on their scores. At 708, the IME 106 chooses one or more pre-selected applications from the pre-selected applications based on their rankings and/or scores. For example, the IME 106 may choose the one or more pre-selected applications whose scores are higher than a preset score threshold or whose rankings are higher than a preset ranking threshold. Alternatively, the IME 106 may take into account both the score and the ranking. For example, the IME may not choose a pre-selected application if its score is lower than the preset score threshold even if its ranking is higher than a preset ranking threshold.
At 710, the IME adjusts scores or rankings of one or more pre-selected applications based on a selection of a candidate or an application launched by the user. For example, after the IME chooses one or more applications to provide candidates to the user, the user may select a candidate generated by a particular application. The user may also directly launch an application as shown in
The IME may use machine-learning techniques to automatically implement the adjusting. The IME will use the adjusted values and properties to update the manifests of the corresponding applications and the user may view such changes or further edit the changes through the application configuration interface as shown in
In the example of
Example Informative Candidate Scenario
In the example of
The input window 808 displays one or more input objects 810 that are to be input into the chatting window 804 as the objects 806. The input objects 810 include text object “theoretical.” An input indication 812, represented by “|,” represents an indication of a place to insert the next input object 810.
User interface 814 of an IME enables a user to select and input one or more candidates into the input window 808 of the host application 802. In the example of
If the user wants to learn more information of the text candidate 820 “physics,” the user may use any input techniques such as a mouse to hover over the text candidate 820 or press a key at a keyboard to activate the functionality of the IME to provide informative candidates. The IME, after receiving the indication from the user, checks whether there is a corresponding informative candidate 824 of the text candidate “physics” and displays the informative candidate 824, if any, at the informative candidate window 826. In the example of ,
,” which is a simplified Chinese definition of physics.
In the example of
In one example, the application that provides the respective candidate, i.e. the text candidates 820 in which the user shows interest, will have a priority in ranking to be selected as the application to provide the informative candidate. Thus the informative candidate is highly relevant to the respective candidate. In the example of
The IME may also set a default application to provide informative candidates under different combinations of parameters. For example, when the text candidate selected by the user is a terminology, a default application such as a dictionary application or an encyclopedia application may be used to provide the informative candidates. Similarly, when the text candidate selected by the user is a stock name, a stock application may be used to provide stock information such as stock charts of the stock.
Example Stand-Alone Application
At 902, one or more applications are associated with an IME. At 904, the IME provides an interface for a user to directly select one or more of the associated applications. In one example, the interface may be the composition window 142 as shown in
The user may launch as many applications as he/she likes. In one example, the user interfaces may be decoupled from functionalities of the IME. The representation styles of the user interfaces, such as skins, of the launched applications may be different from each other and may be different from that of the IME. In another example, the skin of the IME may control the user interface of the launched applications such as via global Cascading Style Sheets (CSS) rules. It is appreciated that the developer of the application does not need to worry about creating the interface of the independently launched application. Rather, the application interface can be generated automatically without requiring the application developer to write user interface source code. For example, the stand-alone interface of the application may be generated based on the manifest and/or the CSS rules. For another example, the stand-alone interface of the application may be selected by the user from a plurality of available interfaces such as skins presented to the user.
At 906, the user selects one or more applications from the user interface by using any inputting techniques, and the IME receives an indication of such selection. Based on the selection of the respective application, the IME will bypass calculation of the scores and rankings of the associated applications as shown in
At 908, the IME launches the selected one or more applications. In one example, the applications may be running in the background without presenting their standalone interfaces at the client device. The candidates provided by the applications are presented to the user through the user interface of the IME. In another example, the launched application may have its independent standalone interface independent and separate from that of the IME such as its own composition window to receive a query. For instance, a homepage or user interface of the application is rendered at the user interface of the client device. The display element in the manifest of the selected application, such as a sub-element for the stand-alone user interface attribute, will be called. If none is provided, a default stand-alone user interface or homepage will be displayed.
At 910, the IME collects multiple parameters relating to the user input. The one or more parameters include the query input by the user, the scenario of the user input such as the scenario of the host application, a context of the user input, and/or a language mode.
At 912, the IME requests the selected one or more applications to provide one or more candidates to the user at least partly based on one or more of the collected parameters. In an example implementation, a generated or identified rich candidate will load through an embedded web browser with a text field automatically placed above it. A selected candidate may be copied to a clipboard of the computing system and then pasted at the user interface of the host application as the user input.
Example Insertion of Selected Candidate
After the candidates provided by the IME and/or the selected or launched applications are displayed at the user interface of the IME or the independent user interfaces of the applications, the user may select one or more of the candidates to insert into the user interface of the host application. The IME or the selected application may, based on the formats supported by the host application, form the selected candidates in an optimal format supported by the host application and insert such optimized selected candidate into the host application.
An Exemplary Computing System for IME
There may be various configurations of the IME.
Memory 1304 may include volatile memory, non-volatile memory, removable memory, non-removable memory, and/or a combination of any of the foregoing. Generally, memory 1304 contains computer executable instructions that are accessible and executable by the one or more processors 1302. The memory 1304 is an example of computer-readable media. Computer-readable media includes at least two types of computer-readable media, namely computer storage media and communications media.
Computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, phase change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by a computing device.
In contrast, communication media may embody computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transmission mechanism. As defined herein, computer storage media does not include communication media.
Any number of program modules, applications, or components can be stored in the memory 1304, including by way of example, an operating system, one or more computer applications such as the IME 1306, a chatting application, a document-editing application, a gaming application, an email application, other program modules, program data, and computer executable instructions or components. For brevity, the applications other than the IME application 1306 are not shown in
The IME 1306 may include three tiers of components, i.e. an interface component 1308, a client logic component 1310, and a service component 1312. The interface component 1308 includes one or more user interfaces to interact with the user. The client logic component 1310 manages the user interfaces of the interface component 1308 and communicates with the service component 1312. For example, the client logic component 1310 collects one or more parameters relating to the user input, sends such parameters to the service component 1312 for further processing, receives returned candidates from the selected engines, and presents the candidates at the user interfaces of the interface component 1308. The service component 1312 selects one or more engines from the IME and/or multiple applications to provide candidates based on the collected parameters, and sends the generated or identified candidates to the client logic component 1310 that manages to display them at one or more user interfaces of the interface component 1308. The division of functionalities between the three components, particularly between the client logic component 1310 and the service component 1312 may be different in different computing systems.
The interface component 1308 may reside on a client device while the client logic component 1310 and/or the service component 1312 may reside at either the client device or one or more servers. In other words, the computing system 1300 may or may not be fully located at the client device. Alternatively, the computing system 1300 may use a client-server structure or a distributed system where more than one server may be utilized to implement the functionalities. The IME 1306 may be used in an environment or in a configuration of universal or specialized computer systems. Examples include a personal computer, a server computer, a handheld device, a portable device, a tablet device, a multi-processor system, a microprocessor-based system, a set-up box, a programmable customer electronic device, a network PC, and/or a distributed computing environment including any system or device above. In a distributed computing environment, a task is executed by remote processing devices which are connected through a communication network. In the distributed computing environment, the modules may be located in storage media (which include data storage devices) of local and remote computers.
The interface component 1308 may include a composition window 1314 that receives a query input by the user, a text candidate window 1316 that displays text candidates, a rich candidate window 1318 that displays rich candidates, and an informative candidate window 1320 that displays informative candidates. The interface component 1308 may also include a stand-alone application interface 1322 that is a user interface of an application directly selected and launched by the user. The interface component 1308 may also include an application configuration interface 1324 that enables the user to manage one or more characteristics of associated applications. The application configuration interface 600 in
The client logic component 1310 may include an application loader 1326, an application pool manager 1328, a candidate manager 1330, an insertion manager 1332, and an informative candidate manager 1334. The application loader 1326 installs application packages corresponding to associated applications and generates the application configuration interface 1324 based on manifests of the associated applications in the application packages. The application pool manager 1328 receives parameters from the interface component 1308 and characteristics of the applications from the application loader 1326 to provide an enhanced query to be further processed by the service component 1312. For example, the application pool manager 1328 receives the query input from the composition window 1314 and other parameters such as the scenarios and contexts from the interface component 1308. The application manger 1328 may also receive the identity of the particular application and the query input from the stand-alone application interface 1322. The characteristics from the application loader 1326, for example, may be in a de-serialized XML manifest format.
The candidate manager 1330 receives candidates provided from an input engine selector 1336 at the service component 1312 and manages the display of the provided candidates. For example, the candidate manager 1330 displays text candidates at the text candidate window 1316 and rich candidates at the rich candidate window 1318. The candidate manager 1330 may also receive an indication that the user is interested in learning information associated with a particular candidate at the rich candidate window 1318 or the text candidate window 1316 and passes such indication to the informative candidate manager 1334 to request a corresponding informative candidate if any. When the candidate manager 1330 receive a user's selection of a particular candidate from the rich candidate window 1318 or the text candidate window 1316, the candidate manager 1330 may send the selected candidate to the insertion manager 1332. For example, the selected candidate may be processed to be in the form of XML. The insertion manager 1332 inserts the selected candidate into the host application in a format, such as an optimal format, supported by the host application.
The informative candidate manager 1334 passes the indication that the user is interested in learning information associated with the particular candidate to an informative candidate engine selector 1338, receives one or more informative candidates corresponding to the particular candidate from the informative candidate engine selector 1338, and sends the informative candidates to the informative candidate window 1320 for display.
The service component 1312 may include the input engine selector 1336, the informative candidate engine selector 1338, and an engine pool 1340. The input engine selector 1336 receives the enhanced query from the application pool manager 1328 and selects one or more engines of the IME and the applications from the engine pool 1340. The input engine selector 1336 receives candidates provided by the selected applications from the engine pool 1340, and returns the provided candidates to the application pool manager 1328. The informative candidate engine selector 1338 may select the same engine that provides the particular candidate in which the user indicates interest and/or other engines from the engine pool 1340 to provide the informative candidates. The engine pool 1340 includes one or more engines of the IME and the applications that can be used to provide candidates. The one or more engines in the engine pool 1340 may be located at the client device, the server, or both.
For example, the common language of data between the interface component 1308, the client logic component 1310, and the service component 1312 may be XML, which represents the applications via manifests, the responses of the engines, and the candidate's representations in the display. The candidates may be transformed into either HTML or text format for display at the interface component 1308.
In an example implementation, data flow between the interface component 1308, the client logic component 1310, and the service component 1312 is described below in view of the example running states that include a query state, a service state, a display state, and an insertion state. It is appreciated that the data flow may be different in different examples.
In the query state, a user inputs a query at the composition window 1314 or the stand-alone application interface 1322. The following information is transmitted to the application pool manager 1328 in a scoring mechanism to pre-select qualifying applications for servicing:
The output is an ordered list of applications, e.g., the “Contenders Vector,” to be further selected by the input engine selector 1336 and served by the engine pool 1340. The applications in the pool are loaded into memory as objects from their original XML manifest representation for performance reasons so that they can be queried in real-time. For example, the manifest of the application, instead of all components of the application, needs to be loaded. By reference to
At the service state, the client logic component 1310, for example at a client device, identifies the engine pool 1340 and asynchronously sends the query, which may be dependent on the “engine” element in the manifest. The service component 1312, for example at a server, decides which engine to use. In other words, the client logic component 1310 submits a request and the service component 1312 decides which engine to use, if any. At the service component 1312, there is an interface, which is called IAppEngine herein, which each of the engines implements. Table 4 lists example functions of the IAppEngine interface.
If there are engines located at the local client device, such engines may also have a main.js file that has the same functions as above.
The enhanced query including multiple parameters to the engine pool 1340 may be as follows:
APPID represents an ID of a particular application. VER represents a version of the application. The CLIENT_SCORE represents a score of a particular application calculated by the client logic component 1310 of the IME 1306 from the query state as described above.
The input engine selector 1336 selects potential cloud engines at the server from the engine pool 1340. For example, the input engine selector 1336 considers engines that match the APPID values from the contender's vector, and pings each engine with the VER information (e.g., the IAppEngine interface exposes the SupportsClient function). For each engine E that is supported, the input engine selector 1336 adds it to ENGINE_CANDIDATES. The ENGINE_CANDIDATES represents the one or more engine candidates to be considered by the input engine selector 1336 to provide candidates.
For each engine in the ENGINE_CANDIDATES, the input engine selector 1336 calculates the E.SERVER_SCORE. The E.SERVER_SCORE represents a score of an engine candidate calculated by the server component 1312 of the IME 1306.
The input engine selector 1336 calls a selection function to decide which engine to call. The selection function interface may be as follows:
FeatureN(E), where N can be any integer, indicates that there exists a number (N) of feature points, or signals, that can be used in the ranking of engines. In this example formula, each of the feature functions, such as Feature1(E), Feature2(E), FeatureN(E), returns a score between 0 and 1. Likewise, the NormalProbability functions normalize the probabilities provided by the server and client scores to between 0 and 1 so that a consistent calculation can be made.
The input engine selector 1336 queries the engines and returns XML responses as follows:
With regard to the XML response, an example engine query function (implemented in IAppEngine) returns a response that includes the ID of the engine, for example <Response ID=APPID> . . . </Response>. The candidates that are returned by this XML fit capabilities of the scenario of the host application. For example, in MESSENGER_CHATBOX, a chat window of the user interface of the MSN® Messenger application, a movie image payload may be Animated Gif, while in another chat application with less capabilities, the image may be a JPG file. The IME 1306 may have pre-packaged XML for at least some scenarios. When a request comes in, the insertion manager 1332 may use Extensible Stylesheet Language Transformations (XSLT) to convert the candidates to the needs of a particular scenario.
With regard to engine quality and relevance, machine-learning techniques may be applied. For example, on another process, a diagnostics monitor may watch the performance of the engines and if a particular engine is running slow, such engine may be penalized in terms of ranking (added as an additional feature point), or may be disabled. Additionally or alternatively, global learning from multiple users may be used. For example, the input engine selector 1336 may consider the overall popularity (or accuracy) of engines among different users and use the overall popularity as another feature in the ranking.
If one or more engines are located at the client device, the client component of the IME will direct the query to the local engines of the engine pool 1340. The architecture/flow is the same as when implemented using a remote server. The local engines may be implemented in Jscript scripting language. In addition, for security reason, the local engines may not be configured to use ActiveX. Rather, certain Object Models may be exposed to use. Example object model functions are shown in Table 5.
At the display state, upon receiving an XML response from the service (either local or remote), the candidate manager 1330 processes the response and transforms it as necessary depending on the target of the selected applications and whether the candidates are text candidates to be displayed at the text candidate window 1316 or rich candidates to be displayed at the rich candidate window 1318. The transformation happens via XSLT, and the precise XSLT is determined based on scenario and targets provided in the XML manifest.
For example, the rich candidates may be displayed in an embedded web browser. There is a special object model to communicate with the IME in order to eventually insert the rich candidate into the host application. An example function is SendInput which takes an XML string. This string will be interpreted at the insertion state. Table 6 lists example functions of CanddidateOM.JS
At the insertion state, the insertion manager 1332 will receive an XML response from the candidate manager 1330 described in the display state. The insertion manager 1332 is responsible for transforming this XML into the optimal format for the application host. It works by looking up the appropriate XSLT to use according to the current scenario, in the manifest. Then it transforms the candidate and ultimately inserts the candidate into the host application. There can be multiple implementation methods for insertion. For example, the insertion manager 1332 may leverage the clipboard as a communication channel between applications. The insertion manger 1332 first temporarily copies whatever is existing in the clipboard to a temporary location, then copies the HTML for rich candidates or text for text candidates into the clipboard, sends a paste command to paste the candidate at the host application, and then restores the data at the temporary location to the clipboard. Other implementations, for example, may leverage COM automation to get a hold of the object model of the host application.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the claims.
This application is a continuation of PCT International Application No. PCT/CN2012/077436, filed Jun. 25, 2012, which is incorporated herein by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4559604 | Ichikawa et al. | Dec 1985 | A |
| 5303150 | Kameda | Apr 1994 | A |
| 5386556 | Hedin et al. | Jan 1995 | A |
| 5485372 | Golding et al. | Jan 1996 | A |
| 5655129 | Ito | Aug 1997 | A |
| 5734749 | Yamada et al. | Mar 1998 | A |
| 5778361 | Nanjo et al. | Jul 1998 | A |
| 5787422 | Tukey et al. | Jul 1998 | A |
| 5796866 | Sakurai et al. | Aug 1998 | A |
| 5832478 | George | Nov 1998 | A |
| 5873107 | Borovoy et al. | Feb 1999 | A |
| 5896321 | Miller et al. | Apr 1999 | A |
| 5926652 | Reznak | Jul 1999 | A |
| 5963671 | Comerford et al. | Oct 1999 | A |
| 5987415 | Breese et al. | Nov 1999 | A |
| 5995928 | Nguyen et al. | Nov 1999 | A |
| 6014638 | Burge et al. | Jan 2000 | A |
| 6076056 | Huang et al. | Jun 2000 | A |
| 6085160 | D'hoore et al. | Jul 2000 | A |
| 6092044 | Baker et al. | Jul 2000 | A |
| 6141655 | Johnson et al. | Oct 2000 | A |
| 6236964 | Tamura et al. | May 2001 | B1 |
| 6247043 | Bates et al. | Jun 2001 | B1 |
| 6339776 | Dayani-Fard et al. | Jan 2002 | B2 |
| 6363342 | Shaw et al. | Mar 2002 | B2 |
| 6377965 | Hachamovitch et al. | Apr 2002 | B1 |
| 6408266 | Oon | Jun 2002 | B1 |
| 6424358 | DiDomizio et al. | Jul 2002 | B1 |
| 6460015 | Hetherington et al. | Oct 2002 | B1 |
| 6490563 | Hon et al. | Dec 2002 | B2 |
| 6519599 | Chickering et al. | Feb 2003 | B1 |
| 6556990 | Lane | Apr 2003 | B1 |
| 6564213 | Ortega et al. | May 2003 | B1 |
| 6573844 | Venolia et al. | Jun 2003 | B1 |
| 6654733 | Goodman et al. | Nov 2003 | B1 |
| 6658404 | Cecchini | Dec 2003 | B1 |
| 6687734 | Sellink et al. | Feb 2004 | B1 |
| 6731307 | Strubbe et al. | May 2004 | B1 |
| 6732074 | Kuroda | May 2004 | B1 |
| 6785677 | Fritchman | Aug 2004 | B1 |
| 6801190 | Robinson et al. | Oct 2004 | B1 |
| 6801661 | Sotak et al. | Oct 2004 | B1 |
| 6801893 | Backfried et al. | Oct 2004 | B1 |
| 6934767 | Jellinek | Aug 2005 | B1 |
| 6941267 | Matsumoto | Sep 2005 | B2 |
| 6963841 | Handal et al. | Nov 2005 | B2 |
| 7069254 | Foulger et al. | Jun 2006 | B2 |
| 7089266 | Stolte et al. | Aug 2006 | B2 |
| 7089504 | Froloff | Aug 2006 | B1 |
| 7099876 | Hetherington et al. | Aug 2006 | B1 |
| 7107204 | Liu et al. | Sep 2006 | B1 |
| 7133876 | Roussopoulos et al. | Nov 2006 | B2 |
| 7165032 | Bellegarda | Jan 2007 | B2 |
| 7171353 | Trower, II et al. | Jan 2007 | B2 |
| 7181450 | Malloy et al. | Feb 2007 | B2 |
| 7194538 | Rabe et al. | Mar 2007 | B1 |
| 7224346 | Sheng | May 2007 | B2 |
| 7225200 | Chickering et al. | May 2007 | B2 |
| 7236923 | Gupta | Jun 2007 | B1 |
| 7277029 | Thiesson et al. | Oct 2007 | B2 |
| 7308439 | Baird et al. | Dec 2007 | B2 |
| 7349981 | Guerrero | Mar 2008 | B2 |
| 7353247 | Hough et al. | Apr 2008 | B2 |
| 7360151 | Froloff | Apr 2008 | B1 |
| 7370275 | Haluptzok | May 2008 | B2 |
| 7383299 | Hailpern et al. | Jun 2008 | B1 |
| 7389223 | Atkin et al. | Jun 2008 | B2 |
| 7395203 | Wu et al. | Jul 2008 | B2 |
| 7447627 | Jessee et al. | Nov 2008 | B2 |
| 7451152 | Kraft et al. | Nov 2008 | B2 |
| 7490033 | Chen et al. | Feb 2009 | B2 |
| 7505954 | Heidloff et al. | Mar 2009 | B2 |
| 7505985 | Kilroy | Mar 2009 | B2 |
| 7512904 | Matthews et al. | Mar 2009 | B2 |
| 7539656 | Fratkina et al. | May 2009 | B2 |
| 7555713 | Yang | Jun 2009 | B2 |
| 7562082 | Zhou | Jul 2009 | B2 |
| 7565157 | Ortega et al. | Jul 2009 | B1 |
| 7599915 | Hill et al. | Oct 2009 | B2 |
| 7617205 | Bailey et al. | Nov 2009 | B2 |
| 7676517 | Hurst-Hiller et al. | Mar 2010 | B2 |
| 7689412 | Wu et al. | Mar 2010 | B2 |
| 7725318 | Gavalda et al. | May 2010 | B2 |
| 7728735 | Aaron et al. | Jun 2010 | B2 |
| 7752034 | Brockett et al. | Jul 2010 | B2 |
| 7769804 | Church et al. | Aug 2010 | B2 |
| 7844599 | Kasperski et al. | Nov 2010 | B2 |
| 7881934 | Endo et al. | Feb 2011 | B2 |
| 7917355 | Wu et al. | Mar 2011 | B2 |
| 7917488 | Niu et al. | Mar 2011 | B2 |
| 7930676 | Thomas | Apr 2011 | B1 |
| 7953730 | Bleckner et al. | May 2011 | B1 |
| 7957955 | Christie et al. | Jun 2011 | B2 |
| 7957969 | Alewine et al. | Jun 2011 | B2 |
| 7983910 | Subramanian et al. | Jul 2011 | B2 |
| 8161073 | Connor | Apr 2012 | B2 |
| 8230336 | Morrill | Jul 2012 | B2 |
| 8285745 | Li et al. | Oct 2012 | B2 |
| 8498864 | Liang et al. | Jul 2013 | B1 |
| 8539359 | Rapaport et al. | Sep 2013 | B2 |
| 8564684 | Bai | Oct 2013 | B2 |
| 8597031 | Cohen et al. | Dec 2013 | B2 |
| 8762356 | Kogan | Jun 2014 | B1 |
| 8996356 | Yang et al. | Mar 2015 | B1 |
| 9489458 | Haugen | Nov 2016 | B1 |
| 20010048753 | Lee et al. | Dec 2001 | A1 |
| 20020005784 | Balkin et al. | Jan 2002 | A1 |
| 20020045463 | Chen et al. | Apr 2002 | A1 |
| 20020099536 | Bordner et al. | Jul 2002 | A1 |
| 20020099581 | Chu et al. | Jul 2002 | A1 |
| 20020103809 | Starzl et al. | Aug 2002 | A1 |
| 20020116528 | Vale | Aug 2002 | A1 |
| 20020123994 | Schabes et al. | Sep 2002 | A1 |
| 20020138479 | Bates et al. | Sep 2002 | A1 |
| 20020156779 | Elliott et al. | Oct 2002 | A1 |
| 20020188603 | Baird et al. | Dec 2002 | A1 |
| 20030041147 | van den Oord et al. | Feb 2003 | A1 |
| 20030061027 | Weise et al. | Mar 2003 | A1 |
| 20030160830 | DeGross | Aug 2003 | A1 |
| 20030179229 | Van Erlach et al. | Sep 2003 | A1 |
| 20030212702 | Campos et al. | Nov 2003 | A1 |
| 20030220917 | Copperman et al. | Nov 2003 | A1 |
| 20040128122 | Privault et al. | Jul 2004 | A1 |
| 20040201607 | Mulvey et al. | Oct 2004 | A1 |
| 20040210844 | Pettinati | Oct 2004 | A1 |
| 20040220925 | Liu et al. | Nov 2004 | A1 |
| 20040243415 | Commarford et al. | Dec 2004 | A1 |
| 20040260534 | Pak et al. | Dec 2004 | A1 |
| 20040267737 | Takazawa et al. | Dec 2004 | A1 |
| 20050050095 | Hurtis et al. | Mar 2005 | A1 |
| 20050102131 | Trower et al. | May 2005 | A1 |
| 20050144064 | Calabria et al. | Jun 2005 | A1 |
| 20050144162 | Liang | Jun 2005 | A1 |
| 20050203738 | Hwang | Sep 2005 | A1 |
| 20050216253 | Brockett | Sep 2005 | A1 |
| 20050283468 | Kamvar et al. | Dec 2005 | A1 |
| 20060026147 | Cone et al. | Feb 2006 | A1 |
| 20060047701 | Maybury et al. | Mar 2006 | A1 |
| 20060085392 | Wang et al. | Apr 2006 | A1 |
| 20060123338 | McCaffrey et al. | Jun 2006 | A1 |
| 20060146028 | Chang | Jul 2006 | A1 |
| 20060163337 | Unruh | Jul 2006 | A1 |
| 20060167857 | Kraft et al. | Jul 2006 | A1 |
| 20060190822 | Basson et al. | Aug 2006 | A1 |
| 20060204142 | West et al. | Sep 2006 | A1 |
| 20060206324 | Skilling et al. | Sep 2006 | A1 |
| 20060206454 | Forstall et al. | Sep 2006 | A1 |
| 20060242608 | Garside | Oct 2006 | A1 |
| 20060248074 | Carmel et al. | Nov 2006 | A1 |
| 20060290535 | Thiesson et al. | Dec 2006 | A1 |
| 20070011154 | Musgrove et al. | Jan 2007 | A1 |
| 20070016422 | Mori et al. | Jan 2007 | A1 |
| 20070033269 | Atkinson et al. | Feb 2007 | A1 |
| 20070050339 | Kasperski et al. | Mar 2007 | A1 |
| 20070052868 | Chou et al. | Mar 2007 | A1 |
| 20070088686 | Hurst-Hiller et al. | Apr 2007 | A1 |
| 20070089125 | Claassen | Apr 2007 | A1 |
| 20070124132 | Takeuchi | May 2007 | A1 |
| 20070136256 | Kapur et al. | Jun 2007 | A1 |
| 20070143262 | Kasperski | Jun 2007 | A1 |
| 20070147269 | Ettle et al. | Jun 2007 | A1 |
| 20070150279 | Gandhi et al. | Jun 2007 | A1 |
| 20070162281 | Saitoh et al. | Jul 2007 | A1 |
| 20070164782 | Church et al. | Jul 2007 | A1 |
| 20070167689 | Ramadas et al. | Jul 2007 | A1 |
| 20070168469 | Church et al. | Jul 2007 | A1 |
| 20070185859 | Flowers et al. | Aug 2007 | A1 |
| 20070192710 | Platz et al. | Aug 2007 | A1 |
| 20070208738 | Morgan | Sep 2007 | A1 |
| 20070213983 | Ramsey | Sep 2007 | A1 |
| 20070214164 | MacLennan et al. | Sep 2007 | A1 |
| 20070233651 | Deshpande et al. | Oct 2007 | A1 |
| 20070233692 | Lisa et al. | Oct 2007 | A1 |
| 20070255552 | Thiesson et al. | Nov 2007 | A1 |
| 20070255567 | Bangalore et al. | Nov 2007 | A1 |
| 20070282811 | Musgrove | Dec 2007 | A1 |
| 20080046405 | Olds et al. | Feb 2008 | A1 |
| 20080052152 | Yufik | Feb 2008 | A1 |
| 20080071740 | Jhala et al. | Mar 2008 | A1 |
| 20080104101 | Kirshenbaum et al. | May 2008 | A1 |
| 20080115046 | Yamaguchi | May 2008 | A1 |
| 20080133752 | Liekens et al. | Jun 2008 | A1 |
| 20080140519 | Thiesson et al. | Jun 2008 | A1 |
| 20080158201 | Yoshida | Jul 2008 | A1 |
| 20080167858 | Christie et al. | Jul 2008 | A1 |
| 20080171555 | Oh et al. | Jul 2008 | A1 |
| 20080189628 | Liesche et al. | Aug 2008 | A1 |
| 20080195645 | Lapstun et al. | Aug 2008 | A1 |
| 20080195980 | Morris | Aug 2008 | A1 |
| 20080201326 | Cotter et al. | Aug 2008 | A1 |
| 20080208567 | Brockett et al. | Aug 2008 | A1 |
| 20080221893 | Kaiser | Sep 2008 | A1 |
| 20080288474 | Chin et al. | Nov 2008 | A1 |
| 20080294982 | Leung et al. | Nov 2008 | A1 |
| 20080312910 | Zhang | Dec 2008 | A1 |
| 20090002178 | Guday et al. | Jan 2009 | A1 |
| 20090043584 | Philips | Feb 2009 | A1 |
| 20090043741 | Kim | Feb 2009 | A1 |
| 20090055386 | Boss et al. | Feb 2009 | A1 |
| 20090063404 | Hacigumus et al. | Mar 2009 | A1 |
| 20090077464 | Goldsmith et al. | Mar 2009 | A1 |
| 20090106224 | Roulland et al. | Apr 2009 | A1 |
| 20090128567 | Shuster et al. | May 2009 | A1 |
| 20090154795 | Tan et al. | Jun 2009 | A1 |
| 20090187515 | Andrew et al. | Jul 2009 | A1 |
| 20090187824 | Hinckley | Jul 2009 | A1 |
| 20090210214 | Qian et al. | Aug 2009 | A1 |
| 20090216690 | Badger | Aug 2009 | A1 |
| 20090222437 | Niu et al. | Sep 2009 | A1 |
| 20090249198 | Davis et al. | Oct 2009 | A1 |
| 20090300546 | Kwok | Dec 2009 | A1 |
| 20090313239 | Wen et al. | Dec 2009 | A1 |
| 20100005086 | Wang | Jan 2010 | A1 |
| 20100100369 | Shetty et al. | Apr 2010 | A1 |
| 20100122155 | Monsarrat | May 2010 | A1 |
| 20100125811 | Moore et al. | May 2010 | A1 |
| 20100138210 | Seo et al. | Jun 2010 | A1 |
| 20100146407 | Bokor et al. | Jun 2010 | A1 |
| 20100169770 | Hong et al. | Jul 2010 | A1 |
| 20100180199 | Wu et al. | Jul 2010 | A1 |
| 20100217581 | Hong | Aug 2010 | A1 |
| 20100217795 | Hong | Aug 2010 | A1 |
| 20100222056 | Wu | Sep 2010 | A1 |
| 20100231523 | Chou | Sep 2010 | A1 |
| 20100245251 | Yuan et al. | Sep 2010 | A1 |
| 20100251304 | Donoghue et al. | Sep 2010 | A1 |
| 20100306139 | Wu et al. | Dec 2010 | A1 |
| 20100306248 | Bao et al. | Dec 2010 | A1 |
| 20100309137 | Lee | Dec 2010 | A1 |
| 20110014952 | Minton | Jan 2011 | A1 |
| 20110022621 | Luo et al. | Jan 2011 | A1 |
| 20110041077 | Reiner | Feb 2011 | A1 |
| 20110060761 | Fouts | Mar 2011 | A1 |
| 20110066431 | Ju et al. | Mar 2011 | A1 |
| 20110087483 | Hsieh et al. | Apr 2011 | A1 |
| 20110107265 | Buchanan et al. | May 2011 | A1 |
| 20110131642 | Hamura et al. | Jun 2011 | A1 |
| 20110137635 | Chalabi et al. | Jun 2011 | A1 |
| 20110145234 | Hu | Jun 2011 | A1 |
| 20110161080 | Ballinger et al. | Jun 2011 | A1 |
| 20110161311 | Mishne et al. | Jun 2011 | A1 |
| 20110173172 | Hong et al. | Jul 2011 | A1 |
| 20110178981 | Bowen et al. | Jul 2011 | A1 |
| 20110184723 | Huang et al. | Jul 2011 | A1 |
| 20110188756 | Lee et al. | Aug 2011 | A1 |
| 20110191321 | Gade et al. | Aug 2011 | A1 |
| 20110201387 | Paek et al. | Aug 2011 | A1 |
| 20110202836 | Badger et al. | Aug 2011 | A1 |
| 20110202876 | Badger et al. | Aug 2011 | A1 |
| 20110219299 | Scalosub | Sep 2011 | A1 |
| 20110258535 | Adler, III et al. | Oct 2011 | A1 |
| 20110282903 | Zhang | Nov 2011 | A1 |
| 20110289105 | Hershowitz | Nov 2011 | A1 |
| 20110296324 | Goossens et al. | Dec 2011 | A1 |
| 20120005184 | Thilagar et al. | Jan 2012 | A1 |
| 20120016678 | Gruber et al. | Jan 2012 | A1 |
| 20120019446 | Wu et al. | Jan 2012 | A1 |
| 20120022853 | Ballinger | Jan 2012 | A1 |
| 20120023103 | Soderberg et al. | Jan 2012 | A1 |
| 20120029902 | Lu et al. | Feb 2012 | A1 |
| 20120035932 | Jitkoff et al. | Feb 2012 | A1 |
| 20120036468 | Colley | Feb 2012 | A1 |
| 20120041752 | Wang | Feb 2012 | A1 |
| 20120060113 | Sejnoha et al. | Mar 2012 | A1 |
| 20120060147 | Hong | Mar 2012 | A1 |
| 20120078611 | Soltani et al. | Mar 2012 | A1 |
| 20120113011 | Wu et al. | May 2012 | A1 |
| 20120117506 | Koch et al. | May 2012 | A1 |
| 20120131032 | Rakshit | May 2012 | A1 |
| 20120143897 | Wei et al. | Jun 2012 | A1 |
| 20120173222 | Wang et al. | Jul 2012 | A1 |
| 20120222056 | Donoghue et al. | Aug 2012 | A1 |
| 20120297294 | Scott et al. | Nov 2012 | A1 |
| 20120297332 | Changuion et al. | Nov 2012 | A1 |
| 20120311480 | Cohen | Dec 2012 | A1 |
| 20130016113 | Adhikari et al. | Jan 2013 | A1 |
| 20130054617 | Colman | Feb 2013 | A1 |
| 20130091409 | Jeffery | Apr 2013 | A1 |
| 20130132359 | Lee et al. | May 2013 | A1 |
| 20130159920 | Scott et al. | Jun 2013 | A1 |
| 20130298030 | Nahumi et al. | Nov 2013 | A1 |
| 20130346872 | Scott et al. | Dec 2013 | A1 |
| 20140040238 | Scott et al. | Feb 2014 | A1 |
| 20150081369 | Sarrazin | Mar 2015 | A1 |
| 20150088927 | Sarrazin | Mar 2015 | A1 |
| 20150106702 | Scott et al. | Apr 2015 | A1 |
| 20150121291 | Scott et al. | Apr 2015 | A1 |
| 20150127748 | Buryak | May 2015 | A1 |
| 20150161126 | Wang et al. | Jun 2015 | A1 |
| 20150370833 | Fey et al. | Dec 2015 | A1 |
| 20160196150 | Jing et al. | Jul 2016 | A1 |
| Number | Date | Country |
|---|---|---|
| 1609764 | Apr 2005 | CN |
| 1851617 | Oct 2006 | CN |
| 1851617 | Oct 2006 | CN |
| 1908863 | Feb 2007 | CN |
| 101183355 | May 2008 | CN |
| 101276245 | Oct 2008 | CN |
| 101286092 | Oct 2008 | CN |
| 101286093 | Oct 2008 | CN |
| 101286094 | Oct 2008 | CN |
| 101587471 | Nov 2009 | CN |
| 101661474 | Mar 2010 | CN |
| 102012748 | Apr 2011 | CN |
| 102056335 | May 2011 | CN |
| 102144228 | Aug 2011 | CN |
| 102193643 | Sep 2011 | CN |
| 102314441 | Jan 2012 | CN |
| 102314461 | Jan 2012 | CN |
| 924594 | Jun 1999 | EP |
| 9204445 | Aug 1997 | JP |
| 11345247 | Dec 1999 | JP |
| 2000148748 | May 2000 | JP |
| 2002506256 | Feb 2002 | JP |
| 2004504674 | Feb 2004 | JP |
| 2006107502 | Apr 2006 | JP |
| 2008505407 | Feb 2008 | JP |
| 2011507099 | Mar 2011 | JP |
| 2012008874 | Jan 2012 | JP |
| 2012094156 | May 2012 | JP |
| 182119 | Nov 2001 | WO |
| WO2010105440 | Sep 2010 | WO |
| WO2010105440 | Sep 2010 | WO |
| Entry |
|---|
| Guo, Hai, and Jing-ying Zhao. “NaXi Pictographs Input Method and WEFT.” Journal of Computers 5.1 (2010): 117-124. |
| Office Action for U.S. Appl. No. 13/109,021, mailed on Mar. 11, 2014, Dyer, A.R., “Network Search for Writing Assistance,” 18 pages. |
| Office action for U.S. Appl. No. 13/315,047, mailed on Apr. 24, 2014, Liu et al., “Sentiment Aware User Interface Customization”, 13 pages. |
| Office action for U.S. Appl. No. 12/693,316, mailed on May 19, 2014, Huang et al., “Phonetic Suggestion Engine”, 22 pages. |
| Office Action for U.S. Appl. No. 13/109,021, mailed on Jun. 19, 2014, Dyer, A.R., “Network Search for Writing Assistance,” 42 pages. |
| “Prose”, Dictionary.com, Jun. 19, 2014, 2 pgs. |
| Damper, “Self-Learning and Connectionist Approaches to Text-Phoneme Conversion”, retrieved on May 26, 2010 at <<ftp://ftp.cogsci.ed.ac.uk/pub/joe/newbull.ps>>, UCL Press, Connectionist Models of Memory and Language, 1995, pp. 117-144. |
| “Database”, Microsoft Computer Dictionary, Fifth Edition, retrieved on May 13, 2011, at <<http://academic.safaribooksonline.com/book/communications/0735614954>>, Microsoft Press, May 1, 2002, 2 pages. |
| “File”, Microsoft Computer Dictionary, Fifth Edition, retrieved on May 13, 2011, at <<http://academic.safaribooksonline.com/book/communications/0735614954>>, Microsoft Press, May 1, 2002, 2 pages. |
| Final Office Action for U.S. Appl. No. 13/109,021, mailed on Jan. 11, 2013, Scott et al., “Network Search for Writing Assistance”, 16 pages. |
| Office action for U.S. Appl. No. 12/693,316, mailed on Oct. 30, 2013, Huang, et al., “Phonetic Suggestion Engine”, 24 pages. |
| Office action for U.S. Appl. No. 12/693,316, mailed on Jun. 19, 2013, Huang et al., “Phonetic Suggestion Engine”, 20 pages. |
| Office Action for U.S. Appl. No. 13/109,021, mailed on Aug. 21, 2012, Scott et al., “Network Search for Writing Assistance”, 19 pages. |
| Office Action for U.S. Appl. No. 13/109,021, mailed on Sep. 25, 2013, Scott et al., “Network Search for Writing Assistance”, 18 pages. |
| Wikipedia, “Soundex”, retrieved on Jan. 20, 2010 at http://en.wikipedia.org/wiki/soundex, 3 pgs. |
| Ciccolini, “Baidu IME More Literate in Chinese Input,” Published Sep. 15, 2011, <<http://www.itnews-blog.com/it/81630.html>>, 4 pages. |
| Scott et al., “Engkoo: Mining theWeb for Language Learning,” In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Systems Demonstrations, Jun. 21, 2011, 6 pages. |
| Sowmya et al., “Transliteration Based Text Input Methods for Telugu,” <<http://content.imamu.edu.sa/Scholars/it/VisualBasic/2009_53.pdf>>, Proceedings of 22nd International Conference on Computer Processing of Oriental Languages. Language Technology for the Knowledge-based Economy (ICCPOL), Mar. 2009, pp. 122-132. |
| U.S. Appl. No. 12/960,258, filed Dec. 3, 2010, Wei, et al., “Wild Card Auto Completion,” 74 pages. |
| U.S. Appl. No. 13/109,021, filed May 17, 2011, Matthew Robert Scott, “Network Search for Writing Assistance,” 43 pages. |
| U.S. Appl. No. 13/331,023, filed Dec. 20, 2011,Tony Hou, Weipeng Liu, Weijiang Xu, and Xi Chen, “Scenario-Adaptive Input Method Editor,” 57 pages. |
| Ramiro, “Baidu IME More literate in Chinese input,” Published Sep. 15, 2011, retrieved at <<http://www.itnews-blog.com/it/81630.html>>, 4 pages. |
| Millward, “Baidu Japan Acquires Simeji Mobile App Team, for added Japanese Typing fun,” Published Dec. 13, 2011, 3 pages. |
| Gamon et al., “Using Statistical Techniques and Web Search to Correct ESL Errors,” Published 2009, retrieved from <<http://research.microsoft.com/pubs/81312/Calico_published.pdf>>, CALICO Journal, vol. 26, No. 3, 2009, 21 pages. |
| “Google launched Input Method editor—type anywhere in your language,” published Mar. 2, 2010, retrieved at <<http://shoutingwords.com/google-launched-input-method-editor.html>>, 12 pages. |
| Lenssen, “Google Releases Pinyin Converter,” Published Apr. 4, 2007, retrieved at <<http://blogoscoped.com/archive/2007-04-04-n49.html>>, 3 pages. |
| “Google Scribe,” retrieved at <<http://www.scribe.googlelabs.com/>>, retrieved date: Feb. 3, 2011, 1 page. |
| Google Transliteration Input Method (IME) Configuration, published Feb. 5, 2010, retrieved at <<http://www.technicstoday.com/2010/02/google-transliteration-input-method-ime-configuration/>>, pp. 1-13. |
| Komasu et al., “Corpus-based Predictive Text Input,” Proceedings of the Third International Conference on Active Media Technology, May 2005, 6 pages. |
| Lo et al., “Cross platform CJK input Method Engine,” IEEE International Conference on Systems, Man and Cybernetics, Oct. 6, 2002, retrieved at <<http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1175680>>, pp. 1-6. |
| “Microsoft Research ESL Assistant,” retrieved at <<http://www.eslassistant.com/>>, retrieved date: Feb. 3, 2011, 1 page. |
| Mohan et al., “Input Method Configuration Overview,” Jun. 3, 2011, retrieved at <<http://gameware.autodesk.com/documents/gfx_4.0_ime.pdf>>, 71 pages. |
| Suematsu et al., “Network-Based Context-Aware Input Method Editor,” Proceedings: Sixth International Conference on Networking and Services (ICNS), Mar. 7, 2010, retrieved at <<http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5460679>>, pp. 1-6. |
| Suzuki et al., “A Comparative Study on Language Model Adaptation Techniques using New Evaluation Metrics,” Proceedings: HLT/EMNLP, Vancouver, Oct. 6-8, 2005, retrieved at <<http://www.aclweb.org/anthology/H/H05/H05-1034.pdf>>, 8 pages. |
| Windows XP Chinese Pinyin Setup, published Apr. 15, 2006, retrieved at <<http://www.pinyinjoe.com/pinyin/pinyin_setup.htm>>, pp. 1-10. |
| “Google Scribe,” retrieved at http://www.scribe.googlelabs.com/, retrieved date: Feb. 3, 2011, 1 page. |
| Wikipedia, “Selection Based Search”, retrieved Mar. 23, 2012 at http://en.wikipedia.org/wiki/Selection based search, 3 pgs. |
| “Microsoft Research ESL Assistant,” retrieved at <<http://www.eslassistant.com/>>, retrieved date Feb. 3, 2011, 1 page. |
| Millward, “Baidu Japan Acquires Simeji Mobile App Team, for added Japanese Typing Fun”, May 23, 2012, 3 pages. |
| “Search Engine”, Microsoft Computer Dictionary, Mar. 2002 , Fifth Edition, pp. 589. |
| Office action for U.S. Appl. No. 13/567,305, mailed on Jan. 30, 2014, Scott, et al., “Business Intelligent In-Document Suggestions”, 14 pages. |
| Office action for U.S. Appl. No. 13/315,047, mailed on Feb. 12, 2014, Liu, et al., “Sentiment Aware User Interface Customization”, 14 pages. |
| International Search Report & Written Opinion for PCT Patent Application No. PCT/US2013/053321, Mailed Date: Oct. 1, 2013, Filed Date: Aug. 2, 2013, 9 Pages. |
| Office action for U.S. Appl. No. 12/693,316, mailed on Oct. 16, 2014, Huang et al., “Phonetic Suggestion Engine,” 24 pages. |
| Office Action for U.S. Appl. No. 13/315,047, mailed on Oct. 2, 2014, Weipeng Liu, “Sentiment Aware User Interface Customization,” 12 pages. |
| Office action for U.S. Appl. No. 13/331,023, mailed on Feb. 12, 2015, Scott et al, “Scenario-Adaptive Input Method Editor,” 20 pages. |
| Office action for U.S. Appl. No. 13/635,219, mailed on Mar. 13, 2015, Scott et al., “Cross-Lingual Input Method Editor,” 21 pages. |
| Office action for U.S. Appl. No. 13/635,306, mailed on Mar. 27, 2015, Scott et al., “Input Method Editor,” 18 pages. |
| Non-Final Office Action for U.S. Appl. No. 13/331,023, mailed Aug. 4, 2014, Matthew Robert Scott et al., “Scenario-Adaptive Input Method Editor,” 20 pages. |
| The European Office Action mailed Oct. 8, 2015 for European patent application No. 12879804.8, a counterpart foreign application of U.S. Appl. No. 13/586,267, 9 pages. |
| The Supplemenary European Search Report mailed Jul. 16, 2015 for European patent application No. 12880149.5, 5 pages. |
| The Supplemenary European Search Report mailed Sep. 14, 2015 for European patent application No. 12879804.8, 5 pages. |
| Miessler, “7 Essential Firefox Quicksearches”, Retrieved from <<https:danielmiessler.com/blog/7-essential-firefox-quicksearches/>>, Published Aug. 19, 2007, 2 pages. |
| Office action for U.S. Appl. No. 13/635,306, mailed on Aug. 14, 2015, Scott et al., “Input Method Editor”, 26 pages. |
| Office action for U.S. Appl. No. 13/315,047, mailed on Sep. 24, 2015, Liu et al., “Sentiment Aware User Interface Customization”, 12 pages. |
| Office action for U.S. Appl. No. 13/635,219, mailed on Sep. 29, 2015, Scott et al., “Cross-Lingual Input Method Editor”, 14 page. |
| U.S. Appl. No. 13/635,219, filed Sep. 14, 2011, Scott, et al., “Cross-Lingual Input Method Editor” 43 pages. |
| Engkoo Pinyin Redefines Chinese Input, Published on: May 13, 2013, Retrieved from: http://research.microsoft.com/en-us/news/features/engkoopinyinime-051313.aspx; 3 pages. |
| “English Assistant”, Published on: Apr. 19, 2013, Retrieved from: http://bing.msn.cn/pinyin/; 2 pages. |
| “Innovative Chinese Engine”, Published on: May 2, 2013, Retrieved from: http://bing.msn.cn/pinyin/help.shtml; 6 pages. |
| “Input Method (IME)”, Retrieved on: Jul. 3, 2013, Retrieved from: http://www.google.co.in/inputtools/cloud/features/input-method.html; 6 pages. |
| International Search Report & Written Opinion for PCT Patent Application No. PCT/CN2013/081156, mailed May 5, 2014; filed Aug. 9, 2013, 14 pages. |
| Office action for U.S. Appl. No. 13/315,047, mailed on Apr. 28, 2015, Liu et al., “Sentiment Aware User Interface Customization”, 12 pages. |
| Office action for U.S. Appl. No. 12/693,316, mailed on Oct. 16, 2014, Huang, et al., “Phonetic Suggestion Engine”, 24 pages. |
| Office Action for U.S. Appl. No. 13/315,047, mailed on Oct. 2, 2014, Weipeng Liu, “Sentiment Aware User Interface Customization”, 12 pages. |
| Office action for U.S. Appl. No. 13/331,023, mailed on Feb. 12, 2015, Scott et al, “Scenario-Adaptive Input Method Editor”, 20 pages. |
| Office action for U.S. Appl. No. 13/635,219, mailed on Mar. 13, 2015, Scott et al., “Cross-Lingual Input Method Editor”, 21 pages. |
| Office action for U.S. Appl. No. 13/635,306, mailed on Mar. 27, 2015, Scott et al., “Input Method Editor”, 18 pages. |
| Non-Final Office Action for U.S. Appl. No. 13/331,023, mailed Aug. 4, 2014, Matthew Robert Scott et al., “Scenario-Adaptive Input Method Editor”, 20 pages. |
| Dinamik-Bot, et al., “Input method”, retrieved on May 6, 2015 at <<http://en.wikipedia.org/w/index.php?title=Input_method&oldid=496631911>>, Wikipedia, the free encyclopedia, Jun. 8, 2012, 4 pages. |
| The European Office Action mailed Jun. 18, 2015 for European patent application No. 12879676.0, a counterpart foreign application of U.S. Appl. No. 13/635,306, 5 pages. |
| The Supplementary European Search Report mailed May 20, 2015 for European Patent Application No. 12879676.0, 3 pages. |
| Office action for U.S. Appl. No. 13/701,008, mailed on May 12, 2015, Wang et al., “Feature-Based Candidate Selection”, 12 pages. |
| Office action for U.S. Appl. No. 13/701,008, mailed on Jun. 15, 2015, Wang et al., “Feature-Based Candidate Selection”, 17 pages. |
| Office action for U.S. Appl. No. 13/331,023, mailed on Jun. 26, 2015, Scott et al., “Scenario-Adaptive Input Method Editor”, 23 pages. |
| Office Action for U.S. Appl. No. 13/109,021, mailed on Sep. 30, 2014, Dyer, A.R., “Network Search for Writing Assistance,” 17 pages. |
| PCT International Preliminary Report on Patentability mailed Mar. 12, 2015 for PCT Application No. PCT/CN2012/080749, 8 pages. |
| Office action for U.S. Appl. No. 13/701,008, mailed on Mar. 17, 2016, Wang et al., “Feature-Based Candidate Selection,” 13 pages. |
| The European Office Action mailed Mar. 1, 2016 for European Patent Application No. 12883902.4, a counterpart foreign application of U.S. Appl. No. 13/701,008, 8 pages. |
| Ben-Haim et al., “Improving Web-based Image Search via Content Based Clustering,” Proceedings of the 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW '06), IEEE, Jun. 17, 2006, 6 pages. |
| Berg et al., “Animals on the Web,” Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '06), vol. 2, IEEE, Jun. 17, 2006, pp. 1463-1470. |
| The European Office Action mailed Nov. 27, 2015 for European patent application No. 12880149.5, a counterpart foreign application of U.S. Appl. No. 13/635,219, 10 pages. |
| The Partial Supplemenary European Search Report mailed Oct. 26, 2015 for European patent application No. 12883902.4, 7 pages. |
| The Supplementary European Search Report mailed Nov. 12, 2015 for European patent application No. 12880149.5, 7 pages. |
| The European Search Report mailed Feb. 18, 2016 for European patent application No. 12883902.4, 7 pages. |
| Office action for U.S. Appl. No. 13/331,023 mailed on Nov. 20, 2015, Scott et al., “Scenario-Adaptive Input Method Editor,” 25 pages. |
| PCT International Preliminary Report on Patentability mailed Feb. 18, 2016 for PCT Application No. PCT/CN2013/081156, 8 pages. |
| Ben-Haim, et al., “Improving Web-based Image Search via Content Based Clustering”, Proceedings of the 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW '06), IEEE, Jun. 17, 2006, 6 pages. |
| Berg, et al., “Animals on the Web”, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '06), vol. 2, IEEE, Jun. 17, 2006, pp. 1463-1470. |
| Office action for U.S. Appl. No. 13/331,023 mailed on Nov. 11, 2015, Scott et al., “Scenario-Adaptive Input Method Editor,” 25 pages. |
| Office action for U.S. Appl. No. 13/635,219, mailed on Mar. 24, 2016, Scott et al., “Cross-Lingual Input Method Editor,” 29 pages. |
| The Japanese Office Action mailed Oct. 31, 2016 for Japanese Patent Application No. 2015-528828, a counterpart foreign application of U.S. Appl. No. 13/701,008. |
| Office action for U.S. Appl. No. 13/635,219, mailed on Nov. 14, 2016, Scott et al., “Cross-Lingual Input Method Editor”, 27 pages. |
| The Chinese Office Action mailed Jun. 28, 2016 for Chinese Patent Application No. 201280074281.4, a counterpart foreign application of U.S. Appl. No. 13/586,267, 25 pages. |
| Translated Chinese Office Action mailed Jun. 3, 2016 for Chinese Patent Application No. 201280074382.1, a counterpart foreign application of U.S. Appl. No. 13/635,219, 18 pages. |
| The European Office Action mailed Jul. 19, 2016 for European Patent Application No. 13891201.9, a counterpart foreign application of U.S. Appl. No. 14/911,247, 7 pages. |
| The European Office Action mailed Jul. 19, 2016 for European patent application No. 12880149.5, a counterpart foreign application of U.S. Appl. No. 13/635,219, 7 pages. |
| The Supplementary European Search Report mailed Jul. 6, 2016 for European patent application No. 13891201.9, 4 pages. |
| Translated Japanese Office Action mailed May 24, 2016 for Japanese patent application No. 2015-528828, a counterpart foreign application of U.S. Appl. No. 13/701,008, 17 pages. |
| Office action for U.S. Appl. No. 13/635,306, mailed on Jul. 28, 2016, Scott et al., “Input Method Editor”, 24 pages. |
| Final Office Action for U.S. Appl. No. 13/635,219, mailed on Aug. 10, 2016, Matthew Robert Scott, “Cross-Lingual Input Method Editor”, 29 pages. |
| The Chinese Office Action dated Jan. 3, 2017 for Chinese patent application No. 201280074383.6, a counterpart foreign application of U.S. Appl. No. 13/635,306, 21 pages. |
| The Chinese Office Action dated Feb. 3, 2017 for Chinese patent application No. 201280074382.1, a counterpart foreign application of U.S. Appl. No. 13/635,219, 12 pages. |
| The European Office Action dated Dec. 22, 2016 for European patent application No. 12880149.5, a counterpart foreign application of U.S. Appl. No. 13/635,219, 11 pages. |
| Office action for U.S. Appl. No. 13/635,219, dated Nov. 14, 2016, Scott et al., “Cross-Lingual Input Method Editor”, 27 pages. |
| Office action for U.S. Appl. No. 13/701,008, dated Nov. 30, 2016, Wang et al., “Feature-Based Candidate Selection”, 21 pages. |
| Office action for U.S. Appl. No. 13/635,306, dated Feb. 25, 2016, Scott et al., “Input Method Editor”, 29 pages. |
| The Chinese Office Action dated Mar. 24, 2017 for Chinese Patent Application No. 201280074281.4, a counterpart foreign application of U.S. Appl. No. 13/586,267, 28 pages. |
| Office action for U.S. Appl. No. 13/635,306, dated Feb. 25, 2017, Scott et al., “Input Method Editor”, 23 pages. |
| The Chinese Office Action dated Jun. 19, 2017 for Chinese Patent Application No. 201280075557.0, a counterpart foreign application of U.S. Appl. No. 13/701,008, 13 pages. |
| “Advanced Querying and Information Retrieval”, In Database System Concepts, 4th Edition, Retrieved Date: Nov. 14, 2007, 78 Pages. |
| “Merriam-Webster Collegiate Dictionary”, In Merriam-Webster, 2000, 3 Pages. |
| “Microsoft Computer Dictionary, Fifth Edition”, Microsoft Press, 2002, 3 Pages. |
| “Microsoft play mobile search ‘wild card’”, Retrieved From: <<https://www.themauritius.com/redirect/index.php?url=http://www.pcw.co.uk/vnunet/news/2155193/microsoft-plays-wild-card>>, May 3, 2006, 3 Pages. |
| “MSN Desktop Search (beta)”, Retrieved From: <<http://www.pcmag.com/article2/0,1759,1771841,00.asp>>, Mar. 2, 2005, 2 Pages. |
| “Prose”, Retrieved From <<https://web.archive.org/web/20070202094243/http://dictionary.reference.com/browse/Prose>>, Feb. 2, 2007, 3 Pages. |
| “Using the Word Wheel”, Retrieved From: <<http://www.isys-search.com/support/techtips/wprdwheel.html>>, Retrieved From: Jun. 23, 2005, 1 Page. |
| “Weighted Finite-State Transducers in Speech Recognition”, In Proceedings of the Computer Speech & Language, vol. 16, Issue 1, 2002, 27 Pages. |
| “Search Report Issued in European Patent Application No. 07754994.7”, dated Feb. 15, 2012, 9 Pages. |
| “Office Action Issued in Korean Patent Application No. 10-2008-7026116”, dated Feb. 20, 2013, 4 Pages. |
| “Non-Final Office Action Issued in U.S. Appl. No. 11/332,954”, dated Mar. 1, 2012, 16 Pages. |
| “Notice of Allowance Issued in U.S. Appl. No. 12/693,316”, dated Apr. 20, 2015, 10 Pages. |
| “Final Office Action Issued in U.S. Appl. No. 12/960,258”, dated Nov. 2, 2012, 13 Pages. |
| “Final Office Action Issued in U.S. Appl. No. 12/960,258”, dated Jan. 25, 2013, 16 Pages. |
| “Non-Final Office Action Issued in U.S. Appl. No. 12/960,258”, dated Jul. 5, 2013, 14 Pages. |
| “Non-Final Office Action Issued in U.S. Appl. No. 12/960,258”, dated Jun. 15, 2012, 11 Pages. |
| “Notice of Allowance Issued in U.S. Appl. No. 12/960,258”, dated Nov. 21, 2013, 15 Pages. |
| “Notice of Allowance Issued in U.S. Appl. No. 13/315,047”, dated Feb. 12, 2016, 15 Pages. |
| “Final Office Action Issued in U.S. Appl. No. 13/567,305”, dated Jun. 13, 2014, 7 Pages. |
| “Notice of Allowance Issued in U.S. Appl. No. 13/567,305”, dated Oct. 3, 2014, 8 Pages. |
| “Notice of Allowance Issued in U.S. Appl. No. 13/567,305”, dated Nov. 6, 2014, 2 Pages. |
| “Notice of Allowance Issued in U.S. Appl. No. 13/567,305”, dated Jan. 12, 2015, 2 Pages. |
| “Office Action Issued in Chinese Patent Application No. 200780003211.9”, dated Jan. 29, 2012, 5 Pages. |
| “Office Action Issued in Chinese Patent Application No. 200780015952.9”, dated Apr. 23, 2012, 6 Pages. |
| “Second Office Action Issued in Chinese Patent Application No. 200780015952.9”, dated Aug. 17, 2011, dated Aug. 17, 2011, 8 Pages. |
| “Office Action Issued in Japanese Patent Application No. 2009-509573”, dated Jun. 22, 2012, 8 Pages. |
| “Office Action Issued in Korean Patent Application No. 9-5-2013-024628057”, dated Apr. 11, 2013, 7 Pages. |
| Barman, et al., “Parsimonious Explanations of Change in Hierarchical Data”, In the Proceedings of the IEEE 23rd International Conference on Data Engineering, Apr. 2007, pp. 1273-1275. |
| Bentley, Jon Louis. “Multidimensional binary search trees used for associative searching”, In Proceedings of the Communications of the ACM, vol. 18, Issue 9, Sep. 1975, pp. 509-517. |
| Blei, et al., “Hierarchical Bayesian Models for Applications in Information Retrieval”, In Proceedings of the Seventh Valencia International Meeting. Oxford University Press, Bayesian Statistics, vol. 7, 2003, pp. 25-43. |
| Church, et al., “The Wild Thing!”, In Proceedings of the ACL on Interactive Poster and demonstration sessions. Association for Computational Linguistics, Jun. 2005, pp. 93-96. |
| Ding, et al., “Computing Geographical Scopes of Web Resources”, In Proceedings of the 26th VLDB Conference, 2000, 23 Pages. |
| Dumais, et al., “Stuff I've seen: A System for Personal Information Retrieval and Re-use”, In Proceedings of the 26th annual international ACM SIGIR conference on Research and Development in Informaion Retrieval, Jul. 28, 2003, pp. 12-79. |
| Jelinek, Frederick “Statistical Methods for Speech Recognition”, In Massachusetts Institute of Technology, 1997, 5 Pages. |
| McLlroy, M D. “Development of a Spelling List”, In Proceedings of the IEEE Transactions on Communications vol. 30, Issue 1, 1982, 14 Pages. |
| Monaci, et al., “Image Compression with Learnt Tree-Structured Dictionaries”, In Proceedings of the IEEE 6th Workshop on Multimedia Signal Processing, 2004, pp. 35-38. |
| “International Search Report and Written Opinion Issued in PCT Application No. PCT/US2007/008570”, dated Sep. 28, 2007, 7 Pages. |
| Suematsu, et al., “Network Based Context Aware Input Method Editor”, In Proceedings of the Sixth International conference on Networking and Services, Mar. 7, 2010, 6 Pages. |
| Vester, et al., “Information Retrieval in Document Spaces Using Clustering”, In Master's Thesis, Department of Informatics and Mathematical Modeling, Technical University of Denmark, Aug. 2005, 266 Pages. |
| Yang, et al., “Detecting Dominant Locations from Search Queries”, In Proceedings of the 28th Annual International ACM SIGIR conference on Research and development in information retrieval, Aug. 15-19, 2005, 8 Pages. |
| Witten, et al., “Managing Gigabytes Compressing and Indexing Document and Images”, In Morgan Kaufmann Publishing, 1999, 65 Pages. |
| Witten, et al., “Managing Gigabytes: Compressing and Indexing Documents and Images(102-166)”, In Morgan Kaufmann Publishing, 1999, 65 Pages. |
| Witten, et al., “Managing Gigabytes: Compressing and Indexing Documents and Images(167-221)”, In Morgan Kaufmann Publishing, 1999, 55 Pages. |
| Witten, et al., “Managing Gigabytes: Compressing and Indexing Documents and Images(271-328)”, In Morgan Kaufmann Publishing, 1999, 58 Pages. |
| Witten, et al., “Managing Gigabytes: Compressing and Indexing Documents and Images(222-270)”, In Morgan Kaufmann Publishing, 1999, 48 Pages. |
| Witten, et al., “Managing Gigabytes: Compressing and Indexing Documents and Images(329-388)”, In Morgan Kaufmann Publishing, 1999, 60 Pages. |
| Witten, et al., “Managing Gigabytes: Compressing and Indexing Documents and Images(37-101)”, In Morgan Kaufmann Publishing, 1999, 65 Pages. |
| Witten, et al., “Managing Gigabytes: Compressing and Indexing Documents and Images(389-452)”, In Morgan Kaufmann Publishing, 1999, 65 Pages. |
| Witten, et al., “Managing Gigabytes: Compressing and Indexing Documents and Images(453 backcover)”, In Morgan Kaufmann Publishing, 1999, 67 Pages. |
| Zhou, et al., “A New Wildcard Search Method for Digital Dictonary Based on Mobile Platfom”, In Proceedings of the 16th IEEE International Conference on Artificial Reality and Telexistence—Workshops, 2006, 5 Pages. |
| “Office Action Issued in Chinese Patent Application No. 201280074281.4”, dated Oct. 25, 2017, 25 Pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20130346872 A1 | Dec 2013 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/CN2012/077436 | Jun 2012 | US |
| Child | 13586267 | US |
