This application includes material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent disclosure, as it appears in the Patent and Trademark Office files or records, but otherwise reserves all copyright rights whatsoever.
The present invention relates to systems and methods for interfaces for web queries, more particularly, to systems and methods for using online handwriting recognition to improve the ease of use of web query entry, particularly for graphic character sets.
Web search services, such as those provided by Yahoo! and Google, are among the most popular and important applications available on the Internet. Access to the full array of content on the Internet is difficult, if not impossible, without the facilities provided by such services. The availability of the Internet has spread to many Asian countries such as, for example, China and Taiwan.
This presents a significant challenge. There is much content that is available in traditional Asian character sets, and searching such content requires the entry of such characters. Electronic devices, such as computers and mobile devices typically provide user interface elements based on the Roman characters. Asian characters can only be entered indirectly using an equivalent based on the Roman character set, such as Pinyin (Chinese), or some other complicated set of key entries. This can pose a barrier to the use of the Internet to large segments of Asian populations, such as the elderly, that are unfamiliar with the Roman alphabet or Asian character equivalent systems.
In one embodiment, the invention is a method. A graphic representation resulting from a user interacting with a user interface operating on a user device is received over a network. The graphic representation corresponds to a portion of a desired graphic character of a graphic character set (e.g. Chinese). The graphic representation is then analyzed, using a computing device, to select a plurality of graphic characters of the graphic character set that are a probable match of the graphic representation. The plurality of probable graphic characters are transmitted, over the network, to the user. A selection of one of the plurality of probable graphic characters is received over the network. An indication is received from the user, over the network, that the selected one of the plurality of probable graphic characters is to be used as a search term to conduct an internet search. An Internet search is then conducted using the search term.
In another embodiment, the invention is a method. A graphic representation resulting from a user interacting with a user interface operating on a user device is received over a network. The graphic representation corresponds to a portion of a desired graphic character of a graphic character set (e.g. Chinese). The graphic representation is then analyzed, using a computing device, to select a plurality of graphic characters of the graphic character set that are a probable match of the graphic representation. The plurality of probable graphic characters are transmitted, over the network, to the user. A selection of one of the plurality of probable graphic characters is received over the network. A plurality of suggested search terms are identified to the user. Each suggested search term comprises at least one graphic character from the graphic character set. One of the plurality of suggested search terms is selected to be used to conduct an Internet search. An Internet search is then conducted using the selected search term.
In another embodiment, the invention is a system. The system comprises one or more modules comprising one or more processors programmed to execute software code retrieved from a computer readable storage medium storing software for various functions. In one embodiment, the system comprises: a graphics receiving module for receiving, over a network, from users, data comprising graphic representations resulting from the users interacting with user interfaces operating on a user devices, the graphic representation corresponding to a portion of a desired graphic character of a graphic character set; a graphic character recognition module for analyzing graphic representations received by the graphics receiving module to select a plurality of graphic characters of the graphic character set that are probable matches of the graphic representations; a graphic character transmission module for transmitting, over the network, pluralities of probable graphic characters to users; a graphic character selection receiving module for receiving, over the network from an indication that the selected one of the plurality of probable graphic characters is to be used as a search term to conduct an internet search; a search indication receiving module for receiving, over the network, an indication that selected probable graphic characters are to be used as a search term to conduct an Internet search; a search term identification module for identifying to users a plurality of suggested search terms, each suggested search term comprising at least one graphic character from the graphic character set; a search term selection receiving module for receiving, over the network, from users, selections of suggested search terms to be used to conduct an Internet search; and a web search module for conducting Internet searches using selected search terms.
The foregoing and other objects, features, and advantages of the invention will be apparent from the following more particular description of preferred embodiments as illustrated in the accompanying drawings, in which reference characters refer to the same parts throughout the various views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating principles of the invention.
The present invention is described below with reference to block diagrams and operational illustrations of methods and devices to select and present media related to a specific topic. It is understood that each block of the block diagrams or operational illustrations, and combinations of blocks in the block diagrams or operational illustrations, can be implemented by means of analog or digital hardware and computer program instructions.
These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, ASIC, or other programmable data processing apparatus, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, implements the functions/acts specified in the block diagrams or operational block or blocks.
In some alternate implementations, the functions/acts noted in the blocks can occur out of the order noted in the operational illustrations. For example, two blocks shown in succession can in fact be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality/acts involved.
For the purposes of this disclosure the term “server” should be understood to refer to a service point which provides processing, database, and communication facilities. By way of example, and not limitation, the term “server” can refer to a single, physical processor with associated communications and data storage and database facilities, or it can refer to a networked or clustered complex of processors and associated network and storage devices, as well as operating software and one or more database systems and applications software which support the services provided by the server.
For the purposes of this disclosure the term “end user” or “user” should be understood to refer to a consumer of data supplied by a data provider. By way of example, and not limitation, the term “end user” can refer to a person who receives data provided by the data provider over the Internet in a browser session, or can refer to an automated software application which receives the data and stores or processes the data.
For the purposes of this disclosure, a computer readable medium stores computer data in machine readable form. By way of example, and not limitation, a computer readable medium can comprise computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other solid-state memory technology, CD-ROM, DVD, or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other mass storage devices, or any other medium which can be used to store the desired information and which can be accessed by the computer.
For the purposes of this disclosure a module is a software, hardware, or firmware (or combinations thereof) system, process or functionality, or component thereof, that performs or facilitates the processes, features, and/or functions described herein (with or without human interaction or augmentation). A module can include sub-modules. Software components of a module may be stored on a computer readable medium. Modules may be integral to one or more servers, or be loaded and executed by one or more servers. One or more modules may grouped into an engine or an application.
The present invention is directed to systems and methods for improving the accessibility of web search services by integrating online recognition of handwritten characters and words into search query entry and search query assistance functions provided by such services. In one embodiment, online, real-time handwriting recognition functions are hosted on servers and are integrated into search query functions users through simple browser interfaces, thereby eliminating the need for complex, specialized software and hardware on end user devices.
If the user does not wish to submit the entered character or a suggested search term to a search engine, the user may continue to enter graphic characters.
In one embodiment, the user can continue to enter any number of characters until the user is satisfied the query in the query entry box 1400 or a suggested search term 1620 properly expresses the user's intent. In the event the characters in the query entry box 1400 express the user's intent, the user can submit the query by selecting the query submission button. In the event a selected search term 1620 express the user's intent, the user can select the term, which can then be automatically submitted to a search engine. While the examples provided in
Web search service providers 2400 can provide one or more user interfaces enabling users 2200 to enter web search queries. Such interfaces may include online graphic character (e.g. handwriting) recognition. Online graphic character recognition can be provided by the web search service provider, or, alternatively, online graphic character recognition services may be provided by a character recognition services provider 2800. In one embodiment, online graphic character recognition providers 2800 maintain one or more character recognition servers 2840 and character recognition databases 2880, such as, for example, databases supporting the recognition of characters from Asian character sets.
The system shown in
The user interface action could be any user interface action that is capable of generating all, or a portion, of a graphic character. In the embodiment illustrated in
As each graphic representation is received, the system analyzes the graphic representation 3200, using at least one computing device, to select a plurality of graphic characters of a graphic character set that are a probable match of the graphic representation. In one embodiment, the analysis 3100 and analysis functions 3200 could be repeated for a plurality of repetitions, allowing a user to build a complex representation of a character comprising multiple elements (e.g. multiple strokes). In one embodiment, each time an element of a graphic character is defined by a user interface action, the character defined by the set of all interface actions is analyzed.
In alternative embodiments, the system could allow a user to enter multiple graphic representations before analyzing the graphic representations. In one embodiment, the analysis of the graphic representation could occur automatically after every user interface interaction (e.g. after every stroke). In one embodiment, the analysis of the graphic representation could be initiated by selection of an analysis function (e.g. an “analyze” button after multiple strokes of a graphic character are entered.)
The number of probable matches could vary based on the analysis of the graphic representations received. The illustrated embodiments in
The plurality of probable graphic characters is then transmitted to the user 3300. In one embodiment, the probable matches are displayed to the user by a user interface. The number of probable matches displayed to a user can vary depending on user interface design. In one embodiment, the user interface displayed to users allows the user to select one of the plurality of graphic characters for use in an Internet query.
A selection of one of the plurality of probable graphic character matches is then received 3400 from the user. The selection can be received after a first set of probable matches is transmitted to the user, or it can be received after multiple sets of probable matches are transmitted to the user in response to multiple user interface actions defining, for example, a complex graphic character. Furthermore, the user may choose to enter and select a plurality of characters, repeating the step 3100 through 3400 described above, thereby building a web query consisting of multiple characters.
In one embodiment, an indication is then received 3500 that the selected probable graphic characters are to be used as a search term to conduct an Internet search. In one embodiment, such an indication could be received after a user selects a “Search” button on a user interface, indicating the user wishes to conduct an Internet search using the selected graphic characters. An Internet search is then conducted 3800 using the selected term.
Alternatively or additionally, the selected character or characters can used to identify 3600 one or more suggested search terms. In one embodiment, existing search assistance functions provided by commercial web search functions, such as those provided by Yahoo! or Google could be used. Such search assistance functions, in one embodiment, match search queries to list of a users' probable intent. Such suggested search terms can then be transmitted to the user. When one of the selected search terms is selected by the user 3700, the selected search term is then used to conduct an Internet search.
Each of the modules could be implement on one or more servers. For example, the graphics receiving module 4100, the graphic character transmission module 4300, the graphic character selection receiving module 4400, a search indication receiving module 4500, the search term identification module 4600, the search term selection receiving module 4700 and the web search module 4800 could be implemented on one or more web search services servers such as shown in 2480 of
In one embodiment, the graphics receiving module 4100 is configured to receive data comprising graphic representations resulting from users interacting with a user interface operating on a user device. The graphic representation corresponds to a portion of a desired graphic character of a graphic character set. The graphic character set could be any currently known, or invented in the future. The user interface action could be any user interface action that is capable of generating all, or a portion, of a graphic character. In the embodiment illustrated in
In one embodiment, the graphic character recognition module 4200, is configured to analyze graphic representations received by the graphics receiving module to select a plurality graphic characters from a graphic character set that are probable matches of the graphic representations received by the module. The number of probable matches could vary based on the analysis of the graphic representations received. The illustrated embodiments in
In one embodiment, the graphic character transmission module 4300 transmits graphic characters from a graphic character set that are probable matches of the graphic representations selected by the graphic character recognition module 4200 to users. In one embodiment, the probable matches are displayed to the user by a user interface. The number of probable matches displayed to a user can vary depending on user interface design. In one embodiment, the user interface displayed to users allows the user to select one of the plurality of graphic characters for use in an Internet query.
In one embodiment, the graphic character selection receiving module 4400 is configured to receive selections of probable graphic character matches transmitted by the graphic character transmission module 4300. Such selections can be received after a first set of probable matches is transmitted to the user, or it can be received after multiple sets of probable matches are transmitted to the user in response to multiple user interface actions defining, for example, a complex graphic character. Furthermore, the user may choose to enter and select a plurality of characters.
In one embodiment, the search indication receiving module 4500 is configured to receive indications that selected probable graphic characters are to be used as search terms to conduct Internet searches. In one embodiment, such indications could be received after a user selects a “Search” button on a user interface, indicating the user wishes to conduct an Internet search using the selected graphic characters. Such searches are conducted, in one embodiment, using the web search module 4800.
In one embodiment, the search term identification module 4600 is configured to use the characters selected by the graphic character selection receiving module to identify suggested search terms. In one embodiment, existing search assistance functions provided by commercial web search functions, such as those provided by Yahoo! or Google could be used. Such search assistance functions, in one embodiment, match search queries to list of a users' probable intent. Such suggested search terms can then be transmitted to the user.
In one embodiment, the search term selection receiving module 4700 receives selected search terms from users and user selected search terms to conduct Internet searches. Such searches are conducted, in one embodiment, using the web search module 4800.
Those skilled in the art will recognize that the methods and systems of the present disclosure may be implemented in many manners and as such are not to be limited by the foregoing exemplary embodiments and examples. In other words, functional elements being performed by single or multiple components, in various combinations of hardware and software or firmware, and individual functions, may be distributed among software applications at either the client level or server level or both. In this regard, any number of the features of the different embodiments described herein may be combined into single or multiple embodiments, and alternate embodiments having fewer than, or more than, all of the features described herein are possible. Functionality may also be, in whole or in part, distributed among multiple components, in manners now known or to become known. Thus, myriad software/hardware/firmware combinations are possible in achieving the functions, features, interfaces and preferences described herein. Moreover, the scope of the present disclosure covers conventionally known manners for carrying out the described features and functions and interfaces, as well as those variations and modifications that may be made to the hardware or software or firmware components described herein as would be understood by those skilled in the art now and hereafter.
Furthermore, the embodiments of methods presented and described as flowcharts in this disclosure are provided by way of example in order to provide a more complete understanding of the technology. The disclosed methods are not limited to the operations and logical flow presented herein. Alternative embodiments are contemplated in which the order of the various operations is altered and in which sub-operations described as being part of a larger operation are performed independently.
While various embodiments have been described for purposes of this disclosure, such embodiments should not be deemed to limit the teaching of this disclosure to those embodiments. Various changes and modifications may be made to the elements and operations described above to obtain a result that remains within the scope of the systems and processes described in this disclosure.
Number | Name | Date | Kind |
---|---|---|---|
5926566 | Wang et al. | Jul 1999 | A |
20020152258 | Zhou | Oct 2002 | A1 |
20050152602 | Chen et al. | Jul 2005 | A1 |
20070050186 | Cheng | Mar 2007 | A1 |
20080158201 | Yoshida | Jul 2008 | A1 |
20090204609 | Labrou et al. | Aug 2009 | A1 |
20100005086 | Wang et al. | Jan 2010 | A1 |
Number | Date | Country |
---|---|---|
1187257 | Jul 1998 | CN |
1896934 | Jan 2007 | CN |
101714141 | May 2010 | CN |
1017141141 | May 2010 | CN |
WO 2010005798 | Jan 2010 | WO |
Entry |
---|
M. C. Ramsey, T. -H. Ong, H. Chen, “Multilingual Input System for the Web-An Open Multimedia Approach of Keyboard and Handwriting Recognition for Chinese and Japanese,” adl, p. 188, Fifth International Forum on Research and Technology Advances in Digital Libraries (ADL '98), 1998. |
Naver Blog Posting (“Chinese Character Handwriting Recognizer Function, ” The Naver Online Chinese Dictionary, Jun. 12, 2009), 8 pages. |
Y. Akiyima, Software Review Application Soft. Journal of Japan Association for East Asian Test Processing, Japan, Koubun Publisher Inc., Oct. 1, 2008; vol. 9. pp. 111-120. |
Number | Date | Country | |
---|---|---|---|
20110295877 A1 | Dec 2011 | US |