The present invention relates generally to an information retrieval service and, more specifically, to an information lookup service that provides information based upon data from a mobile telephone or other computing device.
Over the past several decades, mobile telephones, or cell phones, and wireless computing devices have become ubiquitous. In the year 2003 alone over seventy-five million cell phones were shipped to consumers. Both cell phones and mobile computers have become smaller, more powerful and able to provide increasingly sophisticated services and functionality. Many applications have been developed to take advantage of the large numbers, portability and processing power of these devices. Today, many mobile devices include cameras, keyboards and other accessories to increase their utility.
One example of a service available to users of mobile telephone and computing devices is a music recognition service provided by some foreign and domestic cellular service providers. A user enters a designated code, e.g. “#43,” on the keypad of a cell phone, presses the send button and, after a connection to the service is established, holds the telephone's microphone to a music source for fifteen seconds or so. The service evaluates the music source, determines the song and the artist and sends a text message to the user with the names of the song and the artist. Typically, a small charge is added to the monthly bill of the user for this service although it could also be provided as a free feature of a particular cellular service provider.
Other examples of services available to users of mobile computing devices and telephones include music streaming services that enable a user to tune into any particular music stream such as a radio station and services that enable users to download maps, ring tones and specific songs for playback. Another example is a service that reads special bar codes displayed in conjunction with billboards and performs a lookup and display of information stored in conjunction with the bar code. What is needed is a service that provides a more flexible information retrieval that includes more generalized reference needs than currently available.
Provided is a system and method for providing an information retrieval service on a mobile computing device such as, but not limited to, a wireless computer or cellular telephone. The user of a mobile computing device provides information to the service provider. One example described throughout the Specification regards information concerning a bottle of wine or a list of wines although there is no limit on the type of information that may benefit from the techniques of the claimed subject matter.
With respect to a bottle of wine, the user transmits from a cell phone an image of the label of a particular bottle of wine and/or a textual or vocal description of the bottle. The service scans or otherwise processes the transmitted data to convert the data into a digital form. For example, optical character recognition (OCR) is employed to convert the information displayed on the label into a text file. The service provider performs analytical computations on the data in the text file and, using context and preference rules, produces information of value to the user. For example, the service provider may determine the type of wine, the brand, the year, the region in which the grapes were grown and a rating number for the wine from a commercial wine rating service. The type, brand, year, grape and rating information is then transmitted back to the user in the form of a text message.
Other examples of services provided by the claimed subject matter include, but are not limited to, the transmission of information on the label of a medicine bottle and billboards. With respect to medicine bottles, a user might transmit either an image or a text description of the label and receive information relating to dosage or potential adverse effects the medicine may have with other medications the user is taking. This service may be particularly useful to a user who has trouble reading the label of a bottle. With respect to a billboard, the display does not even have to be in the user's language. If the user is in a foreign country, the service can provide a language translation. In addition, a query may be directed to information displayed on a television or LCD screen.
The service may request additional information from the user for the purpose of better servicing the query. For example, if the OCR is unable to resolve the image, a query may be sent to the user to resolve the character recognition problem. If the service uncovers more information than can be transmitted efficiently in a text message, a message may be transmitted to the user requesting additional information to further restrict the original query.
A query to the claimed service may be modified based upon configuration information stored with respect to a particular user or based upon previous queries from the particular user. For example, a user may specify via a configuration file that the only information transmitted in response to a query corresponding to a wine list is a rating corresponding to the individual wines on the list. This limitation may also be determined based upon previous queries that typically restricted the information returned to the user to ratings information.
This summary is not intended as a comprehensive description of the claimed subject matter but, rather, is intended to provide a brief overview of some of the functionality associated therewith. Other systems, methods, functionality, features and advantages of the invention will be or will become apparent to one with skill in the art upon examination of the following figures and detailed description.
A better understanding of the present invention can be obtained when the following detailed description of the disclosed embodiments is considered in conjunction with the following figures.
Although described with particular reference to a cellular telephone, the claimed subject matter can be implemented in any communication technology (IT) system in which context-based information retrieval is desirable. Those with skill in the communication and computing arts will recognize that the disclosed embodiments have relevance to a wide variety of communication and computing environments in addition to those described below. In addition, the methods of the disclosed invention can be implemented in software, hardware, or a combination of software and hardware. The hardware portion can be implemented using specialized logic; the software portion can be stored in a memory and executed by a suitable instruction execution system such as a microprocessor, personal computer (PC) or mainframe.
In the context of this document, a “memory” or “recording medium” can be any means that contains, stores, communicates, propagates, or transports the program and/or data for use by or in conjunction with an instruction execution system, apparatus or device. Memory and recording medium can be, but are not limited to, an electronic, magnetic, optical, electromagnetic or semiconductor system, apparatus or device. Memory an recording medium also includes, but is not limited to, for example the following: a portable computer diskette, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), and a portable compact disk read-only memory or another suitable medium upon which a program and/or data may be stored.
One embodiment, in accordance with the claimed subject, is directed to a programmed method for providing a context-based information retrieval system. The term “programmed method”, as used herein, is defined to mean one or more process steps that are presently performed; or, alternatively, one or more process steps that are enabled to be performed at a future point in time. The term “programmed method” anticipates three alternative forms. First, a programmed method comprises presently performed process steps. Second, a programmed method comprises a computer-readable medium embodying computer instructions, which when executed by a computer performs one or more process steps. Finally, a programmed method comprises a computer system that has been programmed by software, hardware, firmware, or any combination thereof, to perform one or more process steps. It is to be understood that the term “programmed method” is not to be construed as simultaneously having more than one alternative form, but rather is to be construed in the truest sense of an alternative form wherein, at any given point in time, only one of the plurality of alternative forms is present.
Turning now to the figures,
Each of devices 102, 104, 106 and 108 is coupled to one or more communication mediums, which in this example include a wireless system 110, a plain old telephone service (POTS) 112 and the Internet 114. Also coupled to Internet 114 and POTS 112 is a server computer 120. Server 120 includes a central processing unit (CPU) 122, which is coupled to a monitor 124, a keyboard 126 and a mouse 128, which together facilitate human interaction with server 120. Also included in server 120 and attached to CPU 122 is a data storage component 130, which may either be incorporated into CPU 122 i.e. an internal device, or attached externally to CPU 122 by means of various, commonly available connection devices such as but not limited to, a universal serial bus (USB) port (not shown). Stored on data storage 130 is a context-based information system (CBIS) 132 that implements the claimed subject matter. CBIS 132 is described in more detail below in conjunction with
In this example, cellular telephone 102 and palm computer 106 are communicatively coupled to server 120 via wireless system 110 and Internet 114. Telephone 104 is coupled to server 120 via POTS 112 which has two connections, one via Internet 114 and one via a direct connection. Computer 108 is coupled to server 120 via both POTS 108 and Internet 114. The claimed subject matter may be implemented on many types of communication media. The examples of
In this example, cellular telephone 102 is able to transmit and receive data in text, audio and visual formats. For example, cellular telephone 102 may employ Multimedia Messaging Service (MMS) that allows a user to send and receive messages that include images, audio, video and rich text.
Functionality associated with each of modules 151-160 is described in detail below in conjunction with
OCR module 153, audio module 154 and text module 155 translate images, audio information or text information, respectively, depending upon the format or formats identified by input recognition module 152, into a standardized format for processing by CBIS 132. In the following example, the standardized format is a text file. Parsing module 156 takes standardized input generated by OCR module 153, audio module 154 and text module 155 and determines, based upon general context clues, the information requested. The context includes, but is not limited to, such information as the requesting device, a GPS location of the requesting device, the time, surrounding devices and both user and CBIS 132 defined configuration options. As explained below in conjunction with
Once parsing module 156 has determined the appropriate query, search module 157 retrieves the relevant information. Information may be retrieved from any number of sources such as a database (not shown) located on data storage 130 (
CBIS configuration module 158 stores information relating to the setup and operation of CBIS 132, including, but not limited to, data source locations, user interface screens and user interface default information. This information includes both context and preference rules. Finally, user configuration module 159 stores information related to specific users, including, but not limited to, data concerning setup parameters, login and password information, context information such as scheduling and geographical information, types of devices typically employed by the user, results of previous queries, preferred user interface parameters and so on. Although only one user configuration block 159 is illustrated, in an alternative embodiment, CBIS 132 stores multiple modules 159, each module 159 corresponding to a specific user. In an alternative embodiment, some user configuration information may be stored on user devices in the form of cookies, which should be familiar to those with skill in the computing arts.
Billing module 160 is responsible for maintaining records related to the charging for services related to CBIS 132. For example, CBIS 132 made be configured to access a specific fee for each request a user makes for information or fees made be based upon the amount of data transmitted. Each user may also have access to a limited number of searches for a fixed monthly rate. In addition, billing module 160 may access fees related to various third party vendors to whom a particular user desires access. For example, using the example of retrieving information concerning a bottle of wine, a user may select to receive information form the Wine Spectator and be accessed a access fee for that particular service. The functionality of each of modules 151-160 is also explained below in conjunction with the flowcharts of
Process 200 starts in a “Begin Execute CBIS” block 202 and proceeds immediately to a “Retrieve CBIS Configuration (Config.)” block 204. During block 204, process 200 retrieves data stored in CBIS configuration module 146 (
During a “Wait for Input” block 208, process 200 enters a sleep state to wait for a request for information from a user on a device such as cellular telephone 102, telephone 104 (
During a “User Known?” block 210, process 200 determines whether or not the user who transmitted the query received during block 208 is a known, or registered user. A particular user is verified as a known user based upon the existence of information in user configuration module 159 (
One example of a GUI that enables a user to query for information on wine may look as follows:
The information entered during registration is used to update user configuration module 159. In addition, billing module 160 (
During a “Process Input” block 216, process 200 executes logic to determine an appropriate response to the query received during block 208. Processing associated with block 216 is described in more detail below in conjunction with
Once initiated, process 200 typically executes until intentionally halted or server 120 is powered down. In either case, an asynchronous interrupt 218 is generated to initiate a shut down procedure that includes a “Clean Up Memory” block 220. During block 220, process executes steps to gracefully shutdown process 200 including, but not limited to, cleaning up any memory associated with the execution of process 200. Finally, process 200 proceeds to an “End Execute CBIS” block 229 in which process 200 is complete.
During a “Gather Context” block 236, process 230 collects any information that may be related with the query, both information stored in user configuration module 159 and information transmitted with the query itself. For example, if the query is transmitted from cellular telephone 102 (
During an “Analyze Input” block 238, input recognition module 152 (
During an “Image?” block 240, process 240 determines whether or not the query analyzed during block 238 includes a component in a visual format, e.g. a digital image. If so, process 230 proceeds to an “Optical Character Recognition (OCR)” block 242. During block 242, OCR module 153 (
Once processing of block 246 is complete or, if during block 244, process 230 determines that the query is not in an audio format, control proceeds to a “Text?” block 248 during which process 230 determines whether or whether or not the query analyzed during block 238 includes a component in a textual format, e.g. a typed question. If so, process 230 proceeds to a “Text Processing” block 250 during which, text module 155 (
During a “Parse Input” block 252, parsing module 156 (
During a “Process Query” block 254, process 230 determines an appropriate response to the current query. Block 254 is described in more detail below in conjunction with
In an alternative embodiment, rather than transmitting the response to the device from which the query originated, a response may be place on a web portal (not shown) and retrieved either with the originating device or another device via the Internet 114 (
During an “Archive Query” block 258, process 230 saves both the query and the result in data storage 130 in conjunction with user configuration module 159 and the user who initiated the query. The archiving of data may take the form of Internet 114 enabled files that the user is able to access and share with other users. Such data could be organized in a tree structure and indexed to enable easy lookup. Such a storage structure enables a user to access a subset of the information immediately and yet provide more detailed information later such as when the user has access to a different device better suited to examining additional data. In addition, the archive is able to store user-generated or group-generated comments. For example, if the user drinks a particularly good bottle of wine, a query is generated based upon an image of the label and the user adds comments to the results of the query that make note of the user's opinion. Context information may also be stored with the results. For example, if cellular telephone 102 is the source of the query, GPS or other location data may be sorted in conjunction with the results.
In another embodiment, data is stored in data storage 130 for later processing. For example, if the received query consists of an image of a box or plate of food, the processing of the query may involve a calculation of the total calories of the food, which is then stored in data storage 130 with a listing of the food items to enable the user to maintain a dietary log. Finally, process 230 proceeds to an “End Process Input” block 259 in which process 230 is complete.
During a “Need More Data?” block 276, process 270 determines whether or not the received query contains enough information so that the query can be processed. If not, process 270 proceeds to a “Formulate Request” block 278 during which process 270 generates a message for transmission to, in this example, cellular telephone 102 (
During a “Wait For Reply” block 282, process 270 pauses to enable the user of cellular telephone 102 to formulate and transmit a reply to the request for additional information. Once a reply is received, control proceeds to a transition point A, returning control to Analyze Input block 238 of Process Input process 230, described above in conjunction with
If during block 276, process 270 determines the received query is complete, process 270 proceeds to a “Search For Data” block 284. During block 284, search module 157 (
Once data has been retrieved, process 270 proceeds to a second “Need More Data?” block 286 during which process 270 determines whether or not the query generated either enough of too much data. In the case of too much data, the information set retrieved must be small enough to transmit to cellular telephone 102. In other words, not only must the received query be complete, which is determined during block 276, the query must include some information and be specific enough to prevent too much data form being generated for transmission to cellular telephone 102. If the generated data set is either too small or too large, process 270 proceeds to block 278 and processing continues as described above. For example, a request of additional parameters may take the form of a message that states, “This is a bottle of wine. Do you want me to 1) Find prices; 2) List food pairings; 3) Give reviews: 4) Tell you about the vineyard; 5) Identify specials; 6) other.” An answer to the entry of the number ‘2’ may be a request for additional information such as “1) French; 2) Italian; or 3) Greek?” In other words the claimed subject matter provides the capability of context refinements to search requests.
Another example of a result set that enables refinements to the original query looks as follows:
If during block 286, process 270 determines that the data set is neither too small nor too large, control proceeds to a “Format Reply” block 288. During block 288, process 270 formats the retrieved data into a format as specified by CBIS configuration module 158 and user configuration module 159 (
While the invention has been shown and described with reference to particular embodiments thereof, it will be understood by those skilled in the art that the foregoing and other changes in form and detail may be made therein without departing from the spirit and scope of the invention, including but not limited to additional, less or modified elements and/or additional, less or modified blocks performed in the same or a different order. As explained above, in addition to the described examples, other applications include, but are not limited to, the augmentation of information on such things product and medicine bottle labels and billboards. Services provided may include the translation of foreign languages.