Method and resulting system for integrating a query reformation module onto an information retrieval system

BACKGROUND OF THE INVENTION

[0003] This invention generally relates to a knowledge based technique. More particularly, the present invention provides a way to integrate a query reformulation module to an information retrieval system. Merely by way of example, the present invention is implemented using a conventional information retrieval system coupled to a database, but it would be recognized that the invention has a much broader range of applicability. The invention can be applied to other sources of information from the Internet, a network of computers, and the like.

[0004] Networks, computers, and databases have proliferated the availability of information. Such information includes, among others, newspapers, magazines, advertisements, commercial publications, and commercial products in electronic form. By way of a world wide network of computers, which is known as the Internet, millions if not billions of pieces of information can be accessed through “browser” programs such as those made by Netscape Communications, Inc. of Mountain View, Calif. or Microsoft Corporation of Redmond, Wash. Information retrieval engines such as those made by Yahoo! and others allow a user to access such information using an indexing technique. The indexing technique often uses full-text indexing, in which content words in a document are used as keywords to be searched. Full text index searching has been one of way to retrieve information in conventional retrieval engines. Unfortunately, such full text searching is plagued with many problems. For example, a user of such searching often retrieves thousands of documents or hits or related documents and is therefore not precise. Such searching often requires refinement using a hit or miss strategy, which is often cumbersome and takes time and lacks efficiency. Accordingly, fall text searching has much room for improvement.

[0005] There have also been other attempts to search large quantities of information on systems using natural language techniques. Such natural language techniques often use simple logical forms, which are difficult to scale efficiently and lack precision using large quantities of information. For example, conventional natural language techniques often cause what is known as “combinatorial explosion” when the number of logical forms that are stored as templates grows. Accordingly, natural language techniques have not been able to be scaled for large complex information systems.

[0006] Additionally, such techniques have been separate from each other, where natural language search techniques have not been integrated into keyword search techniques. Even if such techniques have been integrated, integration is often difficult to achieve in an efficient and cost effective manner. Additionally, integration also requires some modification to the pre-existing technique that may influence reliability, operability, and dependability of the technique. Accordingly, there are many limitations with ways to integrate any of the conventional techniques.

[0007] From the above, it is seen that an improved way to acquire information using a knowledge based technique is highly desirable.

SUMMARY OF THE INVENTION

[0008] According to the present invention, a technique including a method and system for a knowledge based technique is provided. More particularly, the invention provides an improved way of integrating a query reformulation module onto a pre-existing information retrieval system. In an exemplary embodiment, the invention provides an enhanced search technique that can be integrated into a conventional information retrieval method and system.

[0009] In a specific embodiment, the invention provides a method for converting a keyword based search engine coupled to a information source into a natural language enhanced search engine. The method includes determining expression based syntax of the keyword based search engine. The method then couples a natural language based search engine to the keyword based search engine based upon the expression based syntax by linking the natural language based search engine to the keyword based search engine.

[0010] In an alternative specific embodiment, the invention provides a method for converting an information retrieval search engine coupled to a information source into a natural language enhanced search engine. The method determines an expression based syntax of the information retrieval search engine. The information retrieval system comprises a graphical user interface coupled to a client device. The method then couples a query reformulation module to the information retrieval search engine. The query reformulation module is adapted to couple a natural language engine to the information retrieval search engine. In one embodiment, the natural language based search engine is trained with a corpus of the information source

[0011] Many benefits are achieved by way of the present invention over conventional techniques. For example, the invention allows a user to implement a natural language search engine overlying conventional search engines, without substantial modification. The invention can be applied using conventional computer software and/or hardware. In certain aspects, the invention can also provide for more directed searching to yield improved searching and the like. Depending upon the embodiment, one or more of these benefits may be achieved. These and other benefits will be described in more throughout the present specification and more particularly below.

[0012] Various additional objects, features and advantages of the present invention can be more fully appreciated with reference to the detailed description and accompanying drawings that follow.

BRIEF DESCRIPTION OF THE DRAWINGS

[0013]
FIG. 1 is a simplified diagram of an knowledge acquisition system according to an embodiment of the present invention;

[0014]
FIG. 1A is a more detailed diagram of a query reformulation system according to an embodiment of the present invention;

[0015]
FIG. 2 is a simplified flow diagram of a query reformulation method according to an embodiment of the present invention;

[0016]
FIG. 3 is a more detailed diagram of a query reformulation method according to an embodiment of the present invention;

[0017]
FIG. 4 is a more detailed diagram of a method for filtering selected non-interesting terms according to an embodiment of the present invention;

[0018]
FIG. 5 is a more detailed diagram of a method for targeted field mapping of database fields according to an embodiment of the present invention;

[0019]
FIG. 6 is a more detailed diagram of a method for adding expansion terms to a query term according to an embodiment of the present invention;

[0020]
FIG. 7 is a more detailed diagram of a method for query normalization according to an embodiment of the present invention;

[0021]
FIG. 8 is a simplified diagram of an illustration of integrating a query reformulation module onto a conventional information retrieval system according to an embodiment of the present invention;

[0022]
FIG. 9 is a simplified system diagram of an integrated query reformation module and information retrieval system according to an embodiment of the present invention;

[0023]
FIG. 9A is an example of an interface that may be used with certain aspects of the invention

[0024]
FIG. 10 is a more detailed diagram of an integrated query reformation module and information retrieval system according to an embodiment of the present invention;

[0025]
FIG. 11 is a simplified flow diagram of a method for integrating a query reformation module onto a conventional information retrieval system according to an embodiment of the present invention; and

[0026] FIGS. 12A-12E are schematic diagrams of an exemplary system described in a design and functional specification provided below.

DESCRIPTION OF THE SPECIFIC EMBODIMENTS

[0027] According to the present invention, a technique including method and system for a knowledge based technique is provided. More particularly, the invention provides an improved way of integrating a query reformulation module onto a pre-existing information retrieval system. In an exemplary embodiment, the invention provides an enhanced search technique that can be integrated into a conventional information retrieval method and system.

[0028]
FIG. 1 is a simplified diagram of a system 100 according to an embodiment of the present invention. This diagram is merely an example, which should not unduly limit the scope of the claims herein. One of ordinary skill in the art would recognize many other variations, modifications, and alternatives. As shown, the system has input 101, where a user inputs a query. The query is generally in a natural language form. The query is indicated as an input query. The input query is provided into an engine 103 to convert the natural language form into a logical form such as a “LexLF” logical form 105 designed by a company called Lexeme, Inc. of Cambridge, Mass. The logical form is preferably one that has semantic information provided into the logical form. The logical form also has key terms of the query, among other information.

[0029] The logical form is derived from an engine developed by LingoMotors, Inc. As merely an example, the engine is described in copending, commonly owned U.S. Appl. No. 09/662,510 by Robert J. P. Ingria et al., filed Sept. 15, 2000, entitled “ANSWERING USER QUERIES USING A NATURAL LANGUAGE METHOD AND SYSTEM” (“the '510 application), and in copending, commonly owned U.S. appl. Ser. No. 09/663,044 by Federica Busa et al., filed Sept. 15, 2000, entitled “NATURAL LANGUAGE TYPE SYSTEM AND METHOD” (“the '044 application”), the entire disclosures of which are herein incorporated by reference in their entireties for all purposes. The engine can also be a variety of other suitable techniques. The output of the engine is indicated as the logical form LexLF. It should be noted that the term “LexLF” is merely intended to be a term for illustration purposes which should not in any way limit the scope of the claims herein.

[0030] The query in the logical form is fed into a query reformulation module 106. As shown, the logical form LexLF is fed into the query reformulation module through connector A 107, 109. The query reformulation module performs one or more operations on the query to make the query more efficient with other information retrieval systems. The query reformulation module feeds an enhanced query 119 into an information retrieval system 121, which is coupled to a data source 124. An answer 123 is outputted from the information retrieval system. Further details of the query reformulation module are provided below.

[0031] Referring to FIG. 1A, the query reformulation module includes a filter module 109, a term expansion module 113, a targeted field information module 111, and a query normalization module 117, and other elements, if desirable. The query reformulation module receives the logical form Lex LF 108. The query reformulation module outputs an enhanced query 119. Each of these modules are coupled to each other in the configuration shown, but can also be in other configurations. Preferably, the modules are coupled to each other in the configuration shown. In some embodiments of the invention, some of the modules can also be eliminated.

[0032] The filter module can be used to identify interesting or non-interesting terms in the query. In a specific embodiment, the filter module can be used to eliminate non-interesting terms, for example. Alternatively, the filter can identify interesting terms. Preferably, the interesting terms are identified using the format of the logical expression provided above. The logical expression identifies, for example, a format and topic of the request. Further details of the filter module are provided in accordance to the Figs. described below.

[0033] The targeted database field information module couples one or more fields of the database to the query to provide a more targeted query. In a specific embodiment, the targeted database information module provides one or more or all of the fields in the database to the query reformulation module. The information module provides the one or more fields of the database. The logical expression provides, for examples, terms that will be used for the query. In a specific embodiment, if a field term matches or is the same as one of the query terms, the matched query term is ignored in the term expansion module, which is described more fully below.

[0034] The term expansion module can provide expansion of terms using sets of synonyms and others. The term expansion is preferably based upon a typing system. An example of such a typing system is described in the '044 application, which has been incorporated by reference. Preferably, the term expansion module expands those terms that are not used as field terms. Here, the concept is to provide expansion for terms that are not expressly identified as a field, which is often implicitly an important term, as identified by the creator of the database, for example. Of course, there can be other ways to expand the terms that will provide other variations to the terms for completeness.

[0035] The query normalization module receives the query, which has been filtered and expanded. The module converts the query into a form that can be processed by an information retrieval system. In a specific embodiment, the query normalization module outputs an enhanced query 119 using a keyword logic technique. For example, the query normalization module will “and” selected terms and “or” expansion terms, which are connected with the “and” to the selected terms. Of course, the type of normalization will depend upon the application.

[0036] As shown, the query reformulation module is coupled to an information retrieval system 121 in FIG. 1. The information retrieval system can be any conventional known or other system. In a specific embodiment, the information retrieval system is a keyword search system using Boolean expressions or the like. The information retrieval system is often coupled to an information source such as a database 124. The database can be any suitable unit that has information that is arranged in some type of logical manner that can be stored and retrieved. An answer 123 based upon a combination of the information retrieval system and enhanced query is output. Further details of methods according to the present system are explained according to the figures. described below.

[0037] Some of the elements can be operated in serial or in parallel manner. Alternatively, the elements can be a combination of serial and parallel operations without departing from the scope of the claims herein. Further, although the above has been described in terms of specific hardware and software features, it would be recognized that there can be many alternatives, variations, and modifications. For example, any of the above elements can be separated or combined. Alternatively, some of the elements can be implemented in software or a combination of hardware and software. Alternatively, the above elements can be further integrated in hardware or software or hardware and software or the like. It is also understood that the examples and embodiments described herein are for illustrative purposes only and that various modifications or changes in light thereof will be suggested to persons skilled in the art and are to be included within the spirit and purview of this application and scope of the appended claims.

[0038] An embodiment of a method according to the present invention may be briefly outlined as follows:

[0039] 1. Provide query in natural language format;

[0040] 2. Perform preprocessing including steps of tokenizing, tagging, and stemming of the query in an engine;

[0041] 3. Perform syntax analysis on the preprocessed query;

[0042] 4. Form a logical form (e.g., LexLF) from the syntax analysis expression;

[0043] 5. Perform filtering step to identify essential terms in query (or eliminate non-essential terms);

[0044] 6. Perform a field information operation on essential terms of the query;

[0045] 7. Perform a term expansion on each of the essential query terms;

[0046] 8. Normalize processed query to a form suitable for an information retrieval system such as Boolean;

[0047] 9. Output an enhanced Boolean expression based upon logical form;

[0048] 10. Query database information based upon enhanced Boolean expression;

[0049] 11. Identify selected information based upon enhanced query; and

[0050] 12. Perform other steps as desirable.

[0051] The above sequence of steps is an example of a way to perform aspects of the present invention. They provide a general query in natural language form. They perform a syntax analysis on the query once the query has been pre-processed. An enhanced Boolean expression is based upon the logical form to provide a more focussed or efficient query to the information retrieval system. Further details of these steps are provided in reference to the Figs. described below.

[0052]
FIG. 2 is a simplified flow diagram 200 of an enhanced query reformulation method according to an embodiment of the present invention. This diagram is merely an example, which should not unduly limit the scope of the claims herein. One of ordinary skill in the art would recognize many other variations, modifications, and alternatives. As shown, the method begins at start, step 201. The method inputs a query 203. The query is generally in a natural language form. The query is indicated as an input query. The input query is provided into an engine for processing 205 to convert the natural language form into a logical form such as the LexLF logical form designed by a company called LingoMotors, Inc. of Cambridge, Mass. The logical form is preferably one that has semantic information provided into the logical form. The logical form also has key terms of the query, among other information.

[0053] The logical form is derived from an engine developed by LingoMotors, Inc. As merely an example, the engine is described in the '510 and '044 applications, which have been incorporated by reference. The engine can also be a variety of other suitable techniques. The output of the engine is indicated as the logical form LexLF. It should be noted that the term “LexLF” is merely intended to be a term for illustration purposes which should not in any way limit the scope of the claims herein.

[0054] The query in the logical form undergoes a process of reformulation, block 207. In a specific embodiment, the reformulation process occurs in a reformulation module, such as the one noted but can be others. The query reformulation module performs one or more operations on the query to make the query more efficient with other information retrieval systems. In a specific embodiment, the query reformulation module includes a filter module, a term expansion module, a targeted field information module, and other elements, if desirable. In some embodiments of the invention, some of the modules can also be eliminated, combined, or others added. Further details of the methods performed in each of these modules are provided below.

[0055] Next, the method processes the reformulated query and normalizes (step 209) it into a format suitable for an information retrieval system. In a specific embodiment, the query normalization process outputs an enhanced query using a keyword logic technique. For example, the query normalization process will “and” selected terms and “or” expansion terms, which are connected with the “and” to the selected terms. Of course, the type of normalization will depend upon the application.

[0056] The enhanced query is processed through an information retrieval process, block 211. The information retrieval process can be any conventional known or other system. In a specific embodiment, the information retrieval process is a keyword search system using Boolean expressions or the like. The information retrieval process uses the enhanced query (block 213) to query a database. The database can be any suitable unit that has information that is arranged in some type of logical manner that can be stored and retrieved. An answer 215 based upon a combination of the information retrieval system and enhanced query is output. The method stops, block 217, once the answer is provided to the user of the method.

[0057] The above sequence of steps is merely illustrative. The steps can be performed using computer software or hardware or a combination of hardware and software. Any of the above steps can also be separated or be combined, depending upon the embodiment. In some cases, the steps can also be changed in order without limiting the scope of the invention claimed herein. One of ordinary skill in the art would recognize many other variations, modifications, and alternatives.

[0058]
FIG. 3 is a more detailed diagram 300 of a query reformulation method according to an embodiment of the present invention. This diagram is merely an example, which should not unduly limit the scope of the claims herein. One of ordinary skill in the art would recognize many other variations, modifications, and alternatives. As shown, the method begins at start, block 301. The method inputs a query, such as the one noted, as well as others. The query is generally in a natural language form. The query is indicated as an input query. Using, for example, a simple illustration of searching for specific types of books in an electronic commerce web site, such as Amazon.com, Inc. or Barnes and Noble.com, Inc., among others. A typical query may be as follows:

[0059] Query=Do you have paperback books on gardening?

[0060] The input query is provided into an engine to convert the natural language form into a logical form such as a LexLF logical form designed by a company called LingoMotors, Inc. of Cambridge, Mass. The logical form is preferably one that has semantic information provided into the logical form. The logical form also has key terms of the query, among other information.

[0061] The logical form is derived from an engine developed by LingoMotors, Inc. As merely an example, the engine is described in the '510 and '044 applications, which have been incorporated by reference. The engine can also be a variety of other suitable techniques. The output of the engine is indicated as the logical form LexLF. It should be noted that the term “LexLF” is merely intended to be a term for illustration purposes which should not in any way limit the scope of the claims herein. An example of a logical form for the above query is as follows:

[0062] LexLF: [utterance=YIN Question,

[0063] type=request for information, lexical item=have

[0064] domain=book retailer, lexical item=book

[0065] format=paperback

[0066] topic of book=gardening]

[0067] The query in the logical form undergoes a process of reformulation. In a specific embodiment, the reformulation process occurs in a reformulation module, such as the one noted but can be others. The query reformulation module performs one or more operations on the query to make the query more efficient with other information retrieval systems. In a specific embodiment, the query reformulation module includes a filter module, a term expansion module, a targeted field information module, and other elements, if desirable. In some embodiments of the invention, some of the modules can also be eliminated, combined, or others added. Further details of the methods performed in each of these modules are provided below.

[0068] In a specific embodiment, the method performs a filter process, block 303, on the logical form. The filter process can be used to identify interesting or non-interesting terms in the query. In a specific embodiment, the filter process can be used to eliminate non-interesting terms. Alternatively, the filter process can identify interesting terms. Preferably, the interesting terms are identified using the format of the logical expression provided above. The logical expression identifies, for example, a format and topic of the request. An example of a filtered query would yield the following expressions from the logical form.

[0069] Format=paperback

[0070] Topic=gardening

[0071] Next, the method performs a field information process, block 305. In a specific embodiment, the targeted database information process provides one or more or all of the fields in the database to the query reformulation process. The information process provides the one or more fields of the database. The logical expression provides, for examples, terms that will be used for the query. In a specific embodiment, if a field term matches or is the same as one of the query terms, the matched query term is ignored in the term expansion process, which is described more fully below. As merely an example, the database fields that were identified in the query have been highlighted in bold below.

[0072] [ utterance=YIN Question type=request for information

[0073] lexical item =have

[0074] domain=book retailer

[0075] lexical item=book

[0076] format=paperback

[0077] topic of book=gardening]

[0078] As shown, the fields include, for example, “domain=book retailer, lexical item=book, format=paperback.” Next, the method performs a term expansion process (block 307) to expand selected terms that have not been identified as field terms. The term expansion process can provide expansion of terms using sets of synonyms and others. The term expansion is preferably based upon a typing method. An example of such a typing method is described in the '044 application, which has been incorporated by reference. Preferably, the term expansion method expands or finds alternative terms for those terms that are not used as field terms. Here, the concept is to provide expansion for terms that are not expressly identified as a field, which is often implicitly an important term, as identified by the creator of the database, for example. Of course, there can be other ways to expand the terms that will provide other variations to the terms for completeness. Using the above example, the term “gardening” has been expanded to include the following other expressions.

[0079] Topic=gardening (not a database field)

[0080] Expanded gardening to also include “horticulture, landscaping, floriculture.”

[0081] Next, the method processes the reformulated query and normalizes (block 309) it into a format suitable for an information retrieval system. In a specific embodiment, the query normalization process outputs an enhanced query using a keyword logic technique. For example, the query normalization process will “and” selected terms and “or” expansion terms, which are connected with the “and” to the selected terms. Of course, the type of normalization will depend upon the application. Using again, the above example, the original query has been converted into an enhanced Boolean expression, which will be used in a conventional information retrieval method.

[0082] Book retailer and paperback and (gardening or horticulture or landscaping or floriculture)

[0083] The enhanced query is processed through an information retrieval process. The information retrieval process can be any conventional known or other system. In a specific embodiment, the information retrieval process is a keyword search system using Boolean expressions or the like. The information retrieval process uses the enhanced query to query a database. The database can be any suitable unit that has information that is arranged in some type of logical manner that can be stored and retrieved. An answer based upon a combination of the information retrieval system and enhanced query is output. The method stops, block 311, once the answer is provided to the user of the method.

[0084] The above sequence of steps is merely illustrative. The steps can be performed using computer software or hardware or a combination of hardware and software. Any of the above steps can also be separated or be combined, depending upon the embodiment. In some cases, the steps can also be changed in order without limiting the scope of the invention claimed herein. One of ordinary skill in the art would recognize many other variations, modifications, and alternatives.

[0085]
FIG. 4 is a more detailed diagram 400 of a method for filtering selected non-interesting terms according to an embodiment of the present invention. This diagram is merely an example, which should not unduly limit the scope of the claims herein. One of ordinary skill in the art would recognize many other variations, modifications, and alternatives. The present method can include a filter process, 400. In a specific embodiment, the method performs a filter process, block 401, on a logical form such as the one described herein or others. The filter process can be used to identify interesting or non-interesting terms in the query. In a specific embodiment, the filter process can be used to eliminate non-interesting terms 403. Alternatively, the filter process can identify interesting terms 404. Preferably, the interesting terms are identified using the format of the logical expression provided above. The filter process can identify or eliminate non-interesting terms from a listing 403 of non-interesting terms, which are provided by a user. For example, the listing can be a “not” list for terms which are eliminated. As merely an example, the not list can include terms such as “looking for,” “where,” “find,” and other conventional query stop words, but also contextually identified terms as a result of linguistic processing of the query, all of which are shown for illustrative purposes only. Alternatively or in combination, the filter process can identify interesting or non-interesting terms based upon the terms identified by the logical expression, such as the example above. Depending upon the embodiment, there can be other ways to filter the logical form.

[0086] The above sequence of steps is merely illustrative. The steps can be performed using computer software or hardware or a combination of hardware and software. Any of the above steps can also be separated or be combined, depending upon the embodiment. In some cases, the steps can also be changed in order without limiting the scope of the invention claimed herein. One of ordinary skill in the art would recognize many other variations, modifications, and alternatives.

[0087]
FIG. 5 is a more detailed diagram 500 of a method for targeted field-information according to an embodiment of the present invention. This diagram is merely an example, which should not unduly limit the scope of the claims herein. One of ordinary skill in the art would recognize many other variations, modifications, and alternatives. In a specific embodiment, the method performs a field information process 500. In a specific embodiment, the method derives field information 501 from a database 503. The field information includes one or more or all of the fields in the database. The fields of the database are processed (block 507) with the terms provided by a logical form 505, which has been derived from an engine and query. Here, the terms in the logical form may be a starting point for terms to be used for the enhanced query. In a specific embodiment, if a field term matches or is the same as one of the query terms, the matched query term is ignored in the term expansion process, which is described more fully below.

[0088] Again, the above sequence of steps is merely illustrative. The steps can be performed using computer software or hardware or a combination of hardware and software. Any of the above steps can also be separated or be combined, depending upon the embodiment. In some cases, the steps can also be changed in order without limiting the scope of the invention claimed herein. One of ordinary skill in the art would recognize many other variations, modifications, and alternatives.

[0089]
FIG. 6 is a more detailed diagram 600 of a method for adding expansions to a query term according to an embodiment of the present invention. This diagram is merely an example, which should not unduly limit the scope of the claims herein. One of ordinary skill in the art would recognize many other variations, modifications, and alternatives. The method performs a term expansion process 600 to expand selected terms 601 that have not been identified as field terms. The term expansion process can provide expansion of terms using sets of synonyms (block 603) and others, which are derived from a library. The term expansion can also be based upon a typing method, block 605, which can be combined with synonyms. An example of such a typing method is described in the '044 application, which has been incorporated by reference. Preferably, the term expansion method expands or finds alternative terms 607 for those terms that are not used as field terms. Here, the concept is to provide expansion for terms that are not expressly identified as a field, which is often implicitly an important term, as identified by the creator of the database, for example. Of course, there can be other ways to expand the terms that will provide other variations to the terms for completeness.

[0090] The above sequence of steps is merely illustrative. The steps can be performed using computer software or hardware or a combination of hardware and software. Any of the above steps can also be separated or be combined, depending upon the embodiment. In some cases, the steps can also be changed in order without limiting the scope of the invention claimed herein. One of ordinary skill in the art would recognize many other variations, modifications, and alternatives.

[0091]
FIG. 7 is a more detailed diagram 700 of a method for query normalization according to an embodiment of the present invention. This diagram is merely an example, which should not unduly limit the scope of the claims herein. One of ordinary skill in the art would recognize many other variations, modifications, and alternatives. In a specific embodiment, the method processes the reformulated query in a logical form 705 and normalizes (block 701) it into a format suitable for an information retrieval method, block 703. In a specific embodiment, the query normalization process outputs an enhanced query (block 707) using a keyword logic technique. For example, the query normalization process will “and” selected terms and “or” expansion terms, which are connected with the “and” to the selected terms. Of course, the type of normalization will depend upon the application.

[0092] The above sequence of steps is merely illustrative. The steps can be performed using computer software or hardware or a combination of hardware and software. Any of the above steps can also be separated or be combined, depending upon the embodiment. In some cases, the steps can also be changed in order without limiting the scope of the invention claimed herein. One of ordinary skill in the art would recognize many other variations, modifications, and alternatives.

[0093] A method according to an embodiment of the present invention for integrating a query reformation module onto an information retrieval system is provided as follows.

[0094] (1) Provide information retrieval (“IR”) system which is coupled to an information source comprising corpus from a customer;

[0095] (2) Determine syntax expression used by the IR system;

[0096] (3) Convert user interface box to make it larger to input natural language input expression;

[0097] (4) Identify database fields from information source that are desirable (e.g., important) for the customer;

[0098] (5) Integrate query reformulation module onto information retrieval system;

[0099] (6) Train query reformulation module with the corpus of the information source from customer;

[0100] (7) Train the filter (e.g., non-interesting terms) in the query reformulation module on query set from the customer; and

[0101] (8) Perform other steps, as desirable.

[0102] The above sequence of steps is used to integrate a query reformulation module onto a conventional information retrieval system. These steps can provide, for example, an enhanced query, which is more accurate and retrieves more selected information. Additionally, the steps are easy to implement and can be used with any conventional technique. Further details of these steps are provided throughout the present specification and more particularly below according to the following figures.

[0103]
FIG. 8 is a simplified diagram 800 of an illustration of integrating a query reformulation module onto a conventional information retrieval system according to an embodiment of the present invention. This diagram is merely an example, which should not unduly limit the scope of the claims herein. One of ordinary skill in the art would recognize many other variations, modifications, and alternatives. As shown, the diagram 800 has an information retrieval system 801, which is coupled to a variety of information sources. Here, the information sources can include a relational database 811, the Internet 819, and text database 817. The information retrieval system 801 is coupled to relational database 811 via line 809. The information retrieval system 801 is coupled to the Internet via line 813. The information retrieval system 801 is coupled to text database via line 815. These lines are provided for illustrative purposes only. The lines can be in the form of hardware such as a hardwire or wireless or a combination of hardwire and wireless.

[0104] A user sends a query 805 from client 803 to the information retrieval system. An answer from the information retrieval system 801 is provided to client 803 via line 807. The client 803 can be a personal computer, a workstation, a mobile communication device, a personal digital assistant, and other client devices. Before integrating a query reformulation module onto the information retrieval system, it is desirable to obtain the following parameters. For example, the method should determine the syntax expression used by the information retrieval system. Here, the output of the query reformulation module would need to provide an enhanced expression in the syntax used by the information retrieval system. Additionally, other parameters that may be useful would be the database fields and knowledge of the corpus of the information source. Knowledge of specific database fields allows for directed identification and extraction of that information from the query. Similarly, training the engine and lexicon on the domain corpus enables more exact and targeted identification of keywords and their reformulations. Further details of integrating such query reformulation module onto the information retrieval system are provided throughout the present specification and more particularly below.

[0105] Although the above has been described in terms of specific hardware and software features, it would be recognized that there can be many alternatives, variations, and modifications. For example, any of the above elements can be separated or combined. Alternatively, some of the elements can be implemented in software or a combination of hardware and software. Alternatively, the above elements can be further integrated in hardware or software or hardware and software or the like.

[0106]
FIG. 9 is a simplified system diagram of an integrated query reformation module and information retrieval system 900 according to an embodiment of the present invention. This diagram is merely an example, which should not unduly limit the scope of the claims herein. One of ordinary skill in the art would recognize many other variations, modifications, and alternatives. As shown, the diagram includes an information source 901, which is a database. An information retrieval system 903 is coupled to the database. The information retrieval system 903 can be any conventional known or other system. In a specific embodiment, the information retrieval system is a keyword search system using Boolean expressions or the like. A query reformulation module 905 couples to the information retrieval system. The query reformulation module takes a natural language query, reformulates it, and sends it to the information retrieval system. The natural language query is provided by a user interface 907. As merely an example, the user interface is larger in size to take on a natural language query. An example of such a user interface is provided in FIG. 9A, for example.

[0107] Although the above has been described in terms of specific hardware and software features, it would be recognized that there can be many alternatives, variations, and modifications. For example, any of the above elements can be separated or combined. Alternatively, some of the elements can be implemented in software or a combination of hardware and software. Alternatively, the above elements can be further integrated in hardware or software or hardware and software or the like.

[0108]
FIG. 10 is a simplified flow diagram 1000 of a method for integrating a query reformation module to a conventional information retrieval system according to an embodiment of the present invention. This diagram is merely an example, which should not unduly limit the scope of the claims herein. One of ordinary skill in the art would recognize many other variations, modifications, and alternatives. As shown, the method includes providing an information retrieval (“IR”) system 1003 which is coupled to an information source 1005 from a customer. The IR system is coupled to user interface 1001.

[0109] Before integrating 1007 a query reformulation module onto the information retrieval system, it is desirable to have certain parameters identified. For example, the method determines 1015 the syntax expression used by the IR system. The method also determines the size 1017 of the text box for the user interface and converts the user interface box to make it larger to input a natural language input expression. The method also identifies database fields 1019 from information source that are desirable (e.g., important) for the customer.

[0110] Next, the method integrates 1007 a query reformulation module 1011 onto information retrieval system. The integrated system includes an improved user interface 1009 coupled to a query reformulation module 1011, which is coupled to the information retrieval system 1003. The information retrieval system is coupled to database 1005.

[0111] The method then performs selected training steps to enhance operation of the integrated system. Here, the method trains 1021 the query reformulation module with the corpus of the information source from customer. Next, the method trains 1023 the filter (e.g., non-interesting terms) in the query reformulation module on query set from the customer. The Example provided below illustrates various features of the method.

[0112] Although the above has been described in terms of specific hardware and software features, it would be recognized that there can be many alternatives, variations, and modifications. For example, any of the above elements can be separated or combined. Alternatively, some of the elements can be implemented in software or a combination of hardware and software. Alternatively, the above elements can be further integrated in hardware or software or hardware and software or the like.

[0113]
FIG. 11 is a more detailed diagram of an integrated a query reformation module and a conventional information retrieval system 1100 according to an embodiment of the present invention. This diagram is merely an example, which should not unduly limit the scope of the claims herein. One of ordinary skill in the art would recognize many other variations, alternatives, and modifications. As shown, the diagram includes an information source 1101, which is a database. An information retrieval system 1103 is coupled to the database. The information retrieval system can be any conventional known or other system. In a specific embodiment, the information retrieval system is a keyword search system using Boolean expressions or the like. A query reformulation module 1105 couples to the information retrieval system. The query reformulation module takes a natural language query, reformulates it, and sends it to the information retrieval system. The query reformulation module includes a normalization module 1109, which provides the enhanced expression in the proper syntax for the information retrieval system. The natural language query is provided by a user interface 1107. As merely an example, the user interface is larger 1113 in size to take on a natural language query. The Example provided below illustrates various features of the method

[0114] Although the above has been described in terms of specific hardware and software features, it would be recognized that there can be many alternatives, variations, and modifications. For example, any of the above elements can be separated or combined. Alternatively, some of the elements can be implemented in software or a combination of hardware and software. Alternatively, the above elements can be further integrated in hardware or software or hardware and software or the like.

EXAMPLE

[0115] To prove the principle and operation of the present invention, we have prepared computer code and implemented the present invention using a database including information for books. The invention as implemented is described in the following design and functional specification. This design and functional specification is merely an example and should not limit the scope of the claims herein. One of ordinary skill in the art would recognize many variations, alternatives, and modifications.

1. Overview

[0116] LingoMotors's TurboSearch enhances conventional search systems by adding language understanding capability. Users can enter a question in ordinary language, and this is translated into a keyword query, in a Boolean format. There are three primary ways in which TurboSearch improves upon keyword search or literal Boolean search:

[0117] A. Ordinary Language Input—users can comfortably type in English. TurboSearch determines which words are part of the key concepts in the question, and which are contextual. Contextual words and phrases, which are useful for understanding queries include: stop phrases (e.g., ‘1 am interested in . . . ’) and function vocabulary (e.g., by, for, etc.). These are used by the engine to build more accurate semantic representations, but they are hidden to the user and are not included in the Boolean format since, as literal words, they add noise and substantially reduce the quality of search.

[0118] B. Field Identification—TurboSearch finds phrases and constructions that map to specific database fields accessible to the target search engine. This allows highly relevant search without requiring the user to work with complex templates. Field recognition uses both syntactic forms and semantic understanding. The specific fields to be identified and the parameters involved are dependent on the application and the search engine's database; these are developed and tested as part of the customization of TurboSearch for a given application.

[0119] C. Term expansion—TurboSearch expands the key concepts in a question into synonym sets (called “synsets”) which are input into the existing search engine. Stop phrases and other contextual words are not expanded but used to enhance interpretation and identification of key concepts. According to the type of key concept, there are distinct ways of creating an expansion. Geographical areas are expanded into locales; other expansions are provided as required for a particular application.

[0120] TurboSearch is a part of an overall system including a search engine, database, and web server. Multiple copies of all components are to be deployed across multiple locations, monitored from multiple Network Operations Centers (NOCs).

[0121] Features of the current version include:

[0122] A. Field Identification

[0123] a. Enhancements to existing fields (Contributor, Format)

[0124] b. New fields (Price, Pub Date)

[0125] B. Stop Phrases

[0126] a. Substantially expanded set of stop phrases

[0127] b. Enhanced testing for consistency & feature interaction

[0128] C. Term Expansion

[0129] a. Tuning “knobs” for synset & locale expansion

[0130] b. Major LingoNet enhancements, tuning and data cleanup

[0131] D. Platform & Performance tuning

[0132] a. Support for Microsoft DataCenter

[0133] b. Performance enhancements to reduce hardware requirements

[0134] E. System Management

[0135] a. Additional Logging and Alarms

[0136] b. Implementation of initial Reports & Stats

[0137] c. Network Management integration

[0138] F. Linguistic feature enhancements

[0139] a. “ing” form improvements

[0140] b. improved resolution of author name/common noun ambiguity

[0141] G. Vocabulary buildup

[0142] a. Domain-specific knowledge acquisition

[0143] b. Substantial tuning and data cleanup

[0144] H. Platform & Performance tuning

[0145] a. Support for Windows 2000

[0146] b. Performance optimization to reduce average and maximum latency

[0147] c. Database enhancements

[0148] d. Load balancing tuning, out-of-service service capability, and performance improvement

[0149] I. New and enhanced Tools

[0150] a. Log analysis

[0151] b. Knowledge acquisition

[0152] c. Version control

[0153] 2. System Context

[0154] TurboSearch is an “add-on” to a conventional search engine. It converts “questions” (either in natural language or in keyword language) into “queries” in annotated Boolean format. The search engine and TurboSearch are deployed as separate components (typically on separate servers), with an XML-RPC interface between them. A schematic figure illustrating the relationship between the Search Engine and TurboSearch Engine is provided in FIG. 8A.

[0155] One deployment includes four sets of servers: Web Servers (hosting both the front-end web pages and a COM object that encapsulates communication), Search Servers, TurboSearch Servers, and Database Servers. The flow of control is shown schematically in FIG. 8B. The Search and TurboSearch components work together to generate a set of answers in the form of product IDs and category IDs; detailed information (book descriptions, cover pictures, etc.) is then fetched from the database.

[0156] The Search, TurboSearch, and Database components are stateless. Sets of servers are deployed across multiple data centers with load balancing hardware between them. Two successive queries from a given end user would typically go to different servers, with session context kept only on the front end.

3. High-Level Architecture

[0157] The high-level architecture for TurboSearch is shown schematically in FIG. 8C. Questions are distilled into “terminal” form through sophisticated linguistic processing that reduces words and phrases to their root forms and identifies their part of speech in context. Powerful proprietary techniques are used to interpret the meaning of the question, and distill it into one or more parses, each of which contains the key concepts and connections between them. These parses are then used to identify fields (specific kinds of concepts in specific connections), stop phrases, and terms to be expanded; the resulting query is formatted and returned to the search engine.

[0158] LingoMotors' linguistic understanding technology is “lexically driven”. Numerous interrelated dictionaries, thesauri, and ontologies are used in the course of processing each question. These are collectively termed “knowledge resources”; they are built using a sophisticated toolset and knowledge acquisition process.

[0159] 4. Interface Description

[0160] TurboSearch has three points of interface, as shown schematically in FIG. 8D.

[0161] A. Query API—this uses XML-RPC to carry a question from a Search Engine to TurboSearch and return the reformulated query.

[0162] B. System Management API—this provides system configuration, software management, and reporting capabilities. This API is typically used by LingoMotors to provide system management on a hosted basis, but may be used by system management staff for self-hosting customers. (Note that this does not replace the application service functions provided by LingoMotors).

[0163] C. Database exchange API—application data is used to enhance and test the Knowledge Resources within TurboSearch. The database exchange may be in nearly any format. This is not a real-time interface; periodic updates are used to keep the system “fresh”.

[0164] A. Query API—Contents

[0165] The question input to TurboSearch is a string in ordinary language, as typed by the user. For example, the following are typical questions:

[0166] “I'm interested in essays on sailing”

[0167] “looking for something to help with a headache”

[0168] “Show me nonfiction books by Isaac Asimov”

[0169] “laptops with large screens under 7 pounds”

[0170] The resulting reformulated query is in Boolean syntax. Key concepts are expanded to synsets, fields are identified, and contextual vocabulary is used but not passed through to the reformulated query. Examples of reformulated queries are:

[0171] [essay story writing] & ([sailing navigation]|[sailing gliding soaring]))

[0172] [medicine medication drug ]& [headache migraine]

[0173] [nonfiction “nonfictional prose” article]& (<author “Isaac Asimov” >)

[0174] [laptop “portable computer”] & (<screen large>) & (<weight under 7 lb>)

[0175] There may be several synsets for each key concept. For example, for the term sail, there might be three meanings (cruise, navigate, canvas) without the contextual words to choose between them. TurboSearch will then include all potentially relevant synsets in the reformulated query. So a question “sail” would result in a reformulated query like:

[0176] ([sail canvas “canvas sheet” ] |[sail navigate] I [sail cruise])

[0177] Applications typically pass the query directly to TurboSearch, although some preprocessing may be provided if desired. Examples of application preprocessing are spell checking, wildcard expansion, user context expansion, and domain tagging. Some preprocessing may result in sets within the input, while some may result in XML tags being included with the question. An example of a set input is:

[0178] User question: “essays on sail*”

[0179] Application wild-card expansion: “essays on [sail sailor sailing sailboat]”

[0180] B. Query API—Syntax

[0181] TurboSearch can produce nearly any Boolean syntax as required by specific search engines. Reformulation can also be done directly to SQL if desired. An example of this syntax is shown below:

[0182] [] to contain a synset or other term expansion. All words and phrases contained within these brackets should be considered OR'd for a search

[0183] ( ) to delimit components and shape precedence

[0184] {} to delimit fields

[0185] | for Boolean OR (disjunction)—items matching on either part are an overall match

[0186] & for Boolean AND (conjunction)—items must match both parts to be an overall match

[0187] ˜ for negation—ANDNOT (applies to the whole set contained in parenthesis)

[0188] <field values> to show field identification within the string

[0189] “” to indicate phrases containing multiple words

[0190] space implied AND

[0191] This syntax supports reformulation of any input query. For an example question of “cheap hotels”, term expansion on each of these words would bring back the following in addition to the original question: “inexpensive hotels”, “inexpensive inns”, “inexpensive hostels”, “cheap inns”, and “cheap hostels”. Collapsing these into a Boolean expression would give us: “(cheap OR inexpensive) AND (hotel OR inn OR hostel)”. In the syntax described above, this would be:

[0192] ([cheap inexpensive] & [hotel inn hostel])

[0193] C. Query API—XML-RPC Format

[0194] This TurboSearch design is stateless, so each query-response pair stands on its own. Moreover, the Query API consists of exchanges of XML documents: a query is an XML document that is well-formed and validated against query.dtd; and a response is an XML document that is well-formed and validated against response.dtd. The query need not be known for the application to understand and process the response, and the previous response needn't be known for TurboSearch to understand and process the next question.

[0195] The Query API is based on XML-RPC, so each query is an RPC call which contains an XML document which contains the question. The XML-RPC protocol is a simple means of remote procedure calling that works over the Internet, or any Intranet or Extranet.

[0196] An XML-RPC message is an HTTP-POST request. The body of the request is an XML document. A procedure executes on TurboSearch and it returns a formatted XML document as a response. Each request has a transaction id generated by the application. In addition, for some applications additional XML tags may be used (for example, to identify user context or domain).

[0197] The simple example below shows an XML document used to structure a question, within an XML-RPC call:

1***POST /RPC2 HTTP/1.0***Mozilla/4.0 (compatible; MSIE 5.01; Windows NT)***Host: xmlrpc.lexeme.com***Content-Type: text/xml***Content-length: 340******<?xml version=“1.0” standalone=“yes”?>***<!DOCTYPE RESPONSE SYSTEM “LingoMotorsQuery.dtd”>***<methodCall>***<methodName>LingoMotorsEngine.forTransactionId:processQuery:</methodName>***<params>***<param><value><int>12345</int></value></param>***<param>How do I prepare bouillabaise?</param>***</params>***</methodCall>The response for this would be the following example:***HTTP/1.1200 OK***Connection: close***Content-type: text/xml***Content-length: 270***Date: Fri, 28 Jul 2000 19:55:07 GMT***Server: xmlrpc.lexeme.com Microsoft-IIS/5.0******<?xml version=“1.0” standalone=“yes”?>***<!DOCTYPE RESPONSE SYSTEM “QrelResponse.dtd”>***<methodResponse>***<params>***<param><value><int>12345</int></value></param>***<param>([prepare cook fix] & [bouillabaisse “fish stew” ])</param>***</params>***</methodResponse>

[0198] 5. Field Identification

[0199] The Search Engine may accommodate a large variety of fields as inputs. Some of these fields may be linguistically derived by TurboSearch; others may be reserved for future use from either a GUI or from TurboSearch.

[0200] The following table summarizes the fields derived by TurboSearch in the current version:

2SearchTypeQuery Element FormatNotesContri-{contributor foo}A Contributor may be used asbutoreither an author or publisherPrice{price ########}All prices are represented asrangesFormat{format paperback}paperback, hardcover. . .aretranslated into{format hardcover}internal codes inside the engine:{format audio}paperback = TP | MM{format ebooks}hardcover = HC{format calendars}ebooks = EA | EB | ED | GB{format largeprint}calendars = C | CA | DK | PG | WLNew{new Y}Audi-{aud_code juvenile}ence{aud_code youngadult}

[0201] A. Contributor

[0202] This feature identifies contributors—people and organizations who are authors or editors—and maps them to the customer's Contributor Field. Turbo Search 1.1 improves what the system returns for queries which are ambiguous between identifying a contributor and specifying other search terms. Examples of phrases which should be recognized as contributors include: books from Oxford University Press→{contributor “oxford university press”}, the works of Jane Austen→{contributor “jane austen”}, books by Jane Austen→ {contributor “jane austen”}and Stephen King's thrillers→{contributor “Stephen king”} & [thrillers thriller].

[0203] B. Format

[0204] The system may allow users to search for books of specific format, e.g., ‘hardcover’ or ‘paperback’. This version of Turbo Search will map English language format expressions to the format field. The goal is to recognize and flag unambiguous expressions of format that are clearly defined in TurboSearch. This version will recognize these expressions and return the format field only, as opposed to returning the format field in disjunction with format term expansion. The treatment of expressions of format which are not explicitly defined as ambiguous will include either term-expansion or term-expansion disjoined with format field identification.

[0205] C. Price

[0206] This field maps English language price expressions to the price field. In this version, only expressions involving explicit numerical values are included. It will, however also recognize modifier adverbs such as under in under five dollars, monetary nouns, such as dollar and verbs expressing price information, such as cost.

[0207] Examples:

[0208] ‘I want a book under 5 dollars’ {price 0ˆ 4.99}

[0209] ‘I want a book over 5 dollars’ (unlikely query) {price 5ˆ inf}

[0210] ‘I want books around 5 dollars’/‘I want a 5 dollar book’—give back arithmetic value {price 5}

[0211] ‘I want a book between 5 and 10 dollars’{price 5ˆ 10}

[0212] D. Pub Date

[0213] This field will allow users to search for new or recently published books, defined as having a publishing date within the past six months. The goal is to recognize and flag user queries that refer to new books and recent editions. The qualifying terms for a specified pubdate field identification of RECENT are ‘new’, ‘newest’, ‘latest’, and ‘recent’.

[0214] E. Audience

[0215] This field will identify only “juvenile” and “youngadult”. Desired result: {audience juvenile} or {audience youngadult}. This field will be used to search for books with these codes in their database.

[0216]

6
. Stop Phrases

[0217] In the current version the list of stop phrases is substantially expanded to further improve on natural language processing. Since the stop phrases are ignored by the system, this expansion allows for many more phrasings by the user, thus making the system even more versatile. However, there will always be many ways to phrase a request and in order to better and more efficiently process this information, this version of TurboSearch introduces pattern matching to the stop phrase feature. Rather than having to exactly match the phrase the user types in, this version can recognize a variety of similar phrasings, making the system more robust as well as more quickly scaleable.

[0218]

7
. Linguistic Features

[0219] Features in the current version include:

[0220] A. Simultaneous adjective and relative clause interpretation; implementing this facility also allows for modifiction by multiple event adjectives and multiple (aka “stacked”) relative clauses

[0221] B. Distinctions between different types of possessive semantics

[0222] C. Full functionality of relative clauses (subject, object, adjunct relatives, and reduced relatives, with/without complementizers)

[0223] D. Generalized “Display This” functionality through Information Builders (general rules replacing long and inevitably incomplete lists of ignorable “stop phrases” that often begin user queries

[0224] E. Full qualia and argument binding capability for event nominals; in particular, binding of theme arguments of a head by a preceding nominal modifier (as in “sword swallowing”)

[0225]

8
. Vocabulary Building

[0226] We distinguish the following three categories of vocabulary:

[0227] A. Content vocabulary: these are the words and phrases that are semantically meaningful on their own, and have entries in our lexicon. (Ex: ‘children’, ‘France’, ‘restaurants’).

[0228] B. Function vocabulary: These words have no well-defined meaning on their own, but they have well-defined database semantics for selecting specific fields. (Ex: ‘in’, ‘by’, ‘during’, ‘books about’, ‘books by’)

[0229] C. Stop vocabulary: These words and phrases are ignored by the system. (Ex: ‘tell me about’, pronouns such as ‘you’, ‘me’, most prepositions, and other requesting or interrogative (questioning) phrases, such as “Do you have”, “where can I find”, and so on.)

[0230] TurboSearch uses these distinctions and the meaning of sentences to expand content vocabulary, discard stop vocabulary, and use function vocabulary in field identification or term expansion.

[0231] The vocabulary is built semi-automatically from product catalogs.

[0232]

9
. Performance and Operational Specs

[0233] TurboSearch is expected to have the following performance characteristics:

[0234] A. Latency (question to query time):

[0235] Average latency 200 milliseconds

[0236] <1% of queries to exceed 500 milliseconds

[0237] B. Throughput (queries per second at peak):

[0238] Total system throughput—100 queries per second

[0239] C. System Availability:

[0240] No downtime (24×7 operation)

[0241] 2 or more disparate data centers (one being lights-out, one DC should be able to handle load)

[0242] D. Startup, reload time and updates:

[0243] Easy frequent reload of new versions with minimal impact to site performance

[0244] Ability to perform quick update of software (under 1 hour)

[0245] Ability to upgrade OS with minimal impact to site performance

[0246] Quick startup time (under 10 minutes)

[0247] 10. Production Architecture

[0248] The diagram is FIG. 8E shows schematically the production (process) architecture of TurboSearch version 1.1. Each machine is a multi-processor computer running Windows 2000. Multiple copies of the TurboSearch process are used on a single machine, sharing a multi-threaded copy of LingoNet. LingoNet is instantiated as an Oracle database, and Oracle native facilities are used for updates, caching and thread management. A load balancer process allows the machine to appear as a single port to the search engine, and also allows for process control within the machine.

[0249] 11. Installation and Packaging

[0250] The components available for installation on the TurboSearch Installation CD are:

[0251] A. Full Product Install (Recommended).

[0252] B. TurboSearch Services. Use this option to install only the TurboSearch engine and use a currently installed version of the LingoNet database.

[0253] C. LingoNet Database. Use this option to install only the LingoNet database and use an existing version of the TurboSearch engine.

[0254] 12. System Management

[0255] The current TurboSearch version is hosted in Windows 2000, and uses the windows event log to report exceptions and error conditions. The following system management constructs are included in this version.

[0256] A. System Management primitives

[0257] a. startup

[0258] b. shutdown/restart

[0259] c. load new code image

[0260] d. load new database image (database machines only)

[0261] B. Logging -using Windows event log

[0262] a. Exceptions, alarms, startup/shutdown messages

[0263] C. Logging—using local file logs

[0264] a. Questions

[0265] b. Queries (reformulations)

[0266] c. Detailed traces/diagnostic output as appropriate

[0267] D. Engine statistics reporting

[0268] a. Query number, average throughput, max throughput, latency

[0269] b. Exceptions - count, type breakdown

[0270] c. Answer statistics - count, averages, type breakdown

[0271] d. Current size (lexical entries, types, memory)

[0272] e. Performance (average speed breakdown, machine utilization)

[0273] E. Database statistics reporting

[0274] a. Include base statistics akin to engine reporting

[0275] b. Current size (synsets, constituents)

[0276] c. Performance (average speed breakdown, machine utilization)

[0277] 13. User Profiles

[0278] The following types of users are expected:

[0279] A. Users

[0280] The users of the system are online customers interested in locating and buying a book.

[0281] These customers are:

[0282] Searching for a particular book

[0283] Looking for particular information

[0284] Browsing for general information

[0285] B. Anticipated Purchasing Behavior

[0286] We anticipate purchase behavior to be one of three modes:

[0287] a. Predetermined—This consumer has an idea of the title, the author, or the subject of the book(s) she wants to buy. She may be in a hurry; she may be buying a gift.

[0288] b. Browse-based—This consumer has more time and may be enticed into browsing many topics that may be unrelated and buying books that were not on his original list.

[0289] c. Impulse—This consumer has a buying profile and responds to suggestions such as best sellers of a certain category, other titles on a similar subject, and new books by a favored author.

[0290] 14. Metrics

[0291] The goals of metrics are:

[0292] Steer ongoing development and maintenance

[0293] Show improvement over conventional systems

[0294] Connect that improvement to business value

[0295] During version 1.1 development, only two search & navigation metrics will be in place: % Correct Reformulation, and First Page Relevance. However, a variety of additional measurements could be used in the deployment of TurboSearch, either simultaneous with TurboSearch deployment or at a later time. These include:

[0296] Business metrics

[0297] Customer Experience metrics

[0298] Search & Navigation metrics

[0299] Systems and operations metrics

[0300] Internal systems metrics

[0301] A. Businness Metrics

[0302] The ultimate goal of implementing new technology is higher business value, measured in dollars. (None of these metrics are part of the system per se, but they are described here because their implementation may occur concurrently with a deployment.)

[0303] Typical business metrics include: Conversion rate, Abandonment rate, Transaction rate, and Sales per transaction.

[0304] Common supporting measurements are: Number of Visits, Number of new customers, Number of repeat visits, Subjective customer loyalty, Brand awareness.

[0305] B. Customer Exerience Metrics

[0306] Overall, the goal is to provide a “knock-your-socks-off” customer impression, with high customer satisfaction. Typically, this is measured through site ratings, customer experience surveys and/or usability studies.

[0307] These may include the following measurements. Subjective ease of use, Overall Customer experience, Impulse purchases, Time spent shopping, Time to first relevant result, Average time to first selection.

[0308] C. Search & Navigation Metrics

[0309] The high-level goals for search and navigation quality are:

[0310] A. “Home run” results—numerous examples of “wow” answers to example questions

[0311] B. Better results than competitive sites on a selected set of “typical” queries

[0312] C. Good consistency—does not return “bizarre” answers

[0313] Two search & navigation metrics will be in place during the current version development:

[0314] A. % Correct Reformulation—this is a score of TurboSearch's output across a reference set of user queries. It will be manually scored at periodic points during development.

[0315] B. First Page Relevance—this is a score across a deployed system including the search engine and front end. The percentage of relevant answers for a reference set of queries will be manually scored across the development system and competitive sites.

[0316] Although the design and functional description is described in terms of specific hardware features, it would be recognized that there can be many alternatives, variations, and modifications. For example, any of the elements described above can be separated or combined. Alternatively, some of the elements can be implemented in software or a combination of hardware and software. Alternatively, the above elements can be further integrated in hardware or software or hardware and software or the like. It is also understood that the examples and embodiments described herein are for illustrative purposes only and that various modifications or changes in light thereof will be suggested to persons skilled in the art and are to be included within the spirit and purview of this application and scope of the appended claims.

[0317] It is understood that the examples and embodiments described herein are for illustrative purposes only and that various modifications or changes in light thereof will be suggested to persons skilled in the art and are to be included within the spirit and purview of this application and scope of the appended claims.

Method and resulting system for integrating a query reformation module onto an information retrieval system

Information

Publication Number

Date Filed

Date Published

Inventors

Original Assignees

CPC

US Classifications

International Classifications

Abstract

Description

Claims

CROSS REFERENCES TO RELATED APPLICATIONS

Provisional Applications (1)