Method and apparatus for construction and use of concept knowledge base

Abstract

A data structure, apparatuses and methods for expanding a search query to be used by a web search engine is provided. The search query is expanded by accessing a concept knowledge base data structure having concept data objects and term data objects with each term data object defining a term and associated with at least one of the concept data objects. Search terms making up the search query are matched to term data objects and a concept set is generated containing concept data objects associated with the term data objects. A second set of term data objects are generated by using the concept data objects to locate term data objects associated with the concept data objects. A user can then select one of the term data objects in the second set to expand the user's search query.

Description

DESCRIPTION OF THE DRAWINGS

While the invention is claimed in the concluding portions hereof, preferred embodiments are provided in the accompanying detailed description which may be best understood in conjunction with the accompanying diagrams where like parts in each of the several diagrams are labeled with like numbers, and where:

FIG. 1 is a data processing system operable to implement the methods disclosed herein;

FIG. 2A is a schematic illustration of the data processing system configured for a user to directly interact with the data processing system;

FIG. 2B is a schematic illustration of the data processing system configured as a server and allowing a user to remotely connect to the data processing system using a remote device;

FIG. 3 is a data structure of a concept knowledge base, in accordance with the present invention;

FIG. 4 is a flowchart illustrating a method of automatically creating an instance of a concept knowledge base;

FIG. 5 is an overview software system for interactive query refinement;

FIG. 6 is a flowchart of a method of generating a query space using a concept knowledge database;

FIG. 7 is an exemplary illustration of a visual representation of a generated query space;

FIG. 8 is an exemplary illustration of a user interface; and

FIG. 9 is an exemplary illustration of a visual representation of a generated query space wherein a concept is compacted.

DETAILED DESCRIPTION OF THE ILLUSTRATED EMBODIMENTS

FIG. 1 illustrates a data processing system 1 suitable for supporting the operation of methods in accordance with the present invention. The data processing system 1 could be a personal computer, server, mobile computing device, cell phone, etc. The data processing system 1 typically comprises: at least one processing unit 3; a memory storage device 4; at least one input device 5; a display device 6 and a program module 8.

The processing unit 3 can be any processor that is typically known in the art with the capacity to run the program and is operatively coupled to the memory storage device 4 through a system bus. In some circumstances the data processing system 1 may contain more than one processing unit 3. The memory storage device 4 is operative to store data and can be any storage device that is known in the art, such as a local hard-disk, etc. and can include local memory employed during actual execution of the program code, bulk storage, and cache memories for providing temporary storage. Additionally, the memory storage device 4 can be a database that is external to the data processing system 1 but operatively coupled to the data processing system 1. The input device 5 can be any suitable device suitable for inputting data into the data processing system 1, such as a keyboard, mouse or data port such as a network connection and is operatively coupled to the processing unit 3 and operative to allow the processing unit 3 to receive information from the input device 5. The display device 6 is a CRT, LCD monitor, etc. operatively coupled to the data processing system 1 and operative to display information. The display device 6 could be a stand-alone screen or if the data processing system 1 is a mobile device, the display device 6 could be integrated into a casing containing the processing unit 3 and the memory storage device 4. The program module 8 is stored in the memory storage device 4 and operative to provide instructions to processing unit 3 and the processing unit 3 is responsive to the instructions from the program module 8.

Although other internal components of the data processing system 1 are not illustrated, it will be understood by those of ordinary skill in the art that only the components of the data processing system 1 necessary for an understanding of the present invention are illustrated and that many more components and interconnections between them are well known and can be used.

FIG. 2A illustrates a network configuration wherein the data processing system 1 is connected over a network 55 to a plurality of servers 50 operating as a search engine. FIG. 2B illustrates a network configuration wherein the data processing system 1 is configured as a server and a remote device 60, such as another computer, a PDA, cell phone or other mobile device connected to the Internet, is used to access the data processing system 1. The data processing system 1 runs the majority of the software and methods, in accordance with the present invention, and accesses the a plurality of servers 50 operating as a search engine to conduct a web search. By having the data processing system 1 configured as a server, the remote client system 60 does not need to have the capacity necessary to contain all the necessary data structures and run all the methods.

Furthermore, the invention can take the form of a computer readable medium having recorded thereon statements and instructions for execution by a data processing system 1. For the purposes of this description, a computer readable medium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk. Current examples of optical disks include compact disk—read only memory (CD-ROM), compact disk—read/write (CD-R/W) and DVD.

Concept Knowledge Base

FIG. 3 illustrates an architectural schematic of a data structure for a concept knowledge base 10, in accordance with an aspect of the present invention. The data structure is stored on a memory and is accessible by an application program being executed by a data processing system, such as the data processing system 1 illustrated in FIG. 1. The data structure contains information that is accessible by the application program.

The concept knowledge base 10 contains information relating to a field of knowledge. For example, the concept knowledge base 10 could contain information related to the field of science. The concept knowledge base 10 contains a number of concept data objects 12, a number of term data objects 14 and a number of edge data objects 16.

Each concept data object 12 contains a concept field 13 containing a concept that is related to a specific concept falling within the field of knowledge of the concept knowledge base 10. The concept field 13 typically contains a text string identifying the concept. For example, if the concept knowledge base 10 is for computer science, there may be concept data objects 12 with the concept field 13 containing the text string of “computer graphics”, another concept data object 12 with the concept field 13 containing the text string of “distributed computing”, another concept data object 12 with the concept field 13 containing the text string “artificial intelligence”, etc.

Each term data object 14 contains a term field 15 containing a text string. The text string contains a word or phrase that describes a concept of one of the concept data objects 12.

Each concept data object 12 is associated with one or more term data objects 14 and each term data object 14 is associated with one ore more concept data objects 12. The association of a concept data object 12 and a term data object 14 is defined by an edge data object 16 which contains a weight field 18. A term data object 14 that is associated with a concept data object 12 contains a term in the term field 15 that describes the concept contained in the concept field 13 of the concept data object 12. The relevancy of the term in the term field 15 of the term data object 14 to the concept in the concept field 13 of an associated concept data object 12 is represented by a weight in the weight field 18 of the edge data object 16.

While it is possible to manually construct the data structure containing the concept knowledge base 10, FIG. 4 illustrates a flowchart of a method of automatically creating a data structure containing a concept knowledge base in accordance with the present invention.

Method 100 comprises the steps of: determining a concept 110; selecting a document describing the concept 120; determining terms in the document to be analyzed 130; determining the frequency of the selected terms 140; checking if there are any remaining documents describing a concept 150; calculating a preliminary weight 160; checking if there are any more concepts 170; and normalizing all of the weights 180.

The method takes a number of documents and/or descriptions in computer readable form that describe a number of different concepts in a knowledge area and uses the documents to automatically generate a data structure of a concept knowledge base 10, as shown in FIG. 3.

The method 100 begins with step 110. A concept falling within the concept knowledge base is determined and a concept data object is created with information identifying the concept contained in the concept field.

Each concept will be described by one or more documents or descriptions in computer readable format. Once a concept has been determined at step 110, one or more documents describing the concept are identified and at step 120 one of these documents is selected to be analyzed.

At step 130, the method 100 determines the terms to be analyzed in the document. For each term to be analyzed, method 100 creates a term data object for each selected term with the term field containing the term, if a term data object containing the term does not already exist. An edge data object indicating the association of the term data object and the concept data object is also created and after the method 100 is completed will contain a weight indicating the relation of the term data object with the associated concept data object containing the concept described by the document being analyzed.

The terms that are analyzed can include all of the words used in the document or only specific words in the documents. For example, common words that are basically non-descriptive, such as “the”, “a”, “this”, etc. may be excluded from the selected terms that are selected for analysis at step 130.

At step 140 the frequency of each of the selected terms in the selected document is determined. The occurrence of each selected term in the document is determined. The occurrence of a selected term t_j in the document being analyzed can easily be determined, via text matching, and is defined by the function:

f(d_ik,t_j)

Each of the terms appearing in the document are then averaged based on the number of occurrences of all of the terms in the document. For example, the averaging could be done using the following equation:

$f^{*}\left(d_{\mathrm{ik}},t_j\right)=\frac{f\left(d_{\mathrm{ik}},t_j\right)}{\sum _{l=1}^mf\left(d_{\mathrm{ik}},t_{l,\mathrm{ik}}\right)}$

where d_ik is the document being analyzed for the set of terms t_ik={t_l,ik, . . . , t_m,ik) with m being the number of terms in document d_ik. This equation simply divides the frequency or tally of a term being analyzed by the total number of terms being analyzed in document d_ik. By conducting this averaging, the eventual weight determined for each association between a term node and a concept node takes into account the number of occurrences of a term in the document and provides a potentially more relevant indicator of the relation between the term data object to the concept data object because words or terms that appear often relative to the total number of terms will be given more weight. This preliminary averaging is used to try to prevent a single large document describing a concept from providing term weights that overshadow the weights provided by a number of smaller documents.

Next, at step 150, the method 100 checks to see if there are any more documents related to the concept that have not been analyzed. If there are more documents to be analyzed related to the concept, the method 100 returns to step 120, selects the next unanalyzed document and repeats steps 130, 140 and 150. As long as more documents related to the concept exist, step 150, causes the method 100 to analyze all of the documents. When there are no more documents related to the concept to be analyzed, the method 100 continues on to step 160.

At step 160 the method 100 calculates a preliminary weight for each of the terms used in the documents related to a single concept. For each term an interim weight w_ij* is calculated taking into account the average term frequency of the documents related to the concept.

$w_{\mathrm{ij}}^{*}=\frac{\sum _{k=1}^nf^{*}\left(d_{\mathrm{ik}},t_j\right)}{n}$

Wherein there are 1 . . . n documents.

This equation, in its entirety, is as follows:

$w_{\mathrm{ij}}^{*}=\frac{\sum _{k=1}^n\frac{f\left(d_{\mathrm{ik}},t_j\right)}{\sum _{l=1}^mf\left(d_{\mathrm{ik}},t_{l,\mathrm{ik}}\right)}}{n}$

This calculation is used to prevent concepts with a large numbers of documents from producing term weights that overshadow term weights from concepts with fewer documents describing the concept.

At step 170, the method 100 checks to see if there are any more concepts left to be evaluated. If there are concepts remaining that have not been analyzed, the method 100 returns to step 110 and the next concept is selected to be analyzed. The method 100 then repeats steps 120, 130, 140, 150 and 160 determining a preliminary weight for each of the terms appearing in the documents describing the selected document. The method 100 continues to analyze each concept repeating steps 110, 120, 130, 140, 150, 160 and 170 until all of the concepts have been analyzed, at which point, the method 100 continues on to step 180.

At step 180 the method 100 determines a normalized weight for each of the terms associated with the concepts. The preliminary weight w_ij* previously determined for each association between a term t_i and a concept is divided by the sum of all of the weights determined for the term t_i connected to r concepts. This equation is shown as follows:

$w_{\mathrm{ij}}=\frac{w_{\mathrm{ij}}^{*}}{\sum _{k=1}^rw_{\mathrm{if}}^{{*}_{\left(k\right)}}}$

Wherein the index f(k) is given by f(x), x=1 . . . r, representing the r concepts to which term i is connected to in the concept knowledge base.

The normalization of the weights is used to prevent common terms that are included in many of the documents for many concepts from having higher weight values than other less common terms. These terms are often of little value in describing a concept. By using normalization, the weights of common terms are significantly reduced. Without this normalization step, common terms that are included in many documents for many different concepts would have a very high weight, even though these terms are of little value in describing the concept. With this normalization step, the weights of these common terms are significantly reduced.

Additionally, rather than using the terms exactly as they appear in the documents or descriptions, in a further aspect of the invention, the stems of the roots of the terms are used to construct the knowledge base allowing terms to be matched based on their stems or roots rather than being based on exact text matches.

Additionally, in some circumstances it may not be necessary to analyze every term in a document. In a further aspect, the method 100 will focus on only specific terms in a document that are highlighted in a particular way, i.e. in an abstract. Alternatively, there could be a list of terms that are not analyzed, such as common terms that are not descriptive of a concepts, for example terms such as the, and, etc. may be excluded from being selected.

At the conclusion of the method 100 a concept knowledge base as illustrated in FIG. 3 will have been automatically constructed by the method 100.

Framework for Visual Refinement Software

FIG. 5 illustrates a software system of a visual query refinement method. The software system 300 comprises: a concept knowledge database 310; a current search query module 320; a query space generation module 330; a query visualization module 340; a search engine preview module 350; a search engine API module 360; a user interface module 370; and a search engine 380.

The search query will comprise one or more search terms. The software system 300 can be implemented on a data processing system, such as the data processing system 1 as shown in FIG. 2A. The data processing system 1 can be a client computer connected to the Internet with the software system 300 being executed completely on the user's client computer, with the exceptions of the search engine API 360 and the search engine 380, which would typically be implemented on one or more of the servers 50. Alternatively, various modules could be implemented on the data processing system 1 configured as a server 50, as shown in FIG. 2B, with the user merely inputting the search query from a remote device 60, i.e. a PDA or mobile phone with an Internet connection, and the software system 300 is primarily implemented on the data processing system 1 with the exception of the user interface module 370 which would be executed on the remote device 60.

The search query is entered into the system at the current search query module 320. From the search query module 320 the search query is passed to the query space generation module 330, which accesses the concept knowledge database 310, to generate a query space of terms a user may wish to add to his or her search query. Typically, the concept knowledge database 310 contains a concept knowledge base data structure as shown in FIG. 3.

From the query space generation module 330 the generated query space is passed to the query visualization module 340 where a visual representation of the query space is generated. The visual representation of the query space is then passed to the user interface module 370.

Additionally, the current search query module 320 also passes the search query to a search engine preview module 350 that has a search engine API 360 conduct a preview of a web search using the search query and passes the results of preview of the web search to the use interface module 370.

The user interface module 370 displays the visual representation of the query space to a user along with the results of a preview search. The user can perform a number of operations using the user interface module 370, such as, submitting a new search query; modify the search query by adding or removing terms; remove a concept; expand or collapse a concept; and sending the search query to the search engine.

Query Space Generation

The software system 300 begins with an initial search query being input to the current search query module 320 which passes the search query to the query space generation module 330. The query space generation module 330 accesses a concept knowledge database 310 and uses the information in the concept knowledge database 310 to generate a query space from the search query.

FIG. 6 illustrates a flowchart of a method for query expansion that is implemented by the query space generation module 330 in FIG. 5, using the concept knowledge database 310. When a search query is passed to the query space generation module 330, the method 400 uses the concept knowledge database 310 to generate a query space to expand the terms used in the search query using relationships to concepts to obtain additional terms that are relevant to the terms in the search query.

Method 400 comprises the steps of: matching terms in the search query to term data objects in the concept knowledge base to obtain a first term set 410; obtaining a concept set of concept data objects associated with the first term set 420; obtaining a second term set of term data objects associated with the concepts objects in the concept set 430; and obtaining an edge set 450.

The method 400 begins with step 410 and the terms in the search query being matched to term data objects in the concept knowledge database 310. The concept knowledge database 310 is accessed and each of the terms making up the search query are matched with any term data objects that have a term in the term field matching the term in the search query. A first term set containing these selected term data objects is obtained. After step 410 is completed, all of the term data objects in the concept knowledge database 310 that have a term in the term field that corresponds to one of the terms in the search query are identified and these term data objects are added to a first term set.

At step 420, the first term set is used to obtain a concept set containing concept data objects from the concept knowledge database 310 associated with one or more term data objects in the first term set. The term data objects making up the first term set are used to obtain a number of concept data objects from the concept knowledge database 310. Concept data objects associated with one or more term data objects in the first term set are selected to form the concept set.

Concept data objects that are not strongly associated with term data objects in the first terms set are excluded from the concept set using a first weight threshold and a term ratio threshold. The first weight threshold is used to exclude concept data objects that are not strongly associated with one of the term data objects in the first term set by comparing the weight assigned to an association between a concept data object and a term data object and excluding the concept data object from the concept set if the weight determined for the association is less than the first weight threshold. By using this first weight threshold, the concept set is limited to only the more relevant concepts. Additionally, a term ratio threshold is used to further exclude concept data objects from the concept set. If a concept data object is associated with one of the term data objects in the first term set with a weight greater than the first weight threshold, the concept data object is evaluated to determine the ratio of all of the term data objects in the first term set to which the concept data object is associated with a weight greater than the first weight threshold. If this ratio is less than the term ratio threshold, the concept data object is excluded from the concept set.

At step 430 a second term set is obtained. Each of the concept data objects in the concept set are evaluated to determine term data objects, in the concept knowledge base 110, associated with each of these concept data objects. Term data objects associated with the concept data objects selected for the concept set are added to the second term set. A second weight threshold is used to exclude term data objects from the second term set if they are associated with concept data objects in the concept sets by a weight that is less than the second weight threshold.

At step 450, an edge set containing edge data objects from the concept knowledge database 310 is obtained. The edge data object defining the association between the term data objects in the first term set and the concept data objects in the concept set along with the edge data objects defining the association between the concept data objects in the concept set and the term data objects in the second term set are placed in the edge set.

At this point, the method 400 ends and there is: a first term set containing term data objects that correspond to terms in the search query; a concept set containing concept data objects associated with term data objects in the first term set, that represent concepts the terms in the search query could be describing; a second term set containing term data objects associated with one or more concept data objects in the concept set, that indicate further terms that may be used to describe the concepts the user may be trying to look for; and an edge set defining the associations between the term data objects and concept data objects in the different sets.

Through experiments, the first weight threshold, term ratio threshold and second weight threshold can be determined. For example, some initial studies found that a first weight threshold of 0.05, a term ratio threshold of 0.51 and a second weight threshold of 0.10 provided satisfactory results.

Referring again to FIG. 5, after the query space (a first term set, a concept set, a second term set and an edge set) is generated by the query space generation module 330, the query space contains: a first term set containing term nodes matching terms in the search query; a concept set, containing concept nodes associated with term nodes in the first term set; a second term set containing term nodes associated with concept nodes in the concept set; and an edge set containing edge data objects defining the association between term data objects and concept data objects. This query space is passed to the query visualization module 340 to generate a visualization representation of the query space.

Visualization of the Query Space

Referring again to FIG. 5, using the query space generated by the query space generation module 330, the query visualization module 340 generates a visual representation of the query space.

FIG. 7 illustrates an example of a visual representation of a generated query space. The visual representation 500 contains: a number of concept nodes 550; selected term nodes 560 and unselected term nodes 570. Concept nodes 550 have one or more connecting lines 580 joining the concept node 550 to either selected term nodes 560 or unselected term nodes 570 that are associated with the concept node 550.

The concept data objects contained in the concept set are used to create the concept nodes 550. Each concept data object in the concept node is used to create a concept node 550 in the visual representation 500 and the concept in the concept field of the concept data object is inserted as text on the concept node 550.

The term data objects contained in the first term set are used to create the selected term nodes 560. Each term data object in the first term set is used to create a single selected term node 560 in the visual representation 500 and the term in the term field of the concept is inserted as text on the term node 560.

The term data objects contained in the second term set are used to create the unselected term nodes 570 in the visual representation 500. An unselected term node 570 is created on the visual representation 500 for each term data object contained in the second term set with the term in the term field of each term data object used as text on the unselected term node.

The edge data objects in the edge set define the associations between the term data objects in the first and second term set and the concept data objects in concept set. Each edge data object in the edge set is used to draw the connecting lines 580 between associated concept nodes 550 and unselected term nodes 560 and unselected term nodes 570. The distance between a concept node 550 and an associated selected term node 560 or associated unselected term node 570 joined by a connecting line 580 is a function of the weight of the association indicated in the edge concept. For example, if a weight of an association between a first unselected term nodes 570A and a concept node 550A is less than the weight of an association between the concept node 550A and a second unselected term nodes 570B, the first unselected term node 570A is positioned in the visual representation 500 further away from the concept node 550A than the second unselected term node 570B.

The concept nodes 550 are rendered in the visual representation 500 so that the concept nodes 550 can be visually distinguished from the selected term nodes 560 and the unselected term nodes 570. Typically, colors are used to make the concept nodes visually distinctive, i.e. the concept nodes 550 being rendered with a red background.

The selected term nodes 560 and unselected term nodes 570 are also rendered in the visual representation 500 to be visibly distinguishable from each other. Typically, this is also done by rendering the selected nodes 560 and unselected term nodes 570 with different background colors from each other. For example, the selected term nodes 560 might be rendered with a yellow background or some other bright color and the unselected term node 570 can be rendered in some neutral color, such as grey.

The visual representation 500 allows users to properly interpret the underlying features of the query space. Users are able to visually distinguish between concept nodes 550, selected term nodes 560 and unselected term nodes 570; along with the relationship between these nodes. Terms the user used in their original search query are shown in the visual representation as selected term nodes 560, allowing a user to easily distinguish between terms in the visual representation 500 that the user used in his or her search query and new terms that were generated and that the user may wish to add to their search query. Additionally, this allows a user to identify whether the terms they have used in their search query are actually appropriate for their information needs. If the concepts shown in the concept nodes 550 are unrelated to the to the information the user is seeking, the search query may not be a proper search query and the user can try a completely new search query. The visual representation 500 can allow a user to determine if the search query they have used have very general terms (i.e. connect to numerous concept nodes) or very specific terms (i.e. connected to very few concepts).

Search Engine Preview

Referring again to FIG. 5, from the current search query module 320, the search query terms are also passed to the search engine preview module 350 to conduct a preview search on the search engine using the search query. The search engine preview module 350 passes the search query to the search engine API 360 and the search engine API 360 returns the results of the search to the search engine preview module 350. These preview results could be a the results of a full search or, alternatively, a subset of the information located in the search such as number of documents returned by the query, the title of the documents and the URL of a set number of these documents.

For example, both Google™ and Yahoo! offer API services that allows the system tp request a search preview.

The results of the search preview are passed from the search engine preview module 350 to the user interface module 370.

User Interface

A user interface module 370 is provided. If the user is using the data processing system 1 as shown in FIG. 2A, the user interface module 370 is executed on the data processing system 1 with a use interface displayed on the display device 6. Alternatively, if the user is accessing the data processing system 1, as shown in FIG. 2B, through the remote client device 60, the user interface module 370 is typically executed on the remote device 60 with a user interface displayed on a screen of the remote device 60.

The user interface module 370 displays to a user a visual representation created by the query visualization module 340 using the query space generated by the query space generation module 330, along with a search preview obtained by the search engine preview module 350.

FIG. 8 illustrates an embodiment of a user interface 600 displayed to the user by the user interface module 370. The user interface 600 comprises: a visual representation 610; a search engine preview 620; a first text field 630 and a second text field 640. The visual representation 610 and the search engine preview allow a user to see the success of his or her search.

The user interface 600 allows a user to: submit a new search query; modify the search query; remove a concept; expand or collapse a concept; and send the query to the search engine.

Submitting a New Search Query

When a user sees the visual representation 610 and the search engine preview 620, if the results are much different than what the user wanted, the user can conduct a completely new search by entering a new search query in the first text field 630 and selecting a search button 635.

Referring again to FIG. 5, when a user enters a new search query, the new search query is passed from the user interface module 370 to the current search query module 320, where the system 300 again generates a new query space with the query space generation module 330, creates a visual representation of the query space with the query visualization module 340, and a search engine preview with the search engine preview module 350 using the new search query.

Modifying the Search Query

A user can also add terms to the search query by selecting unselected terms on the visual representation 610. To add a term a user selects an unselected term node in the visual representation 610 and the term in the term node is added to the terms of the search query.

Referring to FIG. 5, the term is added to the search query to form a new search query and the new search query is passed to the current search query module 320 and modules 330, 340, 350 and 360 to generate an updated visual representation 610 and search preview using the new search query.

Additionally, a user can remove a term from the search query by selecting a selected term node in the visual representation 610. Referring to FIG. 5, the term is removed from the search query to form a new search query and the new search query is passed to the current search query module 320 and modules 330, 340, 350 and 360 to generate an updated visual representation 610 and search preview using the new search query.

Remove a Concept Node from the Visual Representation

Upon seeing the visual representation 610 a user may identify concept nodes illustrated in the visual representation that display concepts the user believes are not relevant to the information the user is trying to obtain in the search. To remove one of these concept nodes from the visual representation, a user selects the concept node in the visual representation 610.

Referring again to FIG. 5, when a user removes a concept node by selecting the concept node on the visual representation, the corresponding concept data object in the concept set is passed to the query visualization module 340 where a new visual representation of the query space is obtained with the concept data object and any term data objects in the second term set that are only associated with the removed concept data object removed. This new visual representation is then passed to the user interface 370.

Expand or Compact a Concept

A user can choose between an expanded and a compacted visual representation of a concept by selecting the node to be expanded or compacted. The user selects a concept node 550A on the visual representation 610 that the user either wishes to expand (if the concept node is compacted) or compact (if the concept node is currently expanded).

Referring to FIG. 5, the query space is passed back to the query visualization module 340 where a new visual representation of the query space is generated with the concept node compacted, such as the visual representation 700 shown in FIG. 9, if the concept node was expanded, or expanded, if the concept node was previously compacted.

Send the Query to the Search Engine

Finally, the user interface 370 allows a user to send the search query to a search engine to conduct a regular web search using the search query. A user selects the search button 645 and, referring to FIG. 5, the software system 300 transmits the search query to a search engine 380 to have the search engine conduct a search based on the search query.

The foregoing is considered as illustrative only of the principles of the invention. Further, since numerous changes and modifications will readily occur to those skilled in the art, it is not desired to limit the invention to the exact construction and operation shown and described, and accordingly, all such suitable changes or modifications in structure or operation which may be resorted to are intended to fall within the scope of the claimed invention.

Claims

1. A memory for storing data for access by an application program being executed on a data processing system, comprising: a data structure stored in said memory, said data structure including information resident in a database used by said application program and including: a plurality of concept data objects stored in the memory, each of said concept data objects containing information related to a concept; anda plurality of term data objects stored in the memory, each term data object containing information related to a term;wherein each of the term data objects is associated with one or more of the concept data objects and the association has an assigned weight.

2. The memory of claim 1 wherein each association between one of the concept data objects and one of the term data objects has an assigned weight.

3. The memory of claim 2 wherein the data structure comprises a plurality of edge data objects, each edge data object defining an association between one of the concept data objects and one of the term data objects and the edge data object containing the assigned weight.

4. A method of automatically generating a concept knowledge base data structure from a plurality of computer readable documents related to a knowledge field, the method comprising: determining a plurality of concepts and for each concept creating a concept data object corresponding to the concept;for each concept, analyzing at least one computer readable document describing the concept and selecting terms in the computer readable document; andfor each selected term, creating a term data object associated with the concept data object.

5. The method of claim 4 further comprising for each selected term determining a weight indicating the relevance of the association between the concept node and the term node.

6. The method of claim 5 wherein each weight is determined by tallying the occurrence of the selected term in the at least one document describing a concept.

7. The method of claim 6 wherein the weight is determined by averaging a frequency term by the frequency of the other selected terms in the document.

8. The method of claim 6 wherein each the weight is normalized.

9. The method of claim 4 wherein an edge data object is created for each association between one of the term data objects and one of the concept data objects.

10. A data processing system for automatically generating a concept knowledge base data structure from a plurality of computer readable documents related to a knowledge field, the data processing system comprising: at least one processing unit;at least one memory storage device operatively coupled to the processing unit; anda program module stored in the at least one memory storage device operative for providing instructions to the at least one processing unit, the at least one processing unit responsive to the instructions of the program module, the program module operative for: determining a plurality of concepts and for each concept creating a concept data object corresponding to the concept on the at least one memory storage device;for each concept, analyzing at least one computer readable document describing the concept and selecting terms in the computer readable document; andfor each selected term, creating a term data object on the at least one memory storage device and associating the term data object with the concept data object.

11. The data processing system of claim 10 wherein the program module is operative for calculating a weight for each association between a term data object and a concept data object, the weight indicating the relevance of the term data object to the concept data object.

12. The data processing system of claim 11 wherein each weight of an association is determined by tallying the occurrence of the selected term in the at least one document describing a concept.

13. The data processing system of claim 11 wherein each weight of an association is determined by averaging a frequency term by the frequency of the other selected terms in the document.

14. The data processing system of claim 11 wherein each weight is normalized.

15. The data processing system of claim 10 wherein an edge data object is created on the at least one memory storage device for each association between one of the term data objects and one of the concept data objects.

16. A computer readable memory having recorded thereon statements and instructions for execution by a data processing system to carry out the method of claim 4.

17. A method of expanding a search query comprising: using a search query comprising a plurality of search terms, accessing a concept knowledge base data structure having a plurality of concept data objects and a plurality of term data objects, each term data object defining a term and associated with at least one of the concept data objects;generating a first term set containing term data objects from the concept knowledge base data structure wherein each term data object in the first term set matches one or more of the search terms;generating a concept set containing concept data objects from the concept knowledge base data structure wherein each concept data object in the concept set is associated with one or more of the term data objects in the first term set;generating a second term set containing term data objects from the concept knowledge base data structure wherein each term data object in the second term set is associated with one ore more of the concept data objects in the concept set; andin response to selecting one of the term data objects in the second term set, adding the term contained in the selected term data object to the search query.

18. The method of claim 17 wherein each concept data object contained in the concept set is associated with at least one of the term data objects in the first term set by a weight greater than a weight threshold.

19. The method of claim 18 wherein each concept data object contained in the concept set has a term ratio greater than a term ratio threshold wherein the term ratio is calculated by determining the ratio of all of the term data objects in the first term set and the number of term data objects in the first term set that the concept data object has an association with greater than the weight threshold.

20. The method of claim 17 wherein each concept data objects contained in the concept set is associated with at least one of the term data objects in the first term set with a weight greater than a first weight threshold and each term data object in the second term set is associated with at least one of the concept data objects in the concept set by a weight greater than a second weight threshold.

21. The method of claim 17 comprising generating a visual representation by graphically representing the term data objects in the first term set and the term data objects in the second term set as term nodes and the concept data objects in the concept set as concept nodes and wherein a user can select a term data object in the second term set to add the term contained in the selected term data object to the search query by selecting the term node corresponding to the selected term data object.

22. The method of claim 21 comprising graphically representing an association between a term data object and a concept data object by displaying a line connecting the term node, representing the term data object, and the concept node, representing the concept data object.

23. The method of claim 22 wherein a weight assigned to the association is represented by a distance between the term node and the concept node.

24. The method of claim 21 further comprising, in response to a user selecting one of the concept nodes in the visual representation, removing the selected concept node and any term nodes representing term data objects associated only with a concept node represented by the selected concept node from the visual representation.

25. The method of claim 21 further comprising requesting search results from a search engine using the search query and displaying the search results to the on the display device in conjunction with the visual representation.

26. A data processing system for expanding a search query, the data processing system comprising: at least one processing unit;at least one memory storage device operatively coupled to the processing unit and containing a concept knowledge base data structure, the concept knowledge base data structure including: a plurality of concept data objects; and a plurality of term data objects, each term data object defining a term and associated with at least one of the concept data objects; anda program module stored in the at least one memory storage device operative for providing instructions to the at least one processing unit, the at least one processing unit responsive to the instructions of the program module, the program module operative for: using a search query containing one or more search terms, generating a first term set containing term data objects from the concept knowledge base data structure wherein each term data object in the first term set matches one or more of the search terms;generating a concept set containing concept data objects from the concept knowledge base data structure wherein each concept data object in the concept set is associated with one or more of the term data objects in the first term set;generating a second term set containing term data objects from the concept knowledge base data structure wherein each term data object in the second term set is associated with one or more of the concept data objects in the concept set; andin response to selecting one of the term data objects in the second term set, adding the term contained in the selected term data object to the search query.

27. The data processing system of claim 26 wherein each concept data object contained in the concept set is associated with at least one of the term data objects in the first term set by a weight greater than a predetermined weight threshold.

28. The data processing system of claim 27 wherein each concept data object contained in the concept set has a term ratio greater than a term ratio threshold wherein the term ratio is calculated by determining the ratio of all of the term data objects in the first term set and the number of term data objects in the first term set that the concept data object has an association with greater than the weight threshold.

29. The data processing system of claim 26 wherein each concept data objects contained in the concept set is associated with at least one of the term data objects in the first term set with a weight greater than a first weight threshold and each term data object in the second term set is associated with at least one of the concept data objects in the concept set by a weight greater than a second weight threshold.

30. The data processing system of claim 26 wherein the data processing system comprises a display device operatively coupled to the data processing system and the program module is operative to direct the processing unit to display a visual representation on the display device by graphically representing the term data objects in the first term set and the term data objects in the second term set as term nodes and the concept data objects in the concept set as concept nodes.

31. The data processing system of claim 30 wherein an association between a term data object and a concept data object is graphically represented on the visual representation by a line connecting the term node representing the term data object, and the concept node representing the concept data object.

32. The data processing system of claim 31 wherein a weight assigned to the association is represented by a distance between the term node and the concept node.

33. The data processing system of claim 32 wherein the term nodes corresponding to the term data objects in the first term set are visually distinctive from the term nodes corresponding to the term data objects in the second term set.

34. The data processing system of claim 33 wherein the term nodes corresponding to the term data objects in the first term set are displayed in a first color and the term nodes corresponding to the term data objects in the second term set are a second color.

35. The data processing system of claim 30 wherein the data processing system further comprises an input device and a term data object is selected by a user using the input device to indicate the term node corresponding to the term data object.

36. The data processing system of claim 30 wherein the program module is operative for, in response to a user selecting one of the concept nodes in the visual representation, removing the selected concept node and any term nodes representing term data objects associated only with a concept node represented by the selected concept node from the visual representation.

37. The data processing system of claim 30 wherein the input device is a computer mouse and the user uses the mouse to select the term node corresponding to the term data object.

38. The data processing system of claim 30 wherein the program modules is operative for: requesting search results from a search engine and displaying the search results on the display device in conjunction with the visual representation.

39. The data processing system of claim 26 wherein the data processing system is operatively connectable to a remote device and wherein a user of the remote device inputs the search query to the data processing system and wherein the program module is operative to direct the processing unit to communicate with the remote device and display a visual representation on the remote device by graphically representing the term data objects in the first term set and the term data objects in the second term set as term nodes and the concept data objects in the concept set as concept nodes.

40. The data processing system of claim 39 wherein an association between a term data object and a concept data object is graphically represented on the visual representation by a line connecting the term node representing the term data object, and the concept node representing the concept data object.

41. The data processing system of claim 40 wherein a weight assigned to the association is represented by a distance between the term node and the concept node.

42. The data processing system of claim 41 wherein the term nodes corresponding to the term data objects in the first term set are visually distinctive from the term nodes corresponding to the term data objects in the second term set.

43. The data processing system of claim 42 wherein the term nodes corresponding to the term data objects in the first term set are displayed in a first color and the term nodes corresponding to the term data objects in the second term set are a second color.

44. The data processing system of claim 39 wherein the program module is operative for, in response to a user selecting one of the concept nodes in the visual representation, removing the selected concept node and any term nodes representing term data objects associated only with a concept node represented by the selected concept node from the visual representation.

45. The data processing system of claim 39 wherein the program modules is operative for: requesting search results from a search engine and displaying the search results on the remote device in conjunction with the visual representation.

46. A computer readable memory having recorded thereon statements and instructions for execution by a data processing system to carry out the method of claim 17.