Patent Application
20070005343
The present exemplary embodiment relates generally to document processing. It finds particular application in conjunction with a method for developing a protocol for identifying text which expresses a given concept and a system for concept matching, and will be described with particular reference thereto. However, it is to be appreciated that the present exemplary embodiment is also amenable to other like applications.
Database and Internet searching are widely used for retrieving documents that are relevant to the information needs of a user. Many document processing problems involve finding text passages which express a given concept. Examples include information retrieval (IR), information extraction (IE), and question answering (QA) problems. Some concepts are generally expressed by a small set of fixed words or expressions and thus are readily easy to detect. For example, they may be detected automatically, using a simple keyword search or a set of regular expressions. Other concepts are more difficult to detect because their expressions are more varied. The problem stems from the fact that language is both productive and ambiguous: the same concept can be expressed by infinitely numerous expressions and, at the same time, the words making up the expressions can have different meanings in other contexts. Keyword searching is thus generally not applicable for concepts conveyed by a wide range of linguistic expressions.
Existing document processing systems typically deal with the productive nature of language by allowing substitution of similar expressions or expression schemata, where similarity is defined such that if one expression is relevant to a user's query, then similar ones can be assumed to be relevant as well. These similarities are usually defined using three levels of linguistic information: morphological equivalences, syntactic equivalences, and lexical semantic equivalences. For morphological equivalences a morphological processing component can detect word forms that are inflected or derived from the same root, e.g.: X acquires Y; X acquired Y; and the acquisition of Y by X. For syntactic equivalences: by using syntactic rules, a system can detect similarity between pairs of expressions such as: X acquired Y; and Y was acquired by X. For lexical semantic equivalences, a system can be provided (by hand or using corpus statistics) with information about various semantic relationships, e.g. synonymy or hyponymy, among lexical units, and this information can be useful for detecting similarities such as: X acquired Y; and X bought Y.
By using generic linguistic resources of the above sorts, a system can allow users to specify a single search pattern that matches a range of different expressions. For example, using currently available linguistic resources, a user searching for descriptions of acquisitions could conceivably write a single pattern that matched all of the above descriptions of transactions. However, the range of expressions that convey a given concept may be even greater. For example, simple morphological, lexical, and syntactic substitutions would not be sufficient to match that same pattern with expressions such as:
Some concepts tend to be expressed in relatively limited ways, and for such concepts, reasonable coverage and precision can be attained using the standard types of linguistic resources described above, if not with a single query pattern then with relatively few of them. The example of commercial transactions appears to be such a concept—in most corpora, the majority of commercial transactions can be matched with one of the patterns “X sell Y” or “Z buy Y” using the standard types of resources. For this reason, the problem of finding purchases of companies is often used as an example in the IE literature. But for other concepts, the number of patterns needed becomes unmanageable.
The challenge, then, is to provide a way of expressing patterns that generalize over the types of variation that cannot be accounted for using traditional morphological, syntactic, and lexical semantic resources. There has been a great deal of theoretical work on the sort of linguistic resources that would be necessary to do this in a general way—for example, the ideal semantic lexicon would contain, in a machine-processable form, the information that a purchase involves a change of ownership, and that an expression following “end up” is the resultant state of a change. A system endowed with such a lexicon could conceivably infer that “Y will end up being owned by X” is likely to indicate a purchase of X by Y. But in practice, the information that could be considered for inclusion in such a lexicon is boundless, and the process of encoding it is time-consuming and error-prone, so practical, general-purpose results do not appear to be forthcoming.
Many systems employ lexical resources, such as a “named entity recognizer,” which is a module that identifies a few types of expressions such as dates, numbers and names (primarily of people, places, and organizations), typically using a combination of fixed word lists and patterns. This allows the user to write extraction patterns in which all entities of a particular type, e.g. company names, are considered “similar” for the purposes of pattern matching. While named entity recognition can be useful for particular tasks, its range of applicability is limited.
Aspects of the present disclosure in embodiments thereof include a method for developing a system for retrieving text related to a selected concept within a text corpus and a method for retrieving text. In one aspect, a method for developing the system includes identifying a set of semantic classes which express the concept and identifying a set of keywords for each of the semantic classes to be used in text searching in a text corpus. Each set of keywords includes at least one keyword. A plurality of syntactic rules are established which are to be applied to retrieved text which includes keywords. Each of the syntactic rules identifies a first of the semantic classes and a second of the semantic classes. A rule is satisfied when a keyword from the first of the semantic classes is in a syntactic relationship with a keyword from the second of the semantic classes. The syntactic relationship can be any one of a plurality of syntactic relationships.
In another aspect, a method for retrieving text related to a selected concept within a text corpus includes identifying a set of semantic classes which, in combinations thereof, express the concept and identifying a set of keywords for each of the semantic classes to be used in text searching in a text corpus. Each set of keywords includes at least one keyword. A plurality of syntactic rules to be applied to retrieved text includes keywords, each of the syntactic rules identifying a first of the semantic classes and a second of the semantic classes. A rule is satisfied when a keyword from the first of the semantic classes and a keyword from the second of the semantic classes are in any one of a plurality of syntactic relationships. The syntactic rules are applied to a text corpus to identify text within the text corpus which satisfies at least one of the syntactic rules.
The system for developing a text retrieval system includes a memory for storing a set of semantic classes which, in combinations thereof, express a concept, a memory for storing keywords for each of the semantic classes to be used in text searching in a text corpus and for storing syntactic rules to be applied to retrieved text which includes keywords, a component which suggests sample sentences to a user which include one or more of the stored keywords, a user input for enabling a user to select sentences from the suggested sentences, and a component which proposes syntactic rules which are met by the selected sentences.
A methodology is described for developing systems that find concepts in natural language text. The method uses syntactic dependencies and task-specific semantic classes. The concept matching system developed using the methodology can be used to perform a search for the concept in a text corpus, such as a document database or internet information source. An exemplary concept matching system developed with such a methodology is described in Sándor and Kaplan, the disclosure of which is totally incorporated herein by reference. The method is particularly suited to developing a system which, with a reasonable assurance of accuracy, uncovers all or at least a preponderance of text/documents considered relevant to the particular search undertaken. In particular, the methodology develops a concept matching system which evaluates a set of documents which form all or a selected portion of a database. The term “document” as used herein may comprise a portion of a document, such as an article abstract, or an entire document. In the case of an abstract, for example, there are typically on the order of ten sentences per document. Each document includes at least one text portion, and generally a plurality of text portions. Each text portion is typically a sentence although it is contemplated that a text portion may be a portion of a sentence, such as a clause or portion in quotations.
In one embodiment, the system developed assigns a relevance to each document. The relevance may be based on criteria other than detection of the desired concept. The relevance may be a binary relevance determination (yes or no) or a weighted relevance determination, allowing the documents in the set to be ranked according to relevance. The concept matching system may be used in conjunction with keyword and/or other searching methods to limit the set of documents in the set to be searched to less than the entire database.
The methodology for developing a system for finding a selected concept in a text corpus can be generally described as including the following operations, which need not be performed in the following order:
Further operating steps may include:
The developed methodology can be applied to a test corpus. Each of these steps will be described in greater detail. As examples, two types of passages to be identified in press releases about new products are given. The first example concerns passages that describe the advantages of a new product over existing or previous solutions. As shorthand, the word “breakthrough” will be used to refer to this complex concept. Here are some sentences (taken from different press releases) that describe breakthroughs:
A second example concerns passages, such as the following, that describe possible uses of new products:
These examples demonstrate that there is no conventional, and thus recurrent, way of conveying the concepts of “breakthrough” and “possible use,” which makes it difficult to define patterns for detecting them. Existing techniques appear to be inadequate for finding these sorts of concept. The present methodology responds to this challenge by providing a framework for describing a target concept and a computational mechanism for matching such descriptions with expressions in text.
Retrieval performance of the concept matching system can be defined in terms of recall and/or precision. Recall is the fraction of all relevant documents that are retrieved. Precision is the fraction of all retrieved documents that are relevant.
Decomposition of Target Concept into Constituent Notions
As noted above, writing general patterns that match a variety of expressions involves an understanding of what makes the expressions similar. The present framework allows a representation of the nature of the similarity among expressions of complex concepts. By way of example, the expressions describing the target concepts “breakthrough” and “possible use,” discussed in the above examples are considered. These and other expressions describing these concepts have aspects in common, despite their varying vocabularies and grammatical structures.
Sentences with similar meanings, even if they use different words, tend to contain words that represent the same constituent notions. For example, the complex concept of “breakthrough” in the sentences above can be paraphrased as “the new product is different from those previously available.” This analysis of the breakthrough concept yields the following constituent notions, listed with some example of keywords (here single words) that express them:
Applying the same kind of analysis to the concept of possible use in the sentences above, this concept can be paraphrased by the expression “a product can be used by users,” which introduces the following constituent notions:
The words that instantiate a given constituent notion form a sort of semantic class. However, the present semantic classes differ from traditional ones in that their members need not share any of the usual morphological, syntactic or semantic properties. For example, in sentence 1, the constituent notion of “contrast” is represented by the verb “cannot,” and in sentence 2, it is represented by the preposition “unlike.” Similarly, the constituent notion of “use” is expressed in sentence 3 by the adjective “suitable,” and in sentence 4, by the verb “fulfill.” It is this heterogeneous nature of the semantic classes that gives the system a power to generalize that surpasses the possibilities of traditional linguistic resources.
Accordingly, the methodology includes developing a set of task-specific semantic classes which, in combination, express a target concept in a search to be performed. These classes will be referred to herein as constituent notions and may be identified by an identifier, such as a word or phrase. The word or phrase selected to identify the constituent notion (the “notion identifier”) may be broader than or narrower than the constituent notion it identifies. It is also contemplated that constituent notions may be identified with alphanumeric characters or the like. While a fully developed concept matching system may have from two to any number of semantic classes, typically at least three, and in one embodiment, at least four semantic classes are employed. For most purposes, up to ten semantic classes are generally sufficient, typically between about four and eight semantic classes provide an effective search tool. During development of the system, fewer or more semantic classes may be selected for evaluation and these classes refined during development, for example by adding or removing one or more semantic classes with the object of improving retrieval.
For each constituent notion within the set of constituent notions that have been identified, a set of keywords which relate to the constituent notion is identified. While particular reference is made to keywords as being single words, the set of keywords may also include hyphenated words, compound words, roots of words (such as acquire or acquir* in place of acquiring, acquired, etc.), and the like. Although the set of keywords may include the notion identifier, as well as synonyms and homonyms of the notion identifier, it need not do. The set of keywords generally includes words which are related to the constituent notion and, in particular, words which may be used in relevant expressions when the constituent notion is being conveyed. The words selected for a particular constituent notion are thus related to each other primarily in that they relate to the same constituent notion, not in that they have the same meaning, although some words may have the same or similar meanings as other words in the set. It is contemplated that a keyword associated with one constituent notion may also be associated with one or more, but generally fewer than all, of the other constituent notions in the set of constituent notions.
The process may be an iterative one, for example, starting with a limited set of keywords, identifying additional keywords and testing the effect the changes have on retrieval.
While the set of keywords for a given constituent notion may include as few as one keyword, it is contemplated that several of the constituent notions in the fully developed concept matching system (e.g., at least three, and in one embodiment, at least four constituent notions) will each be associated with at least five keywords. Typically, most or all of the constituent notions will be associated with a greater number of keywords, e.g., ten or more. The number of keywords in a set is not limited. However, keywords which are found not to improve information retrieval are generally dropped from the set of keywords during development of the concept matching system. During the development of the concept matching system, keywords can be added to or removed from the sets of keywords.
Various ways for developing the set of keywords are contemplated. These include, for example, analysis of a set of documents which have already been determined to satisfy the search criteria, keywords proposed by an interactive computer program, user evaluation of sentences proposed by a computer processing system in response to input keywords, and the like. The words selected for the constituent notions can be later verified by testing the methodology on a test set of documents and refined, as appropriate. For example, the set of keywords is refined by removing keywords which prove to be overly general and/or adding additional keywords which retrieve additional relevant documents, to increase the overall retrieval performance.
Development of Syntactic Rules to be Applied to Text Uncovered
As the example sentences above show, the co-occurrence of keywords (or text units) representing several constituent notions can indicate the presence of the target concept. However, it is generally not the case that any combination of constituent notions indicates the presence of the target concept. Consider, for example, the following sentences, for which the constituent notions are shown in brackets:
The co-occurrence of keywords denoting the constituent notions of time, and availability is not sufficient, in this case to convey the idea of breakthrough in sentence 5. However, a sentence like sentence 6 containing words denoting these three constituent notions as well as the constituent notion of contrast does convey the meaning of breakthrough. The following sentences demonstrate that even when several keywords of appropriate types co-occur in a sentence, the complex concept may not be present:
Sentences 7 and 8 are different from sentences such as 1-4 in that in sentences 1-4, the keywords are in direct syntactic relationships with each other, e.g., in sentence 1, the adjective “conventional” modifies the noun “methods,” whereas in sentences 7 and 8, there is no direct syntactic link connecting any pair of keywords.
Adding a syntactic constraint to the rules for identifying constituent notions in the text can thus improve retrieval performance. In the exemplary embodiment, syntactic dependencies are required among the keywords. It is generally not necessary to specify the type of dependency (subject, object, etc.) that must be present; it is generally sufficient simply to require that there be some syntactic relationship. It has been observed empirically that this improves the precision of the method in two ways. It eliminates sentences where keywords are present but conceptually unrelated to each other, as in sentences 7 and 8, and it can also have the effect of resolving lexical ambiguity. For example, the word “LED” has a technical sense (Light Emitting Diode), and it is also the past participle of the “use” keyword “lead.” Since, unlike the “use” sense, the technical sense tends not to be syntactically related to other “breakthrough” keywords (it tends, rather, to be linked to other technical terms), occurrences of the technical sense do not generally result in false hits.
Although requiring syntactic links results in a large gain in precision, in practice, it has been found that requiring all keywords to be syntactically linked tends to result in unacceptably low coverage. This is because the syntactic links are generally detected by an automatic parser, which can make mistakes, and because some pairs of words can be conceptually related without sharing a direct syntactic link. Therefore, in one embodiment, a relaxed syntactic criterion is employed: most keywords are considered to contribute to the expression of a complex concept if they are in a syntactic relationship with at least one other keyword. Exceptions can be made for words like adverbs and conjunctions that have particular syntactic properties which make it difficult to relate them syntactically to particular words in the sentence.
On the basis of the conceptual framework described above, a series of steps for implementing a search for a particular complex concept can be defined.
An exemplary methodology involves the development of a set of cooccurrence rules (“syntactic rules”) by which text uncovered by searching for the selected keywords is to be further classified by relevance. The rules identify pairs of constituent notions which when present in text in syntactic relationship to one another, are found to improve or anticipated to improve retrieval performance. Once again, this may be an iterative process. For example, a rule is added to or deleted from the set of rules or an existing rule modified and the new set of rules tested on a test corpus, such as a subset of the documents in the database, to assess the retrieval performance of the modified rule set.
Syntactic relationships between words in a punctuated natural language text portion (typically a sentence or portion thereof, such as a clause or portion in quotations) can include, for example, the following relationships:
S1 between verb and its subject
S2 between verb and its object
S3 between adjective or adverb and the noun, verb or adjective it modifies
S4 between a noun and another noun that it premodifies
S5 between a noun and a noun or verb that it modifies by prepositional attachment
For computer implementation of the system, an automated parser can be used to identify syntactic relationships in the documents which are selected by means of the search for keywords. The parser examines the text for syntactic relationships and flags the pairs of words involved in syntactic relationships. The output of the parser is a set of linked words and the syntactic relationship between the linked words. The number of syntactic relationships which may exist is virtually unlimited and in general, the parser searches for only a limited subset of the possible syntactic relationships. An exemplary parser of this type is described, for example in U.S. Patent Application No.2003/0074187, published Apr. 17, 2003, to Ait-Mokhtar, et al., which is incorporated herein in its entirety by reference. For any sentence there may be zero, one or more syntactic relationships identified.
The set of syntactic rules developed identifies pairs of semantic classes which are in any one of a plurality of syntactic relationships. In any rule, each of the two related words (such as a verb and its subject) represents by one of the constituent notions. For example for a set of constituent notions N1, N2, N3, . . . Nx and a set of selected syntactic relationships to be detected S1, S2, S3, S4, S5, . . . Sy, where x is the number of constituent notions in the set of constituent notions, and y is the number of syntactical relations which are accepted, some or all of the constituent notions are incorporated into at least one of the rules developed. For example,
Rule R1={S1, S2, S3, S4, S5, . . . Sy} [N1+N2], i.e., words from constituent notions N1 and N2 are in any syntactic relationship S selected from S1, S2, S3, . . . Sy.
Rule R1 is satisfied for a given document or text portion thereof when any keyword belonging to constituent notion N1 and any keyword belonging to N2 are in syntactic relationship, for example S may be S2, an object/verb syntactic relationship. In this example, a keyword from N1 may be the verb and the keyword from N2, the object, or vice versa. In its broadest application, the syntactic relationship S between the two constituent notions can be any or all of the syntactic relationships to be detected by the parser. Generally, an automated parser is capable of identifying a fixed set of syntactic relationships at a sentence level. In accordance with aspects of the exemplary embodiment, the syntactic relationships which satisfy a given rule are all of those which the parser to be used by the concept matching system is capable of identifying. In an alternative embodiment, a rule may define a specific subset of the syntactic relationships to be applied by the parser, such as one, two, three, four, six, or eight of the syntactic relationships identified by the parser. In one embodiment, all of the rules apply the same set of syntactic relationships.
The rules can be written in the same grammar formalism as used by the parser so that the rules are readily applied to the output of the parser. Alternatively, a different formalism is used for the rules.
The developed syntax rule may be associated with a weighting. The weighting roughly reflects the anticipated or observed relative importance of the rule in the overall retrieval performance. In its simplest form, all rules may be given the same weighting of 1. In more complex weighting schemes, the rules may each be assigned a weighting W of 0<W≦1. For example, some rules which are considered to strongly improve retrieval performance are assigned a weighting of 1 while others, considered to weakly affect retrieval performance, are assigned a weighting of 0.5. A weighted sum over all the rules of the product of number of occurrences that a rule is satisfied within a sentence or other text portion being evaluated (the occurrence rate T) and its weighting W can then be made.
In one embodiment, only documents with a weighted sum of at least 1 are retrieved. For the example above, this is satisfied by one rule with a weighting of 1 or two rules with a weighting of 0.5, which may be in the same sentence or different sentences. In another embodiment, only documents with at least one sentence having a weighted sum of at least 1 are retrieved.
A ranking of the documents can be made based on their scores (e.g., based on the number of times a syntactic rule is satisfied in a text passage). The ranking may also take into account the occurrences of certain specific keywords which are not in syntactic relationship but are nevertheless considered important. The ranking of documents may be performed by a ranking module of the concept matching system.
As an alternative to ranking, the system may list all documents which are considered to be relevant. The minimum relevance criteria may be for example, one pair of constituent notions in syntactic relationship or, where a weighting is applied, a minimum value of the sum of (occurrences x weighting) e.g., a minimum of 1. A minimum of 1 can be obtained, for example, where a “strong” pair of constituent notions (with a weighting of 1) is present or where two “weak” pairs of constituent notions (each with a weighting of 0.5) are present.
An exemplary methodology is illustrated in
A target concept is identified (Step S10). Sample expressions that express the desired concept are found, for example, by manual review of sample text (Step S12), and from them, a first set of keywords is extracted (Step S14). This could be supported by a simple keyword query interface—given a keyword or combination of keywords that the user thinks might be useful (Step S16), the system may provide sample sentences in which that word occurs (Step S18). The user may label retrieved examples as relevant or not relevant (Step S20), for use as training data for step S30 below. In one embodiment, a computer processing unit is connected to a user input, such as a keyboard or touch screen. The user inputs a selected keyword (Step S16) and the processing unit proposes sample sentences (Step S18). Alternatively or additionally the user inputs words to a thesaurus (Step S22) and the processing unit may propose words (Step S24), for example on a display, such as a screen. The user may select, from the proposed words, those which are considered by the user to be most appropriate for the search. These words are then added to the set of keywords.
Iteratively, by examining the keywords collected so far, a number of constituent notions are manually identified that can be combined to express the target concept (Step S26), and classes of keywords are formed such that all of the words in a class express the same constituent notion (Step S28). Each keyword may express one or more constituent notions. This step depends on human intuition and is not readily automated; automatic support may be limited to an interface that facilitates the manual classification.
Rules are written indicating which combinations of constituent notions can be combined to express the desired concept, and how these combinations can be realized syntactically (Step S30). Examples labeled in step S20 can be parsed automatically to give suggestions for such rules that the user can validate or reject. For combinations of keywords that do not occur in the sample expressions, the system may automatically find examples in a corpus and present them to the user for annotation.
The rules developed in step S30 are applied to a test text corpus (Step S18) and the results evaluated. The application of the rules to a corpus can be entirely automated. An efficient way to implement this involves searching the text for sentences that contain the necessary keywords, parsing these sentences, and then filtering the parsed sentences according to the co-occurrences rules. The evaluation of the results is generally performed manually.
Refinement of the rules, classes, and/or keywords may be undertaken to improve retrieval. If coverage is not yet sufficient, additional keywords are identified (and/or keywords removed) and the method repeated from step S28. This may be done by hand by examining sentences retrieved in step S32; the system may also make suggestions automatically by identifying words that frequently occur in syntactic relations with known keywords. If precision is not sufficient, the co-occurrence rules may be refined and the method repeated from step S30. The system may prompt the user to mark sentences that were retrieved but should not be, indicating which rule was responsible for the selection of each such sentence, and allowing the user to edit that rule.
As illustrated in
Once the concept matching system has been developed, as described above, it may be embodied as a “concept matching module” in any suitable electronic information retrieval system, such as a processing unit of a computer system, such as an individual computer or network, either as hardware or stored on software (e.g., CD or DVD). The database or other text corpus to be searched is accessible to the information retrieval system. The database may be provided on a hard disk of a computer system or on particular storage medium as CDs, DVDs, or other digital storage media. Alternatively, the concept matching module and or database may be stored at a remote location connected to the computer system via a data transmission network.
The methodology thus described is designed to develop a concept matching system which finds instances of a concept in text. The applications for which the method is useful do not necessarily fall into any of the classical categories of existing systems: information retrieval (IR) systems, information extraction (IE) systems, and question answering (QA) systems. Although there are similarities with some of these approaches, the differences can lead to increased information retrieval performance. Copending application Ser. No. ______, by Sándor and Kaplan, referred to above and incorporated by reference, provides an example system developed by the present methodology for selecting documents which has some of the objectives of an IR system. Since the present methodology addresses a new sort of search problem, its design differs from those of existing systems. In particular, the present methodology has advantages in the way in which syntax is used to restrict results, and the type of lexical resource that is used.
Most IR engines currently in mainstream use are based on the “bag of words” model, in which a document's content is approximated by the set of words it contains, sometimes augmented with the number of times each word occurs. The present method uses syntactic structure as an indication of conceptual relatedness, which results in increased precision compared to a purely keyword-based approach.
Some IE systems specify a number of keywords that must be present for a match to take place and specify an overall grammatical structure of the sentence. The exemplary methodology described herein employs a pattern which lists a set of constituent notions, and requires that some of these constituent notions be syntactically related, but does not specify the specific syntactic role that each constituent notion must play in the sentence. One of the important discoveries underlying this approach is that for the concept matching problem, at least for certain types of concepts, one can use this less-restrictive type of rule without a significant loss of precision. Since a single rule of this type can match a wider range of linguistic expressions, it makes it possible to achieve broad coverage of concepts that would require prohibitively many rules to cover with a FASTUS-type system.
While identifying particular syntactic relationships is important in a task such as IE or QA where one needs to find out “who did what to whom,” it has been found that in some concept matching applications, this information is not necessary, and that abstracting it away allows more general patterns to be written, using semantic classes that group together words that express the same concept but are of different parts of speech.
Although not required, the methodology may employ thesauri and/or other lexical resources. The thesaurus may be a hand-built thesaurus of the type disclosed in Joyce, et al., or a thesaurus which is automatically generated using corpus statistics, such as that disclosed in Jing, et al, and Qui, et al.
In some cases, coverage by the present methodology may be improved by using information about similarities between words. One kind of lexical resource which addresses similarities is a “named entity recognizer,” which is a module that identifies a few types of expressions such as dates, numbers and names (of people, places, organizations, and the like), typically using a combination of fixed word lists and patterns. Extraction patterns can be written in which all entities of a particular type, e.g., company names, are considered “similar” for the purposes of pattern matching. While named entity recognition can be useful for particular tasks, its range of applicability is limited. The present methodology allows the detection of concepts that do not involve named entities, and for concepts that do, a named entity recognition module could be integrated into the architecture.
The method has application for a variety of disciplines, particularly scientific disciplines. For example, the method can be used to develop a system for retrieving documents from a database, which includes millions of documents. A system developed using documents from one database, including keywords, constituent notions and rules can be used to find documents from another database related to the same complex concept.
It will be appreciated that various of the above-disclosed and other features and functions, or alternatives thereof, may be desirably combined into many other different systems or applications. Also that various presently unforeseen or unanticipated alternatives, modifications, variations or improvements therein may be subsequently made by those skilled in the art which are also intended to be encompassed by the following claims.
Reference is made to copending application Ser. No. ______, filed contemporaneously herewith, entitled CONCEPT MATCHING SYSTEM, by Agnes Sándor and Aaron Kaplan [hereinafter “Sándor and Kaplan”], the disclosure of which is totally incorporated herein by reference.
