DETERMINING RELEVANCE OF A DOCUMENT TO A QUERY BASED ON SPANS OF QUERY TERMS

Abstract

A relevance system determines the relevance of a query term to a document based on spans within the document that contain the query term. The relevance system aggregates the relevance of the query terms into an overall relevance for the document. For each query term, the relevance system calculates a span relevance for each span that contains that query term. The relevance system then aggregates the span relevances for a query term into a query term relevance for that document. The relevance system may aggregate the query term relevances into a document relevance.

Description

BACKGROUND

Many search engine services, such as Google and Overture, provide for searching for information that is accessible via the Internet. These search engine services allow users to search for display pages, such as web pages, that may be of interest to users. After a user submits a search request (i.e., a query) that includes search terms (i.e., query terms), the search engine service identifies web pages that may be related to those search terms. To quickly identify related web pages, the search engine services may maintain a mapping of keywords to web pages. This mapping may be generated by “crawling” the web (i.e., the World Wide Web) to identify the keywords of each web page. To crawl the web, a search engine service may use a list of root web pages to identify all web pages that are accessible through those root web pages. The keywords of any particular web page can be identified using various well-known information retrieval techniques, such as identifying the words of a headline, the words supplied in the metadata of the web page, the words that are highlighted, and so on. The search engine service may generate a relevance score to indicate how relevant the information of the web page may be to the search request based on the closeness of each match, web page importance or popularity, and so on. The search engine service then displays to the user links to those web pages in an order that is based on a ranking determined by their relevance.

Search engines can more generally be used to search a corpus of documents with a web page being one type of document in the corpus. The other types of documents may include articles published in journals, dissertations, technical reports, patents, and so on. With such a corpus, it may be desirable to present the documents ranked based on their relevance to the query. One common technique for ranking the relevance of a document to a query is based on term frequency and inverse document frequency. Term frequency refers to the number of occurrences of a query term within a document, and inverse document frequency refers to the inverse of the number of documents that contain that query term. Generally, a document with a more occurrences of a query term tends to be more relevant, and a query term that occurs in fewer documents is a more important term. One approach for combining term frequency and inverse document frequency into a relevance score for a document is given by the following equation:

$\begin{array}{cc}\sum _{t\in Q}w^{\left(1\right)}\frac{\left(k_1+1\right)·\mathrm{tf}}{K+\mathrm{tf}}& \left(1\right)\end{array}$

where t is a query term of query Q, tf is term frequency of t within the document, k₁ is a constant, and K and w^(l) are defined by the following equations. K is represented by the following equation:

$\begin{array}{cc}K=k_1·\left[\left(1-b\right)+b·\frac{l}{\mathrm{avdl}}\right]& \left(2\right)\end{array}$

where l is the document length, avdl is the average document length in the corpus, and b is a constant. w^(l) is a Robertson/Sparck Jones weight represented by the following equation:

$\begin{array}{cc}\log \frac{N-n+0.5}{n+0.5}& \left(3\right)\end{array}$

where N is the number of documents within the corpus and n is the number of documents containing the query term t within the corpus. Equation 3 is based on inverse document frequency. Thus, the score of relevance given by Equation 1 is based on term frequency, inverse document frequency, and document length.

The relevance of Equation 1 considers each query term independently. It is well known that the proximity of one query term to another query term affects relevance. For example, if the query is “home buying,” then a document that contains the phrase “home buying” may be more relevant than a document that contains the words “home” and “buying” separated by 100 words. One approach for factoring in the proximity of query terms into relevance uses relevance derived from “adjacent” pairs of query terms. Query terms are considered adjacent when the only intervening terms are non-query terms. For example, if the document contains the phrase “at the home page, you can select the buying option for tips” and the query is “home buying tips,” then “home” and “buying” are adjacent query terms that are separated by five non-query terms, a distance of five. However, “home” and “tips” are not adjacent, because the query term “buying” is between them. The relevance of adjacent pairs of query terms is represented by the following equation:

$\begin{array}{cc}\sum _{\left(t_i·t_j\right)\in S}\min \left(w_i^{\left(1\right)},w_j^{\left(1\right)}\right)·\frac{\left(k_1+1\right)·\sum _{\mathrm{occ}\left(t_i,t_j\right)}\mathrm{tpi}\left(t_i,t_j\right)}{K+\sum _{\mathrm{occ}\left(t_i,t_j\right)}\mathrm{tpi}\left(t_i,t_j\right)}& \left(4\right)\end{array}$

where t_i and t_j represent a pair of adjacent query terms and tpi is represented by the following equation:

$\begin{array}{cc}\mathrm{tpi}\left(t_i,t_j\right)=\frac{1.0}{{d\left(t_i,t_j\right)}^2}& \left(5\right)\end{array}$

where d(t_i,t_j) is the distance between the query terms t_i and t_j. The relevance of a document based on query term pairs (i.e., bigrams) is then combined with the relevance based on single query terms (i.e., unigrams) to give the overall relevance of a document.

A disadvantage with combining the unigram relevance and bigram relevance into document relevance is that it is difficult to estimate what their relative contributions should be. Moreover, a linear combination of these relevance scores may be inconsistent with the non-linear nature of traditional term frequency and inverse document frequency metrics.

SUMMARY

A relevance system determines the relevance of a query term to a document based on spans within the document that contain the query term. The relevance system aggregates the relevance of the query terms into an overall relevance for the document. The relevance system initially identifies spans of query terms within a document. A span of query terms is a sequence of terms of the document that includes one or more query terms that satisfy a proximity-based span definition. For each query term, the relevance system calculates a span relevance for each span that contains that query term. The relevance system then aggregates the span relevances for a query term into a query term relevance for that document. The relevance system may aggregate the query term relevances into a document relevance.

This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram that illustrates the identifying of spans by the relevance system in one embodiment.

FIG. 2 is a block diagram that illustrates components of the relevance system in one embodiment.

FIG. 3 is a flow diagram that illustrates the processing of the calculate relevance of documents component of the relevance system in one embodiment.

FIG. 4 is a flow diagram that illustrates the processing of the detect spans component of the relevance system in one embodiment.

FIG. 5 is a flow diagram that illustrates the processing of the find repeated term component of the relevance system in one embodiment.

FIG. 6 is a flow diagram that illustrates the processing of the process repeated term component of the relevance system in one embodiment.

FIG. 7 is a flow diagram that illustrates the processing of the calculate term relevance contribution component of the relevance system in one embodiment.

DETAILED DESCRIPTION

A method and system for determining relevance of a document to a query is provided. In one embodiment, the relevance system determines the relevance of a query term to a document based on spans within the document that contain the query term. The relevance system aggregates the relevance of the query terms into an overall relevance for the document. The relevance system initially identifies spans of query terms within a document. A span of query terms is a sequence of terms of the document that includes one or more query terms that satisfy a proximity-based span definition. For each query term, the relevance system calculates a span relevance for each span that contains that query term. The relevance system then aggregates the span relevances for a query term into a query term relevance for that document. For example, if the document contains the phrase “at the home page, you can select the buying option for tips” and the query is “home buying tips,” then a span for each query term may be “home page you can select the buying option for tips.” Alternatively, a span for “home” may be “home,” and a span for “buying” and “tips” may be “buying option for tips.” The span relevance for a query term may be based on the number of query terms in the span and an inverse of the span width. The span width is the number of terms between the first and last query term of the span. For example, the span width of the span “home page you can select the buying option for tips” is ten and of the span “buying option for tips” is four. Thus, the span relevance of a query term is higher when the span contains a greater number of query terms and is lower when the span width is greater. The relevance system may combine the span relevances into a query term relevance for each query term and then aggregates the query term relevances into a document relevance. Thus, the document relevance is based on the span relevance and is not based on term frequency. More generally, the query term relevance, also referred to as relevance contribution, can in general be used as a replacement for term frequency and can in particular be used in various metrics for calculating document relevance. In this way, the relevance system avoids the linear combination of unigram and bigram metrics and factors the query term proximity into the relevance of a document.

In one embodiment, the relevance system defines a span of query terms as a sequence of terms that includes no repeated occurrences of a query term and the distance between adjacent query terms within the sequence is less than a threshold distance. The relevance system may scan a document to identify sequences of terms that start and end with query terms. When the next query term in the document is the same query term as the last query term in the sequence currently identified, the relevance system terminates the sequence and indicates that the sequence is a span. The relevance system then continues scanning the document with the next query term as the start query term of the new sequence. The relevance system similarly terminates a sequence when the next query term is more than the threshold distance from the last query term of the current sequence. The relevance system also terminates a sequence when the next query term is a repeat of a query term in the sequence (other than the last query term which is terminated as described above). When the relevance system identifies that the next query term is a repeat of a query term already in the sequence, the relevance system may shorten the current sequence and terminate it after the first occurrence of the repeated query term or simply terminate the sequence after the last query term in the sequence. The relevance system shortens the current sequence after the first occurrence of the repeated query term when the distance from that occurrence to the next query term is greater than the distance from the last query term of the sequence to the next query term, which is the repeated query term. Otherwise, the relevance system does not shorten the sequence, but terminates it after the current last query term. In either case, the relevance system starts the next sequence at the next query term after the terminated sequence. For example, if the phrase is “at the home page, you can select the buying option for home tips” and the query is “home buying tips,” the relevance system may identify the spans of “home” and “buying option for home tips.” The relevance system initially considers the sequence “home page you can select the buying option for” when it encounters the second occurrence of “home” as the next query term. Since the second occurrence of “home” is closer to “buying” than the first occurrence of “home,” the relevance system shortens and terminates the sequence after the first occurrence of “home.” The relevance system then starts the next sequence at “buying,” which is the first query term after the shortened sequence.

In one embodiment, the relevance system calculates the relevance of a document to a query by using relevance contribution of query terms based on spans rather than term frequency of query terms. The relevance system first identifies spans of the query terms within the document. The relevance system then calculates the relevance contribution of each query term based on the identified spans that contain that query term. The relevance system then determines the relevance of the document to the query based on the calculated relevance contributions without using a term frequency. For example, the relevance system may use relevance contribution in place of term frequency in a conventional metric (e.g., TF*IDF) for measuring the relevance of a document based on term frequency.

In one embodiment, the relevance system calculates the span relevance of a query term based on the number of query terms within the span and an inverse of the span width. The span relevance may be represented by the following equation:

$\begin{array}{cc}f\left(t,{\mathrm{espan}}_i\right)=\frac{n_i^y}{{\mathrm{Width}\left({\mathrm{espan}}_i\right)}^x}& \left(6\right)\end{array}$

Where t is a query term, espan_i is a span that contains t, n_i is the number of query terms that occur in espan_i, Width(espan_i) is the span width of espan_i, x is an exponent that is used to control the influence of the span width, and y is an exponent that is used to control the influence of the number of query terms in the span. When a span contains only one query term, its span width may be set to the threshold distance. The aggregation of span relevance into a query term relevance or relevance contribution is represented by the following equation:

$\begin{array}{cc}\mathrm{rc}=\sum _if\left(t,{\mathrm{espan}}_i\right)& \left(7\right)\end{array}$

The value relevance contribution rc can be substituted for term frequency in a conventional relevance metric. For example, the substitution of relevance contribution for term frequency in Equation 1 results in the following equation:

$\begin{array}{cc}\sum _{t\in Q}w^{\left(1\right)}\frac{\left(k_1+1\right)·\mathrm{rc}}{K+\mathrm{rc}}& \left(8\right)\end{array}$

FIG. 1 is a diagram that illustrates the identifying of spans by the relevance system in one embodiment. Case 1 illustrates a span that is terminated because the next query term in the document is more than a threshold distance from the last query term in the current sequence. In this example, the query terms are “a,” “b,” “c,” and “d” and the document contains “a . . . b . . . c . . . d,” where the ellipses represent a zero or more non-query terms. In this example, the current sequence is “a . . . b . . . c,” and the distance between “a” and “b” and between “b” and “c” is less than the threshold distance. The distance, however, between “c” and “d” is greater than the threshold distance. As a result, the relevance system terminates the current sequence after “c,” resulting in a span of “a . . . b . . . c,” and starts a new sequence at “d.” Case 2 illustrates a span that is terminated because the next query term in the document is a repeat of the last query term in the current sequence. The current sequence is “a . . . b . . . c,” and the next query term “c” is the same as the last query term in the sequence. As a result, the relevance system terminates the current sequence after “c,” resulting in a span of “a . . . b . . . c,” and starts a new sequence at the second “c.” Case 3 illustrates a span that is terminated because the next query term is a repeat of a query term in the sequence. In this case, the current sequence is “a . . . b . . . c” and next query term is “b.” To ensure that a span includes only one occurrence of each query term, the relevance system could set the first span to “a . . . b” and the second span to start at “c” or the first span to “a . . . b . . . c” and the second span to start at the second “b.” The relevance system selects where to end the current sequence based on the distance between the repeated query term and its next query term within the sequence and the distance between the last query term of the sequence and the next query term of the document. In particular, the relevance system terminates the span where the distance is longer. In this example, if the distance between the first occurrence of “b” and “c” is greater than the distance between “c” and the second occurrence of “b,” then the relevance system shortens and terminates the sequence to after the first occurrence of “b” and starts the next sequence at “c.” If, however, the distance between the first occurrence of “b” and “c” is not greater than the distance between “c” and the second occurrence of “b,” then the relevance system terminates the sequence after “c” and starts the next sequence at the second occurrence of “b.”

FIG. 2 is a block diagram that illustrates components of the relevance system in one embodiment. The relevance system 200 includes a document store 201, an input query component 202, a search component 203, and a rank results component 204. The relevance system inputs a query using the input query component, searches the document store using the search component, and ranks the search results using the rank results component. The rank results component invokes a calculate relevance of documents component 211 to calculate the relevance of each document in the search results to the query. The ranking may be simply based on the relevance. The calculate relevance of documents component invokes a detect spans component 212 to detect the spans of query terms within a document. The calculate relevance of documents component also invokes a calculate term relevance contribution component 213 to calculate the relevance contribution of a query term to the relevance of a document. The relevance system may then rank the search result based on document relevance and display the search results in rank order to the user who input the query.

The computing device on which the relevance system is implemented may include a central processing unit, memory, input devices (e.g., keyboard and pointing devices), output devices (e.g., display devices), and storage devices (e.g., disk drives). The memory and storage devices are computer-readable media that may contain instructions that implement the relevance system. In addition, the data structures and message structures may be stored or transmitted via a data transmission medium, such as a signal on a communications link. Various communications links may be used, such as the Internet, a local area network, a wide area network, or a point-to-point dial-up connection.

The relevance system may be implemented in various operating environments that include personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.

The relevance system may be described in the general context of computer-executable instructions, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, and so on that perform particular tasks or implement particular abstract data types. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.

FIG. 3 is a flow diagram that illustrates the processing of the calculate relevance of documents component of the relevance system in one embodiment. The component is passed a query and generates a relevance score for each document of a corpus (e.g., document store) indicating the relevance of the document to the query. In block 301, the component selects the next document of the corpus. In decision block 302, if all the documents of the corpus have already been selected, then the component completes, else the component continues at block 303. In block 303, the component invokes the detect spans component to detect the spans of query terms within the selected document. In blocks 304-308, the component loops calculating a relevance contribution for each query term and aggregating the relevance contributions into a relevance score for the selected document. In block 304, the component initializes the document relevance score. In block 305, the component selects the next query term. In decision block 306, if all the query terms have already been selected, then the component loops to block 301 to select the next document, else the component continues at block 307. In block 307, the component invokes the calculate term relevance contribution component to calculate the relevance contribution of the selected query term for the selected document. In block 308, the component increases the document relevance score by the relevance contribution and then loops to block 305 to select the next query term.

FIG. 4 is a flow diagram that illustrates the processing of the detect spans component of the relevance system in one embodiment. The detect spans component is passed a document along with a query. The component identifies the query terms within the document and then processes the query terms sequentially to identify the spans. The component uses the variable current first term to identify the first query term in the current sequence of query terms and the variable current last term to identify the current last query term of the sequence. The component decides whether to continue or terminate the sequence based on the proximity of the next query term of the document and based on whether it is a repeat of a query term already in the sequence. If the next query term is farther than a threshold distance from the current last term, the component terminates the sequence at the current last term and starts a new sequence at the next query term. If the next query term is the same as the current last term, the component also terminates the sequence at the current last term and starts a new sequence at the next query term. If, however, the next query term is a repeat of a query term within the current sequence other than the current last term, the component may shorten the sequence based on the distance of the repeated query terms to other query terms within the sequence and then terminates the sequence. In block 401, the component initializes the variable current first term and the variable current last term to the first query term of the document. In blocks 402-410, the component loops selecting and processing each subsequent query term of the document. In block 402, the component initializes the variable next term to the next query term after the current last term. In decision block 403, if there is a next term to select, then the component continues at block 404, else the component continues at block 411 to complete the processing of the document. In decision block 404, if a first termination criterion is satisfied (e.g., case 1 or case 2 as described above), then the component continues at block 405, else the component continues at block 407. The first termination criterion is satisfied when the distance between the current last term and the next term is greater than a threshold distance or when the current last term is the same as the next term. In block 405, the component saves the sequence from the current first term to the current last term as a span. In block 406, the component sets the current first term equal to the next term to start a new sequence and continues at block 410. In block 407, the component invokes the find repeated term component to determine whether the next term occurs within the current sequence. In decision block 408, if the next term is a repeated term, then the component continues at block 409, else the component continues at block 410. In block 409, the component invokes the process repeated term component to terminate the current sequence, possibly shortening it. In block 410, the component sets the current last term to the next term to extend the current sequence and then loops to block 410 to select the next query term. In block 411, the component saves the current sequence as a span and then completes.

FIG. 5 is a flow diagram that illustrates the processing of the find repeated term component of the relevance system in one embodiment. The component determines whether the next term occurs within the current sequence. In block 501, the component sets the variable repeated term to the current first term, which is the beginning of the current sequence. In blocks 502-504, the component loops checking each query term of the sequence to determine if it is a repeat of the next term. In decision block 502, if the repeated term is equal to the current last term, then the component is at the end of the current sequence and returns an indication that no repeated term was found, else the component continues at block 503. In decision block 503, if the repeated term equals the next term, then the component returns the repeated term, else the component continues at block 504. In block 504, the component sets the repeated term to the next query term after the repeated term and then loops to block 502.

FIG. 6 is a flow diagram that illustrates the processing of the process repeated term component of the relevance system in one embodiment. The component shortens the current sequence when the distance between the occurrence of the repeated term within the current sequence and the next query term within the sequence is greater than the distance between the current last term and the next term. Otherwise, the component terminates the current query after the current last term. In block 601, the component sets the repeated next term to the next query term after the repeated term. In decision block 602, if the distance between the repeated term and the repeated next term is greater than the distance between the current last term and the next term, then the component continues at block 603, else the component continues at block 605. In block 603, the component saves as a span the sequence from the current first term to the repeated term, which is a shortening of the sequence. In block 604, the component sets the current first term to the repeated next term to start the new sequence and then returns. In block 605, the component saves as a span the current sequence from the current first term to the current last term. In block 606, the component sets the current first term to the next term to start the next sequence and then returns.

FIG. 7 is a flow diagram that illustrates the processing of the calculate term relevance contribution component of the relevance system in one embodiment. The component is passed a query term and calculates the relevance contribution of that query term based on the spans of a document. In block 701, the component initializes the relevance contribution to zero. In blocks 702-706, the component loops selecting each span and adjusting the relevance contribution based on the relevance of the query term derived from the span. In block 702, the component selects the next span of the document. In decision block 703, if all the spans have already been selected, then the component returns with the accumulated relevance contribution, else the component continues at block 704. In decision block 704, if the selected span contains the passed query term, then the component continues at block 705, else the selected span does not contribute to the relevance contribution of the query term and the component loops to block 702 to select the next span. In block 705, the component calculates the query term relevance derived from the selected span. In block 706, the component increases the relevance contribution by the query term relevance and then loops to block 702 to select the next span of the document.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims. Accordingly, the invention is not limited except as by the appended claims.

Claims

1. A method in a computer system with a processor and a memory for determining relevance of documents to a query, the method comprising: receiving from a user a query having query terms;for each of a plurality of documents, identifying spans of query terms within the document, each identified span including at least two query terms and having a span width indicating distance between the query terms of the span, wherein the distance between the query terms is based on number of terms between the query terms, and wherein a span has a span width that is less than a threshold span width;for each identified span, calculating a span relevance based on the span width of the span;for each query term, calculating a query term relevance for the query term based on the span relevance for each span that includes the query term; andaggregating the calculated query term relevances for the query term into a document relevance indicating relevance of the document to the received query; anddisplaying an indication of the documents in an order based on the document relevances of the documents to the received query.

2. The method of claim 1 wherein the identifying of spans includes identifying spans of query terms wherein a span includes no repeated occurrences of a query term and wherein a distance between adjacent query terms within a span is less than a threshold distance.

3. The method of claim 2 wherein a span includes a repeated occurrence of a query term that is closest to a query term of the span.

4. The method of claim 1 wherein the relevance for the query term for a span is calculated based on the following:

5. The method of claim 1 wherein the aggregating includes summing the calculated relevance for the query term for each span.

6. The method of claim 1 including replacing a term frequency with the aggregated relevance in an equation for calculating relevance of a document to a query.

7. The method of claim 6 wherein the equation before replacing is based on term frequency and inverse document frequency.

8. The method of claim 6 wherein term frequency is not used to calculate relevance.

9. The method of claim 1 wherein the document is a web page.

10. A computer-readable storage medium containing instructions for controlling a computer system to identify spans of query terms of a query within a document of terms for use in determining relevance of the document to the query, by a method comprising: identifying query terms within the document;identifying sequences of query terms within the document wherein a sequence includes at least two query terms such that a distance between the query terms is less than a threshold distance, the distance between the query terms is based on number of terms between the query terms and wherein a sequence is a span of query terms; andcalculating relevance of the document to the query based on the identified sequences of query terms.

11. The computer-readable medium of claim 10 wherein a distance between adjacent query terms within a sequence is less than a threshold distance.

12. The computer-readable medium of claim 11 wherein the distance between two query terms is based on the number of non-query terms between the two query terms.

13. The computer-readable medium of claim 10 wherein when a sequence of query terms is adjacent to two occurrences of the same query term, including within the sequence the query term that is closest to a query term of the sequence.

14. The computer-readable medium of claim 10 including calculating a relevance of query terms based on the following:

15. A method in a computer system with a processor and a memory for calculating relevance of a document of terms to a query of query terms, the method comprising: identifying spans of query terms within the document, a span comprising terms of the document that includes multiple query terms of the query and having a span width that is based on number of non-query terms of the span;calculating a relevance contribution of each query term based on the identified spans that contain the query term;determining relevance of the document to the query based on the calculated relevance contributions without using term frequency; andoutputting an indication of the determined relevance of the document to the query.

16. The method of claim 15 wherein the identifying of spans includes identifying sequences of query terms wherein a sequence includes no repeated occurrences of a query term and wherein at least one sequence includes at least three query terms.

17. The method of claim 16 wherein a distance between adjacent query terms within a sequence is less than a threshold distance.

18. The method of claim 15 wherein the calculating of the relevance contribution of a query term includes calculating relevance of the query term resulting from a span based on the following:

19. The method of claim 15 wherein the determining of relevance is based on an equation that uses inverse document frequency.

20. The method of claim 19 wherein term frequency within the equation is replaced with the calculated relevance contribution.