Patent Grant
7617194
Many search engine services, such as GOOGLE and OVERTURE, provide for searching for information that is accessible via the Internet. These search engine services allow users to search for display pages, such as web pages, that may be of interest to users. After a user submits a search request (i.e., a query) that includes search terms, the search engine service identifies web pages that may be related to those search terms. To quickly identify related web pages, the search engine services may maintain a mapping of keywords to web pages. This mapping may be generated by “crawling” the web (i.e., the World Wide Web) to identify the keywords of each web page. To crawl the web, a search engine service may use a list of root web pages to identify all web pages that are accessible through those root web pages. The keywords of any particular web page can be identified using various well-known information retrieval techniques, such as identifying the words of a headline, the words supplied in the metadata of the web page, the words that are highlighted, and so on. The search engine service identifies web pages that may be related to the search request based on how well the keywords of a web page match the words of the query. The search engine service then displays to the user links to the identified web pages in an order that is based on a ranking that may be determined by their relevance to the query, popularity, importance, and/or some other measure.
Three well-known techniques for page ranking are PAGERANK, HITS (“Hyperlink-Induced Topic Search”), and DirectHIT. PAGERANK is based on the principle that web pages will have links to (i.e., “out links”) important web pages. Thus, the importance of a web page is based on the number and importance of other web pages that link to that web page (i.e., “in links”). Web pages and their links can be represented as a web graph. A web graph represents web sites as vertices of the graph and links between web pages of the web sites as edges between the vertices. The edges are directed to differentiate in and out links. A web graph can be represented as an adjacency matrix Ai where Aij represents the number of out links from web page i to web page j. The importance score wj for web page j can be represented by the following equation:
This equation can be solved by iterative calculations based on the following equation:
ATw=w
where w is the vector of importance scores for the web pages and is the principal eigenvector of AT.
The HITS technique is additionally based on the principle that a web page that has many links to other important web pages may itself be important. Thus, HITS divides “importance” of web pages into two related attributes: “hub” and “authority.” “Hub” is measured by the “authority” score of the web pages that a web page links to, and “authority” is measured by the “hub” score of the web pages that link to the web page. In contrast to PAGERANK, which calculates the importance of web pages independently from the query, HITS calculates importance based on the web pages of the result and web pages that are related to the web pages of the result by following in links and out links. HITS submits a query to a search engine service and uses the web pages of the result as the initial set of web pages. HITS adds to the set those web pages that are the destinations of in links and those web pages that are the sources of out links of the web pages of the result. HITS then calculates the authority and hub score of each web page using an iterative algorithm. The authority and hub scores can be represented by the following equations:
where a(p) represents the authority score for web page p and h(p) represents the hub score for web page p. HITS uses an adjacency matrix A to represent the links. The adjacency matrix is represented by the following equation:
The vectors a and h correspond to the authority and hub scores, respectively, of all web pages in the set and can be represented by the following equations:
a=ATh and h=Aa
Thus, a and h are eigenvectors of matrices AT A and AAT. HITS may also be modified to factor in the popularity of a web page as measured by the number of visits. Based on an analysis of click-through data, bij of the adjacency matrix can be increased whenever a user travels from web page i to web page j.
Although these techniques for ranking web pages based on analysis of links can be very useful, these techniques assume that the underlying data (e.g., web pages and their links) upon which importance is based is both accurate and reliable. Unfortunately, the underlying data can have inaccuracies and the links may not be a reliable representation of importance, resulting in reduced reliability of the determination of importance. Web pages and links may be missing from the underlying data for several reasons, making the data inaccurate. For example, when crawling web sites, some links may be missed due to an HTML parsing error or expiration of a URL. If the importance of scholarly papers is being determined, some citations may be lost due to different formats of abbreviations in the references. The underlying data may also have links that are not reliable representations of importance. For example, the links may include many “spam” links. “Spamming” in general refers to a deliberate action taken to unjustifiably increase the popularity or importance of a web page or web site. In the case of link spamming, a spammer can manipulate links to unjustifiably increase the importance of a web page. For example, a spammer may increase a web page's hub score by adding out links to the spammer's web page. A common technique for adding out links is to create a copy of an existing link directory to quickly create a very large out link structure. As another example, a spammer may provide a web page of useful information with hidden links to spam web pages. When many web pages point to the useful information, the importance of the spam web pages is indirectly increased. As another example, many web sites, such as blogs and web directories, allow visitors to post links. Spammers can post links to their spam web pages to directly or indirectly increase the importance of the spam web pages. As another example, a group of spammers may set up a link exchange mechanism in which their web sites point to each other to increase the importance of the web pages of the spammers's web sites. Spam links may also be present in scholarly papers. For example, an author may prefer to include a citation to a previous paper of the author even though another paper may be more relevant and more authoritative.
A method and system for ranking importance of vertices of a directed graph based on links between the vertices and some prior knowledge of importance of the vertices is provided. A ranking system inputs an indication of the vertices along with an indication of the links between the vertices as the directed graph. The ranking system generates a transition-probability matrix that represents the probability of transitioning from vertex to vertex. The ranking system then generates a ranking of the vertices based on the links between the vertices represented by the stationary distribution of the transition-probability matrix that is minimally perturbed to satisfy the prior knowledge, which may be a partial ranking of the vertices. This ranking is supervised in that the ranking of importance is influenced by the prior knowledge. The ranking system can be used to generate a supervised ranking for any data that can be represented as a directed graph with transition probabilities associated with transitioning from vertex to vertex along the links, such as web pages and their hyperlinks and scholarly articles and their citations.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
A method and system for ranking importance of documents based on links between the documents and some prior knowledge of importance of the documents is provided. In one embodiment, a ranking system inputs an indication of the documents along with an indication of the links between the documents. For example, if the documents are web pages, then the ranking system may represent the web pages and the links between the web pages using an adjacency matrix. The ranking system also inputs some prior knowledge of the ranking of the documents such as a partial ordering of some documents. For example, the ranking system may input from a person an indication of the relative ranking of pairs of web pages. The ranking system may also input other prior knowledge such as the popularity of a web page derived from analysis of click-through data. The popularity of some web pages may indicate a relative ordering of their rankings. The ranking system then generates a ranking of the documents based on the links between the documents while attempting to satisfy the partial ranking constraints of the prior knowledge. This ranking is supervised in that the ranking of importance is influenced by the partial ranking. The ranking system uses the additional partial ranking, which is not susceptible to the same inaccuracies and unreliability of the underlying data. Thus, the ranking system can generate a more accurate and reliable ranking of importance than techniques that rely solely on documents and their links. Moreover, the ranking system can be used to generate a supervised ranking for any data that can be represented as a directed graph with transition probabilities associated with transitioning from vertex to vertex along the links.
In one embodiment, when importance is based on a page rank-type algorithm, the ranking system attempts to perturb (or adjust) the transition-probability matrix so that the stationary distribution, which represents the ranking of the vertices, satisfies the partial rankings. In doing so, the ranking system attempts to minimize the perturbation to the transition-probability matrix so that the ranking reflects, to the extent possible, the influence of the unperturbed transition-probability matrix. Unfortunately, the goal of minimizing the perturbation while calculating the stationary distribution of the transition-probability matrix (i.e., the eigenvector) has a feasible domain that is not convex. The feasible domain is not convex because the minimization is constrained by the stationary distribution that is defined in terms of the perturbation that is to be minimized. The finding of a solution to a problem with a non-convex feasible domain is very computationally complex. To avoid this computational complexity, the ranking system reformulates the optimization problem so that its feasible domain is convex. The ranking system removes this constraint and adds the constraining equation to the objective function that is to be minimized. As described below in detail, the ranking system then uses an interactive optimization algorithm to iteratively and interactively solve the reformulated objective function, which is non-convex. The interactive optimization algorithm iteratively fixes the perturbation and calculates the eigenvector and then fixes the eigenvector and calculates the perturbation until the perturbation and the eigenvector converge on a solution, with the principal eigenvector representing the ranking of the vertices.
In one embodiment, the ranking system may collect prior knowledge of the ranking or importance of documents from various sources. For example, the ranking system can input expert opinions as to which scholarly papers have more impact in a specific domain than others. The ranking system may also use the number of visitors of web pages as an indication of their ranking or importance. The ranking system may represent this prior knowledge by partial order constraints. For example, if vertex 1 is known to be more important than vertex 2, and vertex 3 is known to be more important than vertex 4, the ranking system may represent the partial order constraints represented by the following equations:
Given these constraints, the ranking system perturbs the transition-probability matrix according to the following equation:
P1=P−E (2)
where P1 is the perturbed transition-probability matrix, P is the original transition-probability matrix derived from the adjacency matrix, and E is a perturbation matrix. The ranking system perturbs the transition-probability matrix so that the stationary distribution of the new Markov chain will be consistent with the constraints as much as possible. Many different perturbation matrices can be used to satisfy the partial order constraints. If the transition-probability matrix is perturbed too much, most of the information on importance derived from the directed graph will be lost. Although the information on importance may be to some extent inaccurate and unreliable, much of the information is accurate and reliable. To prevent the transition-probability matrix from being perturbed too much, the ranking system formulates the supervised ranking as an optimization problem in which the perturbation is minimized subject to those partial order constraints as represented by the following equations:
min∥E∥F2 (3)
s.t. (P−E)e=e (3a)
(P−E)≧0 (3b)
πT=πT(P−E) (3c)
eTπ=1 (3d)
π≧0 (3e)
Aπ>Bπ (3f)
where Equation 3 represents the objective function (e.g., the minimization of the perturbation matrix), Equations 3a-3f represent the constraints e represents a vector of 1's, π represents the eigenvector, (P−E)≧0 means that each element of the matrix P−E is non-negative, and ∥E∥F represents the Frobenius norm of the matrix E. Any other matrix norm may also be used although the ranking system may use the Frobenius norm for its computation simplicity. Equations 3a and 3b ensure that the perturbed transition-probability matrix is still a row-stochastic transition-probability matrix. Equations 3c, 3d, and 3e ensure that π is the stationary distribution of the perturbed Markov chain. Equation 3f is the supervision which is given according to the prior knowledge.
The constraints of Equations 3a, 3b, 3d, 3e, and 3f are affine functions of π or E. However, the constraint of Equation 3c contains interweaved terms of π and E, which makes the feasible domain non-convex. Because of the computational complexity of solving optimization problems with feasible domains that are non-convex, the ranking system moves the constraint of Equation 3c to the objective function and reformulates the optimization problem as represented by the following equations:
min∥E∥F2+cπT(I+E−P)(I+E−P)Tπ (4)
s.t. (P−E)e=e (4a)
(P−E)≧0 (4b)
eTπ=1 (4c)
π≧0 (4d)
Aπ>Bπ (4e)
where c in Equation 4 is a trade-off parameter (e.g., 100). The reformulated problem has a feasible domain that is convex.
Although the feasible domain of the problem of Equation 4 is convex, the objective function is not. To solve this problem, the ranking system decomposes this problem into two convex sub-problems. In each sub-problem, the ranking system fixes one parameter (π or E) and finds the optimal value of the other. By iteratively solving these sub-problems one after the other using an interactive optimization algorithm, the ranking system can get a close-to-optimal solution of the original problem. The ranking system represents the first sub-problem by the following equation:
s.t. eT=1 (5a)
π>0 (5b)
Aπ>Bπ (5c)
where the perturbation matrix E is fixed and π is optimized. This sub-problem is a linearly constrained quadratic optimization problem, which can be solved very efficiently. Since both the feasible domain and the objective function are convex, this problem has a global minimum π(E).
The ranking system represents the second sub-problem by the following equations:
s.t. (P−E)e=e (6a)
(P−E)>0 (6b)
where the stationary distribution π is fixed and the perturbation matrix E is optimized. This sub-problem is also a linearly constrained quadratic convex optimization problem, for which the global minimum E(π) can be solved efficiently. The interactive optimization algorithm is represented by the following pseudo code:
Well-known techniques exist for finding the global optimal such as variations of Newton's method. These variations include interior-point methods as described in Boyd and Vandenberghe, Convex Optimization, Cambridge University Press, 2004, Chap. 11. Interior-point methods can achieve specified optimization accuracy with a computational complexity that does not exceed a polynomial of the problem dimensions. The space complexity, however, for the second sub-problem is O(n2). Since there are millions of web pages n, the space complexity is very large. In one embodiment, the ranking system takes advantage of the sparseness of the adjacency matrix representing the directed graph to reduce the space complexity. For example, regardless of the number of web pages, the average is only about ten out links per page. Similarly, regardless of the number of papers in a domain, the average is in the tens of citations per paper. Thus, although there are approximately O(n2) possible vertex pairs, there are actually only O(n) edges. To reduce the space complexity, the ranking system only perturbs the non-constant elements in P (which correspond to those actual edges between vertices). Although this simplification may lead to a drop in effectiveness, it may be a reasonable tradeoff between efficiency and effectiveness.
The computing device on which the ranking system is implemented may include a central processing unit, memory, input devices (e.g., keyboard and pointing devices), output devices (e.g., display devices), and storage devices (e.g., disk drives). The memory and storage devices are computer-readable media that may be encoded with computer-executable instructions that implement the ranking system, which means a computer-readable medium that contains the instructions. In addition, the instructions, data structures, and message structures may be stored or transmitted via a data transmission medium, such as a signal on a communication link. Various communication links may be used, such as the Internet, a local area network, a wide area network, a point-to-point dial-up connection, a cell phone network, and so on.
Embodiments of the ranking system may be implemented in various operating environments that include personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, programmable consumer electronics, digital cameras, network PCs, minicomputers, mainframe computers, cell phones, personal digital assistants, smart phones, personal computers, programmable consumer electronics, distributed computing environments that include any of the above systems or devices, and so on.
The ranking system may be described in the general context of computer-executable instructions, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, and so on that perform particular tasks or implement particular abstract data types. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments. For example, a separate computing system may crawl the web and generate the web graphs. Also, a separate computing system may provide search engine capabilities. Thus, the crawling and search engine functions can be separate from the supervised ranking functions. For example, one company may generate a supervised ranking component and distribute it to other companies for use in various applications, such as ranking web pages, scholarly articles, and so on.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims. The ranking system may be adapted to provide supervised ranking in conjunction with algorithms other than page rank algorithms. For example, the ranking system may be adapted to be used in conjunction with HITS- and DirectHit-based algorithms. The ranking system may also be used to rank vertices of a directed graph regardless of what the vertices and links represent. As described above, the vertices and links may represent web pages and their hyperlinks, scholarly articles and their citations, authors of scholarly articles and authors of their citations, and so on. In general, the ranking system attempts to rank the vertices of a directed graph by modifying the rankings indicated by the links as little as possible to satisfy the prior knowledge of the ranking. The ranking system may effect this minimal modification of the rankings indicated by the links by a minimal perturbation of the transition-probability matrix. Accordingly, the invention is not limited except as by the appended claims.
| Number | Name | Date | Kind |
|---|---|---|---|
| 6285999 | Page | Sep 2001 | B1 |
| 6654742 | Kobayashi et al. | Nov 2003 | B1 |
| 6839682 | Blume et al. | Jan 2005 | B1 |
| 7024404 | Gerasoulis et al. | Apr 2006 | B1 |
| 7026029 | Kamvar et al. | Apr 2006 | B2 |
| 7089252 | Tomlin et al. | Aug 2006 | B2 |
| 7343374 | Berkhin | Mar 2008 | B2 |
| 7464075 | McSherry et al. | Dec 2008 | B2 |
| 20040024752 | Manber et al. | Feb 2004 | A1 |
| 20040111412 | Broder | Jun 2004 | A1 |
| 20050071465 | Zeng et al. | Mar 2005 | A1 |
| 20050216533 | Berkhin | Sep 2005 | A1 |
| 20050256860 | Eiron et al. | Nov 2005 | A1 |
| 20060020607 | Patterson | Jan 2006 | A1 |
| 20060059119 | Canright et al. | Mar 2006 | A1 |
| 20060074910 | Yun et al. | Apr 2006 | A1 |
| 20060248059 | Chi et al. | Nov 2006 | A1 |
| 20080010281 | Berkhin et al. | Jan 2008 | A1 |
| 20080114750 | Saxena et al. | May 2008 | A1 |
| Number | Date | Country | |
|---|---|---|---|
| 20080162453 A1 | Jul 2008 | US |
