Patent Application
20090125529
The present invention relates to computer networks and, more particularly, to techniques for automatically extracting information from documents using a template that has a similar structure to the documents.
The Internet is a worldwide system of computer networks and is a public, self-sustaining facility that is accessible to tens of millions of people worldwide. The most widely used part of the Internet is the World Wide Web, often abbreviated “WWW” or simply referred to as just “the web”. The web is an Internet service that organizes information through the use of hypermedia. The HyperText Markup Language (“HTML”) is typically used to specify the contents and format of a hypermedia document (e.g., a web page).
In this context, an HTML file is a file that contains source code for a particular web page. Typically, an HTML document includes one or more pre-defined HTML tags and their properties, and text enclosed between the tags. A web page is the image or collection of images that is displayed to a user when a particular HTML file is rendered by a browser application program. Unless specifically stated, an electronic or web document may refer to either the source code for a particular web page or the web page itself. Each page can contain embedded references to images, audio, video or other web documents. The most common type of reference used to identify and locate resources on the Internet is the Uniform Resource Locator, or URL. In the context of the web, a user, using a web browser, browses for information by following references that are embedded in each of the documents. The HyperText Transfer Protocol (“HTTP”) is the protocol used to access a web document and the references that are based on HTTP are referred to as hyperlinks (formerly, “hypertext links”).
Through the use of the web, individuals have access to millions of pages of information. However a significant drawback with using the web is that because there is so little organization to the web, at times it can be extremely difficult for users to locate the particular pages that contain the information that is of interest to them. To address this problem, a mechanism known as a “search engine” has been developed to index a large number of web pages and to provide an interface that can be used to search the indexed information by entering certain words or phases to be queried. These search terms are often referred to as “keywords”.
Indexes used by search engines are conceptually similar to the normal indexes that are typically found at the end of a book, in that both kinds of indexes comprise an ordered list of information accompanied with the location of the information. An “index word set” of a document is the set of words that are mapped to the document, in an index. For example, an index word set of a web page is the set of words that are mapped to the web page, in an index. For documents that are not indexed, the index word set is empty.
Although there are many popular Internet search engines, they are generally constructed using the same three common parts. First, each search engine has at least one, but typically more, “web crawler” (also referred to as “crawler”, “spider”, “robot”) that “crawls” across the Internet in a methodical and automated manner to locate web documents around the world. Upon locating a document, the crawler stores the document's URL, and follows any hyperlinks associated with the document to locate other web documents. Second, each search engine contains information extraction and indexing mechanisms that extract and index certain information about the documents that were located by the crawler. In general, index information is generated based on the contents of the HTML file associated with the document. The indexing mechanism stores the index information in large databases that can typically hold an enormous amount of information. Third, each search engine provides a search tool that allows users, through a user interface, to search the databases in order to locate specific documents, and their location on the web (e.g., a URL), that contain information that is of interest to them.
The search engine interface allows users to specify their search criteria (e.g., keywords) and, after performing a search, an interface for displaying the search results. Typically, the search engine orders the search results prior to presenting the search results interface to the user. The order usually takes the form of a “ranking”, where the document with the highest ranking is the document considered most likely to satisfy the interest reflected in the search criteria specified by the user. Once the matching documents have been determined, and the display order of those documents has been determined, the search engine sends to the user that issued the search a “search results page” that presents information about the matching documents in the selected display order.
The Internet today has an abundance of data presented in HTML pages. It, however, is still an arduous task to find informative content from all the other content. Many online merchants present their goods and services in a semi-structured format using scripts to generate a uniform look-and-feel template and present the information at strategic locations in the template. Identifying such positions on a page and extracting and indexing relevant information is key to the success of any data-centric application like search.
With the advent of e-commerce, most webpages are now dynamic in their content. Typical examples are products sold at discounted price that keep changing on sites between Thanksgiving and Christmas every year, or hotel rooms that change their room fares on a seasonal basis. With advertisement and user services critical for business success, it is imperative that crawled content be updated on frequent and near real-time basis.
These examples show that on the Web, especially on large sites, webpages are generated dynamically through scripts that place the data elements from a database in appropriate positions using a defined template. By understanding these templates, one could separate out the more useful information on the pages from the text put in by the script as part of the template.
Information Extraction (IE) systems are used to gather and manipulate the unstructured and semi-structured information on the web and populate backend databases with structured records. Most IE systems are either rule based (i.e., heuristic based) extraction systems or automated extraction systems. In a website with a reasonable number of pages, information (e.g., products, jobs, etc.) is typically stored in a backend database and is accessed by a set of scripts for presentation of the information to the user.
IE systems commonly use extraction templates to facilitate the extraction of desired information from a group of web pages. Generally, an extraction template is based on the general layout of the group of pages for which the corresponding extraction template is defined. One technique used for generating extraction templates is referred to as “template induction”, which automatically constructs templates (i.e., customized procedures for information extraction) from labeled examples of a page's content.
While an example has been provided of using templates to extract information from web pages, templates can be used to extract information from electronic documents having other than an HTML structure. For example, templates can be used to extract information from documents structured in accordance with XML (eXtensible Markup Language).
Any approaches that may be described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.
The present invention is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which:
Techniques are described for automatically generating extraction templates from a training set of similarly structured documents, such as web pages coded in HTML. In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the present invention.
Embodiments of the present invention are described in accordance with the following organization:
a) INITIAL WRAPPER CREATION
b) GENERALIZING THE INITIAL WRAPPER TREE
a) COMPARING WRAPPER TO TRAINING SET
b) GENERALIZING THE WRAPPER BASED ON COMPARISON WITH TRAINING SET
A) Property Based Filter
B) Position Based Filter
C) Range Pruner
D) Contextual Filter
E) Regex Filter
F) Tag-specific Filter
G) Text Manipulation Filter
Techniques are disclosed herein to automatically learn a template that describes a common structure present in documents in a training set. In one embodiment, the training documents are selected from a cluster of structurally similar documents. The cluster can be generated by applying a clustering algorithm to a large set of documents. The documents could be HTML documents (e.g., web pages), XML documents, documents in compliance with other markup languages, or some other structured document.
In one embodiment, the template is expressed as a tree. The structure of the template is compared to the structure of the documents (or at least a part of each document) in the training set, one-by-one, and generalized in response to differences between the template and the document to which the template is currently being compared. Generalizing the template to match a particular document results in a more general template structure that will match the structure of the particular document, while preserving the template's match to documents to which the template was previously matched. Thus, the generalized template describes a common structure present in the documents in the training set.
In one embodiment, a document object model (DOM) tree is constructed for at least a portion of a document to facilitate comparison with the template. Generalizing the template is achieved by generalizing the structure of the template such that its more general structure will match the structure of the DOM for the document, in one embodiment. Various example “generalization operators” are described herein, which may be added to the template to generalize it. If the structure of any particular document is considered too dissimilar from the structure of the template, then the template is not generalized to match the particular document.
After the template is created, the template can be used to extract information from documents outside of the training set. As an example, the template could be learned from a training set of web pages associated with a shopping web site. The learned template could be used to extract information such as product descriptions, product prices, product reviews, product images, etc. Note that some portions of the documents such as banner ads may not be of interest. Thus, the template might only describe the common structure of a portion of the shopping web pages, such as the portion that pertains to the product or products for sale. Because the template can be learned in an automated fashion, templates can be learned across applications to all kinds of script generated websites. Further note that prior to using the template for extraction, there may be some additional modifications. For example, the template could be annotated with attributes that are of interest, wherein those attributes can be extracted from documents that were not used to construct the template.
IIS 110 can be implemented comprising a crawler 112 communicatively coupled to a source of information, such as the Internet and the World Wide Web (WWW). IIS 110 further comprises crawler storage 114, a search engine 120 backed by a search index 126 and associated with a user interface 122.
A web crawler (also referred to as “crawler”, “spider”, “robot”), such as crawler 112, “crawls” across the Internet in a methodical and automated manner to locate web pages around the world. Upon locating a page, the crawler stores the page's URL in URLs 118, and follows any hyperlinks associated with the page to locate other web pages. The crawler also typically stores entire web pages 116 (e.g., HTML and/or XML code) and URLs 118 in crawler storage 114. Use of this information, according to embodiments of the invention, is described in greater detail herein.
Search engine 120 generally refers to a mechanism used to index and search a large number of web pages, and is used in conjunction with a user interface 122 that can be used to search the search index 126 by entering certain words or phases to be queried. In general, the index information stored in search index 126 is generated based on extracted contents of the HTML file associated with a respective page, for example, as extracted using extraction templates 128 generated by template induction 126 techniques. Generation of the index information is one general focus of the IIS 110, and such information is generated with the assistance of an information extraction engine 124. For example, if the crawler is storing all the pages that have job descriptions, an extraction engine 124 may extract useful information from these pages, such as the job title, location of job, experience required, etc. and use this information to index the page in the search index 126. One or more search indexes 126 associated with search engine 120 comprise a list of information accompanied with the location of the information, i.e., the network address of, and/or a link to, the page that contains the information.
As mentioned, extraction templates 128 are used to facilitate the extraction of desired information from a group of web pages, such as by information extraction engine 124 of IIS 110. Further, extraction templates 128 may be based on the general layout of the group of pages for which a corresponding extraction template 128 is defined. For example, an extraction template 128 may be implemented as an HTML file that describes different portions of a group of pages, such as a product image is to the left of the page, the price of the product is in bold text, the product ID is underneath the product image, etc. Template induction 126 processes may be used to generate extraction templates 128. Interactions between embodiments of the invention and template induction 126 and extraction templates 128 are described in greater detail herein.
The diagram in
In this embodiment, a suffix tree 204 is created from the sample HTML 202. A suffix tree 204 is a data-structure that represents suffixes starting from all positions in the sequence, S. The suffix-tree 204 can be used to identify continuous-repeating patterns. However, a structure other than a suffix tree 204 can be used to identify patterns. The suffix tree 204 is analyzed to generate a regular expression (“Regex”) HTML 206. Further details of creating a suffix tree 204 and a regex are discussed below under the heading “initial template creation.”
An initial template 208 is generated from the regex 206. In one embodiment, a template includes HTML nodes and nodes corresponding to defined operators. An example of an HTML node is an HTML tag (e.g., title, table, tr, td, h1, h2, p, etc.). Examples of defined operators include, but are not limited to, STAR, HOOK, and OR. A STAR operator indicates that any subtrees that stem from children of the STAR operator are allowed to occur one or more times in the DOM. A HOOK operator indicates that the underlying subtrees are optional. In one embodiment, a HOOK operator is allowed to have only one underlying subtree. In other words, a HOOK operator is allowed to have only a single child, in one embodiment. An OR operator in the template indicates that only one of the sub-trees underlying the OR operator is allowed to occur at the corresponding position in the DOM. It is not required that the template contain HTML nodes. In one embodiment, the template includes XML nodes and nodes corresponding to defined operators.
Box 210 depicts an example DOM structure for a document in the training set. Box 212 depicts a generalized version of the template 212, which is automatically generated in accordance with an embodiment. As previously mentioned, the template is generalized such that its structure matches that of a common structure of the training documents. To generalize the template 212 to match a particular DOM structure 210, first the template 212 is compared to the DOM 210 to determine what are the differences. Differences are resolved by adding one or more operators to the template 212, which results in matching the template 212 to the current DOM 210 by making the template 212 more general. The changes to the template 212 are made in such a way that the template 212 will still match with DOMs 210 for which the template 212 was previously generalized to match.
The following section describes initial creation of a template, in accordance with one embodiment.
In step 304, a suffix-tree is built on the character sequence “S.”
In step 306, valid patterns are identified. For example, certain tags should have an “open” tag followed, at some point, by a “close” tag. As a particular example, a “bold open tag” should precede a “bold close tag”. This required sequence of tags can be used to identify patterns that are valid and invalid and more prominent in the neighborhood.
In step 308, a regular expression, “R”, is constructed. Step 308 includes several sub-steps including replacing multiple occurrences in the suffix tree with a single occurrence. As an example, the suffix tree has multiple occurrences of “ab”, which are replaced by a single occurrence “ab*”, where the “*” indicates that pattern occurs more than once in the suffix tree. For example, from the character sequence S, a regular expression R is constructed by replacing multiple occurrences of a pattern in S by an equivalent regular expression. In the example from
In step 310, another string, S′, is formed. The new string S′ is formed by neglecting all of the patterns in R having a “*” character, in an embodiment.
Steps 304-310 are repeated on S′ to find more complex and nested patterns. Steps 304-310 may be repeated until no more patterns are available. At the end of this phase, a regular expression, R, is available with multiple occurrences replaced by a starred-single occurrence.
In step 312, all characters in R are replaced by their equivalent HTML tag from step 302.
In step 314, a regular-expression tree is built on R, such that any nested HTML tag is represented as a hierarchy.
<B>(<A><TEXT></A><TEXT>)*</B>
A full regular expression tree serves as the basis for an initial template to be used to compare with documents in a training set, in one embodiment. However, as is discussed in the next section, the initial template can be generalized prior to comparing the template to training documents.
After initial creation, the template may have sub-trees that are approximately, although not exactly, the same. As an example,
In one embodiment, similar sub-trees in the template are merged and generalized using a similarity function on the paths of the template. In an embodiment, this generalization process involves two phases: i) identification of approximation locations and boundary; and ii) approximation methodology.
Initially, a set of candidate nodes in the template are identified for a determination as to whether a sub-tree of a particular candidate node has a similar sub-trees. For example, all STAR nodes are considered candidate nodes. The sub-tree associated with a particular STAR node may be compared with the sibling sub-trees of the same STAR nodes to look for similar sub-trees. The candidate nodes do not have to be STAR nodes, but could be any set of nodes. Typically, the candidate nodes will be the same type of nodes. In the following discussion, the template node whose sub-tree is under consideration for similar sub-trees is referred to as “fpa_node.”
A modified similarity function is used to find the boundary of match, in an embodiment. Initially, all “paths” within the selected template node, fpa_node, are determined. A path from an arbitrary node “p” is defined as a series of HTML tags starting from node p to one of the leaf nodes under node p.
The following example with respect to
Next, paths are computed for the siblings of fpa_node. These will be referred to as “sibling paths”. For example, sibling 611 has three sibling paths. The computed sibling paths are compared to the fpa_node paths to look for path matches. A path match occurs when a fpa_node path matches a sibling path, in an embodiment. In the following discussion, the “current sibling” refers to the sibling whose paths are currently being compared to the fpa_node paths. Based on the number of matching paths, a similarity score is computed, in an embodiment. The numerator is the number of fpa_node paths that have a match in the sibling paths. The denominator is the number of unique fpa_node paths and all sibling paths up until the current sibling. For example, referring to
If the current similarity score is at least a specified threshold, that sibling node is considered to be a “boundary”. As an example, if the threshold were 1/3, then sibling node 611 would be considered to be a boundary.
However, if current similarity score is not at least the specified threshold, then the paths from the next sibling node are combined and a similarity score is computed. Referring to
If there is a HOOK node present in a path under the fpa_node, then the HOOK node is only considered if there is a path under a sibling set that matches this “optional path”, in an embodiment.
Paths containing OR are weighed against each other such that the presence of any one of them is treated as a presence of the entire set, in an embodiment. For example, if there are three children to an OR node, then there will be at least three paths through this OR node—one through each of these three children. Note that there may be more than three paths if these children have a sub-tree below them; however, to facilitate explanation this example assumes there are only three paths. Because an OR node mandates that only one of each of the three paths is allowed, then if any one of this set of three paths is present in the sibling's paths, the entire set is treated as present, in an embodiment. Thus, a count of one is added to the numerator and denominator of the ratio fraction, if at least one of the paths under the OR node matches. Otherwise, a count of one is added only to the denominator.
Once merging happens successfully, the process is repeated for remaining sibling sub-trees. The merging is called “successful”, if the cost of modifying template is less than a cost threshold, otherwise merging is called “failed”. For example, the sub-trees associated with siblings 611 and 612 from
Once the boundary is identified, the template is generalized based on the segments. In an embodiment, generalizing the template based on the segments is performed using techniques discussed herein under the heading “GENERALIZING THE TEMPLATE BASED ON A TRAINING SET OF DOCUMENTS.” That section describes how a template can be generalized to match a single training document or partial document sub-tree. In the present example of generalizing the initial template, a portion of the template, referred to herein as a template component 670, is matched to other portions of the template, referred to herein as template segments or sub-trees. That is, template sub-trees corresponding to segments in the template are matched with the template component 670 to generalize the template component 670. In particular, first the template component 670 is generalized to match the first template segment 652, as shown in
The template includes either HTML nodes or nodes corresponding to one of the defined operators (e.g., STAR, HOOK, OR), in an embodiment.
Given a new document for learning, the DOM of the document is matched with the template in a depth first fashion, in an embodiment. By depth first, it is meant that processing proceeds from a parent node to the leftmost child node of the parent. After processing all of the leftmost child's subtrees in a depthmost fashion, the child to the right of the leftmost child is processed. When there is a mismatch between tags, a mismatch routine is invoked in order to determine whether to match the template to the DOM.
Comparing the template to the DOM depends on the type of operator that is the parent of a sub-tree in the template, in an embodiment. For example, if a STAR operator is encountered in the template, then the sub-tree of the STAR operator is compared to the corresponding portion of the DOM in accordance with STAR operator processing, as described below. Sub-trees having a HOOK operator or an OR operator as a parent node are processed in accordance with HOOK operator processing and OR operator processing respectively, in accordance with an embodiment.
Processing of a sub-tree under a STAR node in the template occurs by traversing the nodes in the sub-tree in a depthmost fashion, comparing the template nodes with the DOM nodes. If all children match at least once, then the STAR sub-tree matches the corresponding sub-tree in the DOM. As an example, referring to
After processing the leftmost subtree in the DOM 210, the rightmost subtree is compared to the template subtree 212, again because template contains a STAR node. Sub-tree 261 matches sub-tree 252. Sub-tree 263 contains three instances of td/u/text. Because of the STAR operator in sub-tree 254, the sub-trees match. That is, the DOM 210 is allowed to have one or more sub-trees td/u/text and be considered a match. Sub-tree 265 matches sub-tree 256. Note that sub-tree 256 has the optional path td/font/strike/text path.
When processing the STAR sub-tree 1505, if there is a mismatch between the STAR sub-tree 1505 and the sub-tree in the DOM under consideration for this cycle, a determination is made as to whether the STAR sub-tree 1505 has matched in the DOM at least once. If the STAR sub-tree 1505 has not matched even once, then the STAR sub-tree 1505 is said to have failed the match, and a mismatch routine is called. The mismatch routine is informed that the STAR sub-tree 1505 failed to match at all, in an embodiment. The mismatch routine is provided with the identity of the nodes which mismatched, in an embodiment. For example, referring to
In the current examples, the STAR node 1502 had a sibling to its right. That is, the STAR node 1502 and the D node are both children of the Z node, in
If the template node is a HOOK, the DOM node is matched with children of the HOOK node.
If the sub-tree under a HOOK node matches only partially with the sub-tree under the corresponding DOM node, the extent of match is recorded. The extent of the match may be based on the number of nodes in the sub-tree that do match and the number that do not match. For example, for the sub-tree of HOOK node 713, nodes C, D, and E match with the DOM sub-tree 721. However, since node G from the DOM sub-tree 721 is not found in the sub-tree of HOOK node 713 it is a mismatch. The extent of the mismatch can be expressed as a ratio, percentage, etc. that reflects that fact that three nodes match and one node does not match. Different nodes can have different weights when computing the extent of match. For example, nodes can be weighted based on their level. In one embodiment, nodes at a higher logical level in the tree are assigned a greater weight.
When a sub-tree in the DOM 704 fails to match a sub-tree in the template 702, it is matched with sub-trees that are rooted at template nodes that are siblings of the template node that was the root of the mismatch. This continues on until the root template node is not a HOOK node. For example, in template 702, the template node that is a mis-match is HOOK node 713. The next node is the F node, as processing is from left to right in this embodiment. Because the F node is not a HOOK node, this is the last node that is compared to the mismatched sub-tree 721 in the DOM 704. If there were more HOOK nodes between HOOK node 713 and node F, the subtrees of each of the HOOK nodes would be matched with the mismatched sub-tree 721. If any of these hypothetical template subtrees are an exact match with the mismatched sub-tree 721, then the mismatched sub-tree 721 would be considered to have matched with the template 702. However, if none of these hypothetical template sub-trees match the mismatched sub-tree 721, then one of the template sub-trees is selected to be modified such that it will match the mismatched sub-tree 721. In one embodiment, the template subtree that comes closest to matching the mismatched sub-tree 721is selected for modification.
Referring to
When comparing a template node to DOM node, if the names (e.g., tag names) do not match, then a mismatch routine is called with an indication of the mismatched template node and DOM nodes. It is possible that a node exists in the template 802 that has no corresponding node in the DOM 804 or vice versa. For example, the G node in the DOM 804 has no corresponding node in the template 802. For this type of mismatch, a mismatch routine is called with an additional indication that one of the two nodes (in DOM and Template) is absent. Note when processing an OR sub-tree, there is no requirement that an OR operator be added. For example, in
When a mismatch routine is called due to a mismatch between the template and the DOM, a determination is made as to whether to resolve the mismatch by generalizing the template. If the template is generalized, the mismatch is ensured to be resolved by adding an appropriate STAR, HOOK, or OR operator, thereby generalizing the template, in an embodiment. In an embodiment, when the mismatch routine is called, a template node “w” and a DOM node “d” are provided to the mismatch routine to indicate where a mismatch occurred. A mismatch can occur in two cases: (i) when the structure of the template and DOM have corresponding nodes, but the nodes not match with each other, and (ii) when the structure is such that a node is absent in either the template or the DOM. If there are corresponding nodes that do not match, then “w” and “d” are the corresponding nodes. If the template structure does not have a node that is present in the DOM, then the mismatch routine is called with “d” as the position under which the missing template structure should be added, with a flag set to indicate this special case. If the DOM structure does not have a node that is present in the template, then the mismatch routine is called with “w” as the position under which the missing DOM structure should be added, with a flag set to indicate this special case.
When a DOM node is to be added into the template, the DOM subtree is first normalized into a regular expression by finding repeated patterns in that subtree, in an embodiment. This is similar to how the regex is learned for the initial template, in an embodiment. Thus, in an embodiment, “adding a DOM node to the template” is accomplished by “adding a regex tree corresponding to the DOM node to the template”.
If a mismatch occurs because there is no DOM node to match a template node, the template node that is missing in the DOM is made optional, in step 912. For example, a HOOK node is added as the parent of the template node that is missing in the DOM.
If a mismatch occurs because there is no template node to match a DOM node, an attempt is made to add a STAR node, in step 922. If STAR node addition fails, then the DOM node that is missing in the template is added to the template as an optional (HOOK) node, in step 924.
The order in which the addition of operators to the template is attempted is in accordance with an embodiment of the present invention. Attempting to add operators in this order may help to generalize the existing structure before adding new changes. However, it is not required to attempt to add operators in the order depicted in
STAR addition is used to generalize the template by allowing, but not requiring, repetition of a group of subtrees, in an embodiment. This generalizing of the repetition includes identifying the largest group of subtrees that repeats, in an embodiment.
STAR addition may also be called when there is no template node to match a DOM node. For example, consider the template 1002 of
The portion of the template 1002 to the left of the boundary point is searched for an exact match to the subtree on d. In this example, the d subtree is represented by the triangle below d; therefore, the search “A” represents a search in the template 1002 for the d-sub-tree. The search continues to the left to the leftmost sibling of the boundary point. If no match is found, then the STAR addition routine returns as failed, and the mismatch routine attempts to solve the mismatch using a HOOK/OR node addition. In
All matches in the searched portion of the template 1002 are processed from the leftmost match first. The sequence of siblings from ti to the boundary point are designated as {ti, si1, si2, . . . , sik, boundaryPt}. The sibling subtrees {si1, si2, . . . , sik, boundaryPt} are matched with sibling subtrees in DOM in sequence. For example, from t1 to boundaryPt in the template 1002, the sibling subtree sequence is A, B, C, A, D, which matches with corresponding sibling subtrees in the DOM 1004.
If the matching succeeds from ti to the boundary point (boundaryPt), then a STAR is added over the template nodes from ti to the boundary point ({ti, si1, si2, . . . , sik, boundaryPt}), and the STAR addition routine returns successfully. For example, in the example in
If, however, the matching fails before the boundary point is reached, then next subtree ti+1 is considered versus the same starting point in the DOM. For example, the sibling subtrees starting at t2 to the boundary point would be compared with sibling subtrees in the DOM 1004 starting at the mismatch point to determine whether there is a match. For example, the sibling subtrees in the template 1002 between t2 to boundaryPt is the sequence A, D. The sequence A, D would be compared to the DOM starting at the mismatch point. The DOM sequence starting at the mismatch point is [A, B, C, A, D, E].
If no match is found for any sibling subtrees starting at any of the points {t1, t2, . . . , tn}, then matching is enforced for the sibling subtree sequence starting from the last subtree tn by calling a mismatch handling routine recursively. The matching continues to further siblings snj (calling mismatch wherever applicable). Finally, when the boundary point is reached, a STAR is added over the template nodes from tn to the boundary point ({tn, sn1, sn2, . . . , snk, boundaryPt}). The STAR addition routine returns as having succeeded.
It may be that a mismatch is “called within itself”. In order to resolve one mismatch (e.g., MMext), there might be another internal mismatch, MMint that needs to be resolved first. In such a scenario, because MMext is already partially resolved by processing the internal mismatch MMint, when handling MMext is not necessary to go all the way to the leftmost sibling, but only until a closer left boundary point is reached.
In one embodiment, if STAR node addition fails, an attempt is made to add a HOOK operator over a mismatched node. The mismatched node may be a node from the DOM or the initial template. In one embodiment, a one-step look-ahead is used. In another embodiment, a multi-step look-ahead is performed. One-step look ahead refers to stepping through the template or DOM only one-step (e.g., one node) for an exact match. For example, if the template is (A,B,C,D) and the DOM is (A,B,C,E,D), then, in one-step look-ahead, the E can be made optional by adding a HOOK over the E. That is, looking ahead one step is sufficient to determine that the D node in the template has a match in the DOM. Adding the HOOK to the template results in a complete match and also results in a relatively small cost of generalizing the template. However, if the DOM is (A,B,C,E,F,D), then one-step look-ahead may not resolve this mismatch as efficiently as multi-step look ahead. Multi-step look ahead refers to looking ahead more than one step (or node). In the present example, looking ahead at least two nodes would result in a determination that the D node in the template has a match in the DOM. However, looking ahead only a single node would not locate the D node in the DOM. Thus, the generalization to the template using one-step look ahead might incur a greater cost. The cost of generalizing the template is discussed in more detail below. In one embodiment, an attempt is made to add a HOOK operator using one-step look ahead rather than performing multi-step look-ahead.
The following example is presented to illustrate modifying the template 1102 by adding a HOOK node. First, a determination is made as to whether wrMismatchPt matches completely with the next sibling of domMismatchPt. Referring to
In some cases, the generalization in both
OR addition is called when both STAR and HOOK additions fail, in an embodiment. In one embodiment, OR addition is used as a last resort to enforce matching. The use of OR addition assures that the template will be matched to all of the DOMs in the training set, in an embodiment.
If the mismatched template node (WrMismatchPt) is already under an OR node in the initial template 1204, or if WrMismatchPt is itself an OR node, then a new OR node is not added to the new template 1206. Rather, the mismatched DOM node (DomMismatchPt) is added as a child of the existing OR node.
The operations defined in the above examples to resolve a mismatch work at the same logical level in the template as that of the mismatch point. By the “same logical level” it is meant that the mismatch is handled by adding operators at the same logical level in the template. As previously mentioned, for purposes of counting logical levels, operators (e.g., HOOK, OR, STAR) are not counted as a logical level. For purposes of discussion, logical levels will be counted upward when moving towards a leaf node.
In one embodiment, a set of operations referred to herein as “Cross Level STAR Addition” (CLSA) and “Cross Level HOOK Addition” (CLHA) are added to the template. The CLSA and CLHA are added by examining the initial template and the DOM at a level other than the level at which the mismatch occurred. In one embodiment, higher levels are examined to attempt to resolve the mismatch between the template and the DOM at a higher level.
When a mismatch occurs, after attempting to add a STAR operator at the same logical level as the mismatch, a determination is made as to whether a STAR operator can be added at a higher level. Referring to
In one embodiment, if attempting to add a HOOK operator at the same logical level as the mismatch fails, then before attempting to add an OR operator at the logical level of the mismatch, an attempt is made to add a HOOK operator at a higher level than the mismatch.
Template 1408 depicts a template that is generalized to match the DOM 1402b by performing CLHA. Note that a single HOOK operator 1422 has been added at the second logical level in order to modify the template to match the DOM 1402b. In this example, instead of adding an OR operator to resolve the mismatch at the third logical level, the mismatch points are first set to their respective parents to check if CLHA is applicable. Referring to DOM 1402b, the DOM mismatch point at the third logical level is moved to the parent at the second logical level. Referring to template 1404b, the template mismatch point at the third logical level is moved to the parent at the second logical level. In this example, CLHA succeeds. The mismatch points can be moved up by more than one level.
If neither CLSA nor CLHA succeeds, the mismatch can be resolved by adding an operator at the same level as the mismatch.
When the template is modified (or proposed to be modified), the template is said to incur a cost of generalization. This cost is the cost of modifying the template to match the current document completely, in an embodiment. A low cost implies that the current document is similar to the other documents in the training set used to build the template. On the other hand, a high cost implies relatively large differences and possibly that the current document is heterogeneous with respect to the rest of the training documents. In an embodiment, a threshold is specified for the cost wherein the template is not modified to match the current document if the cost would be too high. Thus, documents that are too dissimilar from the rest of the training documents are, in effect, removed from the training set.
The following are example factors that can be used to compute the cost. It is not required that all of the factors be used. Each factor can be weighed differently.
1) The size of the changed subtree (number of nodes in the subtree), S. The larger the size of the subtree added/modified, the higher is the cost of change.
2) The height (depth) of the subtree added/modified, H. In principle, on a modified subtree, the nodes added at the top of the subtree have more importance and hence incur higher cost than those at the bottom. It means that a cost of addition of a subtree of size S will be larger if it is a shallow tree (the subtree has lower H).
3) The level in the template which this change occurred, L, computed from the top of the template. The cost decreases exponentially with increasing L. This means that the changes towards the top of the tree incur more cost than those towards the bottom of the tree.
4) The operator added. In one embodiment, the STAR operator does not add any cost, since it generalizes the repetition count. In one embodiment, the OR operator induces cost based on whether it is added as a new node to the template or another disjunction is added to an existing OR node. In one embodiment, the HOOK operator cost depends on whether an existing structure in the template is made optional or a new optional subtree is added to the template.
A particular example of the cost function is Cost=S×101−[(L+H/2)/D], where D is the overall depth (height) of the template and used to normalize the numerator L+H/2. There can be many other such functions.
The cost of change is compared against the sizes of the original template and the current DOM. The size of the current template is computed similar to the one used to compute the cost of change—i.e., every node is weighed proportional to its height H in the template. The current page is said to make a significant change to the template if cost of change induced by the current page is more than a pre-determined fraction (say 30%) of the template and DOM sizes. The template and DOM size can be calculated in many other ways—by simply counting the number of nodes in the template/DOM to weighing them differently by their depth in the tree, relative importance, etc.
Techniques are disclosed herein for extracting attributes (e.g., title, price, description) from documents such as web pages. The documents have a defined structure such as a DOM. To extract an attribute from a new document, first a set of candidate nodes in the new document are identified based on their structural position in the document. The candidate nodes are nodes that might posses the attribute of interest. However, the set of candidate nodes may have “false positives”. That is, some of the candidate nodes might not possess the attribute. Therefore, a set of filters are applied to eliminate the false positives.
The filters are based on characteristics that the attribute has in a set of one or more training documents. For example, in the training document(s) the attribute may be characterized as having the value “bold” for an HTML font property. As another example, the attribute may be characterized as having a contextual format of text 1:text 2. That is, a Name:Value format appears in the text associated with the attribute. Based on the filtered candidate nodes, the attribute may then be extracted from the document. Thus, both the structural position of nodes in the new document and characteristics of the attribute in a set of one or more training documents are used to identify nodes in the new document that have the attribute of interest.
Prior to identifying the candidate nodes in the new document, a set of filters are learned based on one or more training documents. The filters can be learned based on only a single training document or a few training documents, which are labeled with attributes of interest. For example, a user can identify an attribute by labeling a node in a web page as being a title of interest.
To extract information for a particular attribute from a new document, first a set of candidate nodes in the new document are determined. This is achieved by determining which nodes in a DOM for the new document map to a template node that is associated with the attribute. For example, based on the learning phase, it is determined that the position of particular template node corresponds to the position of a node in a DOM that is known to have a title that is of interest. However, multiple DOM nodes could map to this template node. For example, the DOM could have many “title” nodes; however, not all of these are the title that is of interest. The title DOM nodes that map to the template node are identified as candidates for possessing the attribute of interest.
The candidate nodes are input into the filters, and based on the characteristics that the filters learned about the attribute, the filters score each candidate node. Based on the scores that the filters assigned to each candidate, zero or more of the candidate nodes are selected for extraction. In one embodiment, the candidate nodes are ranked based on the scores. In another embodiment, the candidate node having the highest score is identified for extraction.
In an embodiment, a filter assigns a confidence in a learned characteristic, based on analyses of the consistency of the characteristic across different pages. For example, if a filter indicates that a title is nearly always located in the third row of a table, the filter assigns a higher confidence to this characteristic than if the filter learns that the title is located in the third row about 65 percent of the time.
Even if incremental changes are made to the structure of new documents, nodes that posses the attributes can still be reliably identified. For example, the structure of a shopping web page might change by the addition of a new row to a table. The new and old rows will both map to the template because they will both have a “td/tr” format. However, the characteristics that were learned by the filters, such as the color of the title or the context of the title, can be used to accurately determine which of the rows has the attribute of interest.
There are multiple ways in which to capture and transfer annotations. In one embodiment, a human identifies attributes of interest from web pages. The human may mark relevant attributes on a webpage using an annotation tool. For example, using the annotation tool, the user highlights a section of a web page and labels it with an annotation such as “title”, “description”, “text”, “price”, “postal code”, “name”, “rating”, etc. These web page annotations can be transferred as annotations on to the corresponding nodes in the DOM structure of the webpage in accordance with known techniques.
In one embodiment, automated annotation techniques are used to augment the human provided annotations. Automatically annotating the DOMs can be based on information on the page or other appropriate pages. Examples of information that may be used to automatically annotate the page are data represented in a pre-defined schema, such as key-value pairs, labeled columns, etc. Other hints such as links into the page from a listing page, like a browse page or a search result page, are sources of annotation. In still another embodiment, no human annotation is performed.
In one embodiment, the template nodes are annotated with attributes when the template is learned based on a set of training documents. For example, a training set of documents may be used when generalizing the template as discussed in the section “GENERALIZING THE TEMPLATE TREE BASED ON A TRAINING SET OF DOCUMENTS.” A user may annotate nodes of interest in one or more of these training documents. During the template matching phase, the attribute annotations on the DOM nodes are mapped to the template. Thus, the template nodes that structurally correspond to DOM nodes are annotated with attributes of interest.
In step 1606, the training document is analyzed to learn characteristics that the attribute possesses in the training document. In one embodiment, In step 1608, information is stored that associates the attribute with the learned characteristics.
In step 1704, characteristics of the candidate nodes are compared with characteristics that are associated with the attribute. The characteristics are those learned in step 1306 of process 1300, in an embodiment. In step 1706, at least one of the candidate nodes is eliminated from consideration as possessing the attribute, based on the comparison of step 1704. Step 1706 describes the case in which at least one candidate node is eliminated. It is possible that no candidate node is eliminated from consideration.
In step 1708, information is extracted from the document for at least one candidate node that has not been eliminated from consideration as possessing the attribute. Step 1708 describes the case in which there is information to be extracted from the document for at least one candidate node. It is possible that there will not be information to extract for any of the candidate nodes that remain.
A filter 1803 is a module that works to reduce the false positives from a set of generated candidates for an attribute. In the learning phase, each filter 1803 inputs a set of positive candidates (PosCands) and possibly a set of negative candidates (NegCands). The negative candidates are optional. A PosCand is a node that has been marked in a training document 1801 as having the desired attribute and a NegCand is a node that the user has marked as spurious. For example, a user identifies a particular title in a web page and annotates it as a PosCand. The user might annotate a different title in the training document as a NegCand.
The PosCands and the NegCands in the training document(s) 1801(1)-1801(m) map to node(s) in the template 1806. The template 1806 is a tree structure that has been generalized to match the structure of a set of structurally related training documents, in an embodiment. It is possible for multiple nodes in the training document 1801 to map to the same node in the template 1806. It is possible for some such training document nodes to not be labeled as either a PosCand or a NegCand. These document nodes that map to the same template node as either a PosCand or a NegCand are referred to as unlabeled nodes (UnlabCands).
Consider, for example, a filter for a price attribute. A PosCand is a training document node that the user has selected as having the price attribute. Because documents such as web pages may have repeating patterns, there can be more than one training document node that maps to same template node. Because the user has not annotated such nodes, it is unknown whether or not they have the price attribute. NegCands set can be formed in cases where the user specifies the undesirable nodes as well.
The output of each filter 1803 are “stored learnings” 1808. The filters 1803 learn on a per attribute basis. At least one of the filters 1803 is able to assign confidence based on analyses of the consistency of the filter's output across different pages. In other words, the confidence is based on how repetitive the filter output is for different training documents that are eventually considered to posses a particular attribute. For example, if a filter 1803 indicates that a title is nearly always located in the third row of a table, the filter 1803 may assign a higher confidence than a filter 1803 that indicates that a title is located in the third row about 65 percent of the time. The filter 1803 can assign a confidence on a per attribute basis, or a confidence that is independent of attribute. For example, it might be that the filter 1803 works quite well for a title attribute, but not for an address attribute. Also note that a filter 1803 can assign a different weight for each cluster of documents. Examples of different types of filters are described below.
For each attribute of interest, the candidate generation logic 1902 outputs a separate set of candidate nodes from the new document 1901. The new document 1901 is compared with the template 1806 to find the candidate nodes. In particular, at least one of the nodes in the template 1806 is associated with one or more attributes of interest. Steps 1602 and 1604 of process 1600 describe one embodiment for associating a template node with the attribute of interest. The candidate generation logic 192 compares the structure of the new document 1901 with the structure of the template 1806 to identify candidate nodes in the new document 1901. All these candidate nodes are considered as UnlabCands set for the respective attributes, in an embodiment.
In some cases, the attribute of interest may cover multiple nodes in the new document 1901. In such cases, the lowest common ancestor (“lca”) node may be marked as the candidate node and the actual set of nodes is described by mentioning the start and end paths from the lca node. A start (or end) path is a series of node identifiers from the lca node to the start (or end) position of the actual set of nodes.
For purposes of illustration, this section describes a few example filters 1803. During the extraction phase, some of the filters 1803 output a score that is based on a probability that a candidate node possess an attribute of interest. Other filters 1803 perform a “text manipulation”, such as extracting a relevant portion of the text associated with a candidate node. The scoring filters 1803 may base their analysis on the extracted portion of the text, although a scoring filter could also analyze non-extracted text. A filter that performs text manipulation can also output a candidate score.
From the given PosCands, the Property Based Filter finds values of the given format property (e.g., HTML-based text-formatting properties, such as font color, size, stylesheet class, etc.) and stores its confidence across pages. The confidence of a (property, value) pair (p, v) in determining a PosCand may be defined as the probability of the candidate being a PosCand given that the property p takes a specific value v [Pr(class=+ve|property p=value v)]. As an example, the property based filter might learn that bold font is a positive property, blue color is a positive property, red color is a negative property etc. More particularly, the filter may learn that if a candidate node has a blue color, then there is an “x” percent probability that the candidate node has the attribute of interest. Sufficient statistics may be kept to count the number of candidates in which the property was marked as positive/negative by the user such that the probabilities can be learned with desired accuracy.
The Position Based Filter finds the position of the candidate among the candidates generated under the lowest containing STAR node of the template, in one embodiment. As previously discussed, a STAR node in a template indicates multiple occurrences of the underlying template structure are allowed. Hence, if a candidate node maps to a template node under a STAR node, there are potentially many other DOM candidate nodes that map to the same template node. The relative position of the correct candidate in this set is learned by the Position Based Filter. As a particular example, a table in the document may have many rows. Each row is represented by a separate DOM node. However, the template has STAR node and a single node under the STAR to represent that any number of rows are allowed at that structural position. Similar to the Property Based Filter, sufficient statistics may be kept as to where the user-marked PosCands or NegCands are found at a particular DOM node. The confidence may also be determined in a similar fashion, as Pr(class=+ve|position=value v)].
The Range Pruner learns the relative range position of the required text associated with the attribute. The range is defined as the start and end path under the candidate node and the word offsets within the start and end nodes. The learning may be generalized relative to node boundary and number of siblings. The Range Pruner ensures extraction of correct text where a set of nodes form the required text.
The Contextual Filter finds and learns the context around the attribute of interest and outputs a candidate score based on the learned context. Due to the presence of optional information, the position of the desired candidate (in a set of generated candidates) can change from one page to another. For example, the table row that contains a price attribute may vary from one page to the next. Therefore, the position based filter may have a low confidence.
In such cases, the contextual filter may help to detect the correct candidate. An example of such a filter is a Name-Value Pair (NVP) filter. A NVP may occur either as a table or in free text. The table-based NVPs either have names in one column and values in the other (“column major headers”), or have table headers as names and elements in the table as values (“row major headers”). Text-based NVPs have names and values as free text often separated by ‘:’ with names being bold occasionally.
Table based NVP Filters search for a table with row major or column major header, while text based NVPs search for presence of name nodes near the value node and subsequently rely on the Range Pruner to extract the correct text. The presence of a learned context around a candidate on a new page will boost the candidate's overall score. The context filter may be a very strong filter that allows accurate extraction of attributes even if the position of the required text for the attribute varies from one page to the next.
Another kind of Contextual filter is a Prefix—Suffix filter that learns the text that precedes (or succeeds) the text of interest. On finding the preceding and succeeding text on a new page, the content within these is selected as the desired text.
The Regex Filter checks if text associated with an attribute matches a desired data format (e.g., regular expression). Candidates having the desired data format may receive a boost to the scores generated by other filters 1803. The regular expression may be given as a configurable input or, alternatively, may be learned based on the PosCands or NegCands given to the Regex Filter. An example of the regex filter is to learn that a date attribute has the format “dd/mm/yy”, wherein dd is a value between 1 and 31, mm is either a value between 1 and 12 or a textual value corresponding to one of the months, and yy is an integer between 0 and 99.
A filter may perform operations other than scoring. Sometimes, the desired extraction is not what text appears within an HTML tag, but some other aspect of the tag. For example, when an image is selected, a ‘src’ attribute may need to be extracted. Similarly for a hyperlinked text, it may be more appropriate to extract where the link points to (the ‘href’ attribute). The Tag-specific filter performs this task of extracting the appropriate attribute from the specified tag.
In one embodiment, a filter performs a text manipulation operation. An example of a text manipulation is to extract a portion of the text. As a particular example, for a node having the text “this camera sells for $300.00”, the text “$300.00” is extracted. It is possible for other filters 1803 to perform their analysis based on the manipulated version of the text.
Computer system 2100 may be coupled via bus 2102 to a display 2112, such as a cathode ray tube (CRT), for displaying information to a computer user. An input device 2114, including alphanumeric and other keys, is coupled to bus 2102 for communicating information and command selections to processor 2104. Another type of user input device is cursor control 2116, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 2104 and for controlling cursor movement on display 2112. This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane.
The invention is related to the use of computer system 2100 for implementing the techniques described herein. According to one embodiment of the invention, those techniques are performed by computer system 2100 in response to processor 2104 executing one or more sequences of one or more instructions contained in main memory 2106. Such instructions may be read into main memory 2106 from another machine-readable medium, such as storage device 2110. Execution of the sequences of instructions contained in main memory 2106 causes processor 2104 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the invention. Thus, embodiments of the invention are not limited to any specific combination of hardware circuitry and software.
The term “machine-readable medium” as used herein refers to any medium that participates in providing data that causes a machine to operation in a specific fashion. In an embodiment implemented using computer system 2100, various machine-readable media are involved, for example, in providing instructions to processor 2104 for execution. Such a medium may take many forms, including but not limited to storage media and transmission media. Storage media includes both non-volatile media and volatile media. Non-volatile media includes, for example, optical or magnetic disks, such as storage device 2110. Volatile media includes dynamic memory, such as main memory 2106. Transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise bus 2102. Transmission media can also take the form of acoustic or light waves, such as those generated during radio-wave and infra-red data communications. All such media must be tangible to enable the instructions carried by the media to be detected by a physical mechanism that reads the instructions into a machine.
Common forms of machine-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punchcards, papertape, any other physical medium with patterns of holes, a RAM, a PROM, an EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium from which a computer can read.
Various forms of machine-readable media may be involved in carrying one or more sequences of one or more instructions to processor 2104 for execution. For example, the instructions may initially be carried on a magnetic disk of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 2100 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal. An infra-red detector can receive the data carried in the infra-red signal and appropriate circuitry can place the data on bus 2102. Bus 2102 carries the data to main memory 2106, from which processor 2104 retrieves and executes the instructions. The instructions received by main memory 2106 may optionally be stored on storage device 2110 either before or after execution by processor 2104.
Computer system 2100 also includes a communication interface 2121 coupled to bus 2102. Communication interface 2121 provides a two-way data communication coupling to a network link 2120 that is connected to a local network 2122. For example, communication interface 2121 may be an integrated services digital network (ISDN) card or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, communication interface 2121 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links may also be implemented. In any such implementation, communication interface 2121 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.
Network link 2120 typically provides data communication through one or more networks to other data devices. For example, network link 2120 may provide a connection through local network 2122 to a host computer 2124 or to data equipment operated by an Internet Service Provider (ISP) 2126. ISP 2126 in turn provides data communication services through the world wide packet data communication network now commonly referred to as the “Internet” 2128. Local network 2122 and Internet 2128 both use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on network link 2120 and through communication interface 2121, which carry the digital data to and from computer system 2100, are exemplary forms of carrier waves transporting the information.
Computer system 2100 can send messages and receive data, including program code, through the network(s), network link 2120 and communication interface 2121. In the Internet example, a server 2130 might transmit a requested code for an application program through Internet 2128, ISP 2126, local network 2122 and communication interface 2121.
The received code may be executed by processor 2104 as it is received, and/or stored in storage device 2110, or other non-volatile storage for later execution. In this manner, computer system 2100 may obtain application code in the form of a carrier wave.
In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details that may vary from implementation to implementation. Thus, the sole and exclusive indicator of what is the invention, and is intended by the applicants to be the invention, is the set of claims that issue from this application, in the specific form in which such claims issue, including any subsequent correction. Any definitions expressly set forth herein for terms contained in such claims shall govern the meaning of such terms as used in the claims. Hence, no limitation, element, property, feature, advantage or attribute that is not expressly recited in a claim should limit the scope of such claim in any way. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.
In addition, in this description certain process steps are set forth in a particular order, and alphabetic and alphanumeric labels may be used to identify certain steps. Unless specifically stated in the description, embodiments of the invention are not necessarily limited to any particular order of carrying out such steps. In particular, the labels are used merely for convenient identification of steps, and are not intended to specify or require a particular order of carrying out such steps.
This application is related to U.S. patent application Ser. No. 11/481,809, filed on Jul. 5, 2006, entitled “TECHNIQUES FOR CLUSTERING STRUCTURALLY SIMILAR WEB PAGES BASED ON PAGE FEATURES”, the entire content of which is incorporated by reference for all purposes as if fully disclosed herein. This application is related to U.S. patent application Ser. No. 11/481,734, filed on Jul. 5, 2006, entitled “TECHNIQUES FOR CLUSTERING STRUCTURALLY SIMILAR WEB PAGES”, the entire content of which is incorporated by reference for all purposes as if fully disclosed herein. This application is related to U.S. patent application Ser. No. 11/838,351, filed on Aug. 14, 2007, entitled “METHOD FOR ORGANIZING STRUCTURALLY SIMILAR WEB PAGES FROM A WEB SITE”, the entire content of which is incorporated by reference for all purposes as if fully disclosed herein. This application is related to U.S. patent application Ser. No. ______ (Atty. Dkt. 50269-0944) filed on ______, entitled “TECHNIQUES FOR INDUCING HIGH QUALITY STRUCTURAL TEMPLATES FOR ELECTRONIC DOCUMENTS”, the entire content of which is incorporated by reference for all purposes as if fully disclosed herein.
