The present disclosure is related to the following commonly-assigned co-pending U.S. patent applications:
The respective disclosures of these applications are incorporated herein by reference for all purposes.
The present invention relates in general to information delivery systems and methods, and in particular to systems and methods for presenting information based at least in part on publisher-selected labels. The labels are applied to content items by publishers and used to identify related content items in various situations.
The World Wide Web (Web), as its name suggests, is a decentralized global collection of interlinked information—generally in the form of “pages” that may contain text, images, and/or media content—related to virtually every topic imaginable. A user who knows or finds a uniform resource locator (URL) for a page can provide that URL to a Web client (generally referred to as a browser) and view the page almost instantly. Since Web pages typically include links (also referred to as “hyperlinks”) to other pages, finding URLs is generally not difficult.
What is difficult for most users is finding URLs for pages that are of interest to them. The sheer volume of content available on the Web has turned the task of finding a page relevant to a particular interest into what may be the ultimate needle-in-a-haystack problem. To address this problem, an industry of search providers (e.g., Yahoo!, MSN, Google) has evolved. A search provider typically maintains a database of Web pages in which the URL of each page is associated with information (e.g., keywords, category data, etc.) reflecting its content. The search provider also maintains a search server that hosts a search page (or site) on the Web. The search page provides a form into which a user can enter a query that usually includes one or more terms indicative of the user's interest. Once a query is entered, the search server accesses the database and generates a list of “hits,” typically URLs for pages whose content matches keywords derived from the user's query. This list is provided to the user. Since queries can often return hundreds, thousands, or in some cases millions of hits, search providers have developed sophisticated algorithms for ranking the hits (i.e., determining an order for displaying hits to the user) such that the pages most relevant to a given query are likely to appear near the top of the list. Typical ranking algorithms take into account not only the keywords and their frequency of occurrence but also other information such as the number of other pages that link to the hit page, popularity of the hit page among users, and so on.
To further facilitate use of their services, some search providers now offer “search toolbar” add-ons for Web browser programs. A search toolbar typically provides a text box into which the user can type a query and a “Submit” button for submitting the query to the search provider's server. Once installed by the user, the search toolbar is generally visible no matter what page the user is viewing, enabling the user to enter a query at any time without first navigating to the search provider's Web site. Searches initiated via the toolbar are processed in the same way as searches initiated at the provider's site; the only difference is that the user is spared the step of navigating to the search provider's site.
One technique for helping a user find content is to provide an interface via which the user can request “related” pages. Pages can be identified as related based on similarity of their content to that of the currently viewed page and/or whether the pages are published by the same entity. As implemented in existing systems, neither technique is very reliable.
Existing algorithms for identifying related pages based on similarity of content generally rely on overlap of textual elements (words, phrases, etc.) between the current page and the related page. The “best” matches according to such algorithms have the most overlap with the current page; however, the pages with the most overlap are often least interesting to the user, who typically wants to find pages with different information on the same subject. Determining whether two pages relate to the same subject is a difficult task, as it requires determining the subject of each page, which might or might not be evident from the words used.
Identifying pages published by the same entity is sometimes easier but is of limited help to the user. The publisher's own pages can sometimes be identified by URL, on the assumption that URLs beginning with the same domain name are commonly owned, but this assumption is not always reliable. For instance, some domains host content created by multiple independent publishers, and some publishers use multiple domains. Further domain-name matching does not provide a way to identify affiliates of a publisher, since the affiliates typically use different domains.
Therefore, it would be desirable to provide systems and methods for more efficiently identifying related content.
Embodiments of the present invention provide systems and methods allowing publishers to define relationships between their own content items (e.g., World Wide Web pages) and content items created or maintained by other publishers. A central namespace manager provides grouping labels that publishers can insert into their content items, and content items having matching grouping labels are identified as related. Different classes of grouping labels can be used to define different types of relationships, and multiple classes of grouping labels can co-exist. For instance, in one embodiment, one class of grouping labels is used to indicate a personal or business connection between or among publishers and another class is used to indicate a subject-matter relationship between content items. Grouping labels are advantageously not displayed when the content item is displayed for a user but are made available to an information server such as a search server, which uses the labels in identifying content items related to a currently displayed content item. For example, if a user viewing a first content item requests similar or related items, a grouping label included in the first content item can be used to select appropriate items. In another embodiment, in-line or pop-up advertisements to be displayed while the user is viewing a target content item are selected based at least in part on a grouping label included in the target content item. In still another embodiment, grouping labels included by publishers in their content items are used to construct a subject-matter directory of content items.
According to one aspect of the present invention, a method for providing information from a corpus of content items to a client is provided. A number of grouping labels are defined, where each grouping label is includable by publishers in content items in a corpus. An index is automatically built, with the index being configured to identify, for each of the plurality of grouping labels, each of the content items in the corpus that includes that grouping label. A request is received from a client for identification of content items related to a first content item, with the request including a first one of the plurality of grouping labels extracted from the first content item. The index is accessed to identify a second content item having the first one of the plurality of grouping labels included therein, and a reference to the second content item is returned to the client. In some embodiments, the grouping labels are included in a contextual search code block that is part of the content item and are extracted from the code block when the user requests related content.
Various classes, or types, of grouping labels can be defined. For instance, in some embodiments, each grouping label corresponds to a group of affiliated publishers, and in other embodiments, each grouping label corresponds to a category identifier for content items. Multiple classes of grouping labels may coexist in one embodiment, and a given content item might include a grouping label from any, all, or none of the classes at the publisher's discretion.
Where the grouping labels correspond to category identifiers that form a hierarchical taxonomy, a directory of content items might be built based on the hierarchical taxonomy of category identifiers; each content item that has at least one grouping label included therein can be listed in the directory under each category identifier that corresponds to one of the grouping labels included in that content item.
The index can be built in various ways. For instance, if the content items are Web pages, a Web crawler can be operated to locate and process a plurality of Web pages. During processing of each Web page, the presence of any of the plurality of grouping labels is detected, and information about the detected grouping labels for each Web page is added to a page index built by the Web crawler. The page index can be inverted to create an index of the content items by grouping label.
In another embodiment, the index is built as requests are received from clients. For instance, in response to a request for content related to the first content item, the first content item can be added to the index in association with the first grouping label.
According to another aspect of the present invention, a method of retrieving information is provided. A client system displays a first content item, the first content item including an interface element operable to request related content, the first content item further including a first grouping label associated with the interface element. In response to user operation of the interface element, the client system transmits a request for related content to a server system, the request including the grouping label. The client system receives a reference to a related content item selected by the server system in response to the request; the related content item also includes the first grouping label. The client system displays the reference to the related content item.
According to still another aspect of the present invention, a method for providing information from a corpus of content items to a client is provided. A number of grouping labels are defined, with each grouping label corresponding to a node in a hierarchical taxonomy of content categories and each grouping label being includable by a plurality of publishers in a plurality of content items in a corpus. A directory is automatically built; the directory is configured to identify, for each of the grouping labels, each of the content items in the corpus that includes that grouping label. A client interface is provided, the client interface including controls for navigating among the nodes in the hierarchical taxonomy of content categories; upon navigation to a particular node in the hierarchical taxonomy, the client interface presents a listing of at least some of the content items identified in the directory as including the grouping label that corresponds to the particular node in the hierarchical taxonomy.
The directory can be built in various ways. In one embodiment, building the directory includes determining, for each of the plurality of grouping labels, an order of presentation for the content items in the corpus that include that one of the grouping labels. The order of presentation may be based at least in part on user ratings of the content items in the corpus that include the one of the grouping labels, a relevance score (relative to a category name that corresponds to the grouping label) determined for each content item that includes the one of the grouping labels, and/or other information about the content items.
According to a further aspect of the present invention, an information retrieval and communication system includes a namespace data store, an index data store, and an information server. The namespace data store is configured to store a plurality of grouping labels, with each grouping label being includable by publishers in content items in a corpus. The index data store is configured to store an index identifying, for each of the grouping labels stored in the namespace data store, each of the content items in the corpus that includes that grouping label. The information server includes a client interface module and an index interface module. The client interface module is configured to receive requests from a client computer system for identification of content items related to a starting content item, where each request includes a grouping label extracted from the starting content item. The index interface module is configured to access the index to identify a second content item having the grouping label extracted from the starting content item included therein, and the client interface module is further configured to return a reference to the second content item to the client computer system.
In some embodiments, the system also includes a namespace manager communicably coupled to the namespace data store and configured to provide a namespace interface configured to support searching the plurality of grouping labels, the namespace interface being accessible to the publishers. The namespace manager can also be configured to generate a new grouping label in response to a request from one of the plurality of publishers. In some embodiments, the namespace data store is further configured to store, in association with each of the grouping labels, a characteristic value and wherein the namespace manager is further configured to support searching for a desired characteristic value. The characteristic value might be, e.g., a category name or other term descriptive of subject matter of a content item.
In other embodiments, the system includes an index builder configured to create the index and store the index in the index data store. For instance, the index builder might be configured to navigate from one content item to another and generate index information for each content item to which the index builder navigates. In other embodiments, the information server is further configured to maintain a log of requests received from client systems, the log including an identifier of each starting content item in association with the grouping label extracted from the starting content item, and the index builder is further configured to generate index information from the log.
The following detailed description together with the accompanying drawings will provide a better understanding of the nature and advantages of the present invention.
Embodiments of the present invention provide systems and methods allowing publishers to define relationships between their own content items (e.g., World Wide Web pages) and content items created or maintained by other publishers. A central namespace manager provides grouping labels that publishers can insert into their content items, and content items having matching grouping labels are identified as related. Different classes of grouping labels can be used to define different types of relationships, and multiple classes of grouping labels can co-exist. For instance, in one embodiment, one class of grouping labels is used to indicate a personal or business connection between or among publishers and another class is used to indicate a subject-matter relationship between content items. Grouping labels are advantageously not displayed when the content item is displayed for a user but are made available to an information server such as a search server, which uses the labels in identifying content items related to a currently displayed content item. For example, if a user viewing a first content item requests similar or related items, a grouping label included in the first content item can be used to select appropriate items. In another embodiment, in-line or pop-up advertisements to be displayed while the user is viewing a target content item are selected based at least in part on a grouping label included in the target content item. In still another embodiment, grouping labels included by publishers in their content items are used to construct a subject-matter directory of content items.
A. Network Implementation
Several elements in the system shown in
According to one embodiment, client system 20 and all of its components are operator configurable using an application including computer code run using a central processing unit such as an Intel Pentium™ processor, AMD Athlon™ processor, or the like or multiple processors. Computer code for operating and configuring client system 20 to communicate, process and display data and media content as described herein is preferably downloaded and stored on a hard disk, but the entire program code, or portions thereof, may also be stored in any other volatile or non-volatile memory medium or device as is well known, such as a ROM or RAM, or provided on any media capable of storing program code, such as a compact disk (CD) medium, a digital versatile disk (DVD) medium, a floppy disk, and the like. Additionally, the entire program code, or portions thereof, may be transmitted and downloaded from a software source, e.g., from one of server systems 501 to 50N to client system 20 over the Internet, or transmitted over any other network connection (e.g., extranet, VPN, LAN, or other conventional networks) using any communication medium and protocols (e.g., TCP/IP, HTTP, HTTPS, Ethernet, or other conventional media and protocols).
It should be appreciated that computer code for implementing aspects of the present invention can be C, C++, HTML, XML, Java, JavaScript, etc. code, or any other suitable scripting language (e.g., VBScript), or any other suitable programming language that can be executed on client system 20 or compiled to execute on client system 20. In some embodiments, no code is downloaded to client system 20, and needed code is executed by a server, or code already present at client system 20 is executed.
B. Information Retrieval System
Content servers 1501 to 150N store content items such as content item 155 (e.g., Web pages, images, media files) that can be provided on demand to client system 120 via Internet 140. Content servers 1501 to 150N may support various information sharing protocols such as HTTP and/or other known protocols. It is to be understood that any number N of content servers may be present, and each content server may provide any number of content items. Further, client access to some or all of the content items may be limited to authorized users as is known in the art.
Content items are created and stored at content servers 1501 to 150N by publishers; as used herein, the term “publisher” refers to any entity (including persons, organizations and/or computer systems) responsible for creating and/or maintaining a content item on any one of content servers 1501 to 150N. In accordance with an embodiment of the present invention, publishers can select grouping labels such as the “Qgroup” and “Qcat” labels shown in content item 155 to be included in their content items. As described below, grouping labels can be provided to the server and/or client systems of
In some embodiments, content items such as content item 155 may also include a context vector. The “context vector” may contain any data representative of the content (or of some portion thereof). For instance, a context vector can include a list of terms (words or phrases) appearing in the content item, and each term may have an associated weight based on the term frequency (i.e., number of occurrences of the term in the content item), document frequency (i.e., number of content items in the corpus that contain the term), or the like. Examples of context vectors are described, e.g., in above-referenced application Ser. No. 10/903,283 and application Ser. No. 11/033,100.
1. Client System
According to one embodiment, a client application (represented as module 125) executing on client system 120 includes instructions for controlling client system 120 and its components to communicate with server systems 150 and 160 and to process and display data content received therefrom. Client application 125 may be transmitted and downloaded to client system 120 from a software source such as a remote server system (e.g., server systems 150, server system 160 or other remote server system), or client application module 125 may also be provided on any software storage medium (floppy disk, CD, DVD, etc.) that is readable by client system 120 as discussed above. For example, in one aspect, client application 125 may be provided over Internet 140 to client system 120 in an HTML wrapper including various controls such as, for example, embedded JavaScript or Active X controls, for manipulating data and rendering data in various objects, frames and windows.
Client application module 125 advantageously includes various software modules for processing data and media content. In one embodiment, these modules include a specialized search module 126, a user interface module 127, and an application interface module 128. Specialized search module 126 is configured for processing search requests (also referred to herein as queries) to be sent to search server 160 and search result data received from search server 160.
User interface module 127 is configured for rendering data and media content in text and data frames and active windows, e.g., browser windows and dialog boxes. In some embodiments, user interface module 127 includes or communicates with a browser program, which may be a default browser configured on client system 120 or a different browser. Application interface module 128 is configured to support interfacing and communicating between client application 125 and various other applications executing on client 120, such as e-mail applications, instant messaging (M) applications, browser applications, document management applications and others.
User interface module 127 advantageously provides user input interfaces allowing the user to enter queries for processing by search server system 160. For example, where user interface module 127 includes or communicates with a browser, the user may be able to enter a URL or activate a control button to direct the browser to a Web search page (or site) from which the user can submit a query to search server system 160 for processing. In addition or instead, user interface module 127 may include a search toolbar or other interface via which the user can enter and submit a query without first navigating to the search page. Queries entered using user interface module 127 may be preprocessed by specialized search module 126 prior to being sent to search server system 160, e.g., to remove so-called “stop words” (“the,” “and,” etc.), to correct spelling errors, or the like.
In some embodiments, client application 125 may include various features for adding context data (referred to herein as a “context vector”) to the user's queries. For example, specialized search module 126 may be configured to generate context vectors based on content the user is currently viewing at the time a query is entered. As another example, in some embodiments of the present invention, Web pages displayed in the browser may include one or more context vectors that can be used to supplement user-entered queries. User interface module 127 may be configured to detect such context vectors in a page being displayed and use context vector data to supplement a query entered by the user.
2. Search Server System
According to one embodiment of the invention, search server system 160 is configured to provide search result data and media content to client system 120, and content server systems 1501 to 150N are configured to provide data and media content such as web pages to client system 120, for example, in response to links selected by the user in search result pages provided by search server system 160. In some variations, search server system 160 returns content as well as, or instead of, links and/or other references to content.
Search server system 160 references various page indexes 170 that are populated with, e.g., pages, links to pages, data representing the content of indexed pages, etc. Page indexes may be generated by various collection technologies such as an automatic web crawler 172; in addition, manual or semi-automatic classification algorithms and interfaces may be provided for classifying and ranking web pages within a hierarchical category structure. Such technologies and algorithms may be of generally conventional design, and a detailed description is omitted as not being critical to the present invention.
In one embodiment, an entry in page index 170 includes a search term, a reference (e.g., a URL or other encoded identifier) to a page in which that term appears and a context identifier for the page. The context identifier may be used for grouping similar results for search terms that may have different meanings in different contexts. For example, the search term “jaguar” may refer to the British automobile, to an animal, to a professional football team, and so on. The context identifier for a page can be used to indicate which of these contexts is applicable. In one embodiment, the context identifier includes a category for the page, with the category being assigned from a predefined hierarchical taxonomy of content categories. A page reference may be associated with multiple context identifiers, so the same page (or a link thereto) may be displayed in multiple contexts. In some embodiments, context identifiers are automatically associated with page links by the system as users perform various searches; the identifiers may also be modified and associated with links manually by a team of one or more index editors.
Page index 170 may also store the grouping label(s), if any, included in the page, allowing the corpus to be searched by grouping label. As described below, in some embodiments, one or more indices that index the corpus by grouping label are also created; Qcat index 173 is an index of pages by Qcat label, and Qgroup index 174 is an index of pages by Qgroup label. In other embodiments, page index 170 can be searched by Qcat label and/or Qgroup label, and building a separate Qcat index and/or Qgroup index is not required.
Search server system 160 is configured to provide data responsive to various search requests received from a client system 120, in particular from search module 126 and/or user interface module 127. For example, search server system 160 may include a query response module 164 that is configured with search related algorithms for identifying and ranking Web pages relative to a given query, e.g., based on a combination of logical relevance (which may be measured by patterns of occurrence of search terms in the query), context identifiers, page sponsorship, etc.
In some embodiments, query response module 164 is also configured to receive and make use of context vector data that may be provided in association with a query in order to further enhance the response to queries. Query response module 164 may also enhance search result information with additional information (e.g., links and/or advertising copy) obtained from a sponsored content database 162. Sponsored content database 162 may be implemented as part of page index 170 by the inclusion of additional fields in each entry to identify page references that are sponsored and keywords for triggering the display of sponsored content, or it may be implemented in a separate database.
In some embodiments, search server 160 also includes a context processing module 166 that is configured with various algorithms for processing received content to generate a context vector representative of the received content. In general, a context vector may include any data that represents all or part of the content. For example, one embodiment of a context vector for text content may include keywords such as terms (e.g., words or phrases) that appear in the content, and each such term may have an associated frequency count or a weight based on term frequency (the number of times the term occurs in the received content) and/or document frequency (e.g., the number or fraction of documents in the corpus that contain that term) and/or other parameters reflecting the likelihood that a particular term is a reliable indicator of context. Other types of data may also be included, e.g., URLs or other data identifying any links that may be included in the content, the URL or other identifier of the page that contains the content, category data associated with the content or with a page that contains the content, and so on.
3. Labeling and Indexing System
As noted above, content item 155 and other content items can include grouping labels. As used herein, a “grouping label” includes any piece of data (e.g., an alphanumeric string or numerical value) that can be inserted into a document by a publisher and recognized by various servers as indicating that the document belongs to a group of related documents. In preferred embodiments, the grouping label is included in an HTML tag or other control structure so that it is detectable by suitably configured computer systems but does not appear in the displayed page.
Grouping labels can be used to indicate various relationships that can exist between content items. For instance, two documents addressing the same subject matter might be considered related, as might two documents produced by the same publisher, two documents produced by affiliated publishers, and so on. In some embodiments, multiple classes of grouping labels are supported, allowing different types of relationships to be defined.
For purposes of the present description, two classes of grouping labels, referred to herein as Qcat and Qgroup labels, are used. A “Qcat” label is a label indicative of the subject matter of the document. In some embodiments, a hierarchical taxonomy of category names is defined, and each node in the taxonomy is assigned a unique Qcat label. The publisher of a particular content item selects a suitable category name from the taxonomy and includes the corresponding Qcat label in the content item. All content items containing a particular Qcat label are presumed to be related to each other by subject matter.
A “Qgroup” label is a label that indicates a personal or professional affiliation between publishers of different documents. In some embodiments, a publisher (referred to herein as a “founding” publisher) who wants to be affiliated with other publishers creates a unique Qgroup label. The founding publisher includes this Qgroup label in its own content items and invites other publishers to include the same Qgroup label in their items as well. All content items containing a particular Qgroup label are presumed to be affiliated with each other, (i.e., produced by publishers who have an affiliation with each other), and therefore related to each other.
It is to be understood that the classes of grouping labels used herein are illustrative and not limiting. Other classes of grouping labels (e.g., by author, by type of content, etc.) could also be supported in addition to or instead of those used in the examples herein. Further, it is to be understood that only one class of grouping labels might be used, with different types of relationships not being distinguished.
To facilitate consistent use of grouping labels across publishers and content items, each label namespace is advantageously subject to centralized management. In some embodiments, a namespace manager 176 is communicably connected to Internet 140. Namespace manager 176 maintains a labels data store 178 that stores a list of valid grouping label values for each recognized class of grouping labels. Namespace manager 176 communicates, via Internet 140, with publishers of content distributed via content servers 1501 to 150N, allowing the publishers to select grouping labels for inclusion in their content items. In some embodiments, namespace manager 176 can create new grouping labels in response to publisher requests; in other embodiments, namespace manager 176 provides grouping labels from a predefined list.
Further, as described below, namespace manager 176 may also be configured to validate any grouping labels found in content items, e.g., in response to requests from web crawler 172 and/or search server 160. For instance, when web crawler 172 detects a grouping label in a Web page, the value of the detected grouping label may be checked against the list of valid values in labels data store 178. Invalid grouping-label values are advantageously not recorded in page index 170, Qgroup index 174, or Qcat index 173.
4. Ad Server System
In some embodiments, system 110 also includes an advertisement (ad) server 180 coupled to Internet 140. Ad server 180 has access to a sponsored content database 182 that stores references to content items that can be displayed as advertisements (e.g., banner ads or pop-ups or the like) while the user views a related content item. In some embodiments, each entry in sponsored content database 182 can have one or more group labels associated with it, and ads to be displayed while the user views a given content item are selected using the grouping labels, as described below.
It will be appreciated that the search system described herein is illustrative and that variations and modifications are possible. Any of the various systems, including content server, search server, web crawler, namespace manager, and/or ad server, may be part of a single organization, e.g., a distributed server system such as that provided to users by Yahoo! Inc., or they may be part of disparate organizations. Each server system generally includes at least one server and an associated database system, and may include multiple servers and associated database systems, and although shown as a single block, may be geographically distributed. For example, all servers of a search server system may be located in close proximity to one another (e.g., in a server farm located in a single building or campus), or they may be distributed at locations remote from one another (e.g., one or more servers located in city A and one or more servers located in city B). Thus, as used herein, a “server system” typically includes one or more logically and/or physically connected servers distributed locally or across one or more geographic locations; the terms “server” and “server system” are used interchangeably. Further, servers described as separate may be implemented on the same computer system or on different computer systems.
The search server system may be configured with one or more page indexes and algorithms for accessing the page index(es) and providing search results to users in response to search queries received from client systems. The search server system might generate the page indexes itself, receive page indexes from another source (e.g., a separate server system), or receive page indexes from another source and perform further processing thereof (e.g., addition or updating of the context identifiers and/or grouping labels).
In accordance with an embodiment of the present invention, publishers can label their Web pages (or other content items) with grouping labels that are usable to associate pages with each other. As described above, various classes of grouping labels can be used to represent different relationships, e.g., Qcat labels for related subject matter and Qgroup labels for affiliated publishers. The grouping labels selected by the publisher are included in the content items and can be detected by web crawler 172, search server 160 and other systems; thus, information about which pages include which grouping labels can be used in a variety of ways for selecting information to present to a user. This section describes processes for selecting grouping label(s) to include in a content item. Examples of using grouping labels to select information for presentation to a user are described below.
A. Publisher Affiliations: Qgroup Labels
In some embodiments of the present invention, the available grouping labels include a Qgroup label that represents a personal or professional or affiliation among publishers. Using the Qgroup label, publishers can identify their content items as being provided by mutually affiliated publishers.
Qgroup labels are advantageously managed by namespace manager 176. In one embodiment, namespace manager 176 creates a Qgroup label in response to a request from a first publisher, referred to herein as a “founding” publisher. Once a Qgroup label is created, other publishers may use that label at the invitation of the founding publisher. Any number of Qgroup labels may coexist, and labels data store 178 advantageously includes a list of Qgroup labels, their founding publishers, and other publishers who have been invited by the founding publisher. In some embodiments, additional metadata about each Qgroup label may also be stored in labels data store 178.
At step 308, the founding publisher shares the Qgroup label with one or more other publishers. In one embodiment, the Qgroup label is a numerical value, and the founding publisher may provide the Qgroup label directly to other publishers. These other publishers can “join” the group by including the Qgroup label in their own content items. In some embodiments, publishers may be limited to one Qgroup label per content item; in other embodiments, a content item might include multiple different Qgroup labels.
In other embodiments, Qgroup labels might be encoded to ensure that the use of the label by other publishers has been approved by the founding publisher. For instance, each publisher might be given a different, but related, Qgroup label, each of which is separately validated. In one such embodiment, at step 304, the founding publisher receives a first Qgroup label that has a base portion and a publisher-specific portion. At step 308, the founding publisher communicates with namespace manager 176 to issue an invitation to another publisher to obtain a related Qgroup label for inclusion in its own pages. The invitation may be communicated to the other publisher by e-mail or any other suitable channel. If the other publisher accepts, namespace manager 176 generates a second Qgroup label that has the same base portion as the first Qgroup label but a different publisher-specific portion. For purposes of identifying related pages (e.g., as described below), Qgroup labels are considered the same if the base portions are the same. For purposes of validating a Qgroup label, labels data store 178 maintains a list of recognized publisher-specific portions for each base Qgroup label, and a Qgroup label is valid only if the combination of base portion and publisher-specific portion is recognized. Other schemes for regulating which publishers can use a Qgroup label may also be used.
As described below, after the Qgroup labels have been included in the Web pages (or other content items) published by affiliated publishers, these labels can be detected and used to identify the pages having the same Qgroup label (or at least the same base portion) as being related to each other.
In some embodiments, other publishers are allowed to join an existing group by adding the Qgroup label to their content items without an explicit invitation from the founding publisher. Where this is the case, namespace manager 176 advantageously provides metadata about existing groups that enables a publisher to make an informed selection. For instance, when a new Qgroup label is created, the founding publisher can supply a description of a “theme” for the group (e.g., cat breeders, wine enthusiasts, etc.). The theme is stored as metadata, and other publishers can search for Qgroup labels for their content based on theme. In some embodiments, the founding publisher may select between an “open” (any publisher can join) or “closed” (only publishers invited by the founding publisher can join) policy for the label, and namespace manager 176 implements the selected policy.
In another embodiment, namespace manager 176 can infer metadata about a group. For instance, a “theme” for the group can be inferred by analyzing the content of different pages having the same Qgroup label. For this purpose, namespace manager 176 might access Qgroup index 174 to identify such pages.
It is not required that all of a publisher's content items include the same Qgroup label. For instance, a publisher that publishes content in several unrelated areas might prefer to have different content items be affiliated with different groups of publishers.
B. Subject Matter Associations: Qcat Label
In other embodiments of the present invention, the available grouping labels include a Qcat label that represents a subject-matter category of a content item. Using the Qcat label, publishers can identify, by category, the subject matter of their content items. As described below, Qcat labels can be detected and used to identify pages having the same Qcat label as being related to each other.
Qcat labels are advantageously managed by namespace manager 176. In one embodiment, a team of human editors generates a taxonomy of category names. This taxonomy may be hierarchical; in one embodiment, the taxonomy is generally similar to the existing Yahoo! directory. Where a taxonomy of context identifiers is used to categorize pages during web crawling operations, as described above, the same taxonomy may also be used for the category names. Namespace manager 176 defines a Qcat label corresponding to each category name and stores the Qcat labels and associated category names in data store 178. Publishers can interact with namespace manager 176 to view (or browse) the taxonomy of category names and select the one that best describes a particular content item. Namespace manager 176 provides the Qcat label corresponding to the selected category name, and the publisher includes the Qcat label in the content item.
At step 404, a publisher communicates with namespace manager 176 to view the taxonomy of category names. Communication may take place via HTTP requests from the publisher for a document containing all or part of the taxonomy. Namespace manager 176 transmits the requested document, e.g., as an HTML form the publisher can use to select a category. In some embodiments, one document includes the entire taxonomy; in other embodiments, multiple interlinked documents are used. In some embodiments, a search interface is provided via which the publisher can enter keywords and retrieve a list of categories corresponding to those keywords. The publisher browses the taxonomy and selects a category name. The selected name is transmitted to namespace manager 176.
At step 406, namespace manager 176 provides to the publisher the Qcat label corresponding to the selected category name. At step 408, the publisher includes the Qcat label in the content item.
It will be appreciated that the process described herein is illustrative and that variations and modifications are possible. Steps described as sequential may be executed in parallel, order of steps may be varied, and steps may be modified or combined. In one alternative embodiment, the publisher provides to namespace manager 176 a content item to be categorized. Namespace manager 176 executes a text-processing algorithm to select one or more suggested categories for the content item, e.g., based on an analysis of terms appearing in the content item. These suggestions are presented to the publisher, who can select one or reject them all and select a different category, e.g., by browsing the taxonomy starting from one of the suggested categories.
In some embodiments, only one Qcat label per content item is supported. In other embodiments, multiple Qcat labels might be supported. For instance, if a Web page contains sections related to several different subjects (as is often the case for news pages, web logs (blogs) and the like), it might be desirable to include a different Qcat label for each section.
As described below, after the Qcat labels have been included in the Web pages (or other content items) by the publishers thereof, these labels can be detected and used to identify the pages in which they appear as being related to each other.
Grouping labels are used to define relationships among content items. By way of illustration,
Each content item 501-508 includes a Qcat label that was selected by the publisher of that item, e.g., in accordance with process 400 described above. Items with the same Qcat label are regarded as being related in the sense that such items pertain to similar subject matter. Thus, as indicated by the solid lines connecting pairs of related items, items 501, 505 and 506 (all having Qcat=3141) are related, and items 502 and 507 (both having Qcat=13759) are related.
Further, each content item 501-508 includes a Qgroup label selected by the publisher. In this example, publisher A is the founding publisher for a Qgroup label (Qgroup=4379) that has been shared to publisher B. Publisher A's items 501-503 and publisher B's items 504-505 have each been labeled with the same Qgroup label. Items with the same Qgroup label are regarded as being related in the sense that there is some affiliation between the publishers of those items. Thus, as indicated by the broken lines connecting pairs of related items, items 501-505 are all related to each other. Publisher C is not affiliated with publishers A and B, and items 506-508 are not related by Qgroup to any of the other items shown. It is to be understood that the Qgroup label in publisher C's items 506-508 might be shared with other publishers (not explicitly represented in
As
To the extent that the same grouping labels are included in different content items, relationships among content items are established. The grouping label information is advantageously harvested and used to identify related content to present to users, as will now be described.
A. Collecting Relationship Data
In some embodiments, the grouping labels are included as tokens (e.g., HTML tags or other control structures) in the content item. The tokens are advantageously formatted so as not to affect the display of the content item. In some embodiments, however, the tokens are detectable as grouping labels by web crawler 172 (
At step 606, web crawler 172 detects a Qcat label in the page. In preferred embodiments, a Qcat label may appear anywhere within a page, and a page may contain multiple Qcat labels. Web crawler 172 is advantageously configured to detect any number of Qcat labels per page, regardless of where they appear. At step 608, web crawler 172 determines whether the Qcat label is valid, e.g., by consulting labels data store 178. If the Qcat label is valid, then at step 610, the Qcat label is added to the page index entry for the current page. The Qcat label is advantageously in a separate field from other information in the entry. Where a page includes multiple Qcat labels, each Qcat label is added to the page index entry.
At step 612, web crawler 172 determines whether more pages remain to be processed. If so, the next page is selected at step 614 (e.g., using conventional techniques), and process 600 returns to step 602 to process the next page.
After all pages have been processed, at step 616, web crawler 172 builds Qcat index 173, which lists pages containing each valid Qcat label, e.g., by inverting page index 170. In some embodiments, some valid Qcat labels might not be included in any pages, and Qcat index 173 is advantageously arranged such that the absence of pages corresponding to a particular label is correctly handled.
It will be appreciated that the process described herein is illustrative and that variations and modifications are possible. Steps described as sequential may be executed in parallel, order of steps may be varied, and steps may be modified or combined. In addition, a similar process can be used to generate Qgroup index 174 (of
B. Use of Grouping Labels in General Search
In some embodiments, grouping labels are used in processing search queries submitted by users. For example in the embodiment of
For example, in some embodiments where Qcat labels are used, search server 160 matches the search query to a Qcat label, then identifies “category-matching” pages based on Qcat index 173. The category-matched pages can be presented as a separate list of hits, or information about which hits are also category-matched pages can be used in ranking the hits found in a conventional search.
At step 706, search server 160 converts one or more of the query terms to a category name selected from the taxonomy of categories stored in labels database 178. Various algorithms known in the art, including text-string matching algorithms or the like, may be used to match query terms to a category name. At step 708, search server 160 identifies a Qcat label that matches the category name, e.g., by reference to labels data store 178. At step 710, search server 160 identifies “category-matching” pages, i.e., pages that include the matching Qcat label. Step 710 may include consulting Qcat index 173 or page index 170 using the Qcat label as a key.
At step 712, search server 160 incorporates the category-matching pages into the search report. Incorporation may take various forms. In one embodiment, search server 160 ranks the Qcat-matching pages (e.g., based on occurrence of the query terms therein) and provides a separate listing of “category” results in addition to the conventional list of hits. In another embodiment, search server 160 determines whether any of the Qcat-matching pages correspond to the hits and uses that information to rerank the pages, e.g., by boosting the ranking of hits that are also Qcat-matching pages. At step 714, the search result is transmitted to the user.
It will be appreciated that process 700 is illustrative and that variations and modifications are possible. Steps described as sequential may be executed in parallel, order of steps may be varied, and steps may be modified or combined. In some embodiments, not all search queries can be matched to a category name, e.g., due to ambiguity or limitations of the matching algorithm. Where a match is not possible, steps 708, 710 and 712 might be omitted. In some embodiments, where the search query can be matched to multiple category names, the user might be prompted to select the category that best represents the desired information.
In other embodiments, an initial search report is presented to the user without category matching; the search report page includes a section listing the category names of one or more possible matching categories (selected as in step 706 of process 700) and suitable prompts inviting the user to explore one or more of these categories. When the user selects a category, search server 160 responds with a list of category-matching pages, generated, e.g., in accordance with steps 708, 710 and 712 of process 700.
In still other embodiments, when the user submits a query to search server 160, search server 160 initially responds with a listing of hits that is generally conventional except that for each hit that includes a grouping label (e.g., Qcat or Qgroup label), the listing includes a control that the user can operate to view related content. When the user activates this control, the related content can be selected based on the grouping labels included in the hit, e.g., in accordance with steps 710 and 712 of process 700.
C. Use of Grouping Labels in a Contextual Search
In some embodiments, grouping labels are integrated with contextual search interfaces. A contextual search interface is associated with a specific Web page or portion of a Web page. The contextual search interface can appear as an icon in a Web page (referred to herein as a “hosting page”) in association with some or all of the page content. The user can activate the icon to view related content and/or execute a search using a user-supplied query together with a context vector representing the content. A detailed description of contextual search interfaces can be found in above-referenced application Ser. No. 10/903,283, application Ser. No. 11/033,100, and application Ser. No. 11/033,417.
By way of illustration,
Icons 806, 808 are advantageously generated using suitable code blocks in the source code for page 800.
Referring again to
A list of suggested search queries is presented in section 1004, and the user may execute one of these queries by clicking on it. Suggested search queries can be generated based on the context vector, e.g., as described in above-referenced application Ser. No. 10/903,283, application Ser. No. 11/033,100, and application Ser. No. 11/033,417.
Overlay 1002 also includes, in section 1006, a listing of related sites. In one embodiment, the listing in section 1006 is generated using the Qcat label 908 and/or Qgroup label 906 from code block 900 as well as the context vector 904. In this embodiment, overlay 1002 displays only a small number of related sites in section 1006, and the user can operate “More . . . ” button 1008 to view additional related results, e.g., in a new window.
At step 1106, search server 160 logs the received information request. The logged information can be used to build indexes of pages with contextual search interfaces, as described below.
At step 1108, search server 160 determines from the customization tag whether related content is to be displayed in the active contextual search interface. If so, then at step 1110, search server 160 reads the Qcat and/or Qgroup labels from the information request. At step 1112, search server 160 identifies one or more pages with a matching Qcat label (or Qgroup label, or both), e.g., by consulting Qcat index 173 and/or Qgroup index 174.
At step 1114, search server 160 ranks the matching pages. In one embodiment, ranking includes comparing the content and/or context vectors associated with the each matching page to the content and/or context vector of the hosting page. Qgroup and/or Qcat labels may also be used in ranking.
At step 1116, search server 160 selects matching pages to be displayed. The contextual search interface displays, e.g., the top three, top five, or some other number of matching pages, and the highest ranking matching pages are advantageously selected for display. At step 1118, references to the matching pages are added to the contextual search interface, e.g., by including suitably formatted references in a code block defining overlay 1002. As noted above, the overlay may also include a control allowing the user to view pages not selected at step 1116, and this control may also be implemented by including suitable code in the code block defining overlay 1002.
At step 1120, other components of the active contextual search interface, such as suggested queries listing 1004 and query submission box 1412, are generated, e.g., as described in above-referenced application Ser. No. 10/903,283, application Ser. No. 11/033,100, and application Ser. No. 11/033,417. At step 1122, the contextual search interface, in the form of computer-readable code for displaying the page, is transmitted to client system 120, and at step 1124, client system 120 displays the active contextual search interface.
It will be appreciated that the contextual search interface and activation process described herein are illustrative and that variations and modifications are possible. Steps described as sequential may be executed in parallel, order of steps may be varied, and steps may be modified or combined. A contextual search interface may include any combination of elements and can appear in an overlay, in-line in the hosting page, in a new window, in a new browser tab, or elsewhere as desired.
As noted above, information requests generated from contextual search icons can be logged by search server 160. The request logs can be processed to gather information about contextual search interfaces and user behavior associated with such interfaces. For example, an index of pages that have contextual search interfaces can be created. In addition, where the logged information includes grouping labels, an index keyed by grouping label (e.g., Qgroup index 173 or Qcat index 174) can be built from the log. Also, popularity scores for particular pages containing contextual search interfaces can be generated, e.g., based on the number of times an information request is sent from a page containing such an interface. This popularity information can be used, e.g., in ranking related pages selected by grouping label.
D. Use of Grouping Labels for Selecting Advertisements
In another embodiment of the present invention, Qgroup and/or Qcat labels are used to select advertisements (ads) to be displayed when a particular Web page is displayed.
It is known in the art that an advertisement can be dynamically selected when a page is displayed for a user. In accordance with an embodiment of the present invention, the selection includes matching a Qgroup or Qcat label of the page being displayed to a Qgroup or Qcat label assigned to an advertisement by a sponsor thereof. The process may be generally similar to that shown in
E. Generation of a Directory from Grouping Labels
In another embodiment of the present invention, grouping labels can be used to create a browsable directory of labeled pages. For example, in embodiments where the Qcat label represents a category name selected by the publisher from a hierarchical taxonomy, a browsable version of the taxonomy can be displayed for the user. When the user selects a category name from the taxonomy, a listing of pages labeled with the corresponding Qcat label can be displayed.
At step 1208, search server 160 accesses Qcat index 173 to find pages (referred to herein as “hit pages” or “hits”) that have the Qcat label(s) identified at step 1206. At step 1210, the hit pages are ranked. In one embodiment, ranking is based on a relevance score determined by comparing a context vector or other representation of the content of the hit page to the category name, e.g., using a conventional search algorithm with the category name substituted for the search query. In other embodiments, ranking might be based at least in part on page sponsorship information and/or explicit user feedback (e.g., ratings) about the page content or relevance of the page to the category. Such feedback may be collected as users browse the directory and added to the Qcat index. Conventional techniques for collecting user feedback and incorporating such feedback into page ranking may be used.
At step 1212, a listing of hit pages, in rank order, is provided to the user. In one embodiment, the listing is presented in a manner analogous to a conventional search report, with links to the top ten or twenty (or some other number of) hit pages presented in a first screen and navigation buttons for viewing links to additional pages. Each listing may include, e.g., a page title, abstract, and other information (e.g., user feedback), and may be accompanied by prompts and control elements for soliciting and accepting user feedback as to whether the page is useful or relevant to the category.
The listing advantageously is presented together with a navigation panel that allows the user to further browse the taxonomy. The navigation panel may include a list of sub-categories of the current category, an option to move up in the hierarchy, and so on.
While the invention has been described with respect to specific embodiments, one skilled in the art will recognize that numerous modifications are possible. For instance, the Qgroup and Qcat labels are illustrative. Any number of grouping labels may be supported, and a grouping label may indicate any definable relationship.
In addition, some embodiments may include user feedback features and/or an editorial review process for evaluating the appropriateness of the grouping labels assigned to particular pages. Where a grouping label is used inappropriately (e.g., where a Qcat label is included in a page whose content does not relate to the category), the grouping index or other data store can be updated such the label is, in effect, ignored.
The embodiments described herein may make reference to Web sites pages, links, and other terminology specific to instances where the World Wide Web (or a subset thereof) serves as the search corpus. It should be understood that the systems and processes described herein can be adapted for use with a different search corpus (such as an electronic database or document repository) and that results may include content as well as links or references to locations where content may be found.
In addition, the invention has been described with reference to computer systems having particular hardware and software elements. It is to be understood that other systems with different combinations of hardware and/or software components could also be implemented.
Thus, although the invention has been described with respect to specific embodiments, it will be appreciated that the invention is intended to cover all modifications and equivalents within the scope of the following claims.
Number | Name | Date | Kind |
---|---|---|---|
5771378 | Holt et al. | Jun 1998 | A |
6134532 | Lazarus et al. | Oct 2000 | A |
6169992 | Beall et al. | Jan 2001 | B1 |
6327590 | Chidlovskii et al. | Dec 2001 | B1 |
6446065 | Nishioka et al. | Sep 2002 | B1 |
6701311 | Biebesheimer et al. | Mar 2004 | B2 |
6785670 | Chiang et al. | Aug 2004 | B1 |
6789073 | Lunenfeld | Sep 2004 | B1 |
6961731 | Holbrook | Nov 2005 | B2 |
7058626 | Pan et al. | Jun 2006 | B1 |
20010037205 | Joao | Nov 2001 | A1 |
20020004735 | Gross | Jan 2002 | A1 |
20020072997 | Colson et al. | Jun 2002 | A1 |
20020078029 | Pachet | Jun 2002 | A1 |
20020092019 | Marcus | Jul 2002 | A1 |
20030204530 | Anderson et al. | Oct 2003 | A1 |
20030229893 | Sgaraglino | Dec 2003 | A1 |
20040002959 | Alpert et al. | Jan 2004 | A1 |
20040133471 | Pisaris-Henderson et al. | Jul 2004 | A1 |
20040210468 | Rubel et al. | Oct 2004 | A1 |
20050160080 | Dawson | Jul 2005 | A1 |
20050257045 | Bushman et al. | Nov 2005 | A1 |
20060026013 | Kraft | Feb 2006 | A1 |
20060036570 | Schaefer et al. | Feb 2006 | A1 |
20060074853 | Liu et al. | Apr 2006 | A1 |
20060074876 | Kakivaya et al. | Apr 2006 | A1 |
20060242018 | Shulman et al. | Oct 2006 | A1 |
20070106657 | Brzeski et al. | May 2007 | A1 |
20070168346 | Markanthony et al. | Jul 2007 | A1 |