INDEXING AND RETRIEVAL OF BLOGS

Abstract

A system may receive a feed associated with a blog. The system may extract information from the feed and the blog and create a hybrid document based on the extracted information. The system may further use the hybrid document to determine a relevance of the blog to a search query.

Description

FIELD OF THE INVENTION

Implementations consistent with the principles of the invention relate generally to information retrieval and, more particularly, to indexing and retrieval of blogs.

BACKGROUND OF THE INVENTION

The World Wide Web (“web”) contains a vast amount of information. Locating a desired portion of the information, however, can be challenging. This problem is compounded because the amount of information on the web and the number of new users inexperienced at web searching are growing rapidly.

Search engines attempt to return hyperlinks to web pages in which a user is interested. Generally, search engines base their determination of the user's interest on search terms (called a search query) entered by the user. The goal of the search engine is to provide links to high quality, relevant results (e.g., web pages) to the user based on the search query. Typically, the search engine accomplishes this by matching the terms in the search query to a corpus of pre-stored web pages. Web pages that contain the user's search terms are identified as search results and are returned to the user as links.

Over the past few years, a new medium, called a blog, has appeared on the web. Blogs (short for web logs) are publications of personal thoughts that are typically updated frequently with new journal entries, called posts. The content and quality of blogs and their posts can vary greatly depending on the purpose of the authors of the blogs. As blogging becomes more popular, the ability to provide quality blog search results becomes more important.

SUMMARY OF THE INVENTION

In accordance with one implementation consistent with the principles of the invention, a method may include receiving a feed; fetching a blog and one or more posts associated with the feed; extracting information from the feed, the blog, and one or more posts; creating a hybrid document based on the extracted information; and using the hybrid document to determine a relevance of the blog or the one or more posts to a search query.

In another implementation consistent with the principles of the invention, a device includes a memory to store instructions and a processor. The processor executes the instructions to receive a search query, determine a relevance of a blog or a blog post to the search query based on information extracted from the blog or blog post and information extracted from at least one other source, and provide information relating to the blog or the blog post when the blog or the blog post is determined to be relevant to the search query.

In yet another implementation consistent with the principles of the invention, a method may include receiving a search query; determining a relevance of a first set of documents to the search query using a second set of documents, where the first set of documents includes blogs and blog posts and the second set of documents includes hybrid documents created from the first set of documents and at least one other source; and providing information regarding documents in the first set of documents determined to be relevant.

In still another implementation consistent with the principles of the invention, a method may include receiving a search query; identifying a first set of documents to provide in response to the search query based on a second set of documents; and providing information relating to the identified first set of documents.

In yet still another implementation consistent with the principles of the invention, a method may include receiving feeds associated with blogs, extracting first information from the feeds, extracting second information from the blogs and associated posts, creating hybrid documents based on the first information and the second information, receiving a search query, determining a relevance of the blogs or posts to the search query based on the hybrid documents, and providing information relating to the blog or posts determined to be relevant.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate an implementation of the invention and, together with the description, explain the invention. In the drawings,

FIG. 1 is an exemplary diagram illustrating a concept consistent with the principles of the invention;

FIG. 2 is an exemplary diagram of a network in which systems and methods consistent with the principles of the invention may be implemented;

FIG. 3 is an exemplary diagram of a client or server entity in an implementation consistent with the principles of the invention;

FIG. 4 is a diagram of a portion of an exemplary computer-readable medium that may be used by the server of FIG. 2;

FIG. 5 is an exemplary database that may be associated with the server of FIG. 2 in an implementation consistent with the principles of the invention;

FIG. 6 is a flow chart of an exemplary process for creating a hybrid document in an implementation consistent with the principles of the invention;

FIG. 7 is an exemplary configuration of a feed in an implementation consistent with the principles of the invention;

FIG. 8 is an exemplary configuration of a post document in an implementation consistent with the principles of the invention;

FIG. 9 is an exemplary configuration of a blog document in an implementation consistent with the principles of the invention;

FIG. 10 depicts the formation of a hybrid document in an implementation consistent with the principles of the invention; and

FIG. 11 is a flowchart of an exemplary process for presenting search results in an implementation consistent with the principles of the invention.

DETAILED DESCRIPTION

The following detailed description of implementations consistent with the principles of the invention refers to the accompanying drawings. The same reference numbers in different drawings may identify the same or similar elements. Also, the following detailed description does not limit the invention.

Overview

Systems and methods consistent with the principles of the invention combine various documents relating to a blog to produce a hybrid document. When a search query is received, the hybrid document may be analyzed to determine the relevance of the blog to the search query.

FIG. 1 is an exemplary diagram illustrating a concept consistent with the principles of the invention. As illustrated in FIG. 1, information may be extracted from a blog feed 110 for a particular blog. The information may include, for example, time stamps as to when a post within the blog was created and/or updated, the actual content of the post, the post's title, the post's author(s), the blog's author, the blog's title, the profile of the blog author, etc. Information may also be extracted from a document 120 (e.g., a web page) on which the blog is located. In one implementation, document 120 may be obtained based on information in feed 110 (e.g., feed 110 may identify the location of document 120). The information extracted from document 120 may include, for example, the author's profile, the content of the post, a blogroll, etc. In one implementation, information may also be extracted from one or more other documents to which document 120 links (e.g., an author's profile page). The information extracted from these sources may be combined to form a hybrid document 130 for the blog that may be used for determining the relevance of the blog and post to a received search query.

A “document,” as the term is used herein, is to be broadly interpreted to include any machine-readable and machine-storable work product. A document may include, for example, an e-mail, a web site, a file, a combination of files, one or more files with embedded links to other files, a news group posting, a blog or blog post, a web advertisement, etc. In the context of the Internet, a common document is a web page. Web pages often include textual information and may include embedded information (such as meta information, images, hyperlinks, etc.) and/or embedded instructions (such as Javascript, etc.). A “link,” as the term is used herein, is to be broadly interpreted to include any reference to/from a document from/to another document or another part of the same document.

Exemplary Network Configuration

FIG. 2 is an exemplary diagram of a network 200 in which systems and methods consistent with the principles of the invention may be implemented. Network 200 may include multiple clients 210 connected to multiple servers 220-240 via a network 250. Two clients 210 and three servers 220-240 have been illustrated as connected to network 250 for simplicity. In practice, there may be more or fewer clients and servers. Also, in some instances, a client may perform a function of a server and a server may perform a function of a client.

Clients 210 may include client entities. An entity may be defined as a device, such as a personal computer, a wireless telephone, a personal digital assistant (PDA), a laptop, or another type of computation or communication device, a thread or process running on one of these devices, and/or an object executable by one of these devices. Servers 220-240 may include server entities that gather, process, search, and/or maintain documents in a manner consistent with the principles of the invention.

In an implementation consistent with the principles of the invention, server 220 may include a search engine 225 usable by clients 210. In one implementation, search engine 225 may be a blog search engine. Server 220 may crawl a corpus of documents, index the documents, and store information associated with the documents in a repository of documents. Servers 230 and 240 may store or maintain documents that may be crawled or analyzed by server 220.

While servers 220-240 are shown as separate entities, it may be possible for one or more of servers 220-240 to perform one or more of the functions of another one or more of servers 220-240. For example, it may be possible that two or more of servers 220-240 are implemented as a single server. It may also be possible for a single one of servers 220-240 to be implemented as two or more separate (and possibly distributed) devices.

Network 250 may include a local area network (LAN), a wide area network (WAN), a telephone network, such as the Public Switched Telephone Network (PSTN), an intranet, the Internet, or a combination of networks. Clients 210 and servers 220-240 may connect to network 250 via wired, wireless, and/or optical connections.

Exemplary Client/Server Architecture

FIG. 3 is an exemplary diagram of a client or server entity (hereinafter called “client/server entity”), which may correspond to one or more of clients 210 and/or servers 220-240. The client/server entity may include a bus 310, a processor 320, a main memory 330, a read only memory (ROM) 340, a storage device 350, an input device 360, an output device 370, and a communication interface 380. Bus 310 may include a path that permits communication among the elements of the client/server entity.

Processor 320 may include a processor, microprocessor, or processing logic that may interpret and execute instructions. Main memory 330 may include a random access memory (RAM) or another type of dynamic storage device that may store information and instructions for execution by processor 320. ROM 340 may include a ROM device or another type of static storage device that may store static information and instructions for use by processor 320. Storage device 350 may include a magnetic and/or optical recording medium and its corresponding drive.

Input device 360 may include a mechanism that permits an operator to input information to the client/server entity, such as a keyboard, a mouse, a pen, voice recognition and/or biometric mechanisms, etc. Output device 370 may include a mechanism that outputs information to the operator, including a display, a printer, a speaker, etc. Communication interface 380 may include any transceiver-like mechanism that enables the client/server entity to communicate with other devices and/or systems. For example, communication interface 380 may include mechanisms for communicating with another device or system via a network, such as network 250.

As will be described in detail below, the client/server entity, consistent with the principles of the invention, may perform certain document processing-related operations. The client/server entity may perform these operations in response to processor 320 executing software instructions contained in a computer-readable medium, such as memory 330. A computer-readable medium may be defined as a physical or logical memory device and/or carrier wave.

The software instructions may be read into memory 330 from another computer-readable medium, such as data storage device 350, or from another device via communication interface 380. The software instructions contained in memory 330 may cause processor 320 to perform processes that will be described later. Alternatively, hardwired circuitry may be used in place of or in combination with software instructions to implement processes consistent with the principles of the invention. Thus, implementations consistent with the principles of the invention are not limited to any specific combination of hardware circuitry and software.

Exemplary Computer-Readable Medium

FIG. 4 is a diagram of a portion of an exemplary computer-readable medium 400 that may be used by a server 220. In one implementation, computer-readable medium 400 may correspond to memory 330 of server 220. The portion of computer-readable medium 400 illustrated in FIG. 4 may include an operating system 410 and blog indexing software 420.

Operating system 410 may include operating system software, such as the Windows, Unix, or Linux operating systems. Blog indexing software 420 may include software that receives blog feeds and blog documents and predetermined information to be extracted from the feeds and documents. Blog indexing software 420 may also include software that forms hybrid blog documents using the extracted information.

FIG. 5 is an exemplary database 500 that may be associated with server 220 in an implementation consistent with the principles of the invention. Database 500 may be stored locally at server 220, for example, in main memory 330 or storage device 350, or stored external to server 220 at, for example, a possibly remote location. As illustrated, database 500 may store a group of hybrid documents. These hybrid documents may include hybrid blog post documents (i.e., hybrid documents created from posts) and/or hybrid blog documents (i.e., hybrid documents created from blogs). Database 500 may be accessed to determine the relevancy of the blogs and blog posts corresponding to the hybrid documents to a search query received by server 220. It will be appreciated that database 500 may include other information (not shown) that aids in matching blogs and blog posts to search queries.

Moreover, it will be appreciated that database 500 may not actually store hybrid documents as independent entities, but rather may store an index of a corpus of hybrid documents. In this case, the hybrid documents may be virtual entities and not physical entities.

Exemplary Processing for Forming a Hybrid Document

FIG. 6 is a flow chart of an exemplary process for creating a hybrid document in an implementation consistent with the principles of the invention. The foregoing description focuses on creating a hybrid post document. It will be appreciated that the techniques described herein may also be used to create a hybrid blog document.

Processing may begin by receiving a feed associated with a blog (act 610). The feed may, for example, include a Rich Site Summary (RSS) feed, an Atom feed, and/or some other type of blog feed. The feed may include information regarding updates to a particular blog. For example, if an author of the blog adds a new post (i.e., entry) to the blog, a blog feed may be created that includes information regarding the new post. The information may include, for example, timestamps as to when the post was created and/or updated, the actual content of the post, a title of the post, an author of the post, an author of the blog, a title of the blog, a profile for the author of the blog, and/or other information regarding the post and/or blog.

Document(s) relating to the post may be obtained (act 620). For example, a blog document and a post document may be obtained. These documents may include a profile for the author of the blog, the content of the post, a blogroll (i.e., links to other blogs), and/or other information.

Information may be extracted from the blog feed and the blog and post documents (act 630). The information extracted from the blog feed may include, for example, timestamps as to when the post was created and/or updated, the actual content of the post, the title of the post, the author of the post, the author of the blog, the title of the blog, and the profile for the author of the blog, if available. The information extracted from the blog and post documents may include, for example, the profile for the author of the blog, the content of the post, and the blogroll.

In one implementation, information extracted from the blog feed may be compared to information extracted from the blog/post documents to determine whether the feed and/or blog/post documents are legitimate. For example, if the post content extracted from the blog feed does not match the post content extracted from the blog/post documents, this may be an indication that the feed and/or documents are not legitimate. That is, an individual may be attempting to spam a search engine into ranking that individual's blog/post more highly than it would ordinarily be ranked. In this situation, no hybrid document may be formed for this blog/blog post, or be given a very low rating.

In some situations, a blog document may contain links to other documents that may be relevant to the blog and/or post. For example, in some situations, a blog document may link to another document (or documents) that contains profile information for the author of the blog. The profile information may include the author's geographical location, age, gender, etc.

In one implementation, information may be extracted from these other documents (act 640). The information extracted from these other documents may include, for example, author profile information, such as a geographical location of the author. Other information may additionally be extracted.

A hybrid document may be created for the post based on the extracted information (act 650). In one implementation, the information extracted from the feed may be combined with the information extracted from the blog/post documents and the other documents associated with the blog document to create the hybrid document. As will be appreciated by one skilled in the art, the hybrid document is a document that cannot be retrieved by a user in response to a search query. Instead, the hybrid document may be used by a server, such as server 220, to determine the relevance of the post to a received search query.

Once the hybrid document has been created, it may be stored, for example, in database 500. In this manner, database 500 may be populated with hybrid documents relating to a set of posts. In some implementations, storing the hybrid document may include indexing the information extracted from the feed and blog/post documents and storing this extracted information in database 500 as part of an index.

Example

FIG. 7 is an exemplary feed 700 in an implementation consistent with the principles of the invention. In one implementation, feed 700 may be an Extensible Markup Language (XML) document that includes information about a new post within a blog. For explanatory purposes, feed 700, as illustrated in FIG. 7, includes information regarding the blog's title (i.e., “Goldtoe Lemon.Nut”); the author of the post (i.e., “Goldtoe”); timestamps as to when the post was created, modified, and issued; the title of the post; and the content of the post. Some or all of this information may be extracted from feed 700. As disclosed above, a typical feed may include more additional information than illustrated in FIG. 7.

FIG. 8 is an exemplary configuration of a post document 800 in an implementation consistent with the principles of the invention. In one implementation, post document 800 may be a Hypertext Markup Language (HTML) document. Post document 800, as illustrated in FIG. 8, includes information regarding the blog's title (i.e., “Goldtoe Lemon.Nut”), a date of the post, a title of the post, the content of the post, a brief profile for the author, and a link to a full profile for the author. Some or all of this information may be extracted from post document 800. As disclosed above, a typical post document may include additional information than illustrated in FIG. 8.

FIG. 9 is an exemplary configuration of a blog document 900 that contains the blog corresponding to feed 700 in an implementation consistent with the principles of the invention. In one implementation, blog document 900 may be a Hypertext Markup Language (HTML) document. Blog document 900, as illustrated in FIG. 9, includes information regarding the blog's title (i.e., “Goldtoe Lemon.Nut”), a date of the post, a title of the post, the content of the post, a brief profile for the author, a link to a full profile for the author, and a blogroll. Some or all of this information may be extracted from blog document 900. As disclosed above, a typical blog document may include additional information than illustrated in FIG. 9.

FIG. 10 depicts the formation of a hybrid document 1000 in an implementation consistent with the principles of the invention. As illustrated, hybrid document 1000 may be formed from feed 700, post document 800, blog document 900, and/or other documents related to blog document 900. Information may be extracted from any combination of these four sources. For example, hybrid document 1000 may be formed from feed 700, post document 800, and blog document 900; feed 700 and post document 800; feed 700 and blog document 900; post document 800 and blog document 900; or other combinations of these or other sources. Hybrid document 1000 may include some or all of the information extracted from feed 700, post document 800, blog document 900, and other documents related to blog document 900. This extracted information improves the ability of a search engine to determine whether the post is relevant to a received search query. As set forth above, hybrid document 1000 may be a physical entity or a virtual entity stored in database 500.

Presenting Search Results

FIG. 11 is a flowchart of an exemplary process for presenting search results. In one implementation, the processing of FIG. 11 may be performed by one or more software and/or hardware components within server 220. In another implementation, the processing may be performed by one or more software and/or hardware components within another device or a group of devices separate from or including server 220.

Processing may begin with a search query being received (act 1010). For example, the user may provide a search query into a search box associated with a search engine (e.g., entering a search term into a search engine interface or a search box of an add-on toolbar). The web browser (or the add-on toolbar) may send the search query to a search engine, such as search engine 225 associated with server 220.

A list of blogs/blog posts may be identified based on the search query (act 1020). For example, server 220 may determine an information retrieval (IR) score for the blogs/blog posts. The IR score for a blog/blog post may be determined based on a matching of the search terms of a search query to an index created based on the above-discussed hybrid documents. There are a number of known techniques that may be used to determine the IR score for a document. For example, the IR score may be determined based on the number of occurrences of the search terms in the document. Alternatively or additionally, the IR score may be determined based on where the search terms occur within the document (e.g., title, content, etc.) or characteristics of the search terms (e.g., font, size, color, etc.). Alternatively or additionally, a search term may be weighted differently from another search term when multiple search terms are present. Alternatively or additionally, the proximity of the search terms when multiple search terms are present may influence the IR score. Yet other techniques for determining the IR score for a document are known to those skilled in the art.

Server 220 may assign scores to the blogs/blog posts, sort the blogs/blog posts based on their scores, and form a list of search result documents based on the sorted blogs/blog posts. In one implementation, the list of search result documents includes links to the posts or the blogs that host the posts. The list of search result documents do not include links to the hybrid documents since these hybrid documents, as indicated above, are not documents that are retrievable by a user in response to a search query. Instead, these hybrid documents allow for blogs and blog posts to be more accurately matched to received search queries, allowing for higher quality results to be provided to users. Upon selection of a document in the list of search results, the user may be provided with the selected blog or post (act 1030).

As an example of the above processing, assume that a user is interested in blogs from people who live in San Francisco. The user might transmit the search query “San Francisco” to a search engine, such as search engine 225. Search engine 225 may, for example, compare the terms in the search query (or variations thereof) to the index stored in database 500 and identify a set of documents that match the query. Assume that the blog corresponding to document 900 (FIG. 9) is included in the identified set of documents since the author's profile, which is included in a document associated with blog document 900, indicates that the author is located in San Francisco. Search engine 225 may provide the user with a list of search results that include links to blogs and/or blog posts. The list may include post document 800 and/or blog document 900 and not the actual document that includes the author's profile since that document is more than likely not of interest to the user (i.e., the user is likely interested in the blog from a blogger that is located in San Francisco and not a profile document that indicates that the user lives in San Francisco).

CONCLUSION

Implementations consistent with the principles of the invention use hybrid documents to improve blog searching.

The foregoing description of exemplary embodiments of the invention provides illustration and description, but is not intended to be exhaustive or to limit the invention to the precise form disclosed. Modifications and variations are possible in light of the above teachings or may be acquired from practice of the invention.

For example, while series of acts have been described with regard to FIGS. 6 and 11, the order of the acts may be modified in other implementations consistent with the principles of the invention. Further, non-dependent acts may be performed in parallel.

The preceding description refers to a user. A “user” is intended to refer to a client, such as a client 210 (FIG. 2), or an operator of a client.

It will be apparent to one of ordinary skill in the art that aspects of the invention, as described above, may be implemented in many different forms of software, firmware, and hardware in the implementations illustrated in the figures. The actual software code or specialized control hardware used to implement aspects consistent with the principles of the invention is not limiting of the invention. Thus, the operation and behavior of the aspects were described without reference to the specific software code—it being understood that one of ordinary skill in the art would be able to design software and control hardware to implement the aspects based on the description herein.

No element, act, or instruction used in the present application should be construed as critical or essential to the invention unless explicitly described as such. Also, as used herein, the article “a” is intended to include one or more items. Where only one item is intended, the term “one” or similar language is used. Further, the phrase “based on” is intended to mean “based, at least in part, on” unless explicitly stated otherwise.

Claims

1-28. (canceled)

29. A method performed by one or more server devices, the method comprising: extracting, by one or more processors associated with the one or more server devices, first information from a post to a blog that includes a plurality of posts;extracting, from the blog and by one or more processors associated with the one or more server devices, second information, associated with the blog, from a source different than the posts to the blog, where the second information is not included in the posts to the blog; andcomparing, by one or more processors associated with the one or more server devices, one or more terms of a search query to one or more terms included in the first information and the second information to determine a relevance of the post to the search query.

30. The method of claim 29, where the source includes a Rich Site Summary (RSS) feed, an Atom feed, or another structured representation of information contained in the blog.

31. The method of claim 29, where the extracting the second information includes: extracting at least one of a title of the blog, a name of an author of the blog, or a profile of the author of the blog.

32. The method of claim 29, where the source includes one or more documents to which the blog links.

33. The method of claim 29, where the extracting the first information includes identifying at least one of: a timestamp that indicates when the post was created,a timestamp that indicates when the post was updated,content of the post,a title of the post, ora name of an author of the post.

34. The method of claim 29, further comprising: comparing the first information to the second information, anddetermining, based on the comparing, whether at least one of the post or the blog are spam.

35. The method of claim 34, where the determining includes determining that the at least one of the post or the blog are spam when at least a portion of the first information does not match a corresponding portion of the second information.

36. A computer-readable memory device storing computer-executable instructions, the computer-executable instructions comprising: one or more instructions to extract first information from a particular post to a blog that includes a plurality of posts;one or more instructions to extract, from the blog, second information, associated with the blog, from a source different than the posts to the blog, where the second information is not included in the posts to the blog; andone or more instructions to index another post to the blog based on a content of the other post, the first information, and the second information.

37. The computer-readable memory device of claim 36, where the one or more instructions to extract the second information include: one or more instructions to extract the second information from one or more documents to which the blog links.

38. The computer-readable memory device of claim 36, where the second information includes at least one of: information identifying a geographical location of an author of the blog,information identifying an age of the author of the blog, orinformation identifying a gender of the author of the blog.

39. The computer-readable memory device of claim 36, where the computer-executable instructions further comprise: one or more instructions to use the index to determine a relevance of the other post to a received search query.

40. The computer-readable memory device of claim 36, where the source includes a Rich Site Summary (RSS) feed, an Atom feed, or another structured representation of information contained in the blog.

41. The computer-readable memory device of claim 36, where the second information includes at least one of: information identifying a title of the blog,information identifying a name of an author of the blog, orinformation identifying a profile of the author of the blog.

42. The computer-readable memory device of claim 36, where the first information includes at least one of: a timestamp that indicates when the post was created,a timestamp that indicates when the post was updated,content of the post,a title of the post, ora name of an author of the post.

43. The computer-readable memory device of claim 36, further comprising: one or more instructions to compare the first information to the second information, andone or more instructions to determine, based on the comparing, whether at least one of the particular post, the other post, or the blog are spam.

44. A method performed by one or more server devices, the method comprising: identifying, by one or more processors associated with the one or more server devices, a blog that is associated with: one or more posts, anddata that is from a source other than the one or more posts;fetching, by one or more processors associated with the one or more server devices, first information from a particular post of the one or more posts;extracting, by one or more processors associated with the one or more server devices, second information from the data that is from a source other than the one or more posts;determining that the first information is different from the second information, and classifying the particular post as spam in response to the determining; andreducing a rating of the blog post in response to classifying the particular post as spam.

45. The method of claim 44, where the determining includes determining that the first information is different from the second information when at least a portion of the first information does not match a corresponding portion of the second information.

46. The method of claim 44, where the source includes a Rich Site Summary (RSS) feed, an Atom feed, or another structured representation of information contained in the blog.

47. The method of claim 44, where the source includes one or more documents to which the blog links.

48. A system, comprising: a memory to store computer-executable instructions; andone or more processors to execute the computer-executable instructions to: extract first information from a post to a blog that includes a plurality of posts;extract second information, associated with the blog, from a source that is within the blog and that is different than the posts to the blog, where the second information is not included in the posts to the blog; andcompare one or more terms of a search query to one or more terms included in the first information and the second information to determine a relevance of the post to the search query.