Patent Application
20080040301
These and other objects, features and advantages of the present invention will become apparent through consideration of the Detailed Description of the Invention, when considered in conjunction with the drawing Figures, in which:
The present invention will now be described in detail with reference to the drawings, which are provided as illustrative examples of the invention so as to enable those skilled in the relevant art(s) to practice the invention. Notably, the figures and examples below are not meant to limit the scope of the present invention to a single embodiment, but other embodiments are possible by way of interchange of some or all of the described or illustrated elements. Moreover, where certain elements of the present invention can be partially or fully implemented using known components, only those portions of such known components that are necessary for an understanding of the present invention will be described, and detailed descriptions of other portions of such known components will be omitted so as not to obscure the invention. In the present specification, an embodiment showing a singular component should not necessarily be limited to other embodiments including a plurality of the same component, and vice-versa, unless explicitly stated otherwise herein. Moreover, applicants do not intend for any term in the specification or claims to be ascribed an uncommon or special meaning unless explicitly set forth as such. Further, the present invention encompasses present and future known equivalents to the known components referred to herein by way of illustration.
In general, throughout this description, if an item is described as implemented in software, it can equally well be implemented as hardware or a combination of both hardware and software. It is also understood that “data,” as used herein, is either singular or plural, as the context requires. It should be appreciated that computer code for implementing aspects of the present invention in software can be C, C++, HTML, XML, Java, JavaScript, etc. code, or any suitable scripting language (e.g., VBScript), or any other suitable programming language.
There is provided herein new and improved processes and system for use in information management and retrieval including web based information management and retrieval. The invention is described herein with respect to web-based content and the Internet. However, the reader will appreciate that the invention is not limited to web-based content and the Internet, but is equally applicable to other content and networks.
With reference now to
Content tagging empowers users of such websites to add their own contextual tags to website content. Typically, this results in categorization of content in a way that is relevant to the needs of the users. Content tagging generally operates by adding keywords and other user-generated metadata (or ‘tags’) to a web-based document to achieve two essential purposes. First, the added keywords identify a user's interests and secondly, the added keywords capture the meaning of particular content. A user is free to choose any tags of his or her choosing and there is no limitation on the number of tags that may be assigned to a document.
Referring now to
In one embodiment, the system 10 is communicatively coupled over a network 40 to client computers 50. The network 40 can be a local area network (LAN), wide area network (WAN), Internet, or any other network capable of communicating information and data from the system 10 to client computers 50, which may be local or remote. The system 10 may be accessed by a plurality of computers 50, each of which can be any computer device known in the art, for example, a personal computer including components such as a memory device, an input component, such as a keyboard and mouse, a display, and a central processing unit. The computer 50 may also be a laptop, personal digital assistant (PDA), handheld device, such as a mobile telephone or BlackBerry™ or other device capable of receiving data and sending the data over network 40 to the system 10. In general, the client computers 50 have a web browser for receiving and viewing documents written in the Hypertext Markup Language (HTML) and transmitted over the network 40 via Hypertext Transfer Protocol (HTTP) and transmitting requests for HTML documents to the system 10 via HTTP, or any other software for displaying interfaces with information provided from a remote computer, such as server 20.
In operation, the tag file 305 supplies a set ‘m’ of user generated tags, collected over a predetermined time period, as input to the tag identification module. The tag identification 310 is configured to identify distinct tags from among the tags included in the tag file 305. The tag identification module 310 outputs a set of distinct tags, ‘n’, as input to the z-score computing module 315. The z-score computing module 315 is configured to calculate z-scores for a given tag output from the tag identification module 310. The set of sorted or ranked tags ‘n’, output from the z-scoring module, is supplied as input to the rejection module 325 which is configured to reject those tags that have z-scores lower than a threshold, which may be a predetermined threshold. The rejection module outputs the tags whose z-score is at least equal to or greater than the threshold. This set of tags is referred to as the set of statistically significant distinct tags 330.
Process 400 begins at block 405 whereupon the process 400 collects the total number of tags ‘m’ that a given user provides over a time period, which may be a predetermined time period. It is appreciated that the time window is a configurable parameter that may change. For example, a seasonal time period may be used to infer seasonal interests
The process 400 then identifies at block 410, a subset of distinct tags ‘n’ from among the total number of collected tags ‘m’.
The process 400 then moves to block 415 where a z-score is computed for each identified distinct tag ‘n’ at block 410. The z-score is a measure of the observed frequency, freq (i), of a given identified distinct tag ‘i’. As will be described below, the process uses the z-score to compare the observed frequency against a hypothetical frequency in which a user tags web content at random. In the present embodiment, the z-score for a given tag may be computed as follows:
z(i)=[freq(i)−μ]/σ (1)
The term freq (i) in equation (1) represents the observed frequency of occurrence of each distinct tag ‘i’ provided by a user over a predetermined time period. For example, it may be observed that a user provides a “surfing” tag to web-based content a total of 33 times over a period of one year. In this case, freq (i)=33 for the distinct ‘surfing’ tag.
The process, according to the present embodiment, compares an observed frequency of occurrence, freq(i), of a user tagging web content against a hypothetical frequency of occurrence in which a user tags web content at random. This difference is exploited by the present invention to identify those tags that infer a user's interests as opposed to other tags that merely reflect random choice by a user. The z-score, in combination with a threshold α, provides a quantitative means for identifying those tags that infer a user's interests. In general, random tagging behavior, in the context of the invention, may be defined as behavior in which a user provides a total of m tags and n distinct tags and uses the n distinct tags with equal probability of 1/n, independent of every other distinct tag i.
It is well known, to those knowledgeable in the statistical arts, that in random tagging behavior, the number of occurrences X of each distinct tag i has a Gaussian distribution with a mean and standard deviation, calculated as follows:
μ=m/n (2)
and
σ=sqrt (m/n) (3)
These two values are incorporated into equation (1) to derive a z-score for each identified distinct tag ‘i’ which provides a statistical measure of the tag's statistical significance in predicting or inferring a user's interests.
At block 420, the ‘i’ distinct tags are ranked in accordance with their respective z-score values. This is illustrated, by way of example, in the table 500 of
At block 425, a determination is made regarding the statistical significance of each of unique or distinct tag ‘i’. In the present embodiment, this determination is made using a threshold criteria α. Distinct or unique tags having a computed z-score, z(i), greater than the threshold criteria α are considered to be statistically significant in the sense that the tags statistical significance exceeds mere random choice by a user. In other words, the computed z-scores provide an indication that the user is more biased towards, or has an interest in, these distinct tags ‘n’ than mere random chance.
In the present embodiment, the threshold a is determined on the basis of a desired statistical confidence for identifying tags that are true representations of a user's interests. For example, for a 95% statistical confidence, α=1.64 and for a 99% statistical confidence, α=2.33. The α value can be chosen based on the statistical confidence desired. According to one embodiment, the alpha values may be looked up from a standard statistical table for “Areas of a Standard Normal Distribution” for a given confidence level (which may be determined by the user or a software process).
It should also be appreciated that the process of
It is appreciated that the process may consider different start and end times over which the total number of tags ‘m’ is provided by a typical user. Further, the predetermined time may be continuous or non-continuous to predict seasonal or short-term interests of users.
In accordance with one optional embodiment of the present invention, the determination of distinct tags that are representative of a user's true interests, may be improved by editing (pre-processing) the user provided collection of tags ‘m’ to correct for misspellings, synonyms and for removing commonly used words such as, ‘the’ and ‘howto’, for example. In such an embodiment, a pre-processing module may be utilized to operate on the collection of user provided tags ‘m’ and provide, as output, an edited set of tags ‘m’.
In accordance with one optional embodiment of the present invention, for those tags determined to be statistically significant, a post processing module may be utilized in the system 10 to further classify the statistically significant tags into broader categories to generalize the user tags. For example, a tag such as “c++” or “java” may be generalized to a tag having a broader context, such as “programming” by leveraging knowledge of the tag labels to provide more useful results.
In accordance with one optional embodiment of the present invention, it is contemplated to provide a means for comparing tags from two or more users. The comparison may take any number of forms including, for example, comparing the statistically significant tags from each user based on one or more criteria, including, without limitation, the quantity of statistically significant tags, a difference in the z-scores, etc. It should be noted that the two or more users may comprise a social network, and the techniques of the embodiments of the present invention are applicable to sets of tags from social networking sites such as Flickr, Del.icio.us, and other social networking sites know to those of skill in the art.
While the embodiments described herein may make reference to web sites, links, and other terminology specific to instances where the World Wide Web (or a subset thereof) serves as the search corpus, it should be understood that the systems and processes described herein can be adapted for use with a different search corpus (such as an electronic database or document repository) and that results may include content as well as links or references to locations where content may be found. Clickable buttons and other user interface control elements referred to herein may be replaced by any suitable alternative elements.
While the invention has been described in connection with various embodiments, it will be understood that the invention is capable of further modifications. This application is intended to cover any variations, uses or adaptation of the invention following, in general, the principles of the invention, and including such departures from the present disclosure as come within the known and customary practice within the art to which the invention pertains.
