Method and apparatus for responding to request for information-personalization

Description

This application is related to the contemporaneously filed U.S. patent application Ser. No. 11/207,589 entitled “Method and Apparatus for Responding to End-User Request for Information—Collecting,” filed on Aug. 19, 2005 and issued as U.S. Pat. No. 7,444,358 on Oct. 28, 2008, and U.S. patent application Ser. No. 11/207,592, entitled “Method and Apparatus for Responding to End-User Request for Information—Ranking”, also filed on Aug. 19, 2005, and issued as U.S. Pat. No. 7,836,009 on Nov. 16, 2010.

FIELD OF THE INVENTION

The present invention relates to an advanced search engine. The advanced search engine may include a client component for monitoring an end-user's browsing activity, a remote server (may comprise one or more computers) for storing and processing data received from the client component, and a module that process web pages and serves search results to end-users. The advanced search engine may collect web pages for keywords of proven interest, fetch web pages requested by end-users, generate snippets or abstracts of the web pages, eliminate duplicate web pages, rank the importance of the web pages, and provide relevant web pages or links to web pages in response to an end-user search request for information regarding one or more keywords, for example. Technical problems solved, measures used and results obtained are discussed below.

BACKGROUND

One approach to search engines, taken by Google, is to organize the world's information and make it universally accessible and useful. Another approach, once taken by Dogpile, is to have a meta-search engine aggregate the results of other search engines. These approaches create a great haystack of results. For instance, the keyword “cheap travel” returns about 18,000,000 results from Google, about 85,800,000 from Yahoo and 68,377,619 from MSN, as of summer 2005!

Much work has been done to float the “needles” to the top of the results haystack, to devise methods of ranking links returned in response to a query. Google's published patent applications propose, in their titles, to use local inter-connectivity, article information, location awareness and other factors to decide on the position of results. Yahoo, Overture and Microsoft also have worked to refine their presentation of results.

In any set of information, a search term is sometimes not enough to determine what results are sought. In one sense, the search term may be ambiguous, as extensively discussed in Bharat et al., “Generating User Information for Use in Targeted Advertising”, US 2005/0131762 A1 published Jun. 16, 2005 and in Carrasco et al., “Disambiguation of Search Phrases Using Interpreation Clusters”, US 2005/0015366 A1 published Jan. 20, 2005. The term “jaguar” might refer to cars, animals, a football team, or an operating system. Even if a term were unambiguous, different users might prefer to access different information. For instance, teenage travelers, business travelers and luxury travelers look for different travel arrangements and accommodations, potentially using similar search terms.

It is desirable to return the most relevant results, whether in response to a search or, more generally, on an information feed. The growing number of documents published on web sites (and of documents accessible on private servers) invites development of alternative or improved technology to quickly return relevant results responsive to users' queries. In effect, to find the 50 or 100 most relevant web sites for a particular user whose keyword is “cheap travel” and effectively summarize them for the user. This further invites development of technologies that personalize the information returned, whether content, sponsored content or advertising, based on the interests of the user.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of an example computer that may be used in various embodiments.

FIG. 2 shows a schematic diagram of a computing environment in which this technology may operate.

FIG. 3 schematically illustrates the tracking of a conversion process.

FIG. 4 depicts a high level block diagram of components that can function together to produce a search engine that personalizes sponsored and non-sponsored information delivery.

FIG. 5 is an example of a cookie or data record that reports a URL clicked.

FIG. 6 is an example of a cookie that stores the most recent commercial keyword of interest.

FIG. 7 is an example of summary data maintained by the behavior watcher that may be written to persistent memory.

FIG. 8 illustrates a bit string further summarized by category.

FIG. 9 illustrates buckets that may be applied to recency of using search words, clicking through, entering URL destinations or other indications of user interest.

FIG. 10 illustrates buckets that can be applied to frequency of visits.

FIG. 11 illustrates how a category ID can be associated with both the category name and a revenue priority.

FIG. 12 compares the insight that a behavior watching module on a user's machine gives regarding web surfing with the limited view that a server can achieve.

FIG. 13 depicts a web page highly relevant to the keyword “cars”.

FIG. 14 is a counter example of a page weakly relevant to “cars”.

FIG. 15 juxtaposes the results for “cars” from Yahoo!, msn Search and Google with an embodiment of the search engine described herein.

FIG. 16, illustrates user differentiation using results from a behavior watching module.

FIG. 17 depicts traffic to a particular domain, from a particular search engine, by keyword.

FIG. 18 shows an example of a web spider that may be implemented by a Java application operating in a Linux environment.

FIG. 19 provides data regarding the importance of recent behavior in response to banner ads, measured by click through rates.

FIG. 20 depicts how the window scoring routine begins with the first word of text within the document.

DETAILED DESCRIPTION

The following detailed description is made with reference to the figures. Preferred embodiments are described to illustrate, not to limit the scope of the claims. Those of ordinary skill in the art will recognize a variety of equivalent variations on the description that follows. Persons of ordinary skill in the art will recognize, however, that the embodiments described can be practiced without one or more of the specific details. In other instances, well-known details are not shown or described to avoid obscuring aspects of the embodiment.

Being computer-related, it can be appreciated that the components disclosed herein may be implemented in hardware, software, or a combination of hardware and software e.g., firmware). Software components may be in the form of computer readable program code stored in a computer-readable storage medium, such as memory, mass storage device, or removable storage device. For example, a computer readable medium may comprise computer-readable program code for performing the function of a particular component. Likewise, computer memory may be configured to include one or more components, which may then be executed by a processor. Components may be implemented separately in multiple modules or together in a single module.

Embodiments and aspects of embodiments described below can be applied to solve various technical problems. One problem applies to a large network: how to monitor and usefully aggregate patterns of communication among users, search engines and documents accessed. In one scenario, the users are surfing the Internet at widely disbursed personal computers; the search engines include Baidu, Google, Yahoo! and MSN; the documents are pages posted on web sites around the world. In another, employees use an enterprise intranet with an enterprise search engine to locate reference documents exposed on workgroup servers. Addressing these problems may involve positioning a behavior observing module where it can monitor the communication channels in use and report observations to a server, preferably without disrupting the users' routines. In one embodiment, the behavior observing module may run on the user's personal computer (such as a desktop, laptop or handheld computer or media center device). The module can observe both communications and the status of the user's machine when the communications take place. For example, what search terms and results did a user follow to a particular web site? The module can achieve distributed processing and substantially reduce the resources required to aggregate communication behavior by filtering reports and categorically encoding activity. It may report observations to the server as resources are available or when a user browses to an affiliated domain. As part of the ordinary uploading of cookies to domains accessed, observation cookies can be transmitted to the server. The server can aggregate reported patterns of communication. One technical result is receiving reports from communication monitors positioned across a large network and aggregating patterns, including the status or state of individual computers when communications took place. In turn, the aggregated patterns of communications can be used to pre-organize information for retrieval or publication, in anticipation of a query or contact.

A related technical problem is how to organize over-abundant electronic records based on the current and recent status of a particular terminal connected to a network, to enhance the relevance of the first records presented to a user. The over-abundant electronic records may come from web sites world wide, such as the web sites for “jaguar.” Or, they may be documents stored on workgroup servers. They are over-abundant in the sense that they are too numerous to display on a user's screen without repeatedly pressing “page down” or the like. The current and recent status of the terminal, preferably associated with a particular user, may include web sites visited within the last 24 hours (or some other period) before a search query was submitted. Or, it may include a history of documents retrieved from workgroup servers. Either type of activity may be preprocessed and categorically classified. The period for reporting terminal status may precede a query or request for a personalized electronic journal that selects and filters the information based on the current and recent status of the terminal. The technical method again involves a behavior watching module running on the user's computer. In this embodiment, the module summarizes terminal status, publishes the summary to an electronic record (e.g., a cookie) and communicates the record to a search engine or other site that the user contacts. Reporting categorical summaries of status or activity distributes processing and reduces the need for server-based resources. The search engine uses the information, for instance, to determine what sense of “jaguar” is of interest? A highly involved auto category user who searches for “jaguar” would get Jaguar auto related links, while a person with no relevant category involvement would get a mix of auto, animal, etc. links. Category profiles may be developed to further categorize a user's interests. A new parent in the suburbs might be recognized from content accessed and be assigned to a different category for home accessories or cars than a single gen-X 20-something person. Life change events also might be recognized, such as marriage, home buying or parenthood. Like the “jaguar” example, for a “travel Italy” inquiry, the search engine might determine the style of travel that interests the user and organize the over-abundant electronic records accordingly. The technical result is respond to a query or contact based on an electronic report of the status or recent activity of a terminal, selecting from over-abundant electronic records a particular set of records that are most likely relevant to the current and recent status of the terminal.

Web-wide behavioral targeting differs substantially from site-side behavioral targeting. Practicing site-side behavioral targeting, a group of affiliated sites attempt to identify commercial behaviors. The sites typically serve ads, such as portals and news sites. Messages are displayed on the affiliated sites, responsive to behavior that is recognized from visits to the affiliated sites. The results of site-side behavioral targeting are better than non-behaviorally targeted campaigns, but depend on insight that can be gained from a narrow portion of user's behavior, as illustrated by FIG. 12, which is described below. Web-wide behavioral targeting, in contrast to site-side, can identify commercial surfing behavior and, optionally, non-commercial surfing behavior of interest, across all sites, even sites to do not accept advertising. This is significant because 90% of commercial Web activity takes place at sites or on pages that do not display advertising. Therefore, portals see less than 10% of user's total commercial activity, as measured by page views. Non-portal publishers see less than 3% of their users' total commercial activity. A behavior watching module may simplify the relationship with web site operators and allow messages to be displayed across a larger network of publishers than participate in site-side behavioral targeting. Web-wide behavioral targeting produces better results than site-side behavioral targeting.

Referring now to FIG. 1, there is shown a schematic diagram of an example computer that may be used in various embodiments. Depending on its configuration, the computer shown in the example of FIG. 1 may be employed as a client computer, a server computer, or other data processing device. The computer of FIG. 1 may have less or more components to meet the needs of a particular application. As shown in FIG. 1, the computer may include a processor 101, such as those from the Intel Corporation, Advanced Micro Devices or IBM, for example. The computer may have one or more buses 103 coupling its various components. The computer may include one ore more input devices 102 (e-g., keyboard, mouse), a computer-readable storage medium (CRSM) 105 (e.g., floppy disk, CD-ROM), a CRSM reader 104 (e.g., floppy drive, CD-ROM drive), a display monitor 109 (e.g., cathode ray tube, flat panel display), a communications interface 106 (e.g., network adapter, modem) for coupling to a network, one or more data storage devices 107 (e.g., hard disk drive, optical drive, FLASH memory), and a main memory 108 (e.g., RAM). Software embodiments may be stored in a computer-readable storage medium 105 forwarding into a data storage device 107 or main memory 108. Software embodiments in main memory 108 may be executed by processor 101. In the example of FIG. 1, main memory 108 is shown as having a behavior watching and message delivery program 120, which is further described below. Behavior watching and message delivery program 120 and other programs in main memory 108 may be loaded from a computer-readable storage medium 105, a data storage device 107, or over the Internet by way of communications interface 106, for example. Behavior watching and message delivery program 120 and other programs in main memory 108 may be executed by processor 101.

FIG. 2 shows a schematic diagram of a computing environment in which this technology may operate. In the example of FIG. 2, the computing environment includes one or more web server computers 160 (i.e., 160-1, 160-2, ..., 160-n), one or more client computers 110, one or more message server computers 140, and other computers not specifically shown. In the example of FIG. 2, a client computer 110 communicates with server computers (e.g., a web server computer or a message server computer) over the Internet. As such, arrows 201 denote Internet connections in this example. Intermediate nodes such as gateways, routers. bridges, Internet service provider networks, public-switched telephone networks, proxy servers, firewalls, and other network components are not shown for clarity.

A client computer 110 is typically, but not necessarily, a personal computer such as those running the Microsoft Windows™ operating system, far example. A consumer may employ a suitably equipped client computer 110 to get on the Internet and access computers coupled thereto. For example, a client computer 110 may be used to access web pages from a web sever computer 160.

A web server computer 160 may be a server computer hosting a website, which comprises web pages designed to attract consumers surfing on the Internet. A web server computer 160 may include web pages supporting advertisements, downloadable computer programs, products available for online purchase, and so on. As can be appreciated, a website may be on one or more server computers.

A message server computer 140 may include the functionalities of a web server computer 160. In one embodiment, a message server computer 140 further includes a database 170 and a search engine 650. Database 170 may be a commercially available database, such as those available from the Oracle Corporation. Database 170 may store client data received from behavior watching and message delivery programs 120 running in client computers 110. The client data may be transmitted from a client computer 110 to message server computer 140 in a data packet 121. The client data may include navigation and behavioral data obtained by a behavior watching and message delivery program 120 by monitoring a consumer's on-line activities. In the example of FIG. 2, message server computer 140 is shown as communicating with one client computer 110 for clarity of illustration. In practice, message server computer 140 receives data packets 121 containing client data from a plurality of client computers 110, each of which has a behavior watching and message delivery program 120. A message server computer 140 may also include downloadable computer programs and files for supporting, updating, and maintaining software components on a client computer 110.

Web server computers 160 and message server computers 140 are typically, but not necessarily, server computers such as those available from Sun Microsystems, Hewlett-Packard, or International Business Machines. A client computer 110 may communicate with a web server computer 160 or a message server computer 140 using client-sever protocol. It is to be noted that client-server computing is well known in the art and will not be further described here.

As shown in FIG. 2, a client computer 110 may include a web browser 112 and a behavior watching and message delivery program 120. Web browser 112 may be a commercially available web browser or web client. In one embodiment, web browser 112 comprises the Microsoft Internet Explorer.TM. web browser. A web browser allows a consumer on a client computer to access a web page. In the example of FIG. 2, web browser 112 is depicted as displaying a web page 313 from a web server computer 160, A web page, such as web page 113, has a corresponding address referred to as a “URL” (Uniform Resource Locator), Web browser 112 is pointed to the URL of a web page to receive that web page in client computer 110. Web browser 112 may be pointed to a URL by entering the URL at an address window of web browser 112, or by clicking a link 203 (ie., 203-1, 203-2, . . . , 203-n) pointed to that URL, for example.

In one embodiment, behavior watching and message delivery program 120 is downloadable from a message server computer 140 or a web server computer 160. Behavior watching and message delivery program 120 may be downloaded to a client computer 110 in conjunction with the downloading of another computer program. For example, behavior watching and message delivery program 120 may be downloaded to client computer 110 along with a utility program 181 that is provided free of charge or at a reduced cost. Utility program 181 may be a wallet or calendar program, for example. Utility program 181 may be provided to a consumer in exchange for the right to deliver advertisements to that consumer's client computer 110 via behavior watching and message delivery program 120. In essence, revenue from advertisements delivered to the consumer helps defray the cost of creating and maintaining the utility program. Behavior watching and message delivery program 120 may also be provided to the consumer along with free or reduced cost access to an online service, for example.

Behavior watching and message delivery program 120 is a client-side program in that it is stored and run in a client computer 110. Behavior watching and message delivery program 120 may comprise computer readable program code for displaying advertisements in a client computer 110 and for monitoring the online activity of a consumer on the client computer 110. It is to be noted that the mechanics of monitoring a consumer's online activity, such as determining where a consumer is navigating to the URL of web pages received in client computer 110, the domain names of websites visited by the consumer, what the consumer is typing on a web page, what keyword the consumer is providing to a search engine, whether the consumer clicked on a link or an advertisement, when the consumer activates a mouse or keyboard, and the like, is, in general, known in the art and not a further described here. For example, behavior watching and message delivery program 120 may learn of consumer online activities by receiving event notifications from web browser 112.

Behavior watching and message delivery program 120 may record the consumer's online activity for reporting to message server computer 140. The recorded consumer online activity is also referred to as “client data,” and provided to message server computer 140 using data packets 121. Message server computer 140 may use the client data to provide targeted advertisements to the consumer. Message server computer 140 may include the advertisement or data for displaying the advertisement in a message unit 141. In the example of FIG. 2, the targeted advertisement is labeled as advertisement 116 and displayed in a presentation vehicle 115. Presentation vehicle 115 may be a pop-under, pop-up, separate browser window, custom browser window, or other means for displaying an advertisement on a computer screen. Techniques for delivering advertisements to client computers using a client-side program are also disclosed in commonly-owned U.S. application Ser. No. 10/152,204, entitled “Method and Apparatus for Displaying Messages in Computer Systems,” filed on May 21, 2002 by Scott G. Eagle, David L. Goulden, Anthony G. Martin, and Eugene A. Veteska, issued June 27, 2006 as U.S. Pat. No. 7,069,515, which is incorporated herein by reference in its entirety.

As will be more apparent below, behavior watching and message delivery programs are primarily used to obtain client data far building a search engine index, not necessarily to display presentation vehicles in a client computer 110, That is, a behavior watching and message delivery program does not necessarily have to display advertisements in a client computer 110. This is advantageous in that consumers may be allowed to obtain a free or reduced cost utility program 181 (or other benefits) without having to see advertisements from the provider or sponsor of the utility program.

FIG. 3 schematically illustrates the tracking of a conversion process. A “conversion” occurs when a consumer makes an online purchase or registration in response to an advertisement presented to her. As will be more apparent below, the embodiments may use conversion as a factor in ranking a keyword-link combination. In the example of FIG. 3, web pages 202 (i.e., 202-1, 202-2, . . . ) may be sequentially displayed in the same or separate windows of web browser 112. Each web page 202 includes a page identifier 210 (i.e., 210-1, 210-2, 210-3 . . . , 210-4, 210-5), which may be a URL. Behavior watching and message delivery program 120 records the URLs of web pages 202 viewed by the consumer as well the amount of time the consumer spent with each web page as navigation data. For purposes of the present disclosure, the web pages “viewed” by consumers are those web pages received in client computers. In the example of FIG. 3, navigation data 627 comprises log entries 117 (i.e., 117-1, 117-2, . . . , 117-n). Each log entry 117 includes a machine ID anonymously identifying the client computer 110 (or the consumer), a page identifier, and a time stamp indicating when the log entry 417 was made. The time stamps between log entries 117 provide an estimate of the amount of time the consumer spent viewing the indicated web page. A log entry 117 may be created for each web page 202 viewed by the consumer. For example, a log entry 117-1 may be created when the consumer clicks oh a link 501 to receive web page 202-2. In client computer 110, a log entry 117-2 may be created when the consumer receives web page 202-3 in client computer 110, and so on.

In the example of FIG. 3, web page 202-2 is also referred to as a “landing page” because it is the web page directly pointed to by an advertisement having a link in web page 202-1. A website that sells products on line may also have a “confirmation page” 202-5. A confirmation page is a web page provided to the consumer to confirm a just completed online purchase. Alternatively, a confirmation page may involve confirmed submission of registration information. A website may have “intermediate pages” 202-3, 202-4, and so on between a landing page 202-2 and a confirmation page 202-5. An intermediate page may be an online product catalog, shopping cart, and other types of web pages. The page identifiers of landing and confirmation pages of popular or partner websites may be stored in database 170 of message server computer 140, the page identifiers of the landing and confirmation pages may be compared to those in navigation data 627 of client computers 110 to determine the number of consumers making online purchases from a particular web page. As can be appreciated, web pages from which consumers make purchases may be highly relevant, and may thus be deemed a consumer preferred page. Techniques for monitoring consumer purchase behavior are also disclosed in commonly-owned U.S. application Ser. No. 10/464,419, entitled “Generation of Statistical Information in a Computer Network,” filed by David L. Goulden and Dominic Bennett on Jun. 17, 2003, published Dec. 23, 2004 as U.S. Pub. No. 2004-0260744, and issued July 15, 2008 as U.S. Pat. No. 7,401,140, which is incorporated herein by reference in its entirety.

Process Flow

FIG. 4 is a high level block diagram of components that can function together to produce a search engine that personalizes sponsored and non-sponsored information delivery. As background, behavior watching modules currently capture behavioral web-navigation information on over 40 million machines in the U.S. and abroad. The information collected will expand and be adapted as needed. This navigational behavior information is currently used to power web-wide GMT ad vehicles (e.g., pop-ups and slide-unders) as well as power many data-rich applications that give unique insights into consumer behavior across the web. The behavior watching module has the ability to see the entire search life-cycle, from observing the initial keyword entered, to the resultant links displayed, to post-click metrics including dwell times, pages viewed, repeat visits, and post-click purchases or registrations. By aggregating a user's behavior, category profiles and life change events can be determined. User sophistication with search engine usage and degree of category involvement can be determined over time. Using this information, one can select and re-rank the search results, filtered by humans who requested the searches, not by bots.

Data Collection

Web usage statistics are collected (407) using behavior watching modules (120) for users searching on selected search engines across the Internet or an enterprise intranet. The modules send back information related to their searches and how effective their searches were on each particular search engine for each particular keyword. The information is captured in a database, which is loaded daily or at some other concurrent frequency. Information available includes what the user saw and how they responded. URLs seen by the user may be displayed in algorithmic or natural sections the results. Pertinent information available for a single search term includes machine id, keyword, search engine where search was performed, resultant URLs, algorithmic URLs, bidded URLs, paid inclusion URLs, whether or not a URL was clicked, number of pages viewed, dwell time, repeat visits and user metrics such as category involvement and search engine sophistication.

Some statistics from U.S. users can bring to life the analytical reach of data collection and ranking. The one million most frequently used keywords presently account for just more than half of the searches conducted on the major search engines, 53% of the searches. The 10,000 most frequent searches account for 38%. The distribution of keywords entered as searches can be represented by a Zipf distribution, which plots as a straight line on a graph with logarithmic scale on both axes. See, e.g., Jacob Nielson, “Diversity is Power for Specialized Sites”, Alertbox (Jun. 16, 2003) accessed Aug. 13, 2005 at http://www.useit.com/alertbox/20030616.html. It is estimated that 2,000 behavior watching modules will generate enough data to rank the 10,000 most frequent searches. A larger group of behavior watching modules will cover the most frequent searches. A base of 40 million behavior watching modules may capture 30 billion rows of data per month, filtered from 150 billion page views observed. Among the page views observed, on the order of 8 billion commercial events per month are noted, including more than 2.5 million purchases. These numbers and even a passing familiarity with statistics should excite the reader to aggregating the intelligence and behavior of a multiplicity of searchers, and presenting by popular acclaim the most significant web sites on the first page of results from a search engine. A search engine that uses aggregated consumer behavior is well-positioned to rate the authority and popularity of pages/documents as responsive to a search request.

FIGS. 5-11 are examples of data records or cookies that can communicate information from the behavior watching module to a ranking and link-following server and to a search engine that produces personalized results. FIG. 5 is an example of a cookie or data record that reports a URL clicked. A keyword 501, time stamp 502 and list of one or more URLs selected by the user 503 are conveyed, for instance in a cookie when a user reaches a web site affiliated with one of the servers or in a data record when the behavior watching module reports activity, independently of Internet surfing behavior. Additional data regarding conversion status, return conversion status, rank in a list that a user saw, etc. can be included in such a cookie or data record.

FIG. 6 is an example of a cookie from that stores the most recent commercial keyword of interest. This may be the most recent commercial keyword of interest from a search or a keyword associated with a URL or click-through. More than one recent commercial keyword of interest may be stored in one or more cookies, following the example of FIG. 6. One attribute or field of this cookie is the version of the crumb 602. Another is the time written 604, which may be expressed in seconds from 1970 to the time the cookie is written. A commercial flag 606 may be used to categorize the keyword type, for instance between key words that are of current commercial interest and key words that are for sale. The keyword may be included 608, either in text or as a shorthand code. A character set 610 such as UFT-8 may be included to assist with decoding of the keyword 608. This may be important in international settings.

FIG. 7 is an example of summary data maintained by the behavior watcher 120 that may be written to persistent memory 122. In this example, a category ID 802 is a shorthand for a particular category. Bins are allocated for a current period 702, such as a day. The periods may be of equal length for different lengths. In one embodiment, a bit string implements the bins. A history of 60 days may be implemented by 60 bits plus storage for a category ID. Each bin corresponds to a particular time segment. A bit flag indicates whether there was activity 704 related to the category during the particular time segment. At a regular time corresponding to the shortest period represented by any bin, the bit string is updated. Data are shifted into older bins, making room in the most recent bin. Data regarding recent activity is summarized into the most recent bin, labeled zero in the figure. If bins represent unequal amounts of time, special provisions are made at the boundaries between shorter and longer time bins, either algorithmically or by allocating additional storage. For instance, activity during any one-hour segment during a preceding day may result in setting the bit flag in a day-long time segment. The day-long time segment is flag “0” only if the hour segment flag at midnight is “0” when the midnight update takes place. This is an example of an algorithmic provision at the boundary between a one-hour segment and a day-long segment. Alternatively, storage may be allocated for bins to record number of visits in a particular time segment. This may significantly increase the amount of storage required, without necessarily changing how a reported-behavior aware server 132 responds. Compact storage is preferred when there are hundreds of categories and the practical limit for the size of a cookie is 2K to 4K bytes. Cookies can be sent to a server that archives user activity data. The user activity data can be used for analysis or targeting of ads. A bit string can be further summarized by category, shown in FIGS. 8-10.

FIG. 9 illustrates buckets that may be applied to recency of using search words, clicking through, entering URL destinations or other indications of user interest. The most recent visit is recorded by membership in a particular bucket. One of 15 or 16 buckets can be identified with four bits. In the figure, the buckets are of unequal length. A single, most recent visit record such as illustrated by FIG. 6 or an entry kept in memory can be used to assign a category to a particular bucket.

FIG. 19 provides data regarding the importance of recent behavior in response to banner ads, measured by click through rates. A user who had behavior within the last hour in an advertising category was 30 times as likely to respond to a banner ad as one with no behavior over the last 7 days in the category. Behavior in the last hour, two hours or three hours has been observed to generate a huge increase in responsiveness to banner ads. Even behavior within the last seven days correlates strongly with a user's interest. Preferably, the selection of recency bucket sizes reflects the observed power of recent behavior illustrated by the figure.

FIG. 10 illustrates buckets that can be applied to frequency of visits. When the bit string of FIG. 7 is organized by days, activity in a category can be assigned to buckets of different visit frequencies, such as 0, 1, 2, 3, 4 or 5, 6 to 10, 11 to 30, or 31 to 60 days out of the last 60 days in which the user indicated interest. The number of bits set in the bit string is counted and summarized using this categorization.

With the categorical coding of recency and frequency in mind, we turn to FIG. 8. A single cookie may record data for many category IDs 802. For each category ID, three or more metrics of user interest may be recorded. One or more recency categories 804 can be recorded. The recording of recency can be applied to key words submitted to search engines or to other kinds of user behavior, described above. One or more frequency categories 806 can be recorded, applied to key words submitted or other kinds of user behavior. Banner clicks 808 can be separately categorized by recency, frequency or both. The comments in data column 810 are editorial comments to explain the data in the table and are not part of the data. Category ID 98409 experienced heavy and recent user interest. Category ID 65625 experienced heavy user interest last month. Category ID 68530 experienced light, recent user interest. Category 147374 experienced light activity last month, which probably also should be reflected by a nonzero value under frequency 806.

The behavior watcher module 120 preferably sorts the category history information in FIG. 8 so that the most important categories are listed first. Alternatively, the system could leave the category sorting to the reported-behavior responsive server 132. As illustrated in FIG. 11, a category ID 1102 can be associated with both the category name 1104 and a revenue priority 1106. It may be useful to sort category history information by revenue priority 1106, especially if only a portion of the client-sided user behavior data is transmitted to a reported-behavior responsive server 132.

The statistics returned regarding web usage may or may not distinguish between commercial and non-commercial keyword searches. This distinction is made at the time of loading into the database where keywords are checked against the ever-changing list of commercial terms, for instance, terms bidded by Overture. If a keyword is determined to be commercial, then it is assigned a keyword id, which may be compatible with the Overture keyword id list. If the keyword is not deemed commercial, then another id is assigned in the data loading process.

These two types of data (commercial and non-commercial) are loaded into separate sections of a data warehouse. At the time of a new search engine loading run, keyword data is extracted from both the commercial and non-commercial tables. The results are joined and unique keyword ids are assigned from a master table. A keyword can sometimes be found in both the commercial and non-commercial contexts. For instance, if the commercial nature of a keyword is tied to whether it's being bidded upon, a keyword which is not commercial today may become a commercial tomorrow, if it receives a bid. This duality of keywords creates non-unique keyword ids as the same keyword may have both a keyword id created by a bidding process and a second one created through the load process. To eliminate this, it is preferred to create and-maintain a single unique keyword id for every keyword. This list is updated when new keywords are discovered, and assigns existing ids to keywords already in the system.

Some preprocessing may be performed by the behavior watching module to simplify the URLs reported. URLs are unwrapped and cleaned in a separate process. URLs are often wrapped by search engines to enable the serving search engine to track clicks on served URLs. There are many different forms of URL wrapping. For example, a wrapped URL from yahoo.com might be: http://rds.yahoo.com/S=2766679/K=bmw/v=2/SID=w/1=WS1/R=2/SS=100040736/H=1/SHE=0/*-http://www.bmwmotorcycles.com/. Unwrapping the URL produces http://www.bmwmotorcycles.com

From the server's perspective, the process begins with receipt (401) of behavioral information. The server uses whatever information it receives. From the user's perspective, the behavior watching module will report its observations and the user will receive search results ordered aggregating the user's information with others and/or will receive personalized to the user's recent behavior.

De-duping may also occur at the URL level. Information for two URLs which are identical is aggregated into one single URL. If two URLs differ even slightly however, (e.g., by a slash) then the two distinct versions are kept and another attempt at de-duplication is made as described below, for instance, using a combination of title and generated snippet.

Conversion data can be associated to a specific search by an algorithm that ties a search URL click to a specific conversion event, which occurred within a predetermined window. Usage and conversion data are matched for advertiser domains (URLs) that have clicks at the machine id, query time, advertiser domain level. For a particular machine with a click on a particular advertiser domain, if a conversion stat is observed within a predetermined window, then the conversion is attributed to that search click. If the conversion falls outside of the predetermined window, then the search click is not attributed.

For machine id-advertiser domain pairs that have a conversion stat attributed, subsequent future conversions are attributed as repeat conversions. These attributes also may be carried along and are available for use by a ranking algorithm (404). Metrics included with search data include number of visits, time spent (dwell time) and pages viewed.

Domain event data are joined to user data (with conversion metrics) at the machine id-advertiser domain level across sources, for combining search behavior for U.S. machines at google.com, msn.com, and yahoo.com. These results are put into time series order within machine id and advertiser domain. Domain events which occur within a predetermined time period following a search click are assigned as post-click metrics for that search click on that particular advertiser domain. If post-click metrics cannot be assigned to a particular search-click, the record is thrown out.

Several of the domain event data elements are subject to inaccuracies manifested in the client-sent stats. Both time spent and pages viewed are occasionally misreported, and at other times, accurately reported, but in need of logically driven limits to be imposed.

Time spent can be misrepresented by machines having bad or inaccurate clocks. It can be accurately represented but in need of caps in such a case when a machine is left on a particular domain for an extended period of time. In order to cap outliers and to maintain a reasonable threshold for time spent on a site post click, a time limit of 30 minutes has been employed. A cap for pages viewed has also been implemented and set at 5. Other time and pages viewed caps may be substituted. These caps can be implemented on the behavior watching client side or after data are received at a host. In addition, user activity can be monitored by the behavior watching module so that extended periods of inactivity are not counted as dwell time.

When data are joined and aggregated at the machine-id, keyword, and URL level, the resulting data structure may include: machine-id, keyword, keyword_id, URL, URL_ID, domainid (corresponds to the domain of the URL), clicks, dwell_per_click, pages_per click, conversions_per_click, rank_position (from search results list viewed by the user). Optionally, only keywords of predetermined interest (402 may be processed. For instance, keywords having commercial interest, such as bidded keywords, may be processed.

Outliers optionally may be removed (403), to avoid scoring anomalies. One example of an outlier is a link that is returned only once by a search engine and followed with enthusiasm by the user when was returned. A single strong sample point can give a link an unbeatable average score. A link that appeared just before a keyword was rescored could potentially be ranked in the top position for that keyword on the basis of a single sample! Accordingly, one example of optionally removing outliers is to not rank links unless they have been followed a predetermined number of times. For instance, if a link has not been selected by users and followed at least 10 or 100 times, it might remain unranked until its activity level reached the predetermined level or threshold

Ranking Algorithm

A URL ranking algorithm (404) has been developed to identify and rank links for any given keyword. Many variations on combining the aggregated observations have merit, as discussed below. One combination uses URL click rate and dwell time metrics (time spent at the domain and/or number of pages viewed), to select and rank URLs.

Optionally, user responses can be normalized for the position of a URL in a search result set. Position produces an inherent bias in URL click data for search results, which may be desirable or not. URLs occupying higher ranks gamer higher clicks. In order to account for this bias, a normalization algorithm was developed to put clicks on links in disparate positions on equal footing.

Data is aggregated for each position and average click through rates, average time spent, and average pages viewed are calculated. For each rank position (1-n), there are at least three average aggregate measures of that position's importance: click through rate, time spent and pages viewed. Normalization of these measures can be expressed as:

$\begin{matrix} Pages Viewed Inflation \\ Factor for Position n \end{matrix} = \frac{Avg Pages Viewed (Position 1)}{Avg Pages Viewed (Position n)}$

$\begin{matrix} Dwell Time Inflation \\ Factor for Position n \end{matrix} = \frac{Avg Dwell Time (Position 1)}{Avg Dwell Time (Position n)}$

$\begin{matrix} Pages Viewed Inflation \\ Factor for Position n \end{matrix} = \frac{Avg Pages Viewed (Position 1)}{Avg Pages Viewed (Position n)}$

This embodiment calculates and ranks top URLs for any keyword (404) based upon observed user metrics.

This embodiment may re-rank results based upon a time share metric, which corresponds to an individual machine's percentage vote. The algorithm takes into account user web surfing patterns and effectively places users on equal voting for relevant links. The premise is that a user has a certain amount of time which is spent on a site post a search click. These times are totaled to form the individual user's total time value which was spent viewing sites post search clicks. Percentages of the total time are then calculated for each URL click made by that particular user, resulting in a time fraction vote.

In using this methodology, users who in general spend less time surfing the Web have the same voting power as users who tend to spend longer amounts of time. This evens the playing field across all categories. Alternatively, other embodiments may take into account other factors which make up an individual user's profile. Users identified as category experts may have a higher vote. For example, a user highly involved in the electronics category may have his vote count more for links clicked than a user who is new to the category. The voting blocks may take place within a particular category, and not across all categories as a whole.

The following detailed computational example applies to a specific keyword-URL pair through the aggregation and cleansing process.

Links in higher positions garner higher clicks and hence possess higher click through rates. It follows that users also tend to spend longer amounts of time and view more pages at URLs occupying these higher ranks. In order to account for this bias, a normalization process is applied.

Average clicks, average dwell time, and average pages viewed are calculated for each position regardless of keyword or URL combinations. These numbers are shown below as Position Averages. Average fraction clicks, dwell time and page views are also shown for this keyword, URL combination. These average fractions correspond to the percentage of each metric devoted by all machines to each keyword, URL, position grouping.

Position
Position
Position

Average

Average
Average
Average
Average
Average
Fraction

Share of
Share of
Share of
Fraction
Fraction
Page

Kwd
URL
POS
Clicks
Dwell Time
Page Views
Clicks
Dwell Time
Views

01
101
1
0.15
0.14
0.18
.25
.14
.08

01
101
3
0.13
0.12
0.15
.21
.17
.09

01
101
7
0.09
0.11
0.1
.32
.18
.11

Inflation factors are calculated for each position and applied to the appropriate observed metrics, normalizing them for position. The normalizing equation:

$Inflation Factor For Position n = \frac{Average (Position 1)}{Average (Position n)}$

Clicks, dwell time, and page views in lower positions are factored up by the appropriate inflation factor observed for that particular position. In the example below, the average fraction dwell time for position 7 pre-normalization was 0.32. After the application of the inflation factor, the average fraction dwell time was 0.53. This number is now a normalized dwell time.

Clicks
Dwell Time
Page Views

Inflation
Inflation
Inflation

Factor For
Factor For
Factor For
Normalized
Normalized
Normalized

Kwd
URL
POS
Position n
Position n
Position n
Clicks
Dwell Time
Page Views

01
101
1
1.00
1.00
1.00
0.25
0.14
0.08

01
101
3
1.15
1.17
1.20
0.24
0.20
0.11

01
101
7
1.67
1.27
1.80
0.53
0.23
0.20

Weighted averages are calculated for each metric at each position, and totals are calculated across the positions for clicks, and all of the weighted average fraction measurements; clicks, time and page views.

Over time, with new reports of web usage from behavior watching modules, URLs for a particular keyword will adjust their positions. A URL in position 1 this week may be in position 7 the next. Adjusting for position is therefore a cleanup and adjustment process.

Weighted
Weighted
Weighted

Average
Average
Average

Absolute
Fraction
Fraction
Fraction

Kwd
URL
POS
Clicks
Clicks
Time
Page Views

01
101
1
155
38.75
21.70
12.40

01
101
3
124
30.05
24.59
13.39

01
101
7
98
52.27
22.45
19.40

01
101
ALL
377
121.06
68.74
45.20

The final result is the Weighted Average of Normalized Totals for Keyword 01, URL 101, at any position. Computed for every keyword, URL combination a single score is calculated for each metric needed for the ranking algorithm. These metrics are now normalized for position, and for multiple rank occurrences.

Weighted
Weighted
Weighted

Average
Average
Average

Absolute
Fraction
Fraction
Fraction

KWD
URL
POS
Clicks
Clicks
Time
Page Views

01
101
ANY
377
0.32
0.18
0.12

Ranks for scoring can be based on time that a user spends viewing pages on the domain. The information received from the behavior watching module may limit the maximum amount of time that will be assigned for any viewing session or it may track the user's behavior, such as window navigation between programs, mouse clicks or mouse movement, and disregard periods of inactivity when calculating dwell time.

Ranking may ignore links that were selected by users less than a predetermined number of times, which may be predetermined as a fixed number or a function of traffic for the keyword or category. Ignoring outlier links may avoid giving a high ranking to a link that was rarely presented by the search engines and followed only once or twice.

Ranks for scoring also can be based on a combination of click through rate, dwell time and the number of pages or documents viewed after following the link. Combining these factors, in some instances one of the factors will dominate: all or more than two thirds of the ranking weight may be assigned to just one of click through rate, dwell time or number of pages or documents viewed. Alternatively, they may be equally weighted, plus or minus 10%, or the factors may be assigned weighting ratios of approximately 2-1-1, plus or minus 10%, so that one factor is given approximately half of the combined weighting.

Another factor that can be used in ranking is return visits. If the user returns to the domain within a predetermined time after leaving it or within a predetermined number of navigation events, the user's return to the site can be assigned significance. Return to the site may reflect a favorable impression after considering other sites.

Conversion from browser to buyer or registered lead can considered to be particularly worthwhile as a factor. Again, conversion may include both a purchase in the domain and a registration. In some instances, such as car or home purchases, registration may be more realistic measure, because the purchase may be impractical or infrequently completed at a web site. Return conversion also may be taken into account.

Results may be segregated for analysis by search engine and ranks scored. Then, the separate rank scores may be combined into an overall ranking.

Statistical or other analysis can be applied within categories or keywords to determine which combination of ranking factors best attracts users to follow a link responsive to a search. It is anticipated that ranking information will be used differently among categories of keywords. Time spent will be important in the auto is category. Conversions will be much more important music downloads category.

Optionally, click segmentation bands may be applied. These bands give precedence to URLs with high numbers of clicks. Employing these bands may improve the resultant links on selected algorithms.

Segmentation bands are identified based upon total clicks received by a particular URL. For instance:

Tier 1: URLs with 100+ clicks
Tier 2: URLs with between 50 and 99 clicks
Tier 3: URLs with between 10 and 49 clicks
Tier 4: URLs with less than 10 clicks

URLs for a particular keyword are first put into the appropriate segmentation band. Once the band is identified, these URLs are set in descending order by rank score.

A predetermined number of links, such as the top 15 links (4XX), may be selected for data collection, to be followed by a spider engine (4XX).

Three tables are generated as output from the rank process:

Keyword table (KEYWORDID, KEYWORD)

URL table (DOMAINID, URL_ID, URL)

Rank Table (keyword, URL, rank)

Following Links

Traditional crawling programs at other search engines (ex: Slurp at Yahoo!, Googlebot at Google, MSNBot at MSN) crawl the entire web in search of relevant pages to index to be used in determining the rank order of links to display for a given keyword. The embodiment disclosed here, in contrast, is given a succinct number of URLs to crawl, which may optionally be selected (405) from links reported by the behavior watching module. These links are pre-ranked, hence this information retrieval process needs not determine the relative importance of a given URL from its connections to others, but rather to obtain the best possible descriptive information from the URL.

This embodiment takes a specific set of URLs and performs several specific tasks: It strips out all HTML tags and returns first 100k or another predetermined chunk of the text on the page to a file. It takes and stores a mapping from the text object's value into a uniform scalar space to be used as a text signature or text fingerprint. It calculates an MD5 or other fingerprint of the document (with or without html tags). It calculates a summary count of the characters within the text extracted from the document.

This method may be implemented by a Java application which operates in a Linux environment as illustrated by FIG. 18. The application (1811) is a brokered one which allows for a large and scalable number of threads to run simultaneously across multiple Linux servers. In the current configuration, each Linux server (1821, 1822) can handle up to 10 spider instances (1831-33, 1834-36). Each spider in turn runs 200 worker threads (1841-43, 1844-46). At capacity, this configuration is able to spider 800 links a second.

Total threads working for a single broker can be arrived at by the following equation: With i number of spiders each having j number of workers (threads):

$Total threads for i # of Spiders : \sum_{i = 1}^{n} (\sum_{j = 1}^{n} {worker}_{j})$

The dual-broker model (1821, 1822) can segregate keywords by keyword velocity. General keywords are funneled through a robust, heavy duty version of the ranking algorithm. Fast moving keywords (e.g., news, current events) can be processed through a nimble, express version of the ranking algorithm, which uses less history. Keyword velocity is a measure of how quickly the popularity of a keyword changes. The highest velocity keywords can be selected by comparing the number of keyword searches in the last 24 hours (day 0) against the 24 hours before that (day-1). A different time span, such as four or eight hours, can be used, of course. How far the ratio day 0/day-1 varies from “1” is the keyword velocity. If the ratio is less than 1, the keyword is becoming less popular, “old news.” If the ratio is much more than 1, the keyword may relate to a new story. Generally, a predetermined number of relatively high velocity keywords are re-indexed at a predetermined interval or as resources permit. In one embodiment, the top 10,000 keywords are re-indexed each day. While one metric of keyword velocity or volatility has been described, variations are anticipated.

The heavy duty version handles the ranked keyword URL pairs. These ranked keywords URL pairs are made available through an Oracle table on a database. The URL_TABLE includes: DOMAINID, URLID, URL, LENGTH, SIGNATURE_H, SIGNATURE_T, SPIDER_DATE and HOST

The DOMAINID, URLID, and URL fields are populated from a reference database prior to following the links. After the link-following process for a specific URL, the LENGTH, SIGNATURE_H, SIGNATURE_T, SPIDERDATE, and HOST fields are written back to the database.

Brokers use Java Database Connectivity (JDBC) to connect in to the Oracle database. The broker accesses the URL_TABLE from the ranking-process. The broker makes a request for 1/100th of the total number of domains which are available in the URL table for which SPIDER_DATE is null. All URLs associated with these domains are extracted by the broker where they are grouped by domain. Individual spider boxes talk to the Broker via Remote Method Invocation (RMI) requesting URLs for domains 1,000 domains at a time. Domains are then passed from the spider to a worker who takes all of the URLs associated with its domain and operates upon those URLs.

URLs are passed to the workers grouped by domain in order to accommodate generally accepted crawling or link following practices so as not to swamp domains with thousands of requests simultaneously. It is a generally accepted practice to not access a single domain with more than one request at a time. The link following process (406) respects this generally accepted principle by assigning each worker all URLs associated with a given domain.

The link following process (406) is a robust, scalable, application which fetches content and calculates statistics from a specific URL. Once a worker receives a domain and its associated URLs, it accesses that URL using HTTP protocols. If a good response code is received, a link following worker goes to work on that page. The worker receives a 200 response code (status OK) more than 98% of the time. If the page returns an HTTP code indicating a redirect (codes 301, 302, 303 and 307), further action must be taken by the worker or system in order to obtain information about that URL. A worker will follow up to 5 redirects from an initial URL before abandoning. Once the worker reaches an end point, the following tasks take place: Acquire HTTP return code from the URL. If a good response code is acquired: Identify title meta tag if available; calculate an MD5 fingerprint of the entire document (both HTML and text); parse HTML from the page; and write back first 1,000 k of text to disk.

Once the content is parsed and written back to the disk, a subsequent operator takes over. This operator makes several calculations used for the document fingerprint and writes those and other statistics back to the Oracle database. The system writes back the following fields to Oracle: URL_TABLE, DOMAINID, URLID, URL, LENGTH, SIGNATURE_H, SIGNATURE_T, SPIDER_DATE and HOST.

LENGTH is a count of characters in the text of the document (first 1,000 k). This feature can be used for de-duping URLs later in the process (408). SIGNATURE_H is the MD5 hash code signature. SIGNATURE_T is a CRC32 checksum code of the text (first 1000 k). SPIDER_DATE indicates the date and time that the particular URL was accessed. HOST pertains to which spider machine stored the text of the URL.

The following system may create three different measures designed to aid in document de-duplication (409). This de-duplication process aims at identifying documents that are identical or very similar within a given keyword result set. In a prior step not separately shown, URLs are de-duped at the URL level. Easily identified duplicates such as two occurrences of the exact same URL are eliminated. The system attempts to eliminate URLs that do not appear to point to the same page, but in fact do. In one embodiment, mathematical signatures (fingerprints) are taken for each URL and compared to other URLs within a given keyword result set. Three exemplary signatures are a length signature, an MD5 signature and a CRC32 checksum. Other signatures may be substituted.

For the length signature, the character length of the text document is calculated. This measure aids in the de-duping process to aid in giving context to a page which has been identified as a duplicate. For instance, if two sites show identical MD5 and CRC32 signatures, but have very disparate URLs, the signature is analyzed. If the length signature is low, meaning the page is small, it is likely that these two URLs share, for instance, a standard warning screen as would be found prior to entering an adult content site.

An MD5 signature typically is a way to verify data integrity. The MD5 algorithm takes as input a message of arbitrary length and produces as output a 128-bit “fingerprint” or “message digest” of the input. The MD5 algorithm is intended for digital signature applications, where a large file must be “compressed” in a secure manner. The system computes an MD5 signature for the entire document, reducing the identity comparison process to a 128-bit comparison, for instance.

A CRC32 checksum generates the cyclic redundancy checksum polynomial of 32-bit lengths. This is usually used to validate the integrity of data being transmitted. The CRC is a “digital fingerprint” of a file, With CRC32 you can “melt down” a huge 20 MB (or even much bigger) file to have a small, handy reference to it, a single 32-bit number like 7d9c42fb (hexadecimal notation) which reflects the entire contents of this huge file. The system computes a CRC32 signature of the text of the document, giving insights into the text content of the page.

Another signature that can be calculated and used is Rabin's fingerprinting algorithm, for instance Broeder's implementation, which produces a compact checksum.

Any of the checksums or fingerprints can be applied to the whole document, the whole document less HTML tags stripped away, the selected chunk of the document that is cached, the title and snippets or some other predetermined excerpt from the document. More or less than 1,000 k of the document can be used.

The process completes a run for a particular URL with data being written to an Oracle database and a spider box. The Oracle database receives fingerprint information (length, MD5, crc32), spider date/time, and host location information written to URL_TABLE and a spider date/time stamp written to KEYWORD table. The spider box receives files for data links that it followed: URL, title (if it was obtained during the initial fetch from the URL) and text of the document (first 1,000 k) to be used for snippet generation. The text contains elements of the meta description and the body of the document

Snippet Generation

A snippet generation process generates titles and snippets for display (407). The snippet process takes a keyword phrase and URL combination, comes up with the best title describing that URL, and creates the best snippet (i.e., abstract, description) for that URL outlining in a 200 character space the information contained in the URL that pertains to the keyword. Snippet generation follows the link following process. Snippets are created from the text of the document retrieved from the chosen URL.

The keyword “somec bicycles” produces the following sample text for display:

Upland Sports Group: Somec Bicycle Frames

... is a mystique to Italian bicycles; an aura born of precision, innovative

materials and artistry. Societa Mecchanica, better known

as Somec, has contributed to ...

www.uplandsg.com/somec/ - 12k - Aug 17, 2004 - Cached - Similar pages

In this example, the title is “Upland Sports . . . Frames”. The snippet is the two lines following the title. The URL is on the bottom line.

Titles are usually generated from the title of the page retrieved when a link is followed. Most sites annotate the title of the page for search engines through the use of HTML meta tags. A tag identifying the title is present on over 97% of all URLs.

In the 3% of URLs for which the HTML tags do not supply a title, the process composes a title. If there is text available for the URL, the process takes the first approximately 70 characters of text (respecting word boundaries) and creates a title. If there was no text generated from the URL, the domain name is stripped from the URL (all information between www and .com) and displayed as the title.

Snippet generation is a mix of art and science. The process creates snippets leveraging mathematical equations and linguistic science. In one embodiment, snippets can be comprised of 1 single sub-snippet, or up to 3 sub-snippets separated by ellipses (. . . ). A scoring algorithm decides which sub-snippets when combined (or not in the case of a single sub-snippet) produce the best score.

The snippet scoring algorithm is a multi-step process which scores various portions of the document's text. In four parts, it includes keyword tokenization, window scoring, window trading and final determination.

Keyword tokenization is applied because keywords are not always single words. Keywords are often multi-word phrases. The process tokenizes or single outs individual words within a phrase. Identifying individual word tokens typically includes searching for word separators such as spaces, periods, commas, or colon/semicolons. Once the tokenization of the keyword phrase is complete, the window scoring routine can commence.

In one version of window scoring, windows of three different sizes are calculated within the text of the document, for instance, for sub-snippet of lengths 200 characters, 100 characters and 66 characters. FIG. 20 depicts how the window scoring routine begins with the first word of text (2003) within the document(2001). In the first case, the window spans 200 characters. This window receives a score based upon several metrics. Once the score is calculated, the window moves 1 character to the right and calculates another score for the next window. Alternatively, the window may move one word to the right, aligning either the window start or window end with word boundaries. This process iterates (2004, 2005) until a score for the final window is obtained. Intelligence may be built into the algorithm to eliminate the need to compute the score for each entire window, but rather add the additional information to the previous window's score and subtract the information which was lost in the move.

When the process is complete, there may exist:

i windows of length 200 (where i=document length−200)

j windows of length 100 (where j=document length−100)

k windows of length 66 (where k=document length−66)

Window scoring may be based on one or more metrics, such as the number of unique tokens found within the window, the total number of tokens found within the window, and/or the longest string of tokens found within the window. A formula for each window is computed from a combination of these metrics and assigned to that window.

In the case where there is one 200 character snippet, the window with the highest score is chosen. The two highest scoring windows of length 100 are chosen for the two sub-snippet model. The three highest scoring windows of length 66 are chosen for the three sub-snippet model.

The best scores are calculated for each model (1, 2, or 3 sub-snippets). A final algorithm may be applied when 2 or 3 windows are eligible for a snippet. If the global window score can be increased by one window giving up characters to another, then that action is seen as a gain and it is taken. If the global window score cannot be raised in this manner, the snippets are used without trading.

The output from snippet generation may include 5 different scores: Score of single sub-snippet model; score of non-traded two sub-snippet model; score of traded two sub-snippet model; score of the non-traded three sub-snippet model; and/or score of the traded three sub-snippet model. Of these, the single highest score is chosen and that sub-snippet model is applied to that keyword, URL combination.

For a sample keyword=“red dog run”, the following steps may be followed:

Step 1: Tokenize keyword into three tokens:

Token1 = red

Token2 = dog

Token3 = run

Step 2a: Locate instances of the tokens within the text document

-------red----------dog--------dog---------

-----run-------red----------red-------------

-------red-dog-run---------dog-----------

---run----red-------run----run------------

Step 2b: Score the windows and identify the top ones. In this example, the three sub-snippet model, the best 3 windows were calculated.

-------red----------dog--------dog---------

[------run------red--]--------red-------------
(1)

---[----red-dog-run--]-------dog-----------
(2)

--run-----[-red-----run---]-run------------
(3)

Step 3: Allow for trading to occur. In this case, if window I can give up some of the non-token containing characters within it's left edge to window 3. This allows window 3 to expand and include the final token ‘run’, increasing the overall global score of the snippet.

-------red----------dog--------dog---------

------[run------red--]--------red-------------
(1)

---[----red-dog-run--]-------dog-----------
(2)

---run-----[-red-----run----run]------------
(3)

In an alternative embodiment, the snippet generation process may involve the creation of an approximately 200 byte field used as a descriptor for the associated link. Snippet generation takes place post spidering and is created from the complete text of the document associated with the chosen URL or at least the portion of the document stored.

Personalization (410) and Ranking

Within the ranking algorithm, there is the ability to select anonymous users who, based upon their behavioral profile, would have their votes for particular categories of links count more than other users.

Users who are heavy searchers (based upon their observed search behavior) would have their votes count more on links that they click more than the votes of novice searchers on that same link. In this way, the search experts would help produce more relevant ranking results.

Similarly, users who are highly involved in a particular category would have their votes count higher in that category than users who have no involvement in that category. Using behavior watching modules, one can identify users who are highly involved in various categories such as digital cameras, fantasy sports, or automobiles. For example, a user identified as being highly involved in the digital camera category would have his vote count more for links he clicked after a search for ‘Cannon G3’ than a user who is new to the category searching on that same keyword.

Identification of a user's category involvement status also drives personalization. A user with a high degree of involvement in a particular category would get different results from a user identified as less involved. This personalized results serving would require the presence of a cookie like object available on a particular machine. This lifestyle cookie would provide the search engine with a behavioral profile of the user, obtained from the users category navigational patterns. These category specific navigational patterns would be obtained from information contained in a categorization structure that also can be used for targeted advertising. For commercial purposes, a budget category or likely budget can be inferred from sites visited. Visitors to IKEA and Target are likely to have a different budget for apparel than visitors to Sachs Fifth Avenue or Bloomingdale's. Similarly, Hyatt Hotels are in a different budget category than youth hostels.

Personalization based on observed communications is much more powerful than user-entered customization, because research shows that only 8-14 percent of users manually personalized their content. Personalization highly correlates with pages viewed at a domain: users who personalize have been reported to view 130 percent more pages at the domain than users who do not personalize.

Sometimes different behavioral profiles can be leveraged to make a difference in search results (411). Other times, differences between two users' behavioral profiles does not help in the context of a particular search keyword.

Some examples are helpful. First, an ambiguous search terms example: A highly involved auto category user who searches for “jaguar” would get more Jaguar auto related links than jaguar animal related links as compared to a normal mix of auto and animal related links for someone with no identifiable category involvement. Identification and usage of these behaviorally profiles could slant results, without completely replacing results. In the example above, the auto category involved user could get 100% auto results, or just a larger percentage of auto results than found among popular websites.

Next, a sub-category identification example: Three users search for the keyword “rental, car”. Three separate sets of results come up, each personalized for the users. Each user has a particular behavioral profile obtained from their past navigational patterns observed within the travel category. These behaviors are readily identifiable from the observed communications.

User 1: Frequent business traveler—his rental car results would be slanted toward the business traveler car rental results, possibly more about frequent rental points, etc.

User 2: Budget traveler—his rental car results would be slanted toward the budget traveler; rent-a-wreck type results, specials on sub-compact cars etc.

User 3: Luxury Traveler—his rental car results would be slanted toward the high-end luxury traveler; sports car rentals, classic car rentals, etc.

FIG. 12 compares the insight that a behavior watching module on a user's machine gives regarding web surfing with the limited view that a server can achieve. These statistics were generated to compare what a Yahoo! server knows about user behavior, compared to what the behavior watching module sees. The x-axis 1202 identifies categories of web sites. The y-axis is a relative scale, with Yahoo! server observation scaled to “1”. In each stack, Yahoo! 1203 has a visibility of 1. The behavior watching module has a relative score 1204 that is higher and varies strongly between categories. This surprising degree of difference strongly distinguishes the on-machine behavior watching module from visibility that even a widely used search engine with many affiliated sites can achieve.

FIG. 13 depicts a web page highly relevant to the keyword “cars”. Applying a ranking algorithm, the CarsDirect home page scores 97 of 100. Factors used to calculate this score can be categorized by initial visit metrics, subsequent visit metrics and user level metrics. Observing users' response to this site being listed in search results of various search engines responsive to the keyword “cars”, the behavior watching module observed a click through rate of 12 percent. An average user viewed five pages at this web site, following the initial link, before they left the site. The average user spent 55 seconds looking at the pages and the conversion rate, here a registration more likely than a purchase, was 80 percent. On subsequent occasions, within a predetermined time after the search or the first visit, an average returnee viewed 15 pages and spent 300 seconds on the site. Returnees registered with a conversion rate by 15 percent. Among users following this link for this keyword, category (e.g., car category) involvement was high, ranked “Expert”. Search engine usage was “Maven”, indicating that common users were more frequent than computer experts. For this category, among U.S. users, the geographic location of the web site was important. Bet sites in the U.S. were more interesting to users (e.g., car buyers) than overseas web sites.

FIG. 14 is a counter example of a page weakly relevant to “cars”. The title of this article matches the keyword, but the click through rate and other metrics indicate that users considered this page much less relevant to searching for “cars” than the CarsDirect home page. The calculated relevancy score for this page is only 11.

FIG. 15 juxtaposes the results for “cars” from Yahoo!, MSN Search and Google with an embodiment of the search engine described herein. Using a relevancy index, this embodiment produces more relevant search results. The relevancy index is the ranking score, which reflects users' responses to links in search result lists. The relative scores are this embodiment 864, MSN 598, Google 524 and Yahoo! 487. This significantly improved score strongly distinguishes the use of an on-machine behavior watching module and the aggregate of a multitude of users from the ranking algorithms of the leading search engines.

FIG. 16 illustrates user differentiation using results from a behavior watching module. A large sample of user behavior, particularly shopping in the apparel category, is reported in this figure. Users were classified in nine boxes along attributes of amount of browsing in the category (1601) and degree of loyalty (1602) to a particular web domain (alternatively, to a vendor, brand or other category). Amount of browsing was classified as light, medium or heavy. One quarter of users were in each of the heavy and medium usage categories and the remaining half in the light usage category. Loyalty is classified as loyal, a switcher among sources or loyal to competitors. The 3-by-3 arrangement of attributes produces nine boxes. Among loyal shoppers, about one million unique observations were made. The heavy users had a higher click through rate when a particular result was displayed in response to a search, the rate being 7.90%, compared to 5.40% for medium users and just 3.00% for light users. After a click-through event, heavy users converted to purchase or registration at a rate of 8.00%, compared to 7.20% for medium users and 6.50% for light users. Not surprisingly, there are more switchers and competitive loyal unique observations than for the particular domain observed. Personalization of content, such as ads, may depend on which of the nine boxes the user fits. For instance, a loyal, heavy user will get a different message (a la frequent flier programs) than a medium to light switcher. A competitor-loyal heavy user might get a trial offer that would be completely misdirected to a loyal user. The strategy for advertising a heavy, loyal user is to insulate them from competitors. For medium to light loyal users, the strategy may target up-selling and cross-selling. For heavy to medium switchers, the objective is to increase share. Heavy to medium competitive loyal users are targeted for acquisition. Light switchers and light competitive loyal users may be ignored, to focus advertising resources.

Return visit data from the behavior watching module can assist an advertiser in measuring the effectiveness of a particular ad. User differentiation by box can further be associated with selection of ads and evaluation of ad effectiveness.

Cross-browsing of users also can be reported. Users can be selected by follow-through, for instance all click-throughs or all users with conversions. The users with a conversion at a particular domain (or vendor or brand, for instance) can be rated by the frequency of their visits to competitors' domains (or vendors or brands).

FIG. 17 depicts traffic to a particular domain, from a particular search engine, by keyword. This information may be useful to a company devising ads and also when bidding on keywords for sponsored content positioning on search engine result pages. Data from the behavior watching module can yield impressions, click, click-through rate, conversion rate and position on the search engine's results page.

Some Particular Embodiments

The present invention may be practiced as a method or device adapted to practice the method. The same method can be viewed from the perspective of a user at their terminal or personal computer or on the server side, collecting information from users. The invention may be an article of manufacture such as media impressed with logic to carry out computer-assisted method.

A device embodiment, from the user perspective, may be embodied in a module running on the user's computer and collecting behavioral observations, coupled to a server that responds to the behavioral observations with information personalized to the user.

While the present invention is disclosed by reference to the preferred embodiments and examples detailed above, it is understood that these examples are intended in an illustrative rather than in a limiting sense. Computer-assisted processing is implicated in the described embodiments. Accordingly, the present invention may be embodied in methods aggregating of communication patterns, pre-processing links responsive to keyword searches, responding to keyword searches using aggregated communication patterns to rank the responsive links, and responding to keyword searches using recent and current navigation information systems to resolve ambiguities and/or personalize responses based on user characteristics. Other embodiments, as devices, include logic and resources to carry out thes methods. As systems, still other embodiments include behavior watching modules on terminals, servers that process or respond to the behavioral data, or both. Other embodiments include media impressed with logic to carry out the methods, data streams impressed with logic to carry out the methods, or computer-accessible services that carry out the methods. It is contemplated that modifications and combinations will readily occur to those skilled in the art, which modifications and combinations will be within the spirit of the invention and the scope of the following claims.

A further embodiment is a method resolving ambiguity in a keyword search terms submitted by user to a search engine. This method includes receiving, from a behavior watching module operating on the user's computer, information regarding the user's recent navigation behavior. The navigation behavior includes accessing documents that are not all associated with a particular server-side behavior data collection network and further includes at least one of a mouse click-through, enter key stroke or other selection action and at least one keyword derived from context of the selection action. The server determines a plurality of interest categories to which the search term plausibly belongs and selects among the plurality of interest categories using the recent navigation information.

One aspect of the behavior watching modules operation is that it may categorize behaviors by interest categories and recency. The documents accessed may be web pages and the server-side behavior data collection network may be an advertising network.

From this perspective of the behavior watching module, this embodiment may be practiced as a method of supplementing a search request with information that reflects a user's recent navigation behavior among documents accessible via a network. This method includes operating a behavior watching module on the users computer, collecting information regarding the users recent navigation behavior. The recent navigation behavior information includes at least identifying documents electronically accessed by the user; categorizing the documents accessed; and indicating when documents in the categories were accessed. The behavior watching module submits at least a summary of the recent navigation behavior information along with a query to one or more servers including a search engine. The user receives from the search engine a set of results by asked to an interest category corresponding to the submitted recent navigation behavior information. Alternatively, the user may receive from the search engine a set of results by us to a budget category corresponding to the submitted recent navigation behavior information. More generally, any category that can be discerned from the recent navigation behavior can be reflected in a bias of search engine results received in response to the query.

Another embodiment is a method of personalizing content in response to a keyword search term submitted by a user to a search engine. As with the disambiguation method above, this method includes receiving, from a behavior watching module operating on the user's computer, information regarding the user's recent navigation behavior. The navigation behavior includes accessing documents that are not all associated with a particular server-side behavior data collection network and further includes at least one of a mouse click-through, enter key stroke or other selection action and at least one keyword derived from context of the selection action. The server determines a plurality of likely budget categories to which results belong and selects among the likely budget categories using the recent navigation information.

For embodiments using recent navigation behavior, the navigation behavior may reflect recent navigations, including navigations with in the last hour and the last day. Alternatively, it may include recent navigations within periods of time determined to correlate to user responsiveness to information. The user navigation information may be summarized into interest categories were likely budget categories when it is received or after it is received. More generally, any category that can be discerned from the recent navigation behavior can be reflected in a bias of search engine results received in response to the query.

The embodiments and various aspects of the embodiments described above may be practiced as a machine readable memory including instructions to carry out the methods and aspects of methods described or a data stream including the machine-readable instructions. Further, a device may include one or more servers, personal computers or other computer devices having logic and resources adapted to practice the methods and aspects of methods described.

Claims

1. A method of resolving ambiguity in a keyword search term submitted to a search engine, the method comprising: (A) receiving, from a client-side behavior watching module operating on a computer system, recent navigation information regarding recent navigation behavior on the computer system, said navigation information including behavioral data and at least one keyword;(B) checking the at least one keyword against a list of commercial terms;(C) based at least in part on said checking in (B), determining whether the at least one keyword is a commercial keyword;(D) if the at least one keyword is determined in (C) to be a commercial keyword:(d1) assigning the at least one keyword a keyword ID;(d2) generating a first data structure that includes the keyword ID, the behavioral data, and a domain ID that corresponds to a domain of a search location; and(d3) loading the first data structure into a first section of a data warehouse that is associated with commercial keywords;(E) if the at least one keyword is not determined in (C) to be a commercial keyword:(e1) assigning the at least one keyword an other ID;(e2) generating a second data structure that includes the other ID, the behavioral data, and a domain ID that corresponds to a domain of a search location; and(e3) loading the second data structure into a second section of a data warehouse that is associated with non-commercial keywords;(F) ranking links associated with the at least one keyword with results from client-side behavior watching modules on other computer systems; and(G) preferentially providing higher ranked commercial links in response to relevant search criteria.
2. The method of claim 1, wherein the recent navigation information is categorized by interest categories and by recency of behavior.
3. The method of claim 1, wherein the recent navigation information includes data regarding accessing documents that are not all associated with a particular server-side behavior data collection network.
4. The method of claim 3, wherein the documents are web pages and the server-side behavior data collection network is an advertising network.
5. The method of claim 1, wherein the keyword is derived from context of a selection action.
6. The method of claim 1, further comprising assigning the keyword another keyword ID if the keyword is determined to be noncommercial.
7. The method of claim 1, further comprising: identifying a conversion associated with an online purchase or registration made in response to an advertisement; andusing the conversion as a factor in ranking the links.
8. The method of claim 1, further comprising: determining that the keyword search term is submitted from a heavy searcher using the collected information; andweighting the results from the client-side behavior watching module more heavily when ranking links because the keyword search term is submitted from a heavy searcher.
9. The method of claim 1, further comprising: determining that the keyword search term is submitted among a high degree of involvement in a particular category using the collected information; andweighting the results from the client-side behavior watching module more heavily when ranking links associated with the particular category because the keyword search term is submitted among a high degree of involvement in the particular category.
10. The method of claim 1 wherein the client-side behavior watching module is browser independent.
11. The method of claim 1 wherein the client-side behavior watching module is separate from a web browser.
12. The method of claim 1 wherein the client-side behavior watching module runs independently of browser programs on the computer system.
13. A method of supplementing a search request with information that reflects recent navigation behavior among documents accessible via a network, the method comprising: (A) operating a client-side behavior watching module on a computer system to collect recent navigation behavior information regarding recent navigation behavior, wherein the recent navigation behavior information collected includes:(i) identification of documents electronically accessed;(ii) categorization of the documents electronically accessed into various categories;(iii) an indication of when documents in the various categories were electronically accessed; and(iv) data sufficient to identify a conversion from an online purchase or registration in response to an advertisement;(B) submitting at least a summary of the recent navigation behavior information along with a query including a keyword to one or more servers with a search engine, the summary including identification of the conversion in the data sufficient to identify a conversion; and(C) in response to said submitting in (B), receiving from the search engine a list of results biased to an interest category corresponding to the submitted recent navigation behavior information and ranked using results from multiple client-side behavior watching modules on other computer systems, wherein the list of results comprises higher ranked commercial links in response to relevant search criteria of the query if the keyword is associated with the conversion.
14. The method of claim 13, further including the behavior watching module filtering the documents accessed and categorizing the indication of when the categories were accessed.
15. The method of claim 13, wherein the keyword is derived from context of a selection action.
16. The method of claim 13 wherein the list of results is generated based, at least in part, on whether or not the keyword corresponds to a commercial keyword.
17. The method of claim 16 further comprising: determining whether or not the keyword corresponds to a commercial keyword.
18. The method of claim 13 wherein the client-side behavior watching module is browser independent.
19. The method of claim 13 wherein the client-side behavior watching module is separate from a web browser.
20. The method of claim 13 wherein the client-side behavior watching module runs independently of browser programs on the computer system.
21. A method of resolving ambiguity in a keyword search term submitted to a search engine, the method comprising: receiving, from a client-side behavior watching module operating on a computer system, recent navigation information, including a keyword and behavioral data, regarding recent navigation behavior on the computer system;checking the keyword against a list of commercial terms;determining whether the keyword is commercial;if the keyword is determined to be commercial:assigning the keyword a keyword ID;generating a first data structure that includes the keyword ID, the behavioral data, and a domain ID that corresponds to a domain of a search location; andloading the first data structure into a first section of a data warehouse that is associated with commercial keywords;if the keyword is determined to be non-commercial:assigning the keyword an other ID;generating a second data structure that includes the other ID, the behavioral data, and a domain ID that corresponds to a domain of a search location; andloading the second data structure into a second section of a data warehouse that is associated with non-commercial keywords;ranking links associated with the keyword with results from client-side behavior watching modules on other computer systems; andpreferentially providing higher ranked commercial links in response to relevant search criteria.
22. A computer-readable storage medium having stored thereon computer-executable instructions that configure a computing device on execution to: receive, from a client-side behavior watching module operating on a computer system, recent navigation information regarding recent navigation behavior on the computer system, said navigation information including behavioral data and a keyword;check the keyword against a list of commercial terms;based at least in part on the check, determine whether the keyword is a commercial keyword;if the keyword is determined to be a commercial keyword:assign the keyword a keyword ID; andgenerate a first data structure that includes the keyword ID, the behavioral data, and a domain ID that corresponds to a domain of a search location;rank links associated with the keyword with results of other navigation behavior; andpreferentially provide higher ranked commercial links in response to relevant search criteria.
23. A computer-readable storage medium having stored thereon computer-executable instructions that configure a computing device on execution to: collect recent navigation behavior information regarding recent navigation behavior, wherein the recent navigation behavior information collected includes:identification of documents electronically accessed;categorization of the documents electronically accessed into various categories;an indication of when documents in the various categories were electronically accessed; anddata sufficient to identify a conversion from an online purchase or registration in response to an advertisement;submit at least a summary of the recent navigation behavior information along with a query including a keyword to one or more servers with a search engine, the summary including identification of the conversion in the data sufficient to identify a conversion; andin response to said submitting, receive from the search engine a list of results biased to an interest category corresponding to the submitted recent navigation behavior information and ranked using results from multiple client-side behavior watching modules on other computer systems, wherein the list of results comprises higher ranked commercial links in response to relevant search criteria of the query if the keyword is associated with the conversion.

RELATED APPLICATIONS

This application claims the benefit of U.S. provisional Patent Application Nos. 60/603,140 entitled “Method and Apparatus for Responding to End-User Request for Information” by Westover et al. filed on Aug. 19, 2004; 60/637,684 entitled “Method and Device Publishing Cross-Network User Behavioral Data” by Wohlers et al. filed on Dec. 20, 2004; 60/662,680 entitled “Method and Device for Publishing Behavioral Observations to Customers” by Eagle et al. filed on Mar. 17, 2005; and 60/660,798 entitled “Method and Apparatus for Responding to End-User Requests for Information” by Westover et al. filed on Mar. 11, 2005. This application continues-in-part the disclosure of U.S. patent application Ser. No. 11/015,583 entitled “Search Engine for a Computer Network” by Anthony G. Martin filed on Dec. 17, 2004. These identified applications are incorporated by reference for all purposes.

US Referenced Citations (786)

Number	Name	Date	Kind
4455083	Elmes	Jun 1984	A
4575579	Simon et al.	Mar 1986	A
4719567	Whittington et al.	Jan 1988	A
4775935	Yourick	Oct 1988	A
4782449	Brinker et al.	Nov 1988	A
4799146	Chauvel	Jan 1989	A
4850007	Marino et al.	Jul 1989	A
4977594	Shear	Dec 1990	A
5027400	Baji et al.	Jun 1991	A
5029104	Dodson et al.	Jul 1991	A
5093718	Hoarty et al.	Mar 1992	A
5099420	Barlow et al.	Mar 1992	A
5105184	Pirani et al.	Apr 1992	A
5159669	Trigg et al.	Oct 1992	A
5165012	Crandall et al.	Nov 1992	A
5196838	Meier et al.	Mar 1993	A
5202961	Mills et al.	Apr 1993	A
5220420	Hoarty et al.	Jun 1993	A
5220516	Dodson et al.	Jun 1993	A
5220564	Tuch et al.	Jun 1993	A
5231499	Trytko	Jul 1993	A
5247517	Ross et al.	Sep 1993	A
5253341	Rozmanith et al.	Oct 1993	A
5268963	Monroe et al.	Dec 1993	A
5283639	Esch et al.	Feb 1994	A
5283731	Lalonde et al.	Feb 1994	A
5285442	Iwamura et al.	Feb 1994	A
5297249	Bernstein et al.	Mar 1994	A
5305195	Murphy	Apr 1994	A
5307456	MacKay	Apr 1994	A
5313455	van der Wal et al.	May 1994	A
5315580	Phaal	May 1994	A
5319455	Hoarty et al.	Jun 1994	A
5321740	Gregorek et al.	Jun 1994	A
5325423	Lewis	Jun 1994	A
5325483	Ise et al.	Jun 1994	A
5327554	Palazzi et al.	Jul 1994	A
5333237	Stefanopoulos et al.	Jul 1994	A
5347632	Filepp et al.	Sep 1994	A
5355472	Lewis	Oct 1994	A
5355501	Gross et al.	Oct 1994	A
5361091	Hoarty et al.	Nov 1994	A
5361199	Shoquist et al.	Nov 1994	A
5361393	Rossillo	Nov 1994	A
5367621	Cohen et al.	Nov 1994	A
5373375	Weldy	Dec 1994	A
5392447	Schlack et al.	Feb 1995	A
5412720	Hoarty	May 1995	A
5418549	Anderson et al.	May 1995	A
5438518	Bianco et al.	Aug 1995	A
5442771	Filepp et al.	Aug 1995	A
5446919	Wilkins	Aug 1995	A
5469553	Patrick	Nov 1995	A
5481741	McKaskle et al.	Jan 1996	A
5483466	Kawahara et al.	Jan 1996	A
5491785	Robson et al.	Feb 1996	A
5499340	Barritz	Mar 1996	A
5500890	Rogge et al.	Mar 1996	A
5515098	Carles	May 1996	A
5515270	Weinblatt	May 1996	A
5515490	Buchanan et al.	May 1996	A
5517612	Dwin et al.	May 1996	A
5524195	Clanton, III et al.	Jun 1996	A
5524197	Uya et al.	Jun 1996	A
5530472	Bregman et al.	Jun 1996	A
5530759	Braudaway et al.	Jun 1996	A
5530852	Meske et al.	Jun 1996	A
5532735	Blahut et al.	Jul 1996	A
5541986	Hou	Jul 1996	A
5544302	Nguyen	Aug 1996	A
5544320	Konrad	Aug 1996	A
5548745	Egan et al.	Aug 1996	A
5563804	Mortensen et al.	Oct 1996	A
5564043	Siefert	Oct 1996	A
5572643	Judson	Nov 1996	A
5579381	Courville et al.	Nov 1996	A
5583560	Florin et al.	Dec 1996	A
5583763	Atcheson et al.	Dec 1996	A
5584025	Keithley et al.	Dec 1996	A
5590046	Anderson et al.	Dec 1996	A
5594509	Florin et al.	Jan 1997	A
5594779	Goodman	Jan 1997	A
5596718	Boebert et al.	Jan 1997	A
5602905	Mettke	Feb 1997	A
5604542	Dedrick	Feb 1997	A
5606359	Youden et al.	Feb 1997	A
5608850	Robertson	Mar 1997	A
5615131	Mortensen et al.	Mar 1997	A
5615325	Peden	Mar 1997	A
5617526	Oran et al.	Apr 1997	A
5621456	Florin et al.	Apr 1997	A
5627886	Bowman	May 1997	A
5629978	Blumhardt et al.	May 1997	A
5630081	Rybicki et al.	May 1997	A
5635979	Kostreski et al.	Jun 1997	A
5638443	Stefik et al.	Jun 1997	A
5642484	Harrison, III et al.	Jun 1997	A
5649186	Ferguson	Jul 1997	A
5657450	Rao et al.	Aug 1997	A
5664948	Dimitriadis et al.	Sep 1997	A
5673322	Pepe et al.	Sep 1997	A
5675510	Coffey et al.	Oct 1997	A
5680562	Conrad et al.	Oct 1997	A
5682525	Bouve et al.	Oct 1997	A
5684715	Palmer	Nov 1997	A
5684969	Ishida	Nov 1997	A
5701451	Rogers et al.	Dec 1997	A
5704017	Heckerman et al.	Dec 1997	A
5706434	Kremen et al.	Jan 1998	A
5706502	Foley et al.	Jan 1998	A
5708709	Rose	Jan 1998	A
5708780	Levergood et al.	Jan 1998	A
5710884	Dedrick	Jan 1998	A
5710918	Lagarde et al.	Jan 1998	A
5712979	Graber et al.	Jan 1998	A
5715453	Stewart	Feb 1998	A
5717860	Graber et al.	Feb 1998	A
5717923	Dedrick	Feb 1998	A
5721827	Logan et al.	Feb 1998	A
5721908	Lagarde et al.	Feb 1998	A
5724521	Dedrick	Mar 1998	A
5724556	Souder et al.	Mar 1998	A
5724567	Rose et al.	Mar 1998	A
5727129	Barrett et al.	Mar 1998	A
5727159	Kikinis	Mar 1998	A
5734863	Kodosky et al.	Mar 1998	A
5737619	Judson	Apr 1998	A
5737739	Shirley et al.	Apr 1998	A
5740252	Minor et al.	Apr 1998	A
5740549	Reilly et al.	Apr 1998	A
5742768	Gennaro et al.	Apr 1998	A
5742905	Pepe et al.	Apr 1998	A
5745681	Levine et al.	Apr 1998	A
5751956	Kirsch	May 1998	A
5754830	Butts et al.	May 1998	A
5754938	Herz et al.	May 1998	A
5754939	Herz et al.	May 1998	A
5758111	Shiratori et al.	May 1998	A
5758257	Herz et al.	May 1998	A
5760767	Shore et al.	Jun 1998	A
5761601	Nemirofsky et al.	Jun 1998	A
5761662	Dasan	Jun 1998	A
5768508	Eikeland	Jun 1998	A
5768510	Gish	Jun 1998	A
5781894	Petrecca et al.	Jul 1998	A
5787253	McCreery et al.	Jul 1998	A
5787254	Maddalozzo, Jr. et al.	Jul 1998	A
5793972	Shane	Aug 1998	A
5794210	Goldhaber et al.	Aug 1998	A
5794259	Kikinis	Aug 1998	A
5796952	Davis et al.	Aug 1998	A
5801685	Miller et al.	Sep 1998	A
5802320	Baehr et al.	Sep 1998	A
5805735	Chen et al.	Sep 1998	A
5805815	Hill	Sep 1998	A
5809242	Shaw et al.	Sep 1998	A
5809481	Baron et al.	Sep 1998	A
5809512	Kato	Sep 1998	A
5812642	Leroy	Sep 1998	A
5812769	Graber et al.	Sep 1998	A
5819047	Bauer et al.	Oct 1998	A
5819092	Ferguson et al.	Oct 1998	A
5822526	Waskiewicz	Oct 1998	A
5832502	Durham et al.	Nov 1998	A
5835087	Herz et al.	Nov 1998	A
5835092	Boudreau et al.	Nov 1998	A
5835722	Bradshaw et al.	Nov 1998	A
5838458	Tsai	Nov 1998	A
5848246	Gish	Dec 1998	A
5848396	Gerace	Dec 1998	A
5848397	Marsh et al.	Dec 1998	A
5850352	Moezzi et al.	Dec 1998	A
5850433	Rondeau	Dec 1998	A
5854897	Radziewicz et al.	Dec 1998	A
5855008	Goldhaber et al.	Dec 1998	A
5861880	Shimizu et al.	Jan 1999	A
5861883	Cuomo et al.	Jan 1999	A
5862260	Rhoads	Jan 1999	A
5862325	Reed et al.	Jan 1999	A
5864823	Levitan	Jan 1999	A
5870769	Freund	Feb 1999	A
5872850	Klein et al.	Feb 1999	A
5875296	Shi et al.	Feb 1999	A
5877766	Bates et al.	Mar 1999	A
5878231	Baehr et al.	Mar 1999	A
5883955	Ronning	Mar 1999	A
5884025	Baehr et al.	Mar 1999	A
5886683	Tognazzini et al.	Mar 1999	A
5887133	Brown et al.	Mar 1999	A
5890152	Rapaport et al.	Mar 1999	A
5892917	Myerson	Apr 1999	A
5893053	Trueblood	Apr 1999	A
5893118	Sonderegger	Apr 1999	A
5894554	Lowery et al.	Apr 1999	A
5897622	Blinn et al.	Apr 1999	A
5898434	Small et al.	Apr 1999	A
5901287	Bull et al.	May 1999	A
5903892	Hoffert et al.	May 1999	A
5905492	Straub et al.	May 1999	A
5905800	Moskowitz et al.	May 1999	A
5907838	Miyasaka et al.	May 1999	A
5913040	Rakavy et al.	Jun 1999	A
5918012	Astiz et al.	Jun 1999	A
5918013	Mighdoll et al.	Jun 1999	A
5918014	Robinson	Jun 1999	A
5918214	Perkowski	Jun 1999	A
5920697	Masters et al.	Jul 1999	A
5923845	Kamiya et al.	Jul 1999	A
5923853	Danneels	Jul 1999	A
5929850	Broadwin et al.	Jul 1999	A
5930446	Kanda	Jul 1999	A
5930700	Pepper et al.	Jul 1999	A
5930801	Falkenhainer et al.	Jul 1999	A
5931901	Wolfe et al.	Aug 1999	A
5931907	Davies et al.	Aug 1999	A
5933811	Angles et al.	Aug 1999	A
5933832	Suzuoka et al.	Aug 1999	A
5936679	Kasahara et al.	Aug 1999	A
5937037	Kamel et al.	Aug 1999	A
5937390	Hyodo	Aug 1999	A
5937392	Alberts	Aug 1999	A
5937411	Becker	Aug 1999	A
5943478	Aggarwal et al.	Aug 1999	A
5944791	Scherpbier	Aug 1999	A
5946646	Schena et al.	Aug 1999	A
5946664	Ebisawa	Aug 1999	A
5946697	Shen	Aug 1999	A
5948061	Merriman et al.	Sep 1999	A
5951300	Brown	Sep 1999	A
5956693	Geerlings	Sep 1999	A
5956716	Kenner et al.	Sep 1999	A
5958008	Pogrebisky et al.	Sep 1999	A
5958015	Dascalu	Sep 1999	A
5959621	Nawaz et al.	Sep 1999	A
5959623	van Hoff et al.	Sep 1999	A
5960409	Wexler	Sep 1999	A
5961593	Gabber et al.	Oct 1999	A
5961602	Thompson et al.	Oct 1999	A
5961603	Kunkel et al.	Oct 1999	A
5963909	Warren et al.	Oct 1999	A
5963915	Kirsch	Oct 1999	A
5966121	Hubbell et al.	Oct 1999	A
5970473	Gerszber et al.	Oct 1999	A
5974219	Fujita et al.	Oct 1999	A
5974451	Simmons	Oct 1999	A
5978807	Mano et al.	Nov 1999	A
5978833	Pashley et al.	Nov 1999	A
5978836	Ouchi	Nov 1999	A
5978841	Berger	Nov 1999	A
5978842	Noble et al.	Nov 1999	A
5983227	Nazem et al.	Nov 1999	A
5983244	Nation	Nov 1999	A
5983268	Freivald et al.	Nov 1999	A
5987466	Greer et al.	Nov 1999	A
5987606	Cirasole et al.	Nov 1999	A
5991735	Gerace	Nov 1999	A
5991799	Yen et al.	Nov 1999	A
5995597	Woltz et al.	Nov 1999	A
5995943	Bull et al.	Nov 1999	A
5996007	Klug et al.	Nov 1999	A
5996011	Humes	Nov 1999	A
5999526	Garland et al.	Dec 1999	A
5999731	Yellin et al.	Dec 1999	A
5999740	Rowley	Dec 1999	A
5999912	Wodarz et al.	Dec 1999	A
6002401	Baker	Dec 1999	A
6006241	Purnaveja et al.	Dec 1999	A
6006252	Wolfe	Dec 1999	A
6006265	Rangan et al.	Dec 1999	A
6009236	Mishima et al.	Dec 1999	A
6009409	Adler et al.	Dec 1999	A
6009410	LeMole et al.	Dec 1999	A
6011537	Slotznick	Jan 2000	A
6012083	Savitzky et al.	Jan 2000	A
6014502	Moraes	Jan 2000	A
6014638	Burge et al.	Jan 2000	A
6014698	Griffiths	Jan 2000	A
6014711	Brown	Jan 2000	A
6016509	Dedrick	Jan 2000	A
6020884	MacNaughton et al.	Feb 2000	A
6023726	Saksena	Feb 2000	A
6025837	Matthews, III et al.	Feb 2000	A
6025886	Koda	Feb 2000	A
6026368	Brown et al.	Feb 2000	A
6026413	Challenger et al.	Feb 2000	A
6026433	D'Arlach et al.	Feb 2000	A
6026933	King et al.	Feb 2000	A
6029045	Picco et al.	Feb 2000	A
6029141	Bezos et al.	Feb 2000	A
6029145	Barritz	Feb 2000	A
6029182	Nehab et al.	Feb 2000	A
6029195	Herz	Feb 2000	A
6035332	Ingrassia, Jr. et al.	Mar 2000	A
6047318	Becker et al.	Apr 2000	A
6047327	Tso et al.	Apr 2000	A
6049821	Theriault et al.	Apr 2000	A
6052709	Paul	Apr 2000	A
6052717	Reynolds et al.	Apr 2000	A
6052730	Felciano et al.	Apr 2000	A
6055572	Saksena	Apr 2000	A
6058141	Barger et al.	May 2000	A
6061054	Jolly	May 2000	A
6061659	Murray	May 2000	A
6061716	Moncreiff	May 2000	A
6065024	Renshaw	May 2000	A
6065056	Bradshaw et al.	May 2000	A
6067559	Allard et al.	May 2000	A
6067561	Dillon	May 2000	A
6067565	Horvitz	May 2000	A
6070140	Tran	May 2000	A
6073105	Sutcliffe et al.	Jun 2000	A
6073167	Poulton et al.	Jun 2000	A
6073241	Rosenberg et al.	Jun 2000	A
6076166	Moshfeghi et al.	Jun 2000	A
6078916	Culliss	Jun 2000	A
6081840	Zhao	Jun 2000	A
6084581	Hunt	Jul 2000	A
6085193	Malkin et al.	Jul 2000	A
6085226	Horvitz	Jul 2000	A
6085242	Chandra	Jul 2000	A
6088731	Kiraly et al.	Jul 2000	A
6091411	Straub et al.	Jul 2000	A
6092196	Reiche	Jul 2000	A
6094655	Rogers et al.	Jul 2000	A
6094677	Capek et al.	Jul 2000	A
6098064	Pirolli et al.	Aug 2000	A
6098065	Skillen et al.	Aug 2000	A
6101510	Stone et al.	Aug 2000	A
6108637	Blumenau	Aug 2000	A
6108691	Lee et al.	Aug 2000	A
6108799	Boulay et al.	Aug 2000	A
6112215	Kaply	Aug 2000	A
6112246	Horbal et al.	Aug 2000	A
6115680	Coffee et al.	Sep 2000	A
6119098	Guyot et al.	Sep 2000	A
6119165	Li et al.	Sep 2000	A
6122632	Botts et al.	Sep 2000	A
6125388	Reisman	Sep 2000	A
6128655	Fields et al.	Oct 2000	A
6128663	Thomas	Oct 2000	A
6133912	Montero	Oct 2000	A
6133918	Conrad et al.	Oct 2000	A
6134380	Kushizaki	Oct 2000	A
6134532	Lazarus et al.	Oct 2000	A
6138146	Moon et al.	Oct 2000	A
6138155	Davis et al.	Oct 2000	A
6141010	Hoyle	Oct 2000	A
6144944	Kurtzman et al.	Nov 2000	A
6151596	Hosomi	Nov 2000	A
6154738	Call	Nov 2000	A
6154771	Rangan et al.	Nov 2000	A
6157924	Austin	Dec 2000	A
6157946	Itakura et al.	Dec 2000	A
6161112	Cragun et al.	Dec 2000	A
6163778	Fogg et al.	Dec 2000	A
6167438	Yates et al.	Dec 2000	A
6167453	Becker et al.	Dec 2000	A
6178443	Lin	Jan 2001	B1
6178461	Chan et al.	Jan 2001	B1
6182066	Marques	Jan 2001	B1
6182097	Hansen et al.	Jan 2001	B1
6182122	Berstis	Jan 2001	B1
6182133	Horvitz	Jan 2001	B1
6183366	Goldberg et al.	Feb 2001	B1
6185558	Bowman et al.	Feb 2001	B1
6185586	Judson	Feb 2001	B1
6185614	Cuomo et al.	Feb 2001	B1
6191782	Mori et al.	Feb 2001	B1
6192380	Light et al.	Feb 2001	B1
6195622	Altschuler et al.	Feb 2001	B1
6198906	Boetje et al.	Mar 2001	B1
6199079	Gupta et al.	Mar 2001	B1
6202093	Bolam et al.	Mar 2001	B1
6204840	Petelycky et al.	Mar 2001	B1
6208339	Atlas et al.	Mar 2001	B1
6216141	Straub et al.	Apr 2001	B1
6216212	Challenger et al.	Apr 2001	B1
6219676	Reiner	Apr 2001	B1
6222520	Gerszberg et al.	Apr 2001	B1
6223215	Hunt et al.	Apr 2001	B1
6233564	Schulze, Jr.	May 2001	B1
6237022	Bruck et al.	May 2001	B1
6249284	Bogdan	Jun 2001	B1
6253188	Witek et al.	Jun 2001	B1
6253208	Wittgreffe et al.	Jun 2001	B1
6266058	Meyer	Jul 2001	B1
6269361	Davis et al.	Jul 2001	B1
6275854	Himmel et al.	Aug 2001	B1
6279112	O'Toole, Jr. et al.	Aug 2001	B1
6280043	Ohkawa	Aug 2001	B1
6285987	Roth et al.	Sep 2001	B1
6286043	Cuomo et al.	Sep 2001	B1
6286045	Griffiths et al.	Sep 2001	B1
6295061	Park et al.	Sep 2001	B1
6297819	Furst	Oct 2001	B1
6304844	Pan et al.	Oct 2001	B1
6308202	Cohn et al.	Oct 2001	B1
6311194	Sheth et al.	Oct 2001	B1
6314451	Landsman et al.	Nov 2001	B1
6314457	Schema et al.	Nov 2001	B1
6317761	Landsman et al.	Nov 2001	B1
6321209	Pasquali	Nov 2001	B1
6321256	Himmel et al.	Nov 2001	B1
6324553	Cragun et al.	Nov 2001	B1
6324569	Ogilvie et al.	Nov 2001	B1
6324583	Stevens	Nov 2001	B1
6327574	Kramer et al.	Dec 2001	B1
6327617	Fawcett	Dec 2001	B1
6332127	Bandera	Dec 2001	B1
6334111	Carrott	Dec 2001	B1
6335963	Bosco	Jan 2002	B1
6336131	Wolfe	Jan 2002	B1
6338059	Fields et al.	Jan 2002	B1
6338066	Martin et al.	Jan 2002	B1
6341305	Wolfe	Jan 2002	B2
6347398	Parthasarathy et al.	Feb 2002	B1
6351279	Sawyer	Feb 2002	B1
6351745	Itakura et al.	Feb 2002	B1
6353834	Wong et al.	Mar 2002	B1
6356898	Cohen et al.	Mar 2002	B2
6356908	Brown et al.	Mar 2002	B1
6360221	Gough et al.	Mar 2002	B1
6366298	Haitsuka et al.	Apr 2002	B1
6370527	Singhal	Apr 2002	B1
6377983	Cohen et al.	Apr 2002	B1
6378075	Goldstein et al.	Apr 2002	B1
6381735	Hunt	Apr 2002	B1
6381742	Forbes et al.	Apr 2002	B2
6385592	Angles et al.	May 2002	B1
6392668	Murray	May 2002	B1
6393407	Middleton et al.	May 2002	B1
6393415	Getchius et al.	May 2002	B1
6397228	Lamburt et al.	May 2002	B1
6401075	Mason et al.	Jun 2002	B1
6415322	Jaye	Jul 2002	B1
6418440	Kuo et al.	Jul 2002	B1
6418471	Shelton et al.	Jul 2002	B1
6421675	Ryan et al.	Jul 2002	B1
6421724	Nickerson et al.	Jul 2002	B1
6421729	Paltenghe et al.	Jul 2002	B1
6434745	Conley, Jr. et al.	Aug 2002	B1
6437802	Kenny	Aug 2002	B1
6438215	Skladman et al.	Aug 2002	B1
6438578	Schmid et al.	Aug 2002	B1
6438579	Hosken	Aug 2002	B1
6441832	Tao et al.	Aug 2002	B1
6442529	Krishan et al.	Aug 2002	B1
6446128	Woods et al.	Sep 2002	B1
6449657	Stanbach, Jr. et al.	Sep 2002	B2
6452612	Holtz et al.	Sep 2002	B1
6457009	Bollay	Sep 2002	B1
6459440	Monnes et al.	Oct 2002	B1
6460036	Herz	Oct 2002	B1
6460042	Hitchcock et al.	Oct 2002	B1
6460060	Maddalozzo et al.	Oct 2002	B1
6466970	Lee et al.	Oct 2002	B1
6477550	Balasubramaniam et al.	Nov 2002	B1
6477575	Koeppel et al.	Nov 2002	B1
6480837	Dutta	Nov 2002	B1
6484148	Boyd	Nov 2002	B1
6487538	Gupta et al.	Nov 2002	B1
6490722	Barton et al.	Dec 2002	B1
6493702	Adar et al.	Dec 2002	B1
6496931	Rajchel et al.	Dec 2002	B1
6499052	Hoang et al.	Dec 2002	B1
6502076	Smith	Dec 2002	B1
6505201	Haitsuka	Jan 2003	B1
6513052	Binder	Jan 2003	B1
6513060	Nixon et al.	Jan 2003	B1
6516312	Kraft et al.	Feb 2003	B1
6523021	Monberg et al.	Feb 2003	B1
6526411	Ward	Feb 2003	B1
6529903	Smith et al.	Mar 2003	B2
6539375	Kawasaki et al.	Mar 2003	B2
6539424	Dutta	Mar 2003	B1
6564202	Schuetze et al.	May 2003	B1
6567850	Freishtat et al.	May 2003	B1
6567854	Olshansky et al.	May 2003	B1
6570595	Porter	May 2003	B2
6572662	Manohar et al.	Jun 2003	B2
6584479	Chang et al.	Jun 2003	B2
6584492	Cezar et al.	Jun 2003	B1
6584505	Howard et al.	Jun 2003	B1
6594654	Salam et al.	Jul 2003	B1
6601041	Brown et al.	Jul 2003	B1
6601057	Underwood et al.	Jul 2003	B1
6601100	Lee et al.	Jul 2003	B2
6604103	Wolfe	Aug 2003	B1
6606652	Cohn et al.	Aug 2003	B1
6615247	Murphy	Sep 2003	B1
6622168	Datta	Sep 2003	B1
6631360	Cook	Oct 2003	B1
6642944	Conrad et al.	Nov 2003	B2
6643696	Davis et al.	Nov 2003	B2
6665656	Carter	Dec 2003	B1
6665838	Brown et al.	Dec 2003	B1
6678731	Howard et al.	Jan 2004	B1
6678866	Sugimoto et al.	Jan 2004	B1
6681223	Sundaresan	Jan 2004	B1
6681247	Payton	Jan 2004	B1
6686931	Bodnar	Feb 2004	B1
6687737	Landsman et al.	Feb 2004	B2
6691106	Sathyanarayan	Feb 2004	B1
6697825	Underwood et al.	Feb 2004	B1
6701362	Subramonian	Mar 2004	B1
6701363	Chiu et al.	Mar 2004	B1
6714975	Aggarwal et al.	Mar 2004	B1
6718365	Dutta	Apr 2004	B1
6721741	Eyal et al.	Apr 2004	B1
6721795	Eldreth	Apr 2004	B1
6725269	Megiddo	Apr 2004	B1
6725303	Hoguta et al.	Apr 2004	B1
6741967	Wu et al.	May 2004	B1
6757661	Blaser et al.	Jun 2004	B1
6760746	Schneider	Jul 2004	B1
6760916	Holtz et al.	Jul 2004	B2
6763379	Shuster	Jul 2004	B1
6763386	Davis et al.	Jul 2004	B2
6771290	Hoyle	Aug 2004	B1
6772200	Bakshi et al.	Aug 2004	B1
6785659	Landsman et al.	Aug 2004	B1
6785723	Genty et al.	Aug 2004	B1
6801906	Bates et al.	Oct 2004	B1
6801909	Delgado et al.	Oct 2004	B2
6826534	Gupta et al.	Nov 2004	B1
6826546	Shuster	Nov 2004	B1
6827669	Cohen et al.	Dec 2004	B2
6847969	Mathai et al.	Jan 2005	B1
6848004	Chang et al.	Jan 2005	B1
6850967	Spencer et al.	Feb 2005	B1
6853982	Smith et al.	Feb 2005	B2
6857024	Chen et al.	Feb 2005	B1
6874018	Wu	Mar 2005	B2
6877027	Spencer et al.	Apr 2005	B1
6880123	Landsman	Apr 2005	B1
6882981	Philippe et al.	Apr 2005	B2
6892181	Megiddo et al.	May 2005	B1
6892223	Kawabata et al.	May 2005	B1
6892226	Tso et al.	May 2005	B1
6892354	Servan-Schreiber et al.	May 2005	B1
6904408	McCarthy et al.	Jun 2005	B1
6910179	Pennell et al.	Jun 2005	B1
6934736	Sears et al.	Aug 2005	B2
6938027	Barritz	Aug 2005	B1
6957390	Tamir et al.	Oct 2005	B2
6958759	Safadi et al.	Oct 2005	B2
6968507	Pennell et al.	Nov 2005	B2
6973478	Ketonen et al.	Dec 2005	B1
6976053	Tripp et al.	Dec 2005	B1
6976090	Ben-Shaul et al.	Dec 2005	B2
6990633	Miyasaka	Jan 2006	B1
6993532	Platt et al.	Jan 2006	B1
7003734	Gardner et al.	Feb 2006	B1
7016887	Stockfisch	Mar 2006	B2
7039599	Merriman et al.	May 2006	B2
7043526	Wolfe	May 2006	B1
7051084	Hayton et al.	May 2006	B1
7054900	Goldston	May 2006	B1
7065550	Raghunandan	Jun 2006	B2
7069515	Eagle et al.	Jun 2006	B1
7076546	Bates et al.	Jul 2006	B1
7085682	Heller et al.	Aug 2006	B1
7100111	McElfresh et al.	Aug 2006	B2
7133924	Rosenberg et al.	Nov 2006	B1
7136875	Anderson et al.	Nov 2006	B2
7149791	Sears et al.	Dec 2006	B2
7155729	Andrew et al.	Dec 2006	B1
7162739	Cowden et al.	Jan 2007	B2
7181415	Blaser et al.	Feb 2007	B2
7181488	Martin et al.	Feb 2007	B2
7194425	Nyhan	Mar 2007	B2
7254547	Beck et al.	Aug 2007	B1
7283992	Liu et al.	Oct 2007	B2
7346606	Bharat	Mar 2008	B2
7349827	Heller et al.	Mar 2008	B1
7363291	Page	Apr 2008	B1
7421432	Hoelzle et al.	Sep 2008	B1
7424708	Andersson et al.	Sep 2008	B2
7451065	Pednault et al.	Nov 2008	B2
7454364	Shkedi	Nov 2008	B2
7464155	Mousavi et al.	Dec 2008	B2
7512603	Veteska et al.	Mar 2009	B1
7630986	Herz	Dec 2009	B1
7743340	Horvitz et al.	Jun 2010	B2
7844488	Merriman et al.	Nov 2010	B2
20010011226	Greer et al.	Aug 2001	A1
20010029527	Goshen	Oct 2001	A1
20010030970	Wiryaman et al.	Oct 2001	A1
20010032115	Goldstein	Oct 2001	A1
20010037240	Marks et al.	Nov 2001	A1
20010037325	Biderman et al.	Nov 2001	A1
20010037488	Lee	Nov 2001	A1
20010044795	Cohen et al.	Nov 2001	A1
20010047354	Davis et al.	Nov 2001	A1
20010049320	Cohen et al.	Dec 2001	A1
20010049321	Cohen et al.	Dec 2001	A1
20010049620	Blasko	Dec 2001	A1
20010049716	Wolfe	Dec 2001	A1
20010051559	Cohen et al.	Dec 2001	A1
20010053735	Cohen et al.	Dec 2001	A1
20010054020	Barth et al.	Dec 2001	A1
20020002483	Siegel et al.	Jan 2002	A1
20020002538	Ling	Jan 2002	A1
20020004754	Gardenswartz	Jan 2002	A1
20020007307	Miller	Jan 2002	A1
20020007309	Reynar	Jan 2002	A1
20020007317	Callaghan et al.	Jan 2002	A1
20020008703	Merrill et al.	Jan 2002	A1
20020010626	Agmoni	Jan 2002	A1
20020010757	Granik et al.	Jan 2002	A1
20020010776	Lerner	Jan 2002	A1
20020016736	Cannon et al.	Feb 2002	A1
20020019763	Linden et al.	Feb 2002	A1
20020019834	Vilcauskas, Jr. et al.	Feb 2002	A1
20020023159	Vange et al.	Feb 2002	A1
20020026390	Ulenas et al.	Feb 2002	A1
20020032592	Krasnick et al.	Mar 2002	A1
20020035568	Benthin	Mar 2002	A1
20020038363	MacLean	Mar 2002	A1
20020040374	Kent	Apr 2002	A1
20020042750	Morrison	Apr 2002	A1
20020046099	Frengut et al.	Apr 2002	A1
20020049633	Pasquali	Apr 2002	A1
20020052785	Tenenbaum	May 2002	A1
20020052925	Kim et al.	May 2002	A1
20020053078	Holtz et al.	May 2002	A1
20020054089	Nicholas	May 2002	A1
20020055912	Buck	May 2002	A1
20020057285	Nicholas, III	May 2002	A1
20020059094	Hosea et al.	May 2002	A1
20020059099	Coletta	May 2002	A1
20020065802	Uchiyama	May 2002	A1
20020068500	Gabai et al.	Jun 2002	A1
20020069105	do Rosario Botelho et al.	Jun 2002	A1
20020073079	Terheggen	Jun 2002	A1
20020077219	Cohen et al.	Jun 2002	A1
20020078076	Evans	Jun 2002	A1
20020078192	Kopsell et al.	Jun 2002	A1
20020087499	Stockfisch	Jul 2002	A1
20020087621	Hendriks	Jul 2002	A1
20020091700	Steele et al.	Jul 2002	A1
20020091875	Fujiwara et al.	Jul 2002	A1
20020094868	Tuck et al.	Jul 2002	A1
20020099605	Weitzman et al.	Jul 2002	A1
20020099767	Cohen et al.	Jul 2002	A1
20020099812	Davis et al.	Jul 2002	A1
20020099824	Bender et al.	Jul 2002	A1
20020103811	Fankhauser et al.	Aug 2002	A1
20020107847	Johnson	Aug 2002	A1
20020107858	Lundahl et al.	Aug 2002	A1
20020111910	Walsh	Aug 2002	A1
20020111994	Raghunandan	Aug 2002	A1
20020112035	Carey et al.	Aug 2002	A1
20020112048	Gruyer et al.	Aug 2002	A1
20020116494	Kocol	Aug 2002	A1
20020120648	Ball et al.	Aug 2002	A1
02002122065	Segal et al.	Sep 2002
20020123912	Subramanian et al.	Sep 2002	A1
20020128904	Carruthers et al.	Sep 2002	A1
20020128908	Levin et al.	Sep 2002	A1
20020128925	Angeles	Sep 2002	A1
20020152121	Hiroshi	Oct 2002	A1
20020152126	Lieu et al.	Oct 2002	A1
20020152222	Holbrook	Oct 2002	A1
20020154163	Melchner	Oct 2002	A1
20020156781	Cordray et al.	Oct 2002	A1
20020156812	Krasnoiarov et al.	Oct 2002	A1
20020169670	Barsade et al.	Nov 2002	A1
20020169762	Cardona	Nov 2002	A1
20020170068	Rafey et al.	Nov 2002	A1
20020171682	Frank et al.	Nov 2002	A1
20020175947	Conrad et al.	Nov 2002	A1
20020194151	Fenton et al.	Dec 2002	A1
20020198778	Landsman et al.	Dec 2002	A1
20030004804	Landsman et al.	Jan 2003	A1
20030005000	Landsman et al.	Jan 2003	A1
20030005067	Martin et al.	Jan 2003	A1
20030005134	Martin et al.	Jan 2003	A1
20030011639	Webb	Jan 2003	A1
20030014304	Calvert et al.	Jan 2003	A1
20030014399	Hansen et al.	Jan 2003	A1
20030018778	Martin et al.	Jan 2003	A1
20030018885	Landsman et al.	Jan 2003	A1
20030023481	Calvert et al.	Jan 2003	A1
20030023488	Landsman et al.	Jan 2003	A1
20030023698	Dieberger et al.	Jan 2003	A1
20030028529	Cheung et al.	Feb 2003	A1
20030028565	Landsman et al.	Feb 2003	A1
20030028870	Weisman et al.	Feb 2003	A1
20030032409	Hutcheson et al.	Feb 2003	A1
20030033155	Peerson et al.	Feb 2003	A1
20030040958	Fernandes	Feb 2003	A1
20030041050	Smith et al.	Feb 2003	A1
20030046150	Ader et al.	Mar 2003	A1
20030050863	Radwin	Mar 2003	A1
20030052913	Barile	Mar 2003	A1
20030074448	Kinebuchi	Apr 2003	A1
20030088554	Ryan et al.	May 2003	A1
20030105589	Liu et al.	Jun 2003	A1
20030110079	Weisman et al.	Jun 2003	A1
20030110080	Tsutani et al.	Jun 2003	A1
20030115157	Circenis	Jun 2003	A1
20030120593	Bansal et al.	Jun 2003	A1
20030120654	Edlund et al.	Jun 2003	A1
20030131100	Godon et al.	Jul 2003	A1
20030135490	Barrett et al.	Jul 2003	A1
20030135853	Goldman et al.	Jul 2003	A1
20030154168	Lautenbacher	Aug 2003	A1
20030171990	Rao et al.	Sep 2003	A1
20030172075	Reisman	Sep 2003	A1
20030176931	Pednault et al.	Sep 2003	A1
20030182184	Strasnick et al.	Sep 2003	A1
20030195837	Kostic et al.	Oct 2003	A1
20030195877	Ford et al.	Oct 2003	A1
20030206720	Abecassis	Nov 2003	A1
20030208472	Pham	Nov 2003	A1
20030220091	Farrand et al.	Nov 2003	A1
20030221167	Goldstein et al.	Nov 2003	A1
20030229542	Morrisroe	Dec 2003	A1
20040002896	Alanen et al.	Jan 2004	A1
20040024756	Rickard	Feb 2004	A1
20040030798	Andersson et al.	Feb 2004	A1
20040044677	Huper-Graff et al.	Mar 2004	A1
20040068486	Chidlovskii	Apr 2004	A1
20040073485	Liu et al.	Apr 2004	A1
20040078294	Rollins et al.	Apr 2004	A1
20040095376	Graham et al.	May 2004	A1
20040098229	Error et al.	May 2004	A1
20040098449	Bar-Lavi et al.	May 2004	A1
20040117353	Ishag	Jun 2004	A1
20040133845	Forstall et al.	Jul 2004	A1
20040162738	Sanders et al.	Aug 2004	A1
20040162759	Willis	Aug 2004	A1
20040163101	Swix et al.	Aug 2004	A1
20040163107	Crystal	Aug 2004	A1
20040167928	Anderson et al.	Aug 2004	A1
20040181525	Itzhak et al.	Sep 2004	A1
20040181604	Immonen	Sep 2004	A1
20040193488	Khoo et al.	Sep 2004	A1
20040210533	Picker et al.	Oct 2004	A1
20040225716	Shamir et al.	Nov 2004	A1
20040247748	Bronkema	Dec 2004	A1
20040249709	Donovan et al.	Dec 2004	A1
20040249938	Bunch	Dec 2004	A1
20040254810	Yamaga et al.	Dec 2004	A1
20040267723	Bharat	Dec 2004	A1
20050015366	Carrasco et al.	Jan 2005	A1
20050021397	Cui et al.	Jan 2005	A1
20050027821	Alexander et al.	Feb 2005	A1
20050027822	Plaza	Feb 2005	A1
20050033657	Herrington et al.	Feb 2005	A1
20050038819	Hicken et al.	Feb 2005	A1
20050080772	Bem	Apr 2005	A1
20050086109	McFadden et al.	Apr 2005	A1
20050091106	Reller et al.	Apr 2005	A1
20050091111	Green et al.	Apr 2005	A1
20050097088	Bennett et al.	May 2005	A1
20050102202	Linden et al.	May 2005	A1
20050102282	Linden	May 2005	A1
20050125382	Karnawat et al.	Jun 2005	A1
20050131762	Bharat et al.	Jun 2005	A1
20050132267	Aviv	Jun 2005	A1
20050149404	Barnett et al.	Jul 2005	A1
20050155031	Wang et al.	Jul 2005	A1
20050182773	Feinsmith	Aug 2005	A1
20050187823	Howes	Aug 2005	A1
20050203796	Anand et al.	Sep 2005	A1
20050204148	Mayo	Sep 2005	A1
20050216572	Tso et al.	Sep 2005	A1
20050222901	Agarwal et al.	Oct 2005	A1
20050222982	Paczkowski et al.	Oct 2005	A1
20050240599	Sears	Oct 2005	A1
20050273463	Zohar et al.	Dec 2005	A1
20050283469	Veteska et al.	Dec 2005	A1
20050289120	Soulanille et al.	Dec 2005	A9
20060015390	Rijsinghani et al.	Jan 2006	A1
20060026233	Tenembaum et al.	Feb 2006	A1
20060031253	Newbold et al.	Feb 2006	A1
20060053230	Montero	Mar 2006	A1
20060136524	Wohlers et al.	Jun 2006	A1
20060136528	Martin et al.	Jun 2006	A1
20060136728	Gentry et al.	Jun 2006	A1
20060235965	Bennett et al.	Oct 2006	A1
20060253432	Eagle et al.	Nov 2006	A1
20070016469	Bae et al.	Jan 2007	A1
20070038956	Morris	Feb 2007	A1

Foreign Referenced Citations (43)

Number	Date	Country
0631231	Dec 1994	EP
0822535	Feb 1998	EP
1045547	Oct 2000	EP
1154611	Nov 2001	EP
1 207 468	May 2002	EP
1207468	May 2002	EP
343825	Feb 1991	JP
11066099	Mar 1999	JP
2001084256	Mar 2001	JP
2001147894	May 2001	JP
20010222535	Aug 2001	JP
2001312482	Nov 2001	JP
2002024221	Jan 2002	JP
2002032401	Jan 2002	JP
2002073545	Mar 2002	JP
2002092284	Mar 2002	JP
2002524782	Aug 2002	JP
2002259371	Sep 2002	JP
2002334104	Nov 2002	JP
2002334104	Nov 2002	JP
2003058572	Feb 2003	JP
2003141155	May 2003	JP
2003178092	Jun 2003	JP
2003242411	Aug 2003	JP
20030271647	Sep 2003	JP
2004-355376	Dec 2004	JP
WO 9847090	Oct 1998	WO
WO 9938321	Jul 1999	WO
WO 9944159	Sep 1999	WO
WO 9946701	Sep 1999	WO
WO 9955066	Oct 1999	WO
WO 9959097	Nov 1999	WO
WO 0004434	Jan 2000	WO
WO 0054201	Sep 2000	WO
WO 0103028	Jan 2001	WO
WO 0115052	Mar 2001	WO
WO 0139024	May 2001	WO
WO 0144992	Jun 2001	WO
WO 0163472	Aug 2001	WO
WO 0169929	Sep 2001	WO
WO 0190917	Nov 2001	WO
WO 0244869	Jun 2002	WO
WO 03010685	Feb 2003	WO

Related Publications (1)

	Number	Date	Country
	20060041550 A1	Feb 2006	US

Provisional Applications (4)

Number	Date	Country
60603140	Aug 2004	US
60637684	Dec 2004	US
60660798	Mar 2005	US
60662680	Mar 2005	US

Continuation in Parts (1)

	Number	Date	Country
Parent	11015583	Dec 2004	US
Child	11207590		US

Method and apparatus for responding to request for information-personalization

Information

Patent Number

Date Filed

Date Issued

Inventors

Original Assignees

Examiners

Agents

CPC

US Classifications

Field of Search

US

International Classifications

Disclaimer

Term Extension

Abstract