Patent Application
20090063250
The present invention is related to the field of advertising and is more particularly directed toward controlled targeted experimentation.
The Internet provides a mechanism for merchants to offer a vast amount of products and services to consumers. Internet portals provide users an entrance and guide into the vast resources of the Internet. Typically, an Internet portal provides a range of search, email, news, shopping, chat, maps, finance, entertainment, and other Internet services and content. Yahoo, the assignee of the present invention, is an example of such an Internet portal.
When a user visits certain locations on the Internet (e.g. web sites), including an Internet portal, a system can capture the user's online activity. This information may be recorded and, analyzed to determine patterns and interests of the user. In turn, these patterns and interests may be used to target the user to provide a more meaningful and rich experience. For example, if interests in certain products and services of the user are determined, content and advertisements, pertaining to those products and services, may be served to the user. Advertisements are usually provided by advertisers or marketers, who research and develop campaigns for the market. Content is typically provided by a network of publishers, often in conjunction with a portal provider. A system that serves well targeted advertisements benefits both the advertiser/marketer, who provides a message to a target audience, and a user who receives advertisements in areas of interest to the user. Similarly, publishers and portals are benefited by increased relevance and/or traffic. In this document the a publisher will include publisher web sites and Internet Portals.
Currently, advertising through computer networks such as the Internet is widely used along with advertising through other mediums, such as television, radio, or print. In particular, online advertising through the Internet provides a mechanism for merchants to otter advertisements for a vast amount of products and services to online users. In terms of marketing strategy, different online advertisements have different objectives depending on the user toward whom an advertisement is targeted.
Often, an advertiser will carry out an advertising campaign where a series of one or more advertisements are continually distributed over the Internet over a predetermined period of time. Advertisements in an advertising campaign are typically branding advertisements but may also include direct response or purchasing advertisements.
A method of advertising selects an attribute value for a test. The attribute value is for representing a relationship between a user activity and one or more of a user segment, a publisher group, and an ad category. The method optionally constructs a test case that includes the attribute value and one or more of: the user segment, the publisher group, and the ad category. The method selectively places the test case in all inventory location of the publisher group, presents the test case to the user segment, and monitors the status of the attribute value based on user activity.
The method of some embodiments tracks a confidence metric for measuring the reliability of the attribute value. If, for instance, the confidence metric is below a predetermined threshold, then the method performs the test. Alternatively, or in conjunction with the foregoing, the method applies a set of rules for determining the importance of the test. If the importance of the test according to the set of rules is low, then the method advantageously forgoes the test.
The attribute value typically includes at least one of a propensity, an affinity, and a rate, relative to one or more user activities comprising: impressions, clicks, leads, and acquisitions. In some cases, for example, a first attribute value includes a propensity to click and a second attribute value comprises a click rate. If the recorded attribute value is near an initial value for the attribute such as a calculated average value, for example, then the method increases the confidence metric for the attribute. If the recorded attribute value varies significantly from the average value for the attribute, then the method decreases the confidence metric for the attribute.
In some embodiments, selecting a test case includes making a combination of users, publishers, marketers, and/or advertisements. For instance, the combination includes at least two of a user segment, a publisher group, and an ad category. Preferably, the method receives data from a network that includes at least one of user data, publisher data, and marketer data. Each of the user data, publisher data, and marketer data often includes historic and/or current data. The method of a particular implementation optimizes advertising across a first portal site and a publisher network by targeting a first advertisement to a first user in a first context. The first context involves a number of features such as time and/or a specific page, or inventory within a page. The first advertisement has a determined relevance to at least one of the first context and the first user. The ad category of one implementation includes an ad campaign. In this implementation, the method advantageously determines ad campaign performance metrics for one or more of: the user segment, the publisher category, and one or more types of advertisements within the ad campaign.
Hence, the methods, of different embodiments optimize a combination for at least two of an advertisement, a user segment, a context, a web page, and a time. Optimizing the combination in some cases matches a first advertisement to a first user segment by using the attribute value when the confidence metric for the attribute value exceeds the predetermined threshold. The optimizing alternatively includes matching the first advertisement to a first publisher site by using the attribute value when the confidence metric for the attribute value exceeds the predetermined threshold. Preferably, users are segmented based on user behavior, demographics and/or location, which are stored in a user profile, including information that is intrinsic to a user segment. Similarly, publishers and pages having inventory are preferably grouped based on intrinsic content within the pages, while advertisements are categorized, such as based on an ad campaign, for example.
Some embodiments identify a set of attributes that relate the ad categories to at least one of: the user segments and the publisher groups, determine a set of one or more values for each attribute that indicate the strength of a relationship, and generate a hierarchical structure for the attributes. The hierarchical structure preferably has clusters of increased relevance. For instance, the hierarchical structure advantageously organizes the clusters into rows, and normalizes the values of an attribute. Accordingly, the organization of data advantageously reduces input data points for a matching problem.
Particular embodiments recommend a creative to a marketer, recommend an inventory purchase to a marketer, and/or expand inventory for a particular ad campaign by proposing an alternative advertisement to a marketer. These embodiments achieve the foregoing based on an attribute value relating the alternative advertisement to at least one of a user segment, a publisher category, and/or a time of day. Hence, various embodiments improve marketer advertising, and/or improve ad targeting thereby increasing revenue relating to a user activity comprising one or more of impressions, clicks, leads, and acquisitions.
Some tests determine, for example, the relevance of an ad category to a predetermined user segment, the performance of an ad placement for a predetermined publisher group, or an ad placement for a particular inventory location, and/or a presentation of an ad at a predetermined time. These embodiments advantageously increase a relevance to a user, thereby improving user experience. These embodiments may perform one or more of the following: select an advertisement that has a relevance to the user segment for the user, place an advertisement at a high priority location within a page and/or within inventory, place an advertisement on a publisher group that is relevant to the user, and/or present an advertisement to the user at a predetermined time that has a determined importance to the user. Generally, these embodiments provide better ad targeting, use inventory more effectively, and further increase demand for inventory.
An alternative method of targeting receives data from a network of resources. The data includes user data, publisher data; and marketer data. Each of the user data, publisher data, and marketer data may include historic and current data. The method uses the received data for optimizing advertising across a first portal site and a publisher network by targeting a first advertisement to a first user in a first context, the first context comprising a time and a page. The method, selectively performs a test. The test comprises selecting a test case that includes an attribute value and one or more of: a user segment, a publisher group, and an ad category. The method places the test case in an inventory location of the publisher group, presents the test case to a first user segment, and monitors the status of the attribute value associated with the confidence metric.
The method of some embodiments receives a confidence metric for the accuracy of the attribute value. If the confidence metric exceeds a predetermined thresholds then the method forgoes the test, and if the confidence metric is below: the predetermined threshold, then the method performs the test. Alternatively, or in conjunction with the forgoing, some embodiments apply a set of rules for determining the importance of the test case. For instance, if the importance of the test case is low according to the set of rules, then the method advantageously forgoes the test. In this example, the set of rules describe a set of undesirable test cases. In some implementations, the test case includes presenting a first advertisement to the first user segment. The first user segment has a commonality with a second user segment. The second user segment has a known relationship with the first advertisement. The test case of some implementations includes placing a first advertisement within a first inventory location. The first inventory location has a commonality with a second inventory location. Advantageously, the second inventory location has a known relationship with the first advertisement.
The method uses the test to determine a performance for the first advertisement and a first inventory location. The performance is determined by the monitoring of the attribute value. In some instances, behavioral targeting, demographic targeting and/or geographic targeting are used for determining whether a first advertisement performs for the user segment, and for selectively matching the first advertisement to the user segment. Content matching is optionally used for selectively matching a first advertisement to a publisher group. Alternatively, search matching is used for determining the performance of a search keyword in relation to a first advertisement.
A system for targeting includes a user module, a publisher module, a marketer or ad module, and a matching engine. The user module is for receiving a plurality of users and segmenting the users into user segments such as a first user segment. The publisher module is for receiving a plurality of publishers' inventory and grouping the publishers' inventory into publisher groups such as a first publisher group. The first publisher group has a first inventory location for the presentation of advertising. The marketer-ad module is for receiving advertisements and categorizing the advertisements into ad categories such as a first ad category. The matching engine is for matching the first ad category to one or more of the first publisher group and the first user segment. The matching engine also places within the first inventory location a first advertisement from the first ad category. The system further includes a set of desirable combinations and a set of undesirable combinations. The system is configured to distinguish the set of desirable combinations from the set of undesirable combinations, and forgo the set of undesirable combinations.
The matching engine is configured to perform one or more of: user behavioral match; user demographic match; user geographic match; domain match; and content match. Some systems include a confidence column, for tracking a confidence in the reliability and/or accuracy of an attribute value. The test usually includes an element having unknown data. For instance, the element includes one or more of an advertisement, an inventory location, user data, and/or a demographic. For instance, the demographic includes one of a geographic location, a time zone, a time of day, and a day of week. The test further optionally includes one or more of: a new advertisement, a new inventory location, a new user, and a new demographic, for which little data has been collected or is known.
The novel features of the invention are set forth in the appended claims. However, for purpose of explanation, several embodiments of the invention are set forth in the following figures.
In the following description, numerous details are set forth for purpose of explanation. However, one of ordinary skill in the art will realize that the invention may be practiced, without the use of these specific details. In other instances, well-known structures and devices are shown in block diagram form in order not to obscure the description of the invention with unnecessary detail.
In direct response advertising, the overall goal is to elicit an action or response from a customer. In some instances targeting provides user profile data for direct response advertising. For example, an advertisement displayed on a web page that includes a link for the user to “click” is an example of a direct response advertisement. The last, and most focused part of the funnel, is the customer cycle of purchase intention. In this stage, the user is actively shopping, and intends to make a purchase.
Accordingly, a company or business unit can have a variety of marketing or target objectives along a continuum of activities directed toward conversion. For example, a company may desire to attract users to a web site. Data processing that is performed for targeting may generate user profiles to acquire users to the publisher. For example, users from one business unit of the publisher or one zone of the publisher's site may be targeted to acquire users in a different business unit area or zone based on the user's profile. Similarly, the user profiles may be generated for the target objective of engaging users to visit the web site more frequently. Furthermore, the processing may generate user profiles to retain users that have previously visited the web site.
The marketing goals and objectives elicit user profile data to facilitate advertisers and marketers. Some of the targeting systems disclosed herein also have application to provide user profile information to facilitate a user's online experience. In general, the goal of these systems is to match the right content, including ads, to the right user at the right time (i.e., stimulus).
A. Ad Targeting
Embodiments of the invention include a method of targeting an appropriate advertisement to a particular user. Preferably, the advertisement is presented to the user in the appropriate context, and preferably, at an optimized time. Historical data are advantageously used for the targeting and optimization, which improves the advertising provided by a publisher, such as Yahoo.com; and other publisher networks. Hence, particular implementations target the right advertising (what), to the right user (who), in the right context (where), at the right time (when), by optimizing an advertisement-user-context-time combination.
Particular implementations use extensive user, publisher, and marketer data that are preferably compiled from a network of resources. Some embodiments collect the data in an initial phase, while some embodiments receive data that are compiled or precompiled from external sources. These data generally include both historic and current data. For a marketer, the data include advertisement purchase information such as placement, targeting, cost, advertisement performance and conversion information. The data are preferably from multiple forms of advertising, including graphical type advertisements, precision match type ads, content match type ads, and/or domain match type advertisements. For a publisher, the data include a variety of content and an array of sites, domains, pages, zones and ad inventory. For a user, the data include: user profile information such as demographic, geographic, behavioral, device used, and other user type records and information. The various collected data from marketers, publishers, and/or users forms a massive targeting and/or matching problem.
Preferably, a second phase reduces the massive targeting and/or matching problem size. Particular implementations employ a form of categorization for one or more of the user base, the publisher network, and/or one or more marketers. For each category, some of these embodiments identify attributes and data values for the attributes. Hence, these embodiments further generate a hierarchical grouping of attributes.
For instance, users are advantageously categorized based on their intrinsic information, and/or on their behaviors. Typically, this information is collected at various times for storage and/or retrieval from a user profile. The profiles are often complex and include a unique identification for organization and management.
Publishers are preferably categorized based on content, such as the intrinsic content presented by web pages managed by the publisher.
Advertisements are preferably categorized based on type and/or content, on a marketer and/or on an ad campaign. In these embodiments, marketers generally develop an ad campaign, that includes one or multiple advertisements for certain brands and/or products. Hence, multiple advertisements may relate to an ad campaign, and similarly, several ad campaigns may relate to one advertisement.
As mentioned above, the attributes and data values for the categories are preferably organized into hierarchical clusters. Further, associations are determined for each of the categories across the user, publisher, and/or marketer data. Some implementations construct a framework having rows to represent the associations. A particular implementation specifies the strength of the association with data values. These data values can be normalized to a scale from 0 to 100, or alternatively, from 0.0 to 1.0, or to another normalization scale.
At a third phase, embodiments of the invention use optimization algorithms to determine campaign performance metrics for each advertisement and category of advertisement with each category of user, and publisher. Advantageously, the performance of the advertisements and/or ad campaigns for the users and publishers is measured. The matching of ad categories to user segments and publisher groups improves campaign performance for marketers, improves traffic for publishers, and also, improves user experience. An optional implementation further recommends optimized inventory purchases, to marketers, based on the information collection and processing described above. Similarly, implementations expand inventory for particular campaigns by proposing reasonable alternatives to existing campaigns, or by directing new campaigns. Ultimately, these various optimizations and improvements further increase revenue for a publisher, the marketer and ad network that collects, manages, and processes the various data, through its sites and partners. The improvements include, for example, better ad targeting, more effective use of inventory, higher demand for inventory, which, in turn, increases inventory price.
For the marketer, advertising and spend is optimized in a number of ways. For example, embodiments provide better ad targeting and placement, more efficient purchase of inventory, and information that allows recommendations regarding creatives. Publishers also benefit from better ad targeting, more traffic, and/or revenue from non owned/operated advertising. Further, user satisfaction is improved by increasing site relevance and usefulness to the user.
B. Additional Ad Targeting Test, Experimentation, and/or Verification
Some embodiments include a method of controlled ad targeted experimentation and verification to determine and/or improve the effectiveness of advertising, and further, optimize advertising across publisher sites, zones, domains and networks. Similar to the embodiments described above, these embodiments improve ad targeting by using combinations of user, context, and/or time values. However, these embodiments perform additional steps to gather advertising performance data. For example, some implementations systematically and continuously analyze existing and new combinations to determine and/or update the efficacy and confidence in the ad-user-context optimization, thereby ensuring the quality of the recommendations.
In these cases, one or more marketers, publishers, and/or users are categorized, in a first stage. Each of these groups of categories can also be indexed in a hierarchical framework that preferably includes several data values associated with each category. At a second stage, useful combinations of each of the groups including marketers, publishers, and/or users, are identified. A statistical relevance of past performance data is calculated for each marketer, publisher, and/or user category combination. Some embodiments further calculate a margin of error.
These embodiments identify, by using a third stage, the category combinations that have no performance data or performance data with high margins of error. Some embodiments further employ a system of rules to decide combinations that have greater significance or value, to the system. Then, in a fourth stage, the identified category combinations are tested to improve the quality and breadth of the performance data.
Alternatively, test combinations are selected based on other criteria. For instance, new creatives, hard-to-find inventory, different targeting options, and other combinations are selected, for which performance data are more particularly gathered. These cases use the performance data to empirically determine the conversion efficacy of various combinations. Further, as mentioned above, the performance data of these test cases are further used in conjunction with optimization algorithms to match advertisements, to users and to publishers, which improves both campaign performance and user experience. Marketers are further benefited by receiving recommendations for inventory purchases, or by suggestions to expand inventory for particular campaigns. Ultimately, quality and value are optimized for marketers, while traffic and revenue are optimized for the publisher web pages.
The selection and/or presentation of advertising through the inventory is; a non trivial process. The inventory are typically distributed across many varied sites, zones, domains and pages. There are many different users and types of users, and marketers, advertisements, and ad campaigns are usually numerous and varied as well. Timely, relevant, appropriate and/or coherent matching and delivery of content such as advertising is a problem that can have millions of input data points, or more.
Hence,
More specifically,
For instance, the user base is divided into user segments based on data for each user within the user base. The data for each user are typically stored within a user profile that has a unique reference identifier, and a variety of information about the user such as demographic, geographic, behavioral, and other user information, for example. For instance, one user segment might include females, about 30 years of age, with above average income, located near or within a metropolitan area, that showman affinity for cats, and who may be shopping for cars such as by frequenting car buying sites. As evidenced above, the user segmentation of some embodiments has variable granularity, from broad demographic divisions, to more specific finer distinctions between users.
Publishers usually provide sites, zones, domains and/or pages having content and inventory. The system 500 organizes the publishers and the available inventory, by using the content of each site or page. Examples of publisher sites and/or content include news sites, and car buying sites. As mentioned above, in relation to, the user segments, the groupings of some embodiments are optionally broad and coarse grained, or are alternatively fine grained. For instance, the groupings for car buying includes a group for sites that provide guidance or advice regarding buying cars, in addition to a group for performing car purchase and acquisition, and sites for the purchase of specific car accessories.
Similarly, the system 500 organizes the data for marketers in various ways. For instance, advertisements and/or campaigns are categorized based on type or nature of the advertisement. Examples of advertisements and/or campaigns include advertisements for cat food, or alternatively ads for cars, or for car accessories. Generally, the more detailed, finer grained, and/or closely tuned the ad categories, the publisher groups, and the user segments, the better the targeting performance of the system. However, finer grain data processing generally comes with a tradeoff cost, as the number of data points increases. Some embodiments, balance the performance of the system with the costs of fine grain data processing.
In
The user base 601 includes user segment 1, which has a user who is logged or tracked by a unique identifier UserID1. Also shown in
The relevance and/or propensity of the advertisement is preferably calculated for several and/or all combinations of ad category, publisher group, and user segment. For instance, as illustrated by example in
Preferably, the relationships are determined and/or the associations are generated by using an offline or batch process. Some embodiments use an extensive set of data in the form of web activity logs that are obtained from user activity in relation to a set of publisher sites, zones, domains and pages.
The embodiments described above typically operate by using imperfect or incomplete data. Stated differently, some calculations and/or populated values are determined by using more input data, while other calculations are undesirably based on less input data. Hence, the reliability and/or accuracy vary for the values expressing the different relationships. Accordingly, some embodiments identify relationship values having lower confidence, and test the underlying relationship to adjust the relationship value and thereby adjust the confidence in the determined relationship and its calculated value. In
In the embodiments where test cases are performed, each of the test cases generally has a cost in terms, of lost revenue, as the performance of the ad campaign will be suboptimal. In these cases, data are gathered for the system. Hence, embodiments of the invention only selectively assemble and perform certain test cases, to minimize the cost of this type of system optimization. Some embodiments further apply a set of rules to decide whether to test existing combinations and/or novel combinations, in this type of system value verification and/or optimization. For instance, certain combinations are known and/or predetermined to have lesser value to marketers, publishers, and/or users. Such combinations are generally expected to produce little result, and/or a well established result that generally does not warrant the use of system resources.
One example is presenting a user segment that has an age range below 15 years of age in the state of California with an advertisement and/or campaign that is designed to elicit an acquisition response for luxury automobiles. It is typically suboptimal to employ system resources to select and place ads of this nature, to this user segment, within any publisher group context. Moreover, it is undesirable to test cases such as these. Accordingly, some embodiments employ rules to forego inefficient combinations and/or testing. These rules are optionally complex. For instance, instead it may be highly useful to test the efficacy of presenting 16 year olds an advertisement that is designed to elicit an impression regarding a particular car manufacturer. The campaign, advertisement, or creative may be further cobranded with a particular consumption good such as a soft drink, or another product, that has immediate synergy for conversion in either of the brands or categories.
Once data are collected at the step 1302, the process 1300 transitions to the step 1304, where users are segmented. The users are preferably segmented from a user base into user segments by using the data collection described above. Then, at the step 1306, the process 1300 groups the publishers. The groupings are typically based on the publisher content aid/or inventory description. At the step 1308, advertisements are categorized. The categorizations is based optionally on the nature or type of advertising, on the marketer, and/or on the campaign. Some embodiments create the segments, groups and categories concurrently or iteratively. After or while user segmentation, publisher grouping, and/or advertisement categorization are preformed, the process 1300, at the step 1310, selects an attribute to monitor and/or analyze. The attribute preferably involves one or more conversion activities such as those described above in relation to
Then, the process 1300 transitions to the step 1314, where a determination is made whether to continue, such as part of batch and/or a real time process, for example. If the determination is made to continue, then the process 1300 returns to the step 1302. Otherwise, the process 1300 concludes.
Once a selection or recommendation is determined at the step 1402, the process 1400 transitions to the step 1404 where the selected or recommended advertisement is placed. Preferably, the placement is within the inventory of the publisher site, near an optimized location for the user's attention. Some embodiments place one or more advertisements from a particular category or campaign within the publisher site's inventory in real time, as the user navigates the publisher's site pages. Some embodiments place advertisements based on timing such as time of day, independently and/or in conjunction with demographic, geographic, and other data.
When the selected advertisement is placed, the process 1400 transitions to the step 1406, where the placement and advertisement is presented to the user. The advertisement is typically in the form of a creative designed to elicit a response from the user, such as a conversion type activity. Some embodiments further monitor or record the user's activities and/or response to the presentation at the step 1406. These collected data may be incorporated into further targeting determinations and calculations, or may be included in the user's profile information. Then, the process 1400 concludes. In some other embodiments placement and presentation of the ad occur at the same time.
After the rule(s) are applied at the step 1502, the process 11500 transitions to the step 1504, where a confidence level is received. The confidence level is an indicator of the reliability of a value measuring a relationship within a system. Preferably, the relationship value measures, a propensity of a user from a particular user segment and within a particular publisher context to perform a desired action for a particular marketer or ad campaign category. The action preferably involves conversion type activities such as, impressions, clicks, leads, and/or acquisitions, for example. The relationship value further optionally includes affinity measures, and other collected data concerning the user, a publisher, and the campaign(s) of a marketer. Stated differently, the relationship value indicates the probability of an event based on these input data.
At the step 1506, it is determined whether the confidence level is below a predetermined threshold. If at the step 1506, the confidence is above the threshold, then the process 1500 transitions to the step 1510, and forgoes the testing at the step 1508, which is deemed of lesser benefit.
If the confidence level is lower than the threshold, then the process 1500 transitions to the step 1508, where a test is performed. The test checks the value of the relationship such as the propensity. The test is performed a variety of different ways, and for different numbers of iterations depending on the underlying relationship being verified. For instance, where the relationship activity being tested, comprises impressions, the test may include iterations in the range of 10,000 to 20,000 impressions. When the activity comprises clicks, the testing may involve 2,000 to 3,000 clicks. Data collected during the testing and/or iterations are advantageously used. For instance, if the propensity values of each test or iteration do not change significantly during the testing, then the confidence in this value and combination in the system increases. If the propensity values do change significantly during the testing, then the confidence in these values decreases. Additional steps are then optionally performed such as, for example, to reduce use by the system of the unreliable combination, removal or modification of the combination, and/or additional testing of the combination and its associated data values.
After the testing is performed, the process 1500 transitions to the step 1510, where a determination is made whether to continue. If the process 1500 should continue, then the process 1500 returns to the step 1502. Otherwise, the process 1500 concludes.
Once the test case is selected, the process 1600 transitions to the step 1604, where the test case is assembled and/or placed at a location within the inventory for a particular user segment. Since inventory is used for the purpose of data collection, rather than optimal revenue generation/conversion, some embodiments use care in the selection and/or placement of the test case. Further, some embodiments use a set of rules to filter less important test cases and/or identify more important cases.
After the placement, the process 1600 transitions to the step 1606, where the selected test case is presented for a particular publisher group and/or for a particular user segment. As mentioned above, the selection and placement are tuned accordingly, and optionally include such factors as demographic, geographic, behavioral, time, or other data. Then, the process 1600 transitions to the step 1608, where attribute value(s) are recorded for the test case. Such values include, for example, impressions, clicks, leads, acquisitions, rates, propensities, and/or other behaviors, activities and metrics. The data are advantageously stored and/or used to determine the reliability and performance of the test case, such as the efficacy of the combination, for example. After the data are collected, the process transitions to the step 1610, where a determination is made whether to continue. If the process 1600 should repeat, then the process 1600 returns to the step 1602. Otherwise, the process 1600 concludes.
Embodiments of the invention select and/or place advertisements within particular inventory locations. The selection is preferably based on an advantageous combination of the advertisement, a user, a contexts, and/or a time. The selected combination improves improves marketer's advertising, improves publisher's (non owned and operated) advertising, improves advertiser network revenue and/or improves a user satisfaction or experience when visiting a publisher site. For instance, the selected combination preferably achieves better ad targeting, which results in more effective use of inventory such as, for example, by better advertisement placement and/or more efficient purchase of inventory. Improved targeting further increases demand for inventory and/or inventory price.
In a specific example, “revenue-order” banner advertisements are advantageously improved. In the search context, revenue ordering is performed to optimize revenue per search. Banner ad displays are preferably targeted and/or optimized for targeting, and especially for direct marketing ads. In these implementations, advertisements are categorized along a number of dimensions. The “best possible” banner ads are selected and presented for each user browse and/or context. Once performance of advertisements improves, the price charged for advertising increases.
Some embodiments improve campaigns for marketers such as by recommending creatives for campaigns. Particular implementations optimize advertising campaigns for marketers who may wish to place advertisements on publisher sites across the Internet. These implementations are advantageously used for many different kinds of advertising such as search, banner, CPA/CPL, and other types of advertisements. Generally, more revenue is generated from the improvements.
A. Data
Preferably, extensive data are used from a network or resources. Hence the data include large amounts of user, publisher, and/or marketer data. These large input data sets are typically difficult to manage. However, preferred embodiments reduce the massive numbers of inputs into organized data sets. These embodiments further apply optimization algorithms to determine and/or select particularly efficacious combinations from among the data sets, including user, publisher, and/or marketer data sets. Further some embodiments optionally use iterative testing and/or verification techniques to improve the selected combinations and ensure the confidence in their efficacy.
The data advantageously include historic and/or current data, for a variety of events, and for one or more of the marketer, publisher, and/or user data sets. Preferably, the data are for all forms of advertising including graphical ads, precision match, content match, domain match, and other advertising.
For the marketer and/or publisher, the historic and current data often include advertisement performance and purchase information such as placement, targeting, and cost, for example. Marketers typically have fine-grained campaign goals. To address these goals, embodiments of the invention initially use high-level categories such as branding and direct response categories to identify sets of company, industry, and/or sub-industry categories. Some embodiments might use Dun and Bradstreet codes and/or SIC codes, in the identification of categories. Categories are further advantageously based on creative types, or advertisement types, or other attributes. Creative types include text advertisements, banner ads, video ads, and/or landing page type advertisements. Creative attributes include ad size, primary image, and other attributes.
Publisher data preferably includes contextual attributes such as, for example, the web site context or nature, keywords searched, content of the web page, position and/or placement of ads, within the content and/or within the context.
User data includes demographic type attributes, geographic location information, behavioral attributes of a users or a group of users, and/or system attributes. Demographic attributes might include age, gender, and tenure, while system attributes might include browser information, and connection speed, and geographic attributes involve address targeting, by using IP, MAC, or another type of addressing, and user behavioral attributes involves user interests, categories, affinities, and other behavioral data.
B. Reduce Problem Size
Some of the implementations described above collect a large amount of granular data relating to many different attributes for each type of data set. Advantageously, embodiments of the inventions aggregate these data into increasingly manageable sets, and further organize the data within a framework having rows of data values. In a particular step, hierarchical groupings of attributes are identified. One implementation segments users based on the user data, groups publishers based on site, zone, domain or page content, and/or categorizes advertisements based on advertisement, marketer, and/or ad campaign. Allis implementation then organizes the different data sets into hierarchical clusters of rows, calculates a set of values associated with each set, and identifies particularly useful combinations of data sets.
C. Optimization
Once the data are organized into sets and preferably into a framework for identifying selected combinations, some embodiments determine campaign performance metrics for each type of advertisement for each user segment and/or for each publisher group. Moreover, these embodiments advantageously match advertisements within the marketer data set to particular users and/or publishers. The matching is optimized for efficacy to a particular result such as, for example, impressions, clicks, leads, acquisitions, ad/campaign/marketer performance, and/or another conversion activity. Accordingly, implementations of the invention preferably increase revenue in relation to costs and provide other benefits for marketers, and/or publishers, while providing improved user experience. More specifically, particularly beneficial inventory purchases are recommended to marketers, and/or suggestions based on empirical data are provided to expand inventory for particular campaigns.
D. Additional Optimization, Testing, and Verification
Some embodiments continuously run carefully selected, test cases to gather data systematically. These embodiments ensure the efficacy and confidence in the selected combinations, which assists the optimization system in making recommendations regarding targeting, delivery, placement, timing, selecting creatives, and other recommendations. Test cases are preferably designed to sample new creatives, hard-to-find inventory, different targeting options, and the like, to empirically measure the performance of these test cases.
These and other implementations provide additional features such as marketer analytics in the form of cost per lead or cost per action metrics, recommendations for advertising sales, and for sales operations. The marketer analytics and metrics are advantageously used to establish benchmark practices and generate industry reports such as for the on-line advertising industry. As described above, some implementation use a phased approach. In a first phase, existing or historical data are analyzed to obtain insights. In a second phase, a system or framework is constructed by using the historical data. In a third phase, selective determinations are performed by using specific test cases. Moreover, implementations optionally perform error checking by systematically comparing recommendations made by the optimizations described above with external recommendations such as those made by experts. This verification further improves the performance of the system and/or framework.
While the invention has been described with reference to numerous specific details, one of ordinary skill in the art will recognize that the invention can be embodied in other specific forms without departing from the spirit of the invention. For instance, the examples given above often relate to online media. However, targeting across a multiple of media types is applicable as well. Thus, one of ordinary skill in the art would understand that the invention is not to be limited by the foregoing illustrative details, but rather is to be defined by the appended claims.
