Patent Application
20100250332
The present invention is directed towards management of on-line advertising contracts based on adaptive bidding.
The marketing of products and services online over the Internet through advertisements is big business. Advertising over the Internet often seeks to reach individuals within a target group having very specific demographics (e.g. male, age 40-48, graduate of Stanford, living in California or New York, etc). This targeting of very specific demographics is in significant contrast to print and television advertisement that is generally capable only of reaching an audience within some broad, general demographics (e.g. living in the vicinity of Los Angeles, or living in the vicinity of New York City, etc.). The single appearance of an advertisement on a web page is known as an online advertisement impression. Each request of a web page by a user via the Internet represents an impression opportunity to display an advertisement in some portion of the web page to the individual Internet user. Often, there may be significant competition among advertisers for a particular impression opportunity for delivering an advertisement to an individual Internet user.
To participate in this competition, some advertisers enter auctions in order to win some desired number of impressions over a desired time period or to bid as often and as much as necessary so as to meet or optimize some objective (e.g. lowest average cost per impression) subject to some constraints (e.g. exhaust the budget by the end of the campaign period).
In traditional print or mass media advertising, such an optimization might be trivial. For example, to reach 2,000,000 impressions, an advertiser might advertise for some specific period (e.g. four weekly issues) in some popular periodical (e.g. the Midtown edition of the New Yorker Magazine). The price of the placements might be deemed fair (or even optimal) by the advertiser. In contrast to traditional print or mass media advertising, an Internet advertisement is able to specify detailed targeting criteria. For example, an advertiser may desire to spend $15,000 over the month of June for impressions made to males living in Midtown New York City who are over 30 years of age and have an interest in sports. This situation then precipitates an avalanche of questions: Does the advertiser guess at a fair price for the spots, and be satisfied with the first M impressions that exhaust the budget? Over what time frame would those M impressions occur? Will there be days when there are no impressions at all? Are some impressions that are priced above or below the advertiser's guess at a fair price actually more effective at achieving other of the advertiser's campaign objectives? If the guess were guessed at a higher price, would there be enough forecasted impression inventory of the higher-priced impression opportunities in order to exhaust the budget within the campaign period?
Even if the advertiser's guess were correct (i.e. equal to a winning bid on a spot market) at some moment in time, the guess might be wrong (i.e. too high) at another moment in time, resulting in possible over-spending. Or the guess might be too low, possibly resulting in failure to achieve the campaign objectives for a minimum number of impressions.
Especially when bidding might be done in a very narrow market (i.e. where the target specification is very narrow), an initial guess might be wildly wrong. Thus, what is needed is a system that automatically bids to learn a fair price for an impression opportunity, and then adapts bids over time in order to optimize an advertising campaign's objective.
Other automated features and advantages of the present invention will be apparent from the accompanying drawings and from the detailed description that follows below.
A system for performing adaptive bidding to secure Internet advertising impressions in an auction. Included are systems for analyzing advertising campaign objectives, including a campaign period, a target number of impressions, a target budget. An exemplary technique defines a bidding agent for performing the adaptive bidding seeking the minimum target spending of the budget. Objective results of the campaign such as average cost per won impression, total campaign duration relative to desired campaign period, and total expenditure relative to campaign budget can be optimized based on an empirically determined forecast. Techniques for adapting bids include statistically modeling winning bids during an exploration bidding phase, performing iterations for adjustment of the bid amounts using learn-while-bid adaptive bidding, learn-then-bid adaptive bidding, and guess-double-adjust adaptive bidding. Bidding tactics, especially those used to identify a minimum target bid and a maximum target bid used in bidding on an exchange are employed, and the results analyzed using statistical models.
A brief description of the drawings follows:
In the following description, numerous details are set forth for purpose of explanation. However, one of ordinary skill in the art will realize that the invention may be practiced without the use of these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to not obscure the description of the invention with unnecessary detail.
In the context of Internet advertising, bidding for placement of advertisements within an Internet environment (e.g. system 100 of
In the slightly more sophisticated model of
As earlier indicated, the supply of an impression opportunity that meets some target criteria, (e.g. males living in Midtown New York City who are over 30 years of age and have an interest in sports) might be known in the statistical sense, but the actual impression opportunity existence generally is not known until the Internet user actually requests a particular web page that could contain an advertisement. Only then can the demographic specifics of the target be known (e.g. the Internet user is a male living in Midtown New York City who is over 30 years of age and has an interest in sports), and thus the bidding can begin. Of course the Internet user is then waiting to receive the requested web page, so the auction begins and ends quickly—a latency of only a matter of fractions of a second are acceptable to most Internet users. Accordingly bidding agents (bots or other agent entities that prosecute an online advertising campaign) are employed to bid on behalf of the advertiser.
In addition to a target demographic description, a bidding agent might be provided with additional information that characterizes an advertising campaign (e.g. a period of time that the campaign should run, total budget, expected number of impressions, etc.). As well, the bidding agent might be provided with objective functions and constraints such as “minimize cost per impression while running the campaign through the end of the period.” Of course exchanges also may support bidding by other than a bot (e.g. by a natural person) although, for latency reasons, this is reasonable only for pricing blocks of advertisements or for pre-arranged or guaranteed delivery contracts. In temporally relaxed situations the bidding agent could either be an advertiser's employee on behalf of the advertiser, or an intermediary agent acting on behalf of the advertiser.
Now, following a typical bidding scenario, the bidding agent is tasked to acquire a given number of impressions with a given target budget (e.g. advertising spend), when the highest external bid in the marketplace is drawn from an unknown distribution P. The number of impressions/advertisements and spend are usually provided as givens within the campaign description due to the historical fact that display ads are usually sold on a cost per impression (CPM) basis. These given values might be used as constraints in the set-up of an optimization problem.
It should be noted that the act of bidding itself provides information used to adapt future bids. The act of bidding on a display advertising exchange at least partially answers the question, “What price are others willing to pay for an impression”. That is, before the first bid is won, the distribution of winning bids is unknown. However, repeated bidding, and recording and correlating those results yields information as to the market value of an impression of some particular characteristics. And knowing the distribution allows the system to calculate the minimum target spend to achieve the desired number of impressions. Similarly, a maximum target spend to achieve the desired number of impressions can also be known to a statistical certainty. Of course, although the advertiser may specify a target budget, the advertiser desires to achieve the other campaign objectives while spending the minimum amount of the budget. Embodiments of the invention herein seek to optimize (i.e. minimize the actual spend) while achieving other campaign objectives.
Now, in terms of defining and solving the aforementioned optimization problem, one must consider the optimization problem in at least two settings: (1) the transparent market where the winning price in each auction is announced, and (2) the partially observable market where only the winner obtains information about the winning bid. Repeating, the bidding agent is tasked to acquire a given number of impressions with a given target budget, when the highest external bid in the marketplace is drawn from an unknown distribution P. It should be axiomatic that if the distribution of winning bids were known or knowable a priori, then the optimization problem is a very simple optimization problem indeed. Still, if at least some statistical measurements could be taken in order to fairly gauge the distribution over time, then the optimization problem could be solved, at least within some statistical confidence level. Thus it is an aspect of the embodiments of the invention herein to take statistical measurements of the distribution of bids in both the case of a transparent market and the case of a partially observable market.
The methods employable in each of these two cases differ. In particular, the penalty incurred by the agent while attempting to learn the distribution is generally higher in a partially observable market than in a transparent market. The intuition around this is that in the partially observable case, a winning bid must be placed, (i.e. a portion of the budget actually spent) in order to collect observations of points in the distribution. Embodiments herein disclose techniques useful for both settings. Further, the performance of such techniques can be proven to fall within statistical bounds. Thus, an advertiser may have confidence that using the techniques herein to carry out adaptive bidding throughout an advertising campaign will achieve results that very nearly optimize at least one campaign objective under the constraints (e.g. achieve the lowest average cost per impression while achieving the target number of impressions, exhaust the budget while spending through the end of the campaign period, acquire as many impressions as possible within the campaign period, spend substantially all of the budget allocated within the campaign period).
Various systems, each implementing unique approaches corresponding to the transparent and partially observable markets are disclosed herein. Under simulation, the disclosed systems perform very well with respect to both target delivered impression quantity and achieving that within the campaign budget. Further, the systems for the partially observable case perform nearly as well as the systems for the fully observable case despite the higher penalty incurred during learning.
A cost per impression-based (CPM-based) campaign typically defines a target quantity of impressions that needs to be acquired over a certain time period with certain targeting characteristics. The targeting can include a wide variety of demographics, possibly including details about the Internet user, and possibly the Internet user's recent history, possibly even including information both about the web page on which the ad will appear, and even as well as the user's recently viewed web pages. In addition to the target impression quantity, a typical constraint in a CPM-based campaign is a budget constraint on the total spend. The problem statement includes the constraint that the bidding agent should exhaust (rather than simply stay within), the allocated budget. The intuition corresponding to this constraint is that advertisers prefer to exhaust their budgets allocated to a campaign. Having postulated this, one might recognize that not all impressions are regarded by advertisers as equally valuable, even though the target description might be identical. That is, the marketplace has built within it pieces of information (i.e. beyond merely the target demographic description) regarding how valuable a particular impression might be, and high bids for an impression might reflect this information; thus a high price in a marketplace might indicate high value. Accordingly some of the systems disclosed here assume that an agent implementing a campaign on behalf of an advertiser would also want to deliver high-value impressions subject to the chosen budget. Alternatively, in cases where the bidding tactics favor achievement of the number of impressions over achievement of average cost per impression when they are not simultaneously feasible, the agent can deliver the lower cost impressions if desired.
Consider a bidding agent that needs to win d impressions and has a total budget T. Assume that the bidding agent is at any point in time capable of retrieving a forecast of the total supply n of impressions that occur within the remaining campaign period and also satisfies the targeting requirements. While such a perfectly accurate forecast may not ever be available, a lower bound (e.g. a conservative forecast) of the supply can be used. Define f to be a fraction of the supply that the agent needs to win; thus, f=d/n. Define t=T/d to be target spend per impression won. Suppose that the highest bids from other bidders are drawn as independent and identically distributed (aka “i.i.d.”) from a distribution with a cumulative distribution function (CDF) P, and that each impression is sold using a second price auction. In general, a given target quantity and spend need not be simultaneously feasible for the distribution P. However, if it happens that a given target quantity and spend are simultaneously feasible, the agent can choose in favor of either the target quantity or in favor the target spend (depending on the favor preference as indicated by the advertiser's campaign description).
If the distribution P is known to the bidding agent, then the problem is simple to describe and simple to solve. Let z*=P−1(t) be the bid that would win fraction f of the supply. Define p* such that EP[X|X≦p*]=t, that is, p* is the bid that achieves the expected target spend t. If z*≦p* then bidding p* (with probability A=f/P(p*)) independently on each available impression achieves both of the expected supply and spend targets. Otherwise, favoring achievement of the fraction of the supply, a bid of z* achieves the expected desired fraction of supply.
In practice of bidding on impression opportunities, the existence of which specific opportunities are not known a priori except in the statistical, forecasting sense, the distribution P or estimation of the distribution P is also not known a priori to the bidding agent prior to the time the first bids are made. The problem is therefore one of learning (i.e. by bidding) the unknown distribution P in order to meet the target quantity and spend constraints. However, learning incurs a penalty, leading to an explore-exploit tradeoff. More specifically, a longer learning period would tend to result in a higher confidence level of the correct bidding strategy, however a longer learning period (the ‘explore’ period) during a campaign of finite duration implies a shorter time remaining in which to secure the desired number of forecasted impressions (the ‘exploitation’ period). And in fact a shorter exploitation period would tend to increase the fraction of the forecasted supply required to meet the campaign objectives as a later entry into the exploitation phase would imply there had been some impression opportunities counted in d that had occurred (during the learning phase), never to occur again, and thus effectively reducing the d available in the remainder of the campaign.
As regards any sort of learning or estimation of the distribution P by collecting observations, after collecting empirically observed samples to determine a distribution function, the “Dvoretzky-Kiefer-Wolfowitz inequality” (DKW inequality) might be used to aid in predicting how close the empirically determined distribution function will be to the distribution from which the empirical samples are drawn. In particular, the DKW inequality is used to bound the error in estimates as to the performance of the system. The nature and extent of the penalty depends in part on the assumptions made about the extent of information available to the bidding agent. Consider again the two marketplace settings:
As an overview, below are disclosed possible tactics for agent bidding: The Learn-Then. Bid tactic, the Learn-While-Bid tactic and the Guess-Double-Adjust tactic.
The Learn-Then-Bid tactic is applicable to at least the above-described fully observable exchange. The disclosure models performance guarantees using non-asymptotic uniform convergence results for empirical processes. In simple terms, the Learn-Then-Bid tactic observes (without placing bids, since this is a fully observable exchange) and then bids according to the observed (empirical) distribution.
The Learn-While-Bid tactic is a natural improvement to the Learn-Then-Bid tactic in that Learn-While-Bid continues to learn during the exploitation phase.
The Guess-Double-Adjust tactic is especially useful in the partially observable case. As earlier indicated, the partially observable case is more constrained since a cost must be paid for every bid that is observed from the distribution. The Guess-Double-Adjust system explores the distribution gradually based on a “guess-then-double” pattern in order to control expenditure of budget during the learning phase. As will be shown, a simple guess-then-double algorithm (that is, absent the adjust techniques) can be improved upon since the distribution may have concentration right above a guess, which could lead to overspending. An adjust phase limits the overspending and also allows for performance guarantees.
It can be foreshadowed that both the Learn-Then-Bid tactic and its improved version Learn-While-Bid tactic, and also the Guess-Double-Adjust tactic, perform very well for a wide range of target fractions and spends in the applicable scenarios.
Of course, each of the tactics Learn-Then-Bid, Learn-While-Bid tactic, and also Guess-Double-Adjust perform some exploration. An exploration phase might be conducted over some time period not longer than the campaign period. And this time period might be divided into an exploration length m. An initial value for m might be calculated as a proportion of the total length of the campaign, or based upon a number of impressions desired to be won, for example the average time expected to secure 20% of d. Or an initial value for m might be retrieved from some saved value or values as a result of prior exploration runs. In a more sophisticated case, the initial value of m can be calculated as a function of the desired accuracy. That is, given a run with an accuracy goal of “90% Confidence”, an initial value for m might be twice the value for m for a run with an accuracy goal of “45% Confidence”.
Consider a fully observable exchange where at least the amount of the winning bid is revealed to all bidders after every auction.
Expend Budget to Obtain Samples for a Distribution: In the partially observable case, the winning bid information is revealed only to the winner of an auction. Thus the bidder must pay a cost (i.e. at least the cost in the amount of a winning bid) in order to obtain information regarding any specific winning bid, and moreover might pay a cost multiple times in order to gather the needed number of samples for use in estimating the distribution. In other words, a system operating under the rules of a partially-observable exchange cannot learn about the auction simply by bidding zero for a while.
Explore Aggressively: A possible theoretical approach is to submit bids of value b set to infinity or some arbitrarily large but finite amount (e.g. an amount greater than 100 times the calculated target average cost per impression) for an exploration period, and thus quickly gather samples (since the large bid is nearly guaranteed to win the auction). Although using the tactic of bidding arbitrarily large amounts such that every bid wins in auction indeed has the effect of returning samples for a distribution, using that tactic that can cause overspending by a factor of almost b/t when the target fraction is small. Instead, the techniques disclosed herein seek results at the end of the campaign that are approximately optimal.
Explore Blindly: Consider the following tactic: Bid 2t (i.e. an amount twice the calculated target average cost per impression) blindly until the correct number of impressions are obtained. Observe that since the minimum bid to get fn impressions is less than the bid that gets an expected price of t, then the expected price paid when a bid is made that obtains fn is less than or equal to t. Therefore, by Markov's inequality, the number of the lowest fn impressions below 2t is fn/2. Moreover, the most that can be spent is 2t*fn, and since the budget is t*fn, this is twice the budget. Therefore, this tactic will obtain only half the required impressions, though it will not dramatically overspend.
Guess-Then-Double Pattern: It is reasonable and possible to improve on the aforementioned bidding/learning tactics. Instead of following either of these extremes (i.e. bidding 2t blindly or aggressively exploring with b), the following discloses the guess-then-double pattern. The guess-then-double pattern is used in a variety of domains. For instance, in computer science, if one wants to create an array of items but how many elements it will contain is not at that moment known, the array can be sized via a ‘good’ a guess. If later, more space is needed, then the size of the array is doubled. Thus, the number of new allocations (i.e. the number of times the array is re-sized) is logarithmic in the number of elements entered, the increase in the number of elements is linear, and the size may be off (i.e. allocated but not actually used elements) by no more than a factor of two. Of course, doubling from the previous guess or amount as in the foregoing example is only one possibility. In the case of the array, multiplying by a factor smaller than 2 will result in an expense of more new allocation steps but with the benefit of greater efficiency in terms of allocated but unused space.
Guess-Double-Adjust Pattern: The Guess-Double-Adjust pattern applies a modification to the Guess-Then-Double pattern, which might then be used in the exploration phase. Define a bid being used during exploration as an exploration bid. A “safe” bid is t, the target cost per impression since it is unlikely to result in a significant overspending of the total budget. Thus the tactic begins the exploration phase with a ‘guess’ of t. Similar as in the Guess-Then-Double pattern, subsequent bids in the exploration phase are doubled from the previous bid, which increases exponentially beginning from the value of the initial exploration bid. Of course each winning bid is a sample that can then be used to learn something about the distribution (i.e. at least the portion of the distribution below the highest bid). At some point during this exploration phase (i.e. as the bid is successively doubled in each iteration), the exploration bid exceeds p*.
A naïve approach is to simply test for this condition at each iteration, and then react to our experience at the end of each phase of the exploration (i.e. remove lines 8-9). However, this can result in a large amount of overspending, as the following example shows. The target price is 10 cents and we need to obtain 10% of the 1000 impressions. Therefore, we are searching for 100 impressions for $10.00 total. The distribution of bids is: 9.9% are at 1 cent; 0.1% are at $9.01; the remaining 90% of the bids are at $10.00. An ideal bid is $9.01. However, it is difficult to say whether this one bid will be observed. Unless there is a very slow exploration, the bids will likely exceed $10.00 during exploration. Thus, there will be a high penalty where the algorithm will most likely pay 10 times its budget.
The system for adaptive bidding for display advertising recognizes when the exploration bid exceeds p*, and enters into a phase of “cautious exploration”. The intuition of entering and proceeding in this phase is as follows: If the guess-double-adjust bid would predictably result in overspending (i.e. the exploration bid is greater than p*), then momentarily ignore the budget constraint, and operate to achieve the desired number of impressions. As before, continue to increase the bid until it is statistically reliable that there are enough winnable impressions in the exploitation phase, then move into the exploitation phase. In other words, the exploration phase iterates (with or without bid adjustments) until it finds a price Bi* where it estimates the total budget can be exhausted at the same time that it estimates it can win enough impressions (or it has explored at b or above). As with the observable case, this tactic yields a good approximation of the outcome of bidding any price below Bi*, thus the exploration phase can conclude and enter the exploitation phase.
An intuitive description of the explore, adjust, and exploit phases is as follows: In the first phase, termed exploration, the agent explores until it either gets enough information about the distribution, or the budget becomes too tight. In the second (optional) phase, termed adjust, the agent adjusts in the case that the budget becomes too tight, but the bidding has not yet yielded enough information to model the distribution and thus predict if it is feasible to obtain the right number of impressions. This tactic aggressively bids for impressions until it does yield enough information to model the distribution and thus predict if it is feasible to obtain the right number of impressions. In the third phase, termed exploitation, if there is sufficient budget to obtain the desired number of impressions, then the agent tries to exhaust the budget. If there is not sufficient budget remaining to secure the desired number of impressions, the bidding agent tries to get the right number of impressions at a discount price.
Exploring Too Long Situation: It is theoretically possible to spend too much time in the exploration phase (i.e. a solution to the problem becomes infeasible). During the exploration phase, time passes, thus likely reducing the number of forecasted impression opportunities available during the remainder of the campaign. That is, the optimization problem may be insoluble if the exploration phase is too long. Note that in the Guess-Double-Adjust pattern, there will be no more than r=[log φ(b/t)]+1 rounds of exploration, because then the bid will be above b. In each round, there are m bids, so mr is the maximum number of rounds of exploration.
Underspending Situation: It is theoretically possible that the bidding agent underspends against the budget. For instance, suppose that the objective is to obtain a very small number of impressions at a very high price. For instance, suppose 10% of the impressions are at $11.00, 90% are at $1.00, the target spend is $2.00, and the agent wants 1% of the impressions. Therefore, bidding $11.00 is ideal. However, if the agent starts by bidding $2.00, it is possible that before the exploration phase is over, the agent obtains enough impressions, but has spent only half the budget.
The computer system 1100 includes a processor 1102, a main memory 1104 and a static memory 1106, which communicate with each other via a bus 1108. The computer system 1100 may further include a video display unit 1110 (e.g. a liquid crystal display (LCD) or a cathode ray tube (CRT)). The computer system 1100 also includes an alphanumeric input device 1112 (e.g. a keyboard), a cursor control device 1114 (e.g. a mouse), a disk drive unit 1116, a signal generation device 1118 (e.g. a speaker), and a network interface device 1120.
The disk drive unit 1116 includes a machine-readable medium 1124 on which is stored a set of instructions (i.e. software) 1126 embodying any one, or all, of the methodologies described above. The software 1126 is also shown to reside, completely or at least partially, within the main memory 1104 and/or within the processor 1102. The software 1126 may further be transmitted or received via the network interface device 1120 over the network 130.
It is to be understood that embodiments of this invention may be used as, or to support, software programs executed upon some form of processing core (such as the CPU of a computer) or otherwise implemented or realized upon or within a machine or computer readable medium. A machine readable medium includes any mechanism for storing or transmitting information in a form readable by a machine (e.g. a computer). For example, a machine readable medium includes read-only memory (ROM); random access memory (RAM); magnetic disk storage media; optical storage media; flash memory devices; electrical, optical, acoustical or other form of propagated signals (e.g. carrier waves, infrared signals, digital signals, etc.); or any other type of media suitable for storing or transmitting information.
In another embodiment, again in the exemplary environment of a client server network 1200 the client 1290 is operable for receiving advertising campaign objectives 1205 (possibly including a campaign period, a target number of impressions, etc.); launching a bidding agent 1294 for performing adaptive bidding; reporting the winning bid 1241 to a content server 1292; and serving the advertisement 1242. Multiple iterations of the protocol steps 1240, 1241, 1242, 1244, 1245, and 1250 serve as steps for optimizing campaign results based on a statistical forecast of a number of impressions 1244.
While the invention has been described with reference to numerous specific details, one of ordinary skill in the art will recognize that the invention can be embodied in other specific forms without departing from the spirit of the invention. Thus, one of ordinary skill in the art would understand that the invention is not to be limited by the foregoing illustrative details, but rather is to be defined by the appended claims.
