Patent Application
20080114652
1. Field of the Invention
This invention relates generally to digital data processing. More particularly, this invention relates to techniques for predicting the performance of new internet advertising experiments.
2. Description of the Prior Art
Internet advertising campaigns endeavor to enhance the return on investment associated with values (e.g., words) used in campaigns. It would be desirable to predict the performance of new internet advertising campaigns or experiments using data from previous internet advertising experiments.
A computer readable storage medium has executable instructions to identify a new Internet advertising experiment including values. Te effectiveness of the values in prior Internet advertising experiments is evaluated. The performance of the new Internet advertising experiment is predicted based upon the effectiveness of the values in the prior Internet advertising experiments.
The apparatus for predicting the performance of a new Internet advertising experiment comprises a memory device operable with a performance predictor. The memory device is configured to store values of a prior Internet advertising experiment, the performance results of the prior Internet advertising experiment using those values, and new values of a new Internet advertising experiment. The performance predictor is operable to access the memory device to retrieve the values and performance results of the prior Internet advertising experiment, calculate the effectiveness of the values used in the prior Internet advertising experiment, and predict an effectiveness of the new Internet advertising experiment having new values shared with the prior Internet advertising experiment.
The method for launching a new Internet advertising experiment comprises identifying within a memory device values of the new advertising experiment for the plurality of factors. A program then evaluates the effectiveness of prior advertising experiments sharing at least one of the values identified for the new advertising experiment. The program predicts the performance of the new Internet advertising experiment based upon the effectiveness of the values in the prior Internet advertising experiments and launches the new advertising experiment responsive to the predicted performance.
The invention is more fully appreciated in connection with the following detailed description taken in conjunction with the accompanying drawings, in which:
Like reference numerals refer to corresponding parts throughout the several views of the drawings.
The memory 18 stores executable instructions to implement operations of the invention. In particular, the memory 18 stores a performance predictor 20. The performance predictor 20 evaluates a new experiment 22 and prior experiments 24 to establish a prediction of the effectiveness of the new experiment using the techniques described below.
In one embodiment of the invention, advertisements are organized into trials. Trials are parts of experiments. Experiments encompass a set of potential values, each of which belongs to factors. Trials are “realizations” of the combinations of potential values belonging to an experiment. Table 1 illustrates factors and potential values associated with an experiment. Table 2 illustrates trials, factors and values associated with an experiment.
Trials can be designed such that one trial is not required for every possible permutation of potential values in the experiment. This is a concept borrowed from conventional Design of Experiments (DOE) approaches.
With DOE approaches, the goal is to analyze the effectiveness of values within an experiment. Analyzing results across experiments is difficult. This is important because DOE has limits as to how many potential values can be reasonably handled within an experiment. With the present invention, many values across experiments are evaluated. This sharing of values across experiments means that an experiment that already has results can help predict performance for another experiment that has not yet started running.
To make use of this aspect of shared values, a new trial can make use of an existing trials' results—even trials that are from different clients or domains. Each value can be assigned several scores in the context of a campaign. The context of a Campaign is used because a Campaign has a business goal—typically with a ROAS target. (ROAS is Return on Ad Spend, which is equal to Revenue divided by Cost).
Based on the trials that a value has been a part of, each value can have aggregate metrics-not only for cost, clicks, revenue, and conversions, but also for important ratios such as ROAS (revenue/cost), average order value (AOV) (revenue/conversions), conversion rate (conversions/clicks), and cost per click (cost/clicks). There are various ways to aggregate trial metrics for any particular value. For example, if one wants to know an “average” ROAS for a particular value, one could either (1) take the sum of the revenue of all the trials it is a part of and divide it by the sum of the cost of all those trials (“Aggregate ROAS”) or (2) take the mean of all the ROAS values of all of those trials (“Trial-based ROAS Average”).
In these equations, ti refers to the past (or ongoing) trials that a particular value is a part of. A “new” trial—really any trial to be evaluated—has values that are a part of it. Specifically, one for each factor. Scores for each factor should be based on different metrics. Scores that evaluate a landing page, for instance, might depend heavily on the conversion rate and AOV, but not very much on cost—because a landing page cannot really influence cost in a pay-per-click scheme. Keywords, on the other hand, could be evaluated on ROAS, conversion rate, and AOV, but weighted differently across those three metrics. These weights are referred to as “influences”, as shown in Table 3. Note that these influences can be different for different clients.
There are also several ways to combine the scores for each value of the new trial in order to evaluate the trial as a whole. One way is to simply sum up the scores for all the values, then evaluate how “good” the summed score is. Another is to evaluate how “good” each value's score is individually, then combine the “goodness” of those evaluations—we call this the “voted score”. One embodiment of the invention uses the voted score to decide if a new trial is “good,” “bad,” or “neutral,” and the summed score is used as a measure of “volatility”—i.e., a measure of the variance of our prediction.
Another measure of variance can be calculated by tallying the number of trials involved in computing each value score: the more trials involved in each value score computation, the higher the confidence (and the lower the variance) in its predictive power.
Consider a new trial for the client Acme. It is “new” since it has no performance history of its own. Without the current invention, time and money must be spent on testing to get any data at all.
The trial has the keyword “discount stuff,” the advertising headline of “Wonderful Discount Junk,” and a landing page of “http://www.acme.com/discountjunk,” as show in the Table 4. Because Acme has had campaigns running in the past, there have been past trials (Old Trials 1 and 2) that have used “discount stuff” as keywords (though in combination with different values for other factors). Also, Old Trial 3 shares its landing page value with the new trial. The invention may use old trials from the same or different clients.
Unlike the new trial, these three old trials have results. Typically, none of the other trials in the same experiment with the new trial would have any results either, since trials from the same experiment are generated at the same time and would be ideally judged during the same time period. Sample results for the old trials are in the Table 5.
If the influence values of Table 3 are used (reproduced in Table 6, below), calculating the value score for a keyword value for the Acme client involves the following metrics:
Assume that the influence numbers are used as linear weights in determining the value score. To determine the value score for the keyword “discount stuff,” one needs the Aggregate ROAS, the Trial AOV Avg, and the Aggregate Conversion Rate.
Assume that this campaign for Acme is targeting a ROAS of 400%, that Acme's historical Average Order Value has been 35, and that its historical Conversion Rate has been 3.5%. Using these “neutral” numbers, one can evaluate if a particular ROAS, AOV, or Conv Rate figure is “good,” “bad”, or neither—(or perhaps “goodness” can be measured on a scale of 1 to 10, or a formula can be applied to give it a numerical value on a continuum.) One implementation of the invention uses +1 for good, −1 for bad, and 0 for neutral. In this case, 600% is good, 32.5 for AOV is neutral, and 5.8% Conv Rate is good.
Based on the influences values of Table 3, the value score for this keyword for Acme is:
ValueScore(“discountstuff”)=3*(+1)+1*(0)+2*(+1)=+5 (6)
As for the landing page of http://www.acme.com/discountjunk, the influence values from Table 3 are reproduced in Table 7 below:
Since there is only one old trial involved, the calculations are simple.
An AOV of 35 is neutral (same as historical AOV), and a conversion rate of 2% is bad (historical Conv Rate was 3.5%).
ValueScore(“http://www.acme.com/discountjunk”)=3*(0)+2*(−1)=−2 (9)
Now these scores can be combined to see the disposition for the new trial. Summing all the scores for each value yields +3 for the summed score. The voted score is zero (positive for keyword, negative for landing page, no vote for headline).
These scores can be used in several ways. For example, a set of very positive scores can be a signal for the system to launch this trial sooner, with a higher bid, and perhaps on more advertising networks. Negative scores can be a signal to lower the bid for this trial, to let other trials to launch first, or to not launch this trial at all.
An embodiment of the present invention relates to a computer storage product with a computer-readable medium having computer code thereon for performing various computer-implemented operations. The media and computer code may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well known and available to those having skill in the computer software arts. Examples of computer-readable media include, but are not limited to: magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs, DVDs and holographic devices; magneto-optical media; and hardware devices that are specially configured to store and execute program code, such as application-specific integrated circuits (“ASICs”), programmable logic devices (“PLDs”) and ROM and RAM devices. Examples of computer code include machine code, such as produced by a compiler, and files containing higher-level code that are executed by a computer using an interpreter. For example, an embodiment of the invention may be implemented using Java, C++, or other object-oriented programming language and development tools. Another embodiment of the invention may be implemented in hardwired circuitry in place of, or in combination with, machine-executable software instructions.
The foregoing description, for purposes of explanation, used specific nomenclature to provide a thorough understanding of the invention. However, it will be apparent to one skilled in the art that specific details are not required in order to practice the invention. Thus, the foregoing descriptions of specific embodiments of the invention are presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the invention to the precise forms disclosed; obviously, many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, they thereby enable others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated. It is intended that the following claims and their equivalents define the scope of the invention.
Having described and illustrated the principles of the invention in a preferred embodiment thereof, it should be apparent that the invention can be modified in arrangement and detail without departing from such principles. We claim all modifications and variation coming within the spirit and scope of the following claims.
This application claims the benefit from U.S. Provisional Patent Application No. 60/828,366 filed Oct. 5, 2006 whose contents are incorporated herein for all purposes.
| Number | Date | Country | |
|---|---|---|---|
| 60828366 | Oct 2006 | US |
