Patent Application
20140278921
Advertising is a common way for sellers of goods and/or services to increase brand awareness. With traditional advertising media, such as television and print media, an advertisement may be seen by a wide demographic audience. Generally, only a small percentage of the audience will have any interest in ultimately purchasing the goods or services. Also, with traditional media, there is typically a limited supply of space for advertisements. In the art, the amount of resources (e.g., physical space, time, etc.) available for advertising is sometimes referred to as “inventory.” Due to its scarcity, premium inventory for traditional advertising media can be quite expensive.
With the advent of the Internet and the World Wide Web (“web”) the opportunity for online advertising has increased dramatically. Since web technology can generate an advertising message image (called an “impression”) each time a web page (or other platform such as a smartphone, tablet computer, game unit, etc.) is accessed, and since multiple members can access web page content simultaneously, there is also an ever-increasing amount of inventory. This tends to make web advertising more cost-effective than many forms of advertising in traditional media.
Besides cost, web advertising has a number of additional advantages. For one, particular demographic groups can be targeted by the selection of an appropriate website (“publisher”). For another, interaction with a web advertisement can be used to determine viewer interest. Also, web advertising tends to be more flexible with the choice of timing and placement of ads.
Since there is a significant cost to placing advertisements, even over the Internet, advertisers are keenly interested in the effectiveness of their advertisements. In particular, advertisers are interested on whether the awareness of the brand of the product or service that they are selling has increased due to their investment in advertising, and by how much. In this way, advertisers can maximize their return on their advertising campaigns.
Traditionally, the way an advertiser realized that its brand awareness had increased due to an advertising campaign was either to experience a detectible increase in interest in their products or services after the campaign was over or by conducting oral or written surveys of the viewing public. For example, an advertiser could run an advertisement for a period of time and could then see if their sales had increased. If there were no other variables introduced other than the advertisement itself, it could be reasonably surmised that a rise in sales implied an increase in brand awareness by the consuming public. However, lead times for such a process, and errors due to other factors such as the state of the economy, competition of others, prices changes, etc. all tended to obscure the relationship between the advertising campaign and the results. Furthermore, if the advertiser was running a campaign including a number of advertisements (sometimes referred to as “creatives”) it would not be possible to determine which of the advertisements were effective and which were ineffective without costly oral or written surveys. Only then would the advertiser know which advertisements of the campaign should be run and which ones should be omitted.
These and other limitations of the prior art will become apparent to those of skill in the art upon a reading of the following descriptions and a study of the several figures of the drawing.
Various examples are set forth herein for the purpose of illustrating various combinations of elements and acts within the scope of the disclosures of the specification and drawings. As will be apparent to those of skill in the art, other combinations of elements and acts, and variations thereof, are also supported herein.
In an embodiment, set forth by way of example and not limitation, a method for determining changes in brand awareness after exposures to on-line advertisements includes: first selecting and monitoring a test group including a first plurality of members and a control group including a second plurality of members from a pool of users, whereby the first plurality of members and the second plurality of members do not overlap; second exposing the test group, but not the control group, to an advertisement associated with a brand; third conducting surveys of the first plurality of members and the second plurality of members with respect to the brand; and fourth analyzing the monitoring of the test group and the control group to determine a brand lift index with respect to the advertisement.
A method for surveying users of websites for brand awareness includes: assigning cookies randomly for a test group, a control group and a general group during a first period of time; exposing neither the test group nor the control group to a test advertisement for a second period of time; exposing the test group, but not the control group, to the test advertisement for a third period of time; and surveying both the test group and the control group concerning their awareness of a brand associated with the test advertisement during a fourth period of time.
A system for determining changes in brand awareness including: an advertiser server including a processor, memory and a network connection, wherein the advertiser server includes program instructions stored in the memory and executable on the processor including: code segments first selecting and monitoring a test group and a control group over the network connection, where the test group comprises a first plurality of members and the control group comprises a second plurality of members from a pool of users; code segments second exposing the test group, but not the control group, to an advertisement associated with a brand over the network connection; code segments third conducting surveys of the first plurality of members and the second plurality of members with respect to the brand over the network connection; and code segments fourth analyzing the monitoring of the test group and the control group to determine a brand lift index with respect to the advertisement.
These and other examples of combinations of elements and acts supported herein as well as advantages thereof will become apparent to those of skill in the art upon a reading of the following descriptions and a study of the several figures of the drawing.
Several examples will now be described with reference to the drawings, wherein like elements and/or acts are provided with like reference numerals. The examples are intended to illustrate, not limit, concepts disclosed herein. The drawings include the following figures:
The operation servers 12 can be implemented as a single server or as a number of servers, such as a server farm and/or virtual servers, as will be appreciated by those of skill in the art. Alternatively, the functionality of the operation servers 12 may be implemented elsewhere in the system 10 such as on an advertiser computer 14, as indicated at 12A, on the publisher server system 16, as indicated at 12B, on a proxy 18 as indicated at 12C or as part as cloud computing as indicated at 12D, all being non-limiting examples. As will be appreciated by those of skill in the art, the processes of operation servers 12 may be distributed to these systems within system 10.
As used herein, the term “publisher” refers to an entity or entities maintaining platforms which display advertisements (“ads”) to users. The term “advertiser” refers to an entity or entities which provide advertisements to publishers. For example, an “ad agency” or “ad network” is considered to be an advertiser because they provide advertisements to publishers on behalf of their clients.
In an example, the operation servers provide middleman services between the advertisers and the publishers to facilitate the buying and selling of advertisements over the Internet. In other examples, the operation server(s) provide middleman and/or facilitation services for client computers and resource server systems to enhance a variety of e-commerce activities.
In the example of
The publisher server systems 16 can each represent one or more servers, such as a server farm. In the example of
Client computers 17 can access, for example, the publisher servers 16 via the Internet 20. A user of a client computer 16 can, for example, visit a website hosted by a publisher server 16 using a web browser. Proxies 18 can be computers, servers, or clusters of servers which serve as intermediaries or proxies between the operation servers, advertising computers and/or publisher server systems 16. As noted previously, some or all of the functionality of operation servers 12 may be implemented on proxies 18.
It will again be noted that the system 10 as illustrated in
By way of non-limiting example, computer 22 includes a microprocessor 24 coupled to a memory bus 26 and an input/output (I/O) bus 30. A number of memory and/or other high speed devices may be coupled to memory bus 26 such as the RAM 32, SRAM 34 and VRAM 36. Attached to the I/O bus 30 are various I/O devices such as mass storage 38, network interface 40, and other I/O 42. As will be appreciated by those of skill in the art, there are a number of computer readable media available to the microprocessor 24 such as the RAM 32, SRAM 34, VRAM 36 and mass storage 38. The network interface 40 and other I/O 42 also may include computer readable media such as registers, caches, buffers, etc. Mass storage 38 can be of various types including hard disk drives, optical drives and flash drives, to name a few.
With respect to a client computer 17, the other I/O 42 typically includes a monitor, a keyboard, and a mouse (not shown). A client computer 17 includes an operating system, typically stored in mass storage 38 and one or more application programs running on the operating system. One such application program is a web browser available from a number of sources including Microsoft Corporation (Internet Explorer®), Google, Inc. (Chrome®), Apple, Inc. (Safari®) and Mozilla Foundation (Firefox®). Associated with those web browsers are “cookies”, which aid in the web browsing process. These application programs and cookies are also typically stored in mass storage 38.
It should be noted that other computerized devices may be within the scope of the system of
In this non-limiting example, process 44 includes a brand awareness survey process 46 and a brand lift analysis 48. The brand awareness process 46 includes an operation 50 wherein a test group and a control group are selected from a pool of users (e.g. website viewers, game users, “app” users, etc.). The test group includes a first plurality of members and the control group includes a second plurality of member and the pool of users who have visited a test website and requested an advertisement from an advertiser.
As used herein, a test website is a publisher website selected by an advertiser to determine a “brand lift” of a test advertisement. Unique visitors are unique users as opposed to repeat users. Also, the terms “member”, “user”, “viewer” and “individual” are often used synonymously herein to refer to the members of the test group and the control group.
After selecting the members of the test group and control group, process 44 exposes only the test group to the test advertisement in an operation 52. Then, in an operation 54, both the test group and the control group are surveyed concerning the brand being advertised in the test advertisement. In an embodiment, the results of the surveys are stored in logs of the advertiser running the surveys while certain information concerning the surveys are stored the advertiser's third party cookies.
Operation 48 analyzes the test group and control group data to determine a brand lift of the test advertisement. By way of non-limiting example, the survey data of the third party cookies is analyzed such that the brand awareness of the control group and the brand awareness of the test group can be quantified. Since the only significant difference between the control group and the test group is the test group's exposure to the test advertisement, it can be assumed that any increase in brand awareness by the test group is due to the test advertisement.
After the time buffer, the test group, but not the control group, is exposed to the test advertisement that is related to the brand during a third period of time in an operation 60. Members of the test group may be exposed one or more times during this third period to the test advertisement. Next, in an operation 62, both the test group and the control group are offered a survey related to their awareness of the brand during a fourth period of time. Presumably, the test group will have higher brand awareness than the control group due to the test group's exposure to the test advertisement in the third period of time. Members of the test group and the control group who respond to the advertisement are join the general group and are not surveyed again. For members of the test group and the control group who do not take the survey when it is first offered a second opportunity to take the survey will be presented in an operation 64. After the members either take the survey offered the second time or again decline to take the survey, they are put into the general group.
It will be appreciated that there are a number of ways for implementing the processes described with respect to
In this example, a brand awareness survey process conducts a single survey (“test survey”) on a single website (“test website”) with regards to an advertisement (“test advertisement”) related to a brand is set forth by way of example and not limitation. This example process is implemented using a feature of web browsers known as “cookies.” However, it will be appreciated that other technologies than cookie technologies are also suitable for collecting survey data.
As is well known to those of skill in the art, cookies are HTTP headers that are transferred between servers and web browsers over a network, such as the Internet. Cookies can serve a variety of purposes including streamlining many browsing activities allowing for a more personalized user experience. For example, cookies can store preferences and other information about a user which can be accessed by the websites with which they are associated.
A persistent cookie will outlast a user's session at a website. For example, if a persistent cookie has a “Max-Age” set to 1 year then, within the year, the initial value set in that cookie is sent to the advertiser's server each time the user is associated with an ad request. Since persistent cookies can include such information as to how the user came to the website, persistent cookies are sometimes also called “tracking cookies.”
Third party cookies are cookies set with domains different from that shown in the address bar of the browser. Third party cookies are often provided when content from a website is derived from a different domain. By way of non-limiting example, an advertiser server might provide a third party cookie to a website at a publisher server along with advertising content. These third party cookies are subsequently available to the advertising server upon repeat visits by the same user.
In this non-limiting example, third party cookies of the advertiser are used to collect information about the test group and the control group. For example, the following information may be provided in the cookies:
A brand awareness survey process, set forth by way of example but not limitation, assigns members to either a test group, a control group, or a general group from a pool of users by assigning third party cookies to the members. In this example, the individual members of the three groups do not overlap. The number of individuals in each group can be the same, substantially the same, or different.
An example assignment process is as follows. Starting at a time t0, cookies are assigned according to a random mechanism with 3 outcomes: test, control, and general. The cookies contain information about the survey with which they are associated and serve as user IDs. In this example, the cookies are associated with only one survey, but it will be appreciated by those of ordinary skill in the art that the cookies could also be associated with multiple surveys. The “general” cookies, in this example, are unused by the survey process.
A first time period P1 ranges in time from t0 to t1. In this example, cookies with a new format are randomly assigned to the three groups during the first time period P1. After a sufficiently long first time period P1, most of the cookies for the three groups will be new cookies. For example, if P1=4 weeks, the vast majority of the cookies will have the new format. Any cookies not having the new format after the end of the first time period P1 are, in this non-limiting example, assigned to the general group.
A second time period P2 ranges in time from t1 to t2. During this period, neither the test group nor the control group are subjected to and advertisement that is being tested for brand lift, in this non-limiting example. The second time period P2 is a “buffer” or “forgetness” period. The purpose of this period is to lessen the effect's of brand awareness on the test group and the control group should any of the members (“individuals”) have previously been exposed to the test advertisement.
More particularly, in this example, the control group, after time t1 until the surveying process is over will not be subjected to the test advertisement. During the second time period P2, the test group members are also not subjected to the test advertisement. This time interval “buffer” helps members from the test group “forget” the test advertisement in case they were previously exposed to it.
A third time period P3 ranges in time from t2 to t3. In this non-limiting example, test group members may be subject to the test advertisement should they be at the test website during the third time period P3. The third time period P3 can be, by way of non-limiting example, the same length of time as a typical Internet advertising campaign, e.g. six to eight weeks.
A fourth time period P4 ranges in time from t3 to t4. During this period, members of both the test group and the control group are subjected to brief brand-awareness surveys, by way of non-limiting example. By “survey” it is meant that one or more questions are presented to a member for response. In this non-limiting example, the surveys are one or two questions long to encourage participation in the survey process.
It will be appreciated that not all members of the test group and the control group will be subjected to the survey during the fourth time period P4. For example, not all of the members of the two groups may return a publisher's site, or make an ad request, within the fourth time period P4. Also, members may have been subject to an alternative survey and not exposed to the test survey.
It will also be appreciated by those of skill in the art that the members of the two groups should be surveyed as close as possible to each other in time, and ideally at time t3 (right after the second time period P2 ends), but this is generally unpractical. In this non-limiting example, the third time period P3 should be relatively small, e.g. 2 to 7 days.
Should a member of either the test group or the control group answer the survey they will join the general group. That is, once a member of either group answers the survey, they will not be surveyed again. Should a member of either the test group or the control group not answer a survey when it is first presented (referred to herein as “first non-respondents”), they will be presented with the survey a second time if “seen” again by making an ad request. Whether or not they respond to the second survey, the first non-respondents will then join the general group. Data collection is over at time t4.
In this example, the cookies of the survey process of Example 1 are logged in an event log from time t0 to t4. An advertisement request within the time period t0-t4 will be referred to herein as an “event time.” Cookies only seen once in the event log indicate that that user had only requested an advertisement once, or that the cookie had been deleted and recreated.
For cookies “seen” multiple times (e.g. there are multiple event times stored in the cookie), the elapsed times (“inter-visit times”) between the test advertisement requests are calculated. In a non-limiting embodiment, statistical independence of the inter-visit times is checked, as well as statistical independence of the inter-visit times conditional on the total number of visits.
By way of non-limiting example, a probability of the number of revisits (0, 1, 2, . . . ) can be built. If the inter-event times appear to be interdependent, a single distribution can be built for the inter-event times. However, if the inter-events appear to be statistically independent, a distribution can be created for each of the inter-visit times (e.g. between the 1st and 2nd event times, the 2nd and 3rd event times, etc.).
Using the above mentioned probabilities, the expected number of event times from a member within predetermined time intervals can be calculated. Once the time t2 is set, time t3 is chosen such that a sufficiently large number of members of the test group will have the chance to be subjected to the test advertisement two or three times, in this non-limiting example. In alternative examples with multiple campaigns, the time period should be set such that an expected number of event times is more than 20 for most members.
Once t3 is set, the time t4 is picked so that the expected number of visits within the fourth period of time P4 is at least between one and two but preferably higher than two so that a first non-respondent has a good chance to be exposed to the survey a second time, as long as P4 does not get to be too long.
As used herein, the term “brand lift” will refer to the increase in brand awareness of a member of the test group after being exposed to the test advertisement. Brand lift is preferably a positive number, but could also be zero (e.g. when the test advertisement had no effect on the user) or, rarely, negative (e.g. where the user had less brand awareness after being exposed to the test advertisement).
For this calculation, let “a” be a measure of brand awareness of members of the control group, and “β” be a measure of brand awareness of members of the test group. Brand lift, as a percentage increase, is then given by:
As will be appreciated by those of skill in the art, there are a number of ways of calculating these brand awareness variables. The following method is set forth by way of example and not limitation and uses the following conventions:
The variable α can then be defined as:
It should be noted that the above formula assumes that the that the sum in the denominator positive.
For the test group, call N(t; 1) and N(t; 2) the respective number of elements within the sets A(t; 1) and A(t; 2), where the sets are defined similarly but for elements of the test group. Denote N(t; k; 1) and N(t; k; 2) the respective number of elements of sets A(t; k; 1) and A(t; k; 2), where A(t; k; 1) and A(t; k; 2) are the subsets of A(t; 1) and A(t; 2), respectively, whose elements got exposed k times to the test advertisement within the test advertisement-exposure window of time period P3. Call R(t; u; k; 1; 1) the response to the first question of member u from A(t; k; 1), and similarly for R(t; u; k; 2; 1).
The variable β can then be defined as:
For even more granular quantities, the variable β can be defined as:
Where the brand lift is calculated as:
It will be appreciated by those of skill in the art that a number of useful statistics concerning a change in brand awareness with respect to a test advertisement can be obtained by the aforementioned process. For example, an overall “brand lift index” can be calculated as a constant and also as a function of several variables. For example, brand lift relative to the frequency of exposure to the test advertisement or brand lift since the last exposure to the test advertisement can be calculated from the data derived from the cookies.
Although various examples have been described using specific terms and devices, such description is for illustrative purposes only. For example, many of the examples set forth herein refer to the use of cookies associated with a web browser as a user ID. However, it will be appreciated that other devices, such as electronic game units, are provided with unique user IDs that can be used for similar purposes. In any case, information concerning the user's interaction with an advertisement and the results of one or more surveys can be logged at an advertiser's server.
It will also be appreciated that the words used are words of description rather than of limitation. It is to be understood that changes and variations may be made by those of ordinary skill in the art without departing from the spirit or the scope of any examples described herein. In addition, it should be understood that aspects of various other examples may be interchanged either in whole or in part. It is therefore intended that the claims herein and hereafter presented be interpreted in accordance with their true spirit and scope and without limitation or estoppel.
