Patent Application
20180365738
This disclosure generally relates to evaluating users of an online system for presenting content items to. Specifically, this disclosure relates to evaluating users of an online system for presenting content items to, based on an influence of the users with respect to other users of the online system.
In general, a content item is intended to persuade individuals that view the content item to engage in a desired behavior related to the content item. An individual that views a given content item may not only be persuaded to engage in the desired behavior themselves, but the individual may also indirectly persuade additional individuals to engage in the desired behavior. In this way, by presenting a content item to a single individual, other individuals may fulfill behavioral objectives associated with the content item without even viewing the content item themselves.
Evaluating the possible effect that one individual has on other individuals can be a challenge. Thus, content items are typically presented to individuals with no regard for the probability that these individuals will persuade additional individuals to engage in behavioral objectives associated with the content items.
This disclosure describes a method for selecting users of an online system to be presented with content items, based on the expected influence of users on other users' behavior. First, a third party system places a request with the online system to present a given content item to users of the online system. The request may include targeting criteria used in selecting users to present with the content item. Users of the online system that fulfill the specified targeting criteria are selected as subject users. Subject users may be presented with the content item based in part on the expected influence of the subject user with respect to other users. Specifically, if a subject user that fulfills the specified targeting criteria is determined to have a threshold expected influence with respect to other users, the subject user may be presented with the content item.
To determine an expected influence of a subject user with respect to other users, the online system further uses the targeting criteria specified by the third party system to identify a set of connections of the identified subject user. The set of connections of the identified subject user consists of additional users of the online system that have a threshold “affinity” to the subject user. Affinities between users of the online system can be determined by comparing the behaviors of the given users. Generally, users that have a relatively high affinity to one another frequently interact and/or engage similar behaviors.
Next, based on the targeting criteria specified by the third party system, an event coordination score between the subject user and each individual in the subject user's set of connections is determined. An event coordination score is a quantitative measure of the affinity between the subject user and an individual in the subject user's set of connections. Specifically, an event coordination score is a quantitative measure of the synchronization of one or more actions performed by the subject user and the individual in the subject user's set of connections. By comparing the similarity of actions between the two users, an event coordination score measures the influence that the subject user and the connection of the given subject user have on one another. The event coordination score may represent a co-occurrence of actions without respect to timing, or may represent a directional event occurrence, indicating when a user performed an action before another user.
Event coordination scores between the subject user and each individual in the subject user's set of connections are used to determine an influence score for the subject user. An influence score is a measure of the influence that a given subject user has over the actions of all individuals within the given subject user's set of connections.
Based on the subject user's influence score and additional targeting criteria specified by the third party system, the online system determines whether the given subject user is considered to be an influential subject user. Subject users determined to be influential subject users may be presented with content items provided by the third party system.
In alternative embodiments, influence scores for subject users may also be used to rank subject users. These rankings can in turn be used to select subject users to present with content. In further embodiments, a subject user's influence score may be used to assign an expected value of presenting a content item to that subject user. Such expected values can in turn be used in advertising auctions to determine bid amounts or to determine a set of content items to present to the subject user.
By identifying and presenting content items to influential users, the influential users can be persuaded to engage in desired behavior encouraged by the content items. These influential users may then use their influence to persuade additional individuals to engage in the desired behavior associated with the content items. In this manner, a large number of individuals can be indirectly persuaded to engage in desired behavior related to a content item without ever viewing the content item itself. Furthermore, effectively identifying influential users enables better evaluation of the downstream value of presenting content to these users.
The figures depict various embodiments for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles described herein.
The user devices 101 are one or more computing devices capable of receiving user input as well as transmitting and/or receiving data via the network 102. In one embodiment, a user device 101 is a conventional computer system, such as a desktop or a laptop computer. Alternatively, a user device 101 may be a device having computer functionality, such as a personal digital assistant (PDA), a mobile telephone, a smartphone, or another suitable device. A user device 101 is configured to communicate via the network 102. In one embodiment, a user device 101 executes an application allowing a user of the user device 101 to interact with the online system 104. For example, a user device 101 executes a browser application to enable interaction between the user device 101 and the online system 104 via the network 102. In another embodiment, a user device 101 interacts with the online system 104 through an application programming interface (API) running on a native operating system of the user device 101, such as IOS® or ANDROID™.
The user devices 101 are configured to communicate via the network 102, which may comprise any combination of local area and/or wide area networks, using both wired and/or wireless communication systems. In one embodiment, the network 102 uses standard communications technologies and/or protocols. For example, the network 102 includes communication links using technologies such as Ethernet, 802.11, worldwide interoperability for microwave access (WiMAX), 3G, 4G, code division multiple access (CDMA), digital subscriber line (DSL), etc. Examples of networking protocols used for communicating via the network 102 include multiprotocol label switching (MPLS), transmission control protocol/Internet protocol (TCP/IP), hypertext transport protocol (HTTP), simple mail transfer protocol (SMTP), and file transfer protocol (FTP). Data exchanged over the network 102 may be represented using any suitable format, such as hypertext markup language (HTML) or extensible markup language (XML). In some embodiments, all or some of the communication links of the network 102 may be encrypted using any suitable technique or techniques.
One or more third party systems 103 may be coupled to the network 102 for communicating with the online system 104. In one embodiment, a third party system 103 provides content items to the online system 104 for presentation to users of the online system 104. Content items provided by the third party system 103 system may include sponsored content and content campaigns.
Each user of the online system 104 is associated with a user profile, which is stored in the user profile store 201. A user profile includes declarative information about the user that was explicitly shared by the user and may also include profile information inferred by the online system 104. In one embodiment, a user profile includes multiple data fields, each describing one or more attributes of the corresponding online system 104 user. Examples of information stored in a user profile include biographic, demographic, and other types of descriptive information, such as work experience, educational history, gender, hobbies or preferences, location and the like. A user profile may also store other information provided by the user, for example, images or videos. In certain embodiments, images of users may be tagged with information identifying the online system users displayed in an image, with information identifying the images in which a user is tagged stored in the user profile of the user. A user profile in the user profile store 201 may also maintain references to actions by the corresponding user performed on content items in the content store 202 and stored in the action log 204. Furthermore, a user profile in the user profile store 201 may also maintain references to actions by the corresponding user performed on third party systems 103 and stored in the action log 204. These references to actions performed on third party systems 103 may be received by the online system 104 directly from the third party systems 103 or through an alternative means of user tracking.
In one embodiment, the third party system 103 uses a tracking pixel or piece of HTML code placed by the third party system 103 on third party system 103 websites to monitor users visiting the websites that have not opted out of tracking. A tracking pixel might be included on various pages, including on a product page describing a product, on a shopping cart page that the user visits upon putting something into a shopping cart, on a checkout page that the user visits to checkout and purchase a product, etc. For example, a tracking pixel results in a transparent 1×1 image, an iframe, or other suitable object being created for third party system 103 pages. When a user's browser loads a page having the tracking pixel, the tracking pixel results in the user's browser attempting to retrieve the content for that pixel, and the browser contacts the online system 104 to retrieve the content. The request sent to the online system 104, however, actually includes various data about the user's actions taken on the third party system 103 website. The third party system 103 website can control what data is sent to the online system 104. For example, the third party system 103 may include information about the page the user is loading (e.g., is it a product page, a shopping cart page, a checkout page, etc.), about information on the page or about a product on the page of interest to the user (e.g., the SKU number of the product, the color, the size, the style, the current price, any discounts offered, the number of products requested, etc.), about the user (e.g., the third party system 103's user identifier (UID) for the user, contact information for the user, etc.), and other data. In some embodiments, a cookie set by the online system 104 can also be retrieved by the online system 104, which can include various data about the user, such as the online system 104s' UID for the user, information about the client device and the browser, such as the Internet Protocol (IP) address of the client device, among other data. Tracking can also be performed on mobile applications of content providers by using a software development kit (SDK) of the online system 104 or via an application programming interface (API) of the online system 104 to track events (e.g., purchases) that occur by users on the content provider's app that are reported to the online system 104.
The online system 104 can then use this data received about the user to serve better content to the user in the future (e.g., since the online system 104 has further data about products purchased or browsed that might be of interest), and can also use this in retargeting of that product to the user (e.g., where the user went to the product page on the third party system site by clicking on an of the third party system 103 presented to the user in the online system 104). The online system 104 can also use this data in conversion tracking and reporting results of advertising campaigns to the third party system 103. For example, if the third party system 103 has spent money at the online system 104 to have the online system 104 serve ads for its products, and a particular user views an ad on the online system 104 and then purchases the product advertised (possibly at some point later, and possibly on a different device), the online system 104 can link the purchase/conversion to the ad originally shown on the online system 104. Thus, the online system 104 can include this data in its reporting to the third party system 103 of how many conversions the ad campaign received.
While user profiles in the user profile store 201 are frequently associated with individuals, allowing individuals to interact with each other via the online system 104, user profiles may also be stored for entities such as businesses or organizations. This allows an entity to establish a presence on the online system 104 for connecting and exchanging content with other online system users. The entity may post information about itself, about its products or provide other information to users of the online system 104 using a brand page associated with the entity's user profile. Other users of the online system 104 may connect to the brand page to receive information posted to the brand page or to receive information from the brand page. A user profile associated with the brand page may include information about the entity itself, providing users with background or informational data about the entity.
The content store 202 stores objects that each represent various types of content. Examples of content represented by an object include a page post, a status update, a photograph, a video, a link, a shared content item, a gaming application achievement, a check-in event at a local business, a brand page, or any other type of content. Online system users may create objects stored by the content store 202, such as status updates, photos tagged by users to be associated with other objects in the online system 104, events, groups or applications. In some embodiments, objects are received from third party systems 103 separate from the online system 104. In one embodiment, objects in the content store 202 represent single pieces of content, or content “items.” Hence, online system users are encouraged to communicate with each other by posting text and content items of various types of media to the online system 104 through various communication channels. This increases the amount of interaction of users with each other and increases the frequency with which users interact within the online system 104.
One or more content items included in the content store 202 include content for presentation to a user and a bid amount. The content is text, image, audio, video, or any other suitable data presented to a user. In various embodiments, the content also specifies a page of content. For example, a content item includes a landing page specifying a network address of a page of content to which a user is directed when the content item is accessed. The bid amount is included in a content item by a user and is used to determine an expected value, such as monetary compensation, provided by an advertiser to the online system 104 if content in the content item is presented to a user, if the content in the content item receives a user interaction when presented, or if any suitable condition is satisfied when content in the content item is presented to a user. For example, the bid amount included in a content item specifies a monetary amount that the online system 104 receives from a user who provided the content item to the online system 104 if content in the content item is displayed. In some embodiments, the expected value to the online system 104 of presenting the content from the content item may be determined by multiplying the bid amount by a probability of the content of the content item being accessed by a user.
In various embodiments, a content item includes various components capable of being identified and retrieved by the online system 104. Example components of a content item include: a title, text data, image data, audio data, video data, a landing page, a user associated with the content item, or any other suitable information. The online system 104 may retrieve one or more specific components of a content item for presentation in some embodiments. For example, the online system 104 may identify a title and an image from a content item and provide the title and the image for presentation rather than the content item in its entirety.
Various content items may include an objective identifying an interaction that a user associated with a content item desires other users to perform when presented with content included in the content item. Example objectives include: installing an application associated with a content item, indicating a preference for a content item, sharing a content item with other users, interacting with an object associated with a content item, or performing any other suitable interaction. As content from a content item is presented to online system users, the online system 104 logs interactions between users presented with the content item or with objects associated with the content item. Additionally, the online system 104 receives compensation from a user associated with content item as online system users perform interactions with a content item that satisfy the objective included in the content item.
A content item may include one or more targeting criteria specified by the user who provided the content item to the online system 104. Targeting criteria included in a content item request specify one or more characteristics of users eligible to be presented with the content item. For example, targeting criteria are used to identify users having user profile information, edges, or actions satisfying at least one of the targeting criteria. Hence, targeting criteria allow a user to identify users having specific characteristics, simplifying subsequent distribution of content to different users.
The principle embodiment discussed throughout this disclosure is evaluation of users of the online system 104 for presentation with content items from the content store 202. As described above, content items of the content store 202 may include targeting criteria. These targeting criteria may be used in evaluating users of the online system 104 for presentation with content items. Users of the online system 104 that fulfill the specified targeting criteria are selected as subject users. Subject users may be presented with the content item based in part on the expected influence of the subject user with respect to other users of the online system 104.
Influence of a subject user on other users of the online system 104 may be determined based on actions or types of connections between the subject user and other users of the online system 104. A subject user's influence may be determined based on interactions between the subject user and other users performed external to the online system 104, such as on a third party system 103. For example, the influence of a subject user may be determined based on actions performed by that subject user with regard to other users, such as sent a message to another user, used an application, joined a group, left a group, joined an event, generated an event description, purchased or reviewed a product or service using an online marketplace, requested information from a third party system 103, installed an application, or performed any other suitable action. Determining influence of subject users allows users to further refine which subject users are eligible to be presented with content items. As another example, determination of a subject user's influence may include identification of the subject user's connections to another user or object or having a particular type of connection to another user or object. Based on the determined influence of a subject user, the subject user may be presented with content items from the content store 202.
In various embodiments, the content store 202 includes multiple campaigns, which each include one or more content items. In various embodiments, a campaign in associated with one or more characteristics that are attributed to each content item of the campaign. For example, a bid amount associated with a campaign is associated with each content item of the campaign. Similarly, an objective associated with a campaign is associated with each content item of the campaign. In various embodiments, a user providing content items to the online system 104 provides the online system 104 with various campaigns each including content items having different characteristics (e.g., associated with different content, including different types of content for presentation), and the campaigns are stored in the content store.
The action logger 203 receives communications about user actions internal to and/or external to the online system 104, populating the action log 204 with information about user actions. Examples of actions include adding a connection to another user, sending a message to another user, uploading an image, reading a message from another user, viewing content associated with another user, and attending an event posted by another user. In addition, a number of actions may involve an object and one or more particular users, so these actions are associated with the particular users as well and stored in the action log 204.
The action log 204 may be used by the online system 104 to track user actions on the online system 104, as well as actions on third party systems 103 that communicate information to the online system 104. Users may interact with various objects on the online system 104, and information describing these interactions is stored in the action log 204. Examples of interactions with objects include: commenting on posts, sharing links, checking-in to physical locations via a user device 101, accessing content items, and any other suitable interactions. Additional examples of interactions with objects on the online system 104 that are included in the action log 204 include: commenting on a photo album, communicating with a user, establishing a connection with an object, joining an event, joining a group, creating an event, authorizing an application, using an application, expressing a preference for an object (“liking” the object), and engaging in a transaction. Additionally, the action log 204 may record a user's interactions with advertisements on the online system 104 as well as with other applications operating on the online system 104. In some embodiments, data from the action log 204 is used to infer interests or preferences of a user, augmenting the interests included in the user's user profile and allowing a more complete understanding of user preferences.
The action log 204 may also store user actions taken on a third party system 103, such as an external website, and communicated to the online system 104. For example, an e-commerce website may recognize a user of an online system 104 through a social plug-in enabling the e-commerce website to identify the user of the online system 104. Because users of the online system 104 are uniquely identifiable, e-commerce websites, such as in the preceding example, may communicate information about a user's actions outside of the online system 104 to the online system 104 for association with the user. Hence, the action log 204 may record information about actions users perform on a third party system 103, including webpage viewing histories, advertisements that were engaged, purchases made, and other patterns from shopping and buying. Additionally, actions a user performs via an application associated with a third party system 103 and executing on a user device 101 may be communicated to the action logger 203 by the application for recordation and association with the user in the action log 204.
In one embodiment, the edge store 205 stores information describing connections between users and other objects on the online system 104 as edges. Some edges may be defined by users, allowing users to specify their relationships with other users. For example, users may generate edges with other users that parallel the users' real-life relationships, such as connections, co-workers, partners, and so forth. Other edges are generated when users interact with objects in the online system 104, such as expressing interest in a page on the online system 104, sharing a link with other users of the online system 104, and commenting on posts made by other users of the online system 104.
An edge may include various features each representing characteristics of interactions between users, interactions between users and objects, or interactions between objects. For example, features included in an edge describe a rate of interaction between two users, how recently two users have interacted with each other, a rate or an amount of information retrieved by one user about an object, or numbers and types of comments posted by a user about an object. The features may also represent information describing a particular object or user. For example, a feature may represent the level of interest that a user has in a particular topic, the rate at which the user logs into the online system 104, or information describing demographic information about the user. Each feature may be associated with a source object or user, a target object or user, and a feature value. A feature may be specified as an expression based on values describing the source object or user, the target object or user, or interactions between the source object or user and target object or user; hence, an edge may be represented as one or more feature expressions.
The edge store 205 also stores information about edges, such as affinity scores for objects, interests, and other users. Affinity scores, or “affinities,” may be computed by the online system 104 over time to approximate a user's interest in an object or in another user in the online system 104 based on the actions performed by the user. A user's affinity may be computed by the online system 104 over time to approximate the user's interest in an object, in a topic, or in another user in the online system 104 based on actions performed by the user. Computation of affinity is further described in U.S. patent application Ser. No. 12/978,265, filed on Dec. 23, 2010, U.S. patent application Ser. No. 13/690,254, filed on Nov. 30, 2012, U.S. patent application Ser. No. 13/689,969, filed on Nov. 30, 2012, and U.S. patent application Ser. No. 13/690,088, filed on Nov. 30, 2012, each of which is hereby incorporated by reference in its entirety. Multiple interactions between a user and a specific object may be stored as a single edge in the edge store 205, in one embodiment. Alternatively, each interaction between a user and a specific object is stored as a separate edge. In some embodiments, connections between users may be stored in the user profile store 201, or the user profile store 201 may access the edge store 205 to determine connections between users.
The indirect conversions module 206 evaluates users of the online system 104 to target with content items for a campaign based on their level of influence with respect to other users of the online system 104. The influence of one user for one or more additional users is determined using information from the user profile store 201, the content store 202, the action log 204, and the edge store 205. One embodiment of the indirect conversions module is discussed in greater detail with regard to
The web server 207 links the online system 104 via the network 102 to the one or more user devices 101, as well as to the one or more third party systems 103. The web server 207 serves web pages, as well as other content, such as JAVA®, FLASH®, XML and so forth. The web server 207 may receive and route messages between the online system 104 and the user device 101, for example, instant messages, queued messages (e.g., email), text messages, short message service (SMS) messages, or messages sent using any other suitable messaging technique. A user may send a request to the web server 207 to upload information (e.g., images or videos) that are stored in the content store 202. Additionally, the web server 207 may provide application programming interface (API) functionality to send data directly to native client device operating systems, such as IOS®, ANDROID™, or BlackberryOS.
The third party input module 301 receives a request from a third party system 103 to present a content item to one or more users of the online system 104. As discussed above with regard to
The subject user selection module 302 identifies one or more users of the online system 104 as candidates to be targeted with the content specified by a content request. In one embodiment, the subject user selection module 302 uses targeting criteria specified for the content campaign by the third party system 103 when selecting candidates to receive content. These users may be considered as recipients of a content item. The candidates chosen for content targeting by the subject user selection module 302 are referred to as “subject users.” A subject user is a candidate for content targeting. In other words, a subject user may or may not ultimately be presented with a given content. Selection of the subject users that will ultimately be presented with a given content may be determined based on the influence score of a subject user and other factors affecting the expected value of presenting the content item to the subject user.
As one example, a third party system 103 that sells running shoes places a content request with the online system 104. The content request indicates that the content presented to users of the online system 104 should be an image of running shoes. The content request further specifies targeting criteria indicating that users to which the content is presented must engage with content related to running at least three times per month. To determine which users satisfy this targeting criterion, the subject user selection module 302 draws on information from one or more of the user profile store 201, the content store 202, the action log 204, and the edge store 205. For example, the subject user selection module 302 may search for users within the user profile store 201 that have listed running in the user profile data field labeled “hobbies.” The subject user selection module 302 may also identify subject users by searching the content store 202 for photos of users wearing running shoes. The subject user selection module 302 may search the action log 204 to identify users that have indicated they will attend a local marathon. The subject user selection module 302 may also search the edge store 205 to identify users that have “liked” the user profile of an Olympic runner. Through these embodiments as well as others, the subject user selection module 302 can identify subject users that satisfy the targeting criteria specified by the third party system 103.
In some embodiments, subject users may be selected based on data that fulfills targeting criteria specified by the third party system 103, but that was not sourced from the online system 104. In other words, data used in the selection of subject users may be stored and detected on third party systems 103 and then transmitted to the online system 104 for use in selecting subject users. For example, as mentioned with regard to
Selection of a subject user may be dependent upon the type of conversion event. In one embodiment, a given user of the online system 104 may not be chosen as a subject user if the user has already engaged with the same or similar a content item in the desired manner. In another embodiment, users that performed conversion actions for high-end, expensive purchases may be excluded in the targeting criteria for content items similar or related to these purchases. This may hold true even if the similar or related content items are sponsored by a third party system 103 other than the third party system 103 providing the campaign. For example, if the content item specified by the third party system 103 is airline tickets to Tahiti, targeting criteria for the airlines tickets may exclude users that have already purchased airline tickets to Tahiti. In an alternative embodiment in which sponsored content items are less expensive, more common items, subject user targeting criteria may not adhere to such limitations. For example, if the content item specified by the third party system 103 is a tin of cookies, a user that has purchased cookies in the past may still be selected as a subject user. In a further embodiment, a frequency of expected purchases may be used to determine the allotted time period between user purchase of an item and selection of the user as a subject user for receipt of content related to the item. For example, a relatively inexpensive, common item such as a T-shirt may be associated with a high frequency of expected purchase. Thus a user that purchased a T-shirt may be selected as a subject user for receipt of a T-shirt content immediately following the purchase of the given shirt. On the other hand, a relatively expensive, enduring, and exclusive item such as a wedding ring may be associated with a lower frequency of expected purchase. Thus a person that purchased a wedding ring may not be chosen as a subject user for receipt of a wedding ring content.
Following identification of a subject user by the subject user selection module 302, the connection selection module 303 identifies one or more additional users of the online system 104 to be used in evaluating the influence of the subject user. The one or more additional users used in evaluating the influence for a subject user are known as a “set of connections” for the given subject user. To improve the accuracy of influence score calculation for a subject user, the set of connections for a subject user are users of the online system 104 with whom the subject user has a strong affinity.
To determine the set of connections for a subject user, the connection selection module 303 may use one or more targeting criteria that must be fulfilled for a user to be considered a connection of the given subject user. In one embodiment, criteria for selecting connections of a subject user are specified by the third party system 103 via the third party input module 301. In another embodiment, the connection selection module 303 specifies standard, pre-determined criteria for selecting connections of a subject user. In yet another embodiment, the online system 104 may suggest criteria for selecting connections for a subject user based on data from previous content campaigns.
For example, a user may be deemed a connection of a subject user if the user and the subject user engage in at least one interaction per day via the online system 104. In another example, a user may be deemed to be a connection of a subject user if the user and the subject user are listed as friends within the online system 104. In another example, a user may be deemed to be a connection of a subject user if the user and the subject user engage in the same behaviors within a specified range of time or within a specified range of geographic location.
Another example of criteria used in the selection of connections for a subject user is directionality of relationship. A relationship between two users may be unidirectional or bidirectional. In other words, user A may have an affinity to user B without existence of reciprocated affinity, user B may have an affinity to user A without existence of reciprocated affinity, or a level of affinity may be reciprocal between both user A and user B. In one embodiment, a third party system 103 may indicate a required directionality of relationship between a user and a given subject user for the user to be considered a connection of the given subject user.
To determine whether a given user fulfills the criteria required to be considered a connection of a given subject user, the connection selection module 303 may draw on information from one or more of the user profile store 201, the content store 202, the action log 204, and the edge store 205. For example, to determine whether a user and a subject user engaged in the same behavior within a specified range of time, the connection selection module 303 may conduct a search within the action log 204. In another example, the connection selection module 303 may use affinity scores stored in the edge store 205 to determine whether or not a user is a connection of a given subject user.
In some embodiments, connections of a subject user may be selected based on data that fulfills criteria specified by the third party system 103, but that was not sourced from the online system 104. For example, fulfillment of criteria may be detected by the online system 104 based on activity tracked external to the online system 104.
Finally, a connection can be quantitatively scored based on fulfillment of targeting criteria such as the targeting criteria described above. Using the scores calculated for all connections of a subject user, the connections can be ranked. The final set of connections for the subject user can be selected from this ranked list of connections. For example, in some embodiments, all connections with at least a specified threshold score are included in the subject user's set of connections. In another embodiment, a specified number of connections with the highest scores are included in the subject user's set of connections.
In some embodiments, a set of connections of a subject user may include users with whom the subject user has no direct relationship, but with whom another user within the subject user's set of connections has a direct relationship, thereby creating an indirect, extended relationship between the subject user and a connection of the subject user's connection. The subject user's set of connections therefore may include users with a number of degrees of separation from the subject user. Accordingly, though not directly connected to the subject user, these indirectly-connected users may be more likely to be influenced by the subject user's action than a directly connected user that is not closely-connected to the subject user. Criteria input by the third party system 103 via the third party input module 301 may regulate the inclusion of indirect connections within a subject user's set of connections. An example of this embodiment is described in more detail with regard to
The connection selection module 303 may also identify a group of users not based on a connection to a subject user, but based on the qualities of the group itself. For example, an advertiser selling swim caps may wish to identify the most influential member of a swim team expecting that user to influence other members of the team. In this example, a subject user is not selected initially. Instead, the swim team is initially selected as a set of connections, and each user may be evaluated as a subject user with respect to the group. An example of a similar embodiment is discussed in further detail with regard to
Turning back to
In one embodiment, actions used in the calculation of event coordination scores are specified by the third party system 103 via the third party input module 301. In another embodiment, the event coordination scoring module 304 specifies actions considered in calculation of event coordination scores. In yet another embodiment, the online system 104 may suggest actions for inclusion based on data from previous cases.
For example, a third party system 103 may indicate that all actions related to the consumption of food be included in the event coordination score calculation. On the other hand, the third party system 103 may specify that the purchase of almond butter be the only action considered in the calculation of event coordination scores. As another example, a third party system 103 may indicate that all social actions should be included in the calculation of event coordination scores. Examples of social actions include “liking” a specific page, listening to a specific song, following a specific celebrity, replying to a specific comment, or sharing a specific post. As another example, a third party system 103 may indicate that all actions considered in calculation of the event coordination score may be more closely tied to advertising. Some examples of actions tied to advertising may be clicking on a specific advertisement, purchasing a specific item, or watching a specific video advertisement.
To determine whether a given user has completed an action used in the calculation of event coordination scores, the event coordination scoring module 304 may use information from one or more of the user profile store 201, the content store 202, the action log 204, and the edge store 205. For example, to determine whether a user and a subject user watched the same advertisement within a specified range of time, the connection selection module 303 may conduct a search within the action log 204.
In some embodiments, actions that do not occur on the online system 104 can be used in the calculation of event coordination scores. For instance, actions used in event coordination score calculation may be executed and detected on third party systems 103 and then transmitted to the online system 104 for inclusion in the calculation. For example, as mentioned with regard to
The time period within which one user performs an action previously performed by another user is also relevant in the calculation of the event coordination score between the two users, and may be included in the calculation. In one embodiment, a third party system 103 or the event coordination scoring module 304 may specify a maximum threshold of time required for actions executed by two independent users to be considered in the calculation of their event coordination score. In another embodiment, the event coordination score between two users might be weighted more heavily for users that perform the same action within ten minutes of one another than for a different pair of users that perform the same action within one week of one another. Additional embodiments not explicitly mentioned here are also possible.
An event coordination score can either be non-directional or directional. A non-directional event coordination score does not take into account the order of actions or events executed by the individuals in question. In other words, a non-directional event coordination score does not distinguish between which users are the followers and which users and the influencers. One embodiment of the calculation of non-directional event coordination scores is described in greater detail with regard to
On the other hand, a directional event coordination score takes into account the order in which individuals in a group perform a given action. Each individual in the group is assigned a unique event coordination score that indicates the degree to which the individual is an influencer or a follower of each additional individual within the given group. This emphasis on directionality enables a more precise determination of which users initiate actions that others also perform. One embodiment of the calculation of directional event coordination scores is explained in more detail with regard to
The directionality of event coordination scores can be specified by a third party system 103 via the third party input module 301, or by the settings of the event coordination scoring module 304. The event coordination scoring module 304 then evaluates the event coordination scores based on the specified directionality.
The influence scoring module 305 determines an influence score for a given subject user. In the case in which a group of users is selected without regard for a subject user, the influence scoring module 305 determines an influence score for each user within the group of users. An influence score is a measure of the influence that a given individual has over the actions of all individuals in a given group. In the case of a subject user, the subject user's influence score is a measurement of the influence that the subject user has over the actions of all users within the subject user's set of connections. Two embodiments of the calculation of influence scores are shown in
The influential subject user selection module 306 identifies influential subject users based on subject users' influence scores calculated by the influence scoring module 305. “Influential subject users” are the subject users that are ultimately chosen to be presented with a given advertisement. The influential subject user selection module 306 uses criteria specified by the third party system 103 via the third party input module 301 or by the influential subject user selection module 306 in the selection of influential subject users. One example of criteria used in the selection of influential subject users may be a threshold influence score. In this example, a subject user may be considered influential only if their influence score is greater than a given value. Another example of criteria used in the selection of influential subject users may be the total number of influential subject users desired.
In alternative embodiments, influence scores for subject users may also be used to rank subject users. These rankings can in turn be used to select subject users to present with content. In further embodiments, a subject user's influence score may be used to assign an expected value of presenting a content item to that subject user. Such expected values can in turn be used in advertising auctions to determine bid amounts or to determine a set of content items to present to the subject user.
The online system 104 receives 401 targeting criteria for a campaign of a content item, the targeting criteria specifying the targeting criteria for selecting users of the online system 104 to present with the content item. The targeting criteria received may apply to numerous steps of the targeting process.
The online system 104 identifies 402 a subject user meeting the targeting criteria specified by the content campaign in step 401. The identified subject user is a candidate for content targeting. In other words, the subject user may or may not ultimately be presented with the content item introduced in step 401. Selection of subject users that will ultimately be presented with the content item may be determined based influence scores determined for the subject users, as well as other factors affecting the expected value of presenting the content item to the subject users.
The online system 104 identifies 403 a set of connections of the subject user also meeting the targeting criteria specified in step 401. To improve the accuracy of influence score calculation for the subject user, the set of connections for the subject user may be users of the online system 104 with whom the subject user has a strong affinity.
The online system 104 determines 404, based on the targeting criteria, an event coordination score between the subject user and each connection in the set of connections, an event coordination score being determined by a comparison of one or more events co-occurring for the subject user and the connection. Two embodiments for the calculation of event coordination scores are described in greater detail with regard to
The online system 104 determines 405, from the event coordination scores, an influence score for the subject user. Two embodiments for the calculation of an influence score are described in greater detail with regard to
Finally, the online system 104 presents 406 the content item to the subject user based in part on the influence score. Presentation of the content item to the subject user may also depend on the expected value of presenting the content item to the subject user.
In the embodiment depicted in
The events on which the coordinated event scoring of
Additional criteria for event coordination scoring besides event specification may also be provided by the indirect conversions module 206 or by the third party system 103. For example, the third party system 103 may indicate that the purchase of a snowboard is only considered a coordinated event between two users if the two users in question purchase the snowboard within one month of one another. In this case, each arrow connecting two users indicates that both users purchased a snowboard within a one month period of time. Absence of an arrow in this case indicates that one user purchased a snowboard but the other did not, that both users purchased a snowboard but not within a single month, or that neither user purchased a snowboard.
Each arrow of
Each arrow of
To assign a quantitative influence score to the subject user Fred, the embodiment depicted in
While influence score calculation is performed only for the subject user Fred, arrows depicting event coordination scores between other users have been included for use in
In the embodiment depicted in
In the embodiment depicted in
The events on which the coordinated event scoring of
Additional criteria for event coordination scoring besides event specification may also be provided by the indirect conversions module 206 or by the third party system 103. For example, the third party system 103 may indicate that the purchase of a snowboard is only considered a coordinated event between two users if the two users in question purchase the snowboard within one month of one another. In this case, each arrow connecting two users indicates that both users purchased a snowboard within a one month period of time. Absence of an arrow in this case indicates that one user purchased a snowboard but the other did not, that both users purchased a snowboard but not within a single month, or that neither user purchased a snowboard.
Each arrow of
Each arrow of
To assign a quantitative influence score to the subject user Fred, the embodiment depicted in
While influence score calculation is performed only for the subject user Fred, arrows depicting event coordination scores between other users have been included for use in
As a result of the use of directional event coordination scoring, the subject user Fred receives a lower overall influence score compared to the same scenario in which non-directional event coordination scoring was used. In this way, use of directional event coordination scoring enables the indirect conversions modules 206 to pare down the number of subject users labeled as influential subject users, which in turn pares down the number of users to which a given advertisement is presented. This results in more efficient advertising because it ensures that the few individuals to whom an advertisement is presented are the most influential.
Like the example depicted in
In the embodiment depicted in
The events on which the coordinated event scoring of
Additional criteria for event coordination scoring besides event specification may also be provided by the indirect conversions module 206 or by the third party system 103. For example, the third party system 103 may indicate that the purchase of a snowboard is only considered a coordinated event between two users if the two users in question purchase the snowboard within one month of one another. In this case, each arrow connecting two users indicates that both users purchased a snowboard within a one month period of time.
Each arrow of
Each arrow of
The embodiment in
This pattern not only exists for secondary users that are once removed from the influence of subject user Fred, but for users that are removed any plurality of times from Fred's influence. For example, Gene is three times removed from Fred's influence in the following manner: Fred influences Kim, who influences, Lucy, who influences Jimmy, who influences Gene. In this way Gene is indirectly influenced by Fred via a chain of users. In some embodiments, such indirect influence may also be included in Fred's extended influence score.
Gene, Jimmy, and Lucy all appear multiple times within the influence pyramid of
To assign a quantitative influence score to the subject user Fred, the embodiment depicted in
Furthermore, in the embodiment depicted in
Using these principles specific to the embodiment depicted in
In alternative embodiments of
As a result of including Fred's indirect influence over distant users in his influence score, Fred received a greater overall influence score compared to the same scenario in which only Fred's direct influence was accounted for. However, Fred's extended, directional influence score still remains lower than his non-directional influence score. In this way, the inclusion of indirect influence in a subject user's influence score enables calculation of a median influence score between the two extremes of purely non-directional and directional influence scores.
If this group was selected based on a single subject user and thus was not a closed connection group, only the subject user's influence score could be accurately calculated with regard to the given group. The influence scores of all other users within the group must be calculated with regard to each user's unique set of connections. In a majority of real world applications, connection groups are not closed. In other words, each subject user has a unique set of connections, and thus requires a unique set of connections. However, in some instances, it may be more desirable to find the most influential individual within a set group of individuals as in
The data chart in
In the embodiment depicted in
Take the calculation of the non-directional event coordination score between Michael and Pam as one example. Their initial non-directional event coordination score is 0. Then, as seen in
The same type of calculation is conducted for all pairs within the closed group of connections.
The method of calculating non-directional event coordination scores described in
A non-directional influence score of relatively high value indicates that the given user has a relatively great influence over the actions of the other individuals in the closed group of connections. Conversely, a non-directional influence score of relatively low value indicates that the given user has relatively little influence over the actions of the other individuals in the closed group of connections. As seen in
An advertiser may use these influence scores to determine which individuals are the most influential with regard to their group of connections. These influential individuals may then be targeted with a given advertisement in the hope that if they engage with the advertisement, their connections may also be influenced to engage with the advertisement, even if the connections themselves have not seen the ad. In one embodiment, an advertiser may choose to advertise to all individuals with a non-directional influence score greater than a threshold value of 0. In this case, in the embodiment of
If this group was selected based on a single subject user and thus was not a closed connection group, only the subject user's influence score could be accurately calculated with regard to the given group. The influence scores of all other users within the group must be calculated with regard to each user's unique set of connections. In a majority of real world applications, connection groups are not closed. In other words, each subject user has a unique set of connections, and thus requires a unique set of connections. However, in some instances, it may be more desirable to find the most influential individual within a set group of individuals as in
The data chart in
Take the hair gel purchase opportunity as an example. As seen in
The quantitative data of
In the embodiment depicted in
A relatively positive event coordination score for user A with regard to user B indicates that overall, user A influenced the actions of user B. A relatively negative event coordination score for user A with regard to user B indicates that overall, user A had no influence over the actions of user B. The event coordination score for user A with regard to user B only indicates the degree to which user A prompted user B to act similarly. It does not indicate the impact that the actions of user B had on the actions of user A. Unlike the process described with regard to the non-directional event coordination scoring of
Take the calculation of the directional event coordination score for Pam with regard to Michael as one example. Pam's initial directional event coordination score with regard to Michael is 0. Then, as seen in
Again, because the order in which users perform events is taken into account in
The method of calculating directional event coordination scores described in
A directional influence score of relatively high value indicates that the given user has a relatively great influence over the actions of the other individuals in the closed group of connections. Conversely, a directional influence score of relatively low value indicates that the given user has relatively little influence over the actions of the other individuals in the closed group of connections. As seen in
An advertiser may use these influence scores to determine which individuals are the most influential with regard to their group of connections. These influential individuals may then be targeted with a given advertisement in the hope that if they engage with the advertisement, their connections may also be influenced to engage with the advertisement, even if the connections themselves have not seen the ad. In one embodiment, an advertiser may choose to advertise to all individuals with a directional influence score greater than a threshold value of −6. In this case, in the embodiment of
By accounting for directionality of influence, the most influential members of the group may change. For example, without accounting for directionality, as shown in
The foregoing description of the embodiments has been presented for the purpose of illustration; it is not intended to be exhaustive or to limit the patent rights to the precise forms disclosed. Persons skilled in the relevant art can appreciate that many modifications and variations are possible in light of the above disclosure.
Some portions of this description describe the embodiments in terms of algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations are commonly used by those skilled in the data processing arts to convey the substance of their work effectively to others skilled in the art. These operations, while described functionally, computationally, or logically, are understood to be implemented by computer programs or equivalent electrical circuits, microcode, or the like. Furthermore, it has also proven convenient at times, to refer to these arrangements of operations as modules, without loss of generality. The described operations and their associated modules may be embodied in software, firmware, hardware, or any combinations thereof.
Any of the steps, operations, or processes described herein may be performed or implemented with one or more hardware or software modules, alone or in combination with other devices. In one embodiment, a software module is implemented with a computer program product comprising a computer-readable medium containing computer program code, which can be executed by a computer processor for performing any or all of the steps, operations, or processes described.
Embodiments may also relate to an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, and/or it may comprise a general-purpose computing device selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a non-transitory, tangible computer readable storage medium, or any type of media suitable for storing electronic instructions, which may be coupled to a computer system bus. Furthermore, any computing systems referred to in the specification may include a single processor or may be architectures employing multiple processor designs for increased computing capability.
Embodiments may also relate to a product that is produced by a computing process described herein. Such a product may comprise information resulting from a computing process, where the information is stored on a non-transitory, tangible computer readable storage medium and may include any embodiment of a computer program product or other data combination described herein.
Finally, the language used in the specification has been principally selected for readability and instructional purposes, and it may not have been selected to delineate or circumscribe the patent rights. It is therefore intended that the scope of the patent rights be limited not by this detailed description, but rather by any claims that issue on an application based hereon. Accordingly, the disclosure of the embodiments is intended to be illustrative, but not limiting, of the scope of the patent rights, which is set forth in the following claims.
