Patent Application
20200302486
The following relates generally to marketing and more specifically to a method and system for determining optimized customer touchpoints.
The generation of a marketing campaign is typically a manual process. In practice, marketing managers that are subject matter experts leverage various marketing automation tools, basic analytic tools, and marketing channels to deploy marketing campaigns to engage their customers. Some examples include a product coupon emailed to a customer, a message sent via SMS with a link to a landing page containing some advertising communication, and the pushing of relevant product or category content through a mobile application. The intent of these tactics is to attract customers to their stores, be those brick and mortar stores or online stores, and ultimately spend money on the goods and services they sell.
In order to create a marketing campaign, a goal is first identified, such as customer retention. Marketing content is generated by the marketing manager to achieve this goal using business intuition. Marketing content can be any offer or information having innate or assigned attributes that enable its matching to specific customers. It can be used to promote the goods and/or services of a company or group of companies. The marketing content can be, for example, an offer of a discount of $2 on the next purchase over $20. Once the marketing content is set, the marketing manager can use a customer relationship management (“CRM”) system to select a set of customers (i.e., a target audience) to send the marketing content to. Marketing content can also be information related or unrelated to products, such as recipes, articles, etc.
CRM systems generally enable the selection of individuals, groups, and segments of a population to receive marketing communications based on various criteria that are specified by a user. The criteria can be specified via a set of filters. Upon setting the filters, the CRM system selects a subset of the customers in a customer database that match the specified criteria. The CRM system then enables the marketing manager to push the marketing content to the subset of the customers. The marketing content is, however, quite static and is generally not very personalized for each customer.
In an aspect, there is provided a method for determining optimized customer touchpoints on a computer system, the method comprising: assigning attributes to a set of marketing content for a set of customers; generating at least one initial score for the behavior of each of the customers in the set of customers using machine learning techniques based on the attributes; selecting a first subset of the marketing content for each of a first subset of the customers to receive based at least partially on the at least one initial score; receiving feedback data relating to the first subset of the marketing content; determining at least one adjusted score, from the at least one initial score, for at least some of the customers, using reinforcement learning on the feedback data; and selecting a second subset of the marketing content for each of a second subset of the customers to receive based at least partially on the at least one adjusted score.
In a particular case, the feedback data comprises customer behavioral data of at least a portion of the customers in the first subset of the customers.
In another case, the method further comprising receiving at least a portion of the attributes from a user.
In yet another case, the machine learning techniques utilize at least one of: a predictive response model for determining a probability that a particular customer in the set of customers responds to the marketing content; a predictive uplift model for determining a probability that a particular customer in the set of customers responds to the marketing content, who in the absence of such marketing content, would not have responded; a reactive churn model for determining a change in value over a selected period of time for each customer in the set of customers; and a predictive churn model for determining a change in value over multiple periods of time for each customer in the set of customers.
In yet another case, the at least one initial score is generated using historical interaction data from the set of customers.
In yet another case, the method further comprising assigning weightings to at least a portion of the attributes based on priority of a customer objective.
In yet another case, the reinforcement learning comprises an incremental reward structure.
In a further case, the reward structure is based on increments and decrements in customer sales.
In yet another case, the reinforcement learning comprises a weighted reward structure.
In yet another case, receiving the feedback data, determining the at least one adjusted score, and selecting the second subset of the marketing content are performed repeatedly.
In another aspect, there is provided a system for determining optimized customer touchpoints, the system comprising one or more processors and a data storage device, the one or more processors configured to execute: an attributes module for assigning attributes to a set of marketing content for a set of customers; a data science module for generating at least one initial score for the behavior of each of the customers in the set of customers using machine learning techniques based on the attributes; a selection module for selecting a first subset of the marketing content for each of a first subset of the customers to receive based at least partially on the at least one initial score; and a feedback module for receiving feedback data relating to the first subset of the marketing content, the data science module further determines at least one adjusted score, from the at least one initial score, for at least some of the customers, using reinforcement learning on the feedback data, and the selection module further selects a second subset of the marketing content for each of a second subset of the customers to receive based at least partially on the at least one adjusted score.
In a particular case, the feedback data comprises customer behavioral data of at least a portion of the customers in the first subset of the customers.
In another case, the attributes module receives at least a portion of the attributes from a user via an input interface.
In yet another case, the data science module uses machine learning techniques that utilize at least one of: a predictive response model for determining a probability that a particular customer in the set of customers responds to the marketing content; a predictive uplift model for determining a probability that a particular customer in the set of customers responds to the marketing content, who in the absence of such marketing content, would not have responded; a reactive churn model for determining a change in value over a selected period of time for each customer in the set of customers; and a predictive churn model for determining a change in value over multiple periods of time for each customer in the set of customers.
In yet another case, the at least one initial score is generated by the data science module using historical interaction data from the set of customers.
In yet another case, the attributes module assigns weightings to at least a portion of the attributes based on priority of a customer objective.
In yet another case, the reinforcement learning comprises an incremental reward structure.
In a further case, the reward structure is based on increments and decrements in customer sales.
In yet another case, the reinforcement learning comprises a weighted reward structure.
In yet another case, the feedback module repeatedly receives new feedback data such that the data science module repeatedly adjusts the at least one adjusted score based on the new feedback data, and the selection module repeatedly selects the second subset of the marketing content based on the new at least one adjusted score.
These and other aspects are contemplated and described herein. It will be appreciated that the foregoing summary sets out representative aspects of systems and methods to assist skilled readers in understanding the following detailed description.
The features of the invention will become more apparent in the following detailed description in which reference is made to the appended drawings wherein:
Embodiments will now be described with reference to the figures. For simplicity and clarity of illustration, where considered appropriate, reference numerals may be repeated among the Figures to indicate corresponding or analogous elements. In addition, numerous specific details are set forth in order to provide a thorough understanding of the embodiments described herein. However, it will be understood by those of ordinary skill in the art that the embodiments described herein may be practiced without these specific details. In other instances, well-known methods, procedures and components have not been described in detail so as not to obscure the embodiments described herein. Also, the description is not to be considered as limiting the scope of the embodiments described herein.
Various terms used throughout the present description may be read and understood as follows, unless the context indicates otherwise: “or” as used throughout is inclusive, as though written “and/or”; singular articles and pronouns as used throughout include their plural forms, and vice versa; similarly, gendered pronouns include their counterpart pronouns so that pronouns should not be understood as limiting anything described herein to use, implementation, performance, etc. by a single gender; “exemplary” should be understood as “illustrative” or “exemplifying” and not necessarily as “preferred” over other embodiments. Further definitions for terms may be set out herein; these may apply to prior and subsequent instances of those terms, as will be understood from a reading of the present description.
Any module, unit, component, server, computer, terminal, engine or device exemplified herein that executes instructions may include or otherwise have access to computer readable media such as storage media, computer storage media, or data storage devices (removable and/or non-removable) such as, for example, magnetic disks, optical disks, or tape. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. Examples of computer storage media include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by an application, module, or both. Any such computer storage media may be part of the device or accessible or connectable thereto. Further, unless the context clearly indicates otherwise, any processor or controller set out herein may be implemented as a singular processor or as a plurality of processors. The plurality of processors may be arrayed or distributed, and any processing function referred to herein may be carried out by one or by a plurality of processors, even though a single processor may be exemplified. Any method, application or module herein described may be implemented using computer readable/executable instructions that may be stored or otherwise held by such computer readable media and executed by the one or more processors.
The following relates generally to marketing and more specifically to a method and a system for determining optimized customer touchpoints. The method and system provides a method by which custom marketing campaigns can be run in an automated manner using customer data.
Referring now to
The system 20 enables the creation of personalized marketing content that is selected for and delivered to a subset of customers, thereby providing them bespoke experiences as a replacement for traditional direct marketing methods. Through the interaction with digitized point-of-sale machines, loyalty programs, digital communication channels, and various other means, a vast amount of customer data is being generated for each customer. The various customer data is collected mainly through PoS terminals, recurring billing (in the case of contractual services), and e-commerce websites of the transactional variety, and through other means such as market research, 3rd party aggregation, and other data resellers and brokers. The system 20 draws upon this customer data from the various channels in order to better understand the behavior of each customer. The system 20 mines this data and applies statistical and machine learning approaches to yield recommendations and actions that enable deeply personalized experiences.
The solution provided by the system 20 is one that is personalized. Traditional CRM systems marketing enable a marketing campaign to be targeted at customers to which it may be relevant. For example, upon the generation of a promotional email regarding certain products, traditional CRM systems enable the selection of which customers the promotion may be relevant to using marketing intuition or other simplified heuristics. Selection of a group of customers to send the promotional email to can be determined by, for example, ranking customers in order of highest spenders, and providing the top 10% of that list a global discount. In contrast, the system 20 customizes marketing campaigns in an automated manner by controlling the timing of delivery, preferred channel of communication, discounting, taking into consideration the customers share of wallet, and a slew of other aspects of the customer experience. In doing so, the system 20 approaches marketing in a holistic, real-time manner.
It uses a combination of constrained optimization, prediction, and reinforcement learning to directly optimize for key financial objectives (e.g., increased customer value over time, incremental sales in a short period, reduced customer churn, etc.). It leverages predictions about customers and other latent behaviors and selects the right mix of marketing content (that is, offers and other content in order to achieve specified business objectives given certain inputs and constraints. Still further, the system 100 simplifies the process by providing an intuitive interface through which to provide the inputs.
The marketing platform includes a management console accessible via a web interface generated by the web server. The management console enables a marketing manager to define marketing campaigns and view results via a campaign editor, delegate tasks for a campaign to other people, enable notifications such as via email for milestones in the process of defining a campaign, define/upload marketing content, directly integrate the system with marketing automation and execution tools, select a creative template for a campaign based on historical performance of the creative template, etc. During marketing campaign creation, the campaign editor enables the marketing manager to specify the following inputs and constraints:
In addition, the second campaign definition screen 200 also shows projected results for the campaign in a right panel. The right panel enables the visualization of various elements of the system 20 itself. As a user provides inputs to the attributes module 90 via the input interface 68, such as customer objectives, customer quantities, strategic focus selections, the number and allocation of hero and product offers, the system 20 weighs these different inputs and adjusts the model selection accordingly. As an example, if “win-back” is selected as the primary customer objective, the system 20 tends to over-index the selection of richer offers from the offer and action pool.
Hero offers are offers or content that are relatively generic in nature, such as discounts at a total store or category level (e.g., twice the points when you purchase any skincare product, or 10% off when you purchase anything). Product offers are offers that are narrower in scope (e.g., $1 off of a specific box of cereal, 5% off of a narrow range of a particular brand of lipsticks, etc.).
The marketing content pool contains a finite collection of marketing content templates that are available to be used via the marketing platform. An example of marketing content could be an offer, such as: spend $80 on skincare products between Jan. 1, 2016 and Jan. 10, 2016 and receive a 25% discount. The attributes in this example can be broken down into their constituent components:
The offer and action pool can also include content or advice. In the grocery space, for example, the offer and action pool can contain recipes. These recipes can be tagged or labeled based on the specific products they use or foods they create. The system 20 can take the recipes and assign them on a one-to-one basis to customers.
The data science module 96 is a set of data mining and machine learning building blocks that generate predictive or explanatory scores about customers. Both supervised and unsupervised learning techniques can be used in generating the scores. These scores are then used by the selection module 94 to match customers to marketing content. The customer data used to generate these scores is a mix of historical point of sale data, other interaction data and various other data sources, and is stored in the customer database 84. The machine learning techniques used to generate these building blocks are specific to the particular building block. For example, a predictive response model can leverage supervised classification techniques. And, by virtue of the specific conditions of what is being predicted, there are naturally various versions of each building block. The purpose of each version is specific to its use case and the context in which it is actually deployed. The engine leverages a standardized naming convention to determine which model version in each building block should be used.
The following are a number of illustrative building blocks used by the data science module 96:
The selection module 94 determines the best mode of achieving the goals of the marketing campaign specified via the campaign editor. In particular, the selection module 94 takes the inputs and constraints received via the campaign editor, the outputted scores for the building blocks in the data science module 96, and the possible marketing content in the marketing content pool, and combines them to generate an output list of what a customer should receive. The various building blocks are assigned weightings and combined with each other to map to specific marketing content. For example, if “win-back” is the highest priority customer objective, a customer's churn score is weighted heavily and heavy negative weightings map very often to marketing content in the marketing content pool with higher incentives.
The selection module 94 is instantiated for each brand new set of data it receives. Prior customer data can be imported into the system to enable the system 20 to benefit from prior experience. In addition, inputs provided impact the assigned weightings of the building blocks of the data science module 96. This instantiation step is a mix of constrained optimization and conditional rules. The reason the system 20 is instantiated is because it doesn't have a base of decisions from which to learn what maximizes the objectives. There needs to be a base of matches for the reinforcement learning capability to have enough data to actually learn.
Beyond instantiation, the steady state of the marketing platform shifts over to a reinforcement learning hybrid approach. As further customer data, in the form of feedback data, is collected by the feedback module 92, the building blocks are re-trained and re-scored by the data science module 96, and, as a result, new predictions are made for customers by the selection module 94.
The example below illustrates a particular use case for an example data set and will be discussed with reference to the general method 300 of
At 320, the following received user inputs/constraints are captured as exemplified as follows:
At block 330, customers are then scored using the data science module 96. In one particular embodiment, this is performed as follows:
At block 350, the system 20 then performs reinforcement learning and feedback. Once instantiated, the system 20 can be invoked repeatedly to generate personalized touchpoints for customers. As this process continues, various iterations occur whereby customers are contacted, marketing content is deployed, actions are taken and behavioral data captured by the feedback module 92. The initial weighting system in the instantiation process begins to give way to a machine intelligent approach to matching. The machine learning technique used prioritizes online learning in an environment whose outcomes can be considered as being partly due to randomness and partly due to phenomena under the control of a decision maker. For example, while personalized, lucrative offers to perform a certain action have undoubted causal relationship to the action, there are other random factors which may have contributed to that action.
In an embodiment, the reinforcement learning framework operates as follows: each customer-to-offer match the machine creates that leads to a positive action (defined by positive incremental sales above a certain threshold) is rewarded a +1 whereas each match that leads to a negative action (incremental below a certain threshold) is penalized by −1. This reward structure then impacts the instantiated rules by scaling them up or down. In the example above, if a particular threshold offer was recommended to a customer and that offer was not redeemed, that selection gets penalized and another threshold is tried the next time. This is repeated until the threshold that maximizes the objective (activation) is achieved and rewarded.
The reinforcement learning agent can be expressed as a policy π(mt+1|{mt, ot, rt}) where mi is the match (e.g., the offer presented to a customer) performed to customer i at time t, oi is the observation based on that match and ri is the reward the agent receives for performing that match. So, in the formulaic expression, the policy is defined as selection of a match at time t+1, given the entire history of matches, observations and rewards up to time t. Thus, a single customer's interaction history at time t can be expressed as htC=1={mi, ri, mi+1, ri+1, . . . mt, ot, rt}C=1 where C=1 denotes customer 1. As each customer has their own interaction history, if there are N customers, there are N interaction histories. The policy described takes not only a single customer's interaction history to determine the optimal match to perform, but rather the entire set of interaction histories. Furthermore, the policy is selected in a greedy way, such that it selects the match which maximizes the observation.
In embodiments, the components of the system are stored by and executed on a single computer system. In other embodiments, the components of the system 20 are distributed among two or more computer systems that may be locally or globally distributed.
The system can be seeded with historical data from which inferences can be drawn, enabling reinforcement learning to be employed using the historical data prior to the collection of further customer data by the system. That is, where previous interactions or touchpoints between the client and their customer base exist and that data is readily available and interpretable, the instantiation phase can be skipped entirely in favor of implementing the reinforcement learning embodiment.
Incremental sales generated by a particular match are not the only observations that can generate a reward. An example of an alternative reward-generating observation is a change in the customer's reactive churn score, or simply the redemption/usage of the offer sent. Thus, the reinforcement learning embodiment is built in such a way as enable various definitions of both observations and rewards.
The system can be instantiated in accordance with the client or user needs. For example, requisite data to generate all the building blocks may not exist, or there may be a set of customers that the client wishes to exclude from contacting, or a set of products that the customer wishes never to promote. Such case-by-case situations can be dealt with when the instantiated matching rules are generated.
The system 20 can change or re-train the models with which the scores themselves are being calculated.
The system 20 can perform reinforcement learning “concurrently” with the receiving of customer data via various channels, enabling the system 20 to continue to match customers to marketing content and learn new weightings.
The reward structure (+1 for good actions, −1 for bad actions) used by the system 20 for reinforcement learning is not the only way it can learn to optimize its operation. The system 20 can alternatively include a weighted reward that is a function of the metric the system 20 is trying to maximize. That is, the system can over-reward for making good decisions but reduce the punishment for bad decisions.
Turning to
While in most cases, the reinforcement learning agent is set to select the greediest action that maximizes the observation value (e.g., incremental sales, customer lifetime value, etc.), it can similarly be set to select another action with a slightly lower reward value based on other competing parameters.
The reinforcement learning agent can update its base of interaction histories in real-time.
The listed data science building blocks are a subset of the full set of all data science building blocks that can be used. Data science building blocks may be added, removed or under-weighted at any time based on user requests or other issues arising from the operation of the engine.
While, in most cases, it is assumed that a customer's behavior is not impacted by interaction with other customers (i.e., customer behavior is identically distributed and conditionally independent), other modes of the system 20 can be employed where this is not necessarily true.
Although the reinforcement learning element of the system 20 makes no assumptions about the “latent” state of a customer, in that the agent itself assumes that all knowledge about a customer is contained within the set of interaction histories, those interactions are composed of matches based on the data science building blocks. Some of these building blocks do, however, model aspects of a customer that are latent (such as their wants and needs for particular goods or services). For example, the product affinity model tries to infer latent features of a customer based on the products they and other customers purchase.
With enough interaction history, the system 20 operating using the reinforcement learning embodiment post-instantiation can be considered an artificially intelligent agent.
The space of inputs can be changed and customized depending on the particular use case. For example, a retailer may want inputs to allow controls on the number of offers and the product categories to focus on, whereas a financial services firm may choose to allow controls on the term or length of contractual or securitized loan agreements.
The system permits the real-time adjustments of the scores using real-time customer data received. Further, the real-time adjustments can be used to adjust the selection of customers to which marketing content is selected, as well as the marketing content made to each customer, in real time.
The system 20 can be thought of as being channel-agnostic. In this sense, as new channels are deployed, the system 20 can accommodate these new channels immediately.
The front-end tool is not necessary for the capture of inputs. Inputs can be provided through alternate modes, such as flat files, etc.
Similarly, inputs can be automated in the form of “triggers”. For example, when a customer performs an action (e.g., visits a website, engages with an in-store associate equipped with a smart device, etc.), preset inputs can be used to generate a touchpoint for the customer.
Although the invention has been described with reference to certain specific embodiments, various modifications thereof will be apparent to those skilled in the art without departing from the spirit and scope of the invention as outlined in the claims appended hereto. The entire disclosures of all references recited above are incorporated herein by reference.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/CA2017/050383 | 3/28/2017 | WO | 00 |
