Patent Application
20200327577
The disclosed embodiments generally relate to systems and methods for targeted email marketing.
A/B testing is widely applied in email marketing campaigns, as well as other advertising activities, to compare performances between two different email advertising variants. By randomly sending two versions of an email advertisement that differ in email advertisement design and/or content, A/B testing can determine the relative efficacy of the two versions. A strategy for conducting the email marketing campaign can thereby be determined.
However, this approach can be a time consuming process that limits the speed of adjusting an email advertising strategy, especially when there are multiple candidate versions of email advertisements to be tested. In addition, A/B testing may require significant manual effort for evaluation and adjustment of candidate email advertisements, which may not only result in high labor cost, but may also cause bias and low accuracy in determining efficacy. Further, A/B test only provides a one-size-fits-all strategy, which may not be the best solution for a diverse customer base.
Thus, there is a need for systems and methods capable of developing an effective email marketing campaign with greater efficiency and flexibility.
In the following description, certain aspects and embodiments of the present disclosure will become evident. It should be understood that the disclosure, in its broadest sense, could be practiced without having one or more features of these aspects and embodiments. It should also be understood that these aspects and embodiments are merely exemplary.
The disclosed embodiments include a system for targeted email marketing. In some embodiments, the system includes one or more processors, and a memory storing instructions. When executed by the one or more processors, the instructions cause the system to perform operations including: generating email templates respectively corresponding to different customer marketing communications; sending first emails to first customers, in which each of the plurality of first emails includes one of the plurality of email templates in accordance with an initial allocation; compiling response information from the first customers and updating a bandit model in accordance with the response information, the bandit model for determining an allocation of the email templates to customers; determining, by the bandit model, a revised allocation of the email templates to second emails; and sending the second emails to second customers in accordance with the revised allocation.
The disclosed embodiments also include a method for targeted email marketing. The method includes: generating a plurality of email templates respectively corresponding to different customer marketing communications; determining a number of customers for receiving each email template in accordance with an initial allocation, and sending a plurality of first emails to a plurality of first customers, each of the first emails including one of the email templates in accordance with the initial allocation; compiling information about responses of the first customers to the first emails; determining a revised allocation of the email templates by a bandit model in accordance with the response information from the plurality of first customers to the first emails; and determining a number of customers for receiving each of the email templates in accordance with the revised allocation of the email templates, and sending a plurality of second emails to a plurality of second customers.
The disclosed embodiments further include a non-transitory computer-readable medium, storing instructions that are executable by one or more processors to perform a method for targeted email marketing. The method includes: generating a plurality of email templates respectively corresponding to different customer marketing communications; determining a number of customers to receive each of the plurality of email templates in accordance with an allocation, and sending a plurality of emails to a plurality of customers, in accordance with the allocation, intermittently in a plurality of cycles; and in each of the plurality of cycles, updating the allocation of the plurality of email templates to the numbers of customers for a next cycle, by a bandit model, in accordance with responses of the plurality of customers receiving the plurality of emails in a previous cycle.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only, and are not restrictive of the disclosed embodiments, as claimed.
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate several embodiments and, together with the description, serve to explain the disclosed principles. In the drawings:
Traditionally, A/B testing can be applied to provide a comparison between two different “creatives” (A and B) sent in an email marketing campaign to determine which creative has a better performance in the campaign. In addition to the A/B testing, design of experiments (DOE) may also be applied to determine what combination of imagery, wording, and/or positioning works better with customers. For example, by implementing DOE, two or more variants can be provided in the campaign at the same time during a test. Contribution of each part (e.g., the imagery, wording, and/or positioning) to customer responses can be analyzed and estimated after the test. Thus, an optimal design with the combination that is predicted to deliver a high response can only be rolled out in a following stage of the campaign. As used herein, a “creative” refers to content for representing an advertisement, for example, images, characters, videos, and/or sounds for inclusion in an email advertisement.
As used herein, the terms “users” and “customers” are used interchangeably to refer to prospective recipients of email advertisements. Further, “customers” may refer to either of existing or potential customers.
Performance of a creative during testing may be assessed in accordance with performance criteria such as a number of clicks on, responses to, or sales resulting from an email advertisement containing a particular creative. Other criteria may also be used to determine which version, or variant, of a creative achieves a better performance. For example, the performance of the creative may be measured by a responding rate or a click through rate, e.g., the ratio of users who click on a link to the number of total users who view an email, to determine which creative is more effective and/or more profitable.
Typically, A/B testing includes randomly distributing different creatives via email in an initial phase to customers on a mailing list, waiting for feedback results of the testing to be collected from customers, manually evaluating the testing results to determine which creative provides a successful customer experience or set of experiences, based on performance criteria such as described above, and then adjusting the creatives to be sent in a next round of the campaign, in order to send a “winning” creative, i.e., the creative that resulted in the most successful customer experience, to recipients in the mailing list in a next phase.
However, in a manual evaluation of A/B testing, it is difficult for an evaluator to precisely evaluate how the content of the creative, and/or design of a particular segment in the creative, affect the creative performance. That is, actual factors in the winning creative that resulted in the successful customer experience cannot be identified. In addition, a lengthy wait for results in A/B testing may cause a significant delay in making adjustments. Email marketing, by nature, relies on sending content and receiving feedback, and, as a result, is time consuming because of the need to wait until a sample size is large enough to determine the winning creative. In some cases, the process of waiting for feedback from customers may even take several months, which causes high opportunity costs, since many creatives with lower performance and the single winning creative may be randomly distributed among email advertisements sent to customers in this lengthy process.
Moreover, existing processes for adjusting creatives are manual in scope, which means an administrator is involved in the processes of designing creatives, analyzing feedback results, and making judgments to determine the winning creative. The manual process leads to inefficiencies and inaccurate variants being sent to customers. Furthermore, since only one global winning creative is ultimately identified based on an entire population of customers, different needs of different customers are not taken into consideration. That is, the emails sent to customers are not personalized based on characteristics of the receptive customers, which may hinder improvement in advertising performance, since many potential target customers receive email advertisements based on suboptimal creatives.
In some embodiments of the present disclosure, improved methods for targeted email marketing are provided that include a contextual multi-armed bandit engine to identify user segments and strategically find relationships between creatives and different customer segments. Thus, when creatives are added to, or removed from, a marketing campaign conducted using systems and methods consistent with the present disclosure, a determination is made as to which customers are most responsive to which creatives and, based on the determination, personalized marketing emails are sent to customers in an automated process. Furthermore, the contextual multi-armed bandit engine is self-updated in each marketing campaign cycle, and learns from historical data collected from customers in previous cycles, which brings dynamic and real-time response adjustments to the campaign.
Thus, the disclosed systems and methods overcome problems of traditional A/B testing in email marketing practice, such as time consuming, costly manual processing and lack of customer personalization. Accordingly, a marketing campaign utilizing the improved systems and methods disclosed herein can be more successful, with quick adaptability, automated processing, and personalization of marketing content. The following disclosure provides exemplary systems and methods for targeted email marketing for realizing at least some of the above advantages and benefits over conventional systems.
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings and disclosed herein. Wherever convenient, the same reference numbers will be used throughout the drawings to refer to the same or like parts.
Any one of the emails sent by system 100 may include one corresponding “creative,” but the present disclosure is not limited thereto. In some embodiments, an email sent by system 100 may include multiple corresponding creatives associated with advertisements in different categories. For example, an email can include a corresponding creative associated with credit cards, a creative associated with car loans, and a creative associated with saving accounts. Different creatives have creative IDs for identification in system 100. As shown in
The lower portion of
In various embodiments of the present disclosure, system 100 applies an approach driven by a machine learning algorithm to automate the email marketing process and provide personalized creatives and/or email content by building a self-updated model which learns from customer feedback. More particularly, system 100 applies a bandit algorithm, such as a contextual bandit algorithm. The contextual bandit algorithm is an algorithm for solving a contextual multi-armed bandit problem. Information about the customers and the creatives is referred to herein as the “context” for the contextual bandit algorithm.
A multi-armed bandit problem, sometimes also called an N-armed bandit problem, is a problem in which a fixed limited set of resources are allocated between competing or alternative choices, i.e., multiple “arms” of bandits, in a way that maximizes long term expected gain for the resources. Properties of the choices are partially known at the time of allocation of the resource, and may become further understood by allocating resources to the choice. The multi-armed bandit problem models an agent, i.e., a bandit model, that simultaneously attempts to acquire new knowledge, which is also called “exploration”, and to optimize decisions based on existing knowledge, which is called “exploitation.” The agent attempts to balance the exploration and the exploitation in order to maximize a total value over a period of time considered.
A contextual multi-armed bandit problem is a multi-armed bandit problem in which an agent analyzes a multi-dimensional feature vector, i.e., a context vector, associated with a current iteration before choosing between arms in each iteration. A contextual multi-armed bandit model uses these context vectors along with rewards of the arms played in the past to make the choice of the arm to play in the current iteration. Over time, the contextual multi-armed bandit model's aim is to collect enough information about how context vectors and rewards relate to each other, so that choices can be made by analyzing the context vectors.
The following publications include information concerning the bandit model, the bandit algorithm, the multi-armed bandit and the contextual multi-armed bandit, and are incorporated by reference herein as if fully described herein: Russo, D. J., Van Roy, B., Kazerouni, A., Osband, I., & Wen, Z, 2018, “A tutorial on Thompson sampling,” Foundations and Trends in Machine Learning, Vol. 11: No. 1, pp 1-96; Agarwal, A., Hsu, D., Kale, S., Langford, J., Li, L., & Schapire, R, 2014, “Taming the monster: A fast and simple algorithm for contextual bandits,” International Conference on Machine Learning (pp. 1638-1646); and Chapelle, O., Manavoglu, E., & Rosales, R., 2015, “Simple and scalable response prediction for display advertising,” ACM Transactions on Intelligent Systems and Technology (TIST), 5(4), 61.
In some embodiments, system 100 builds the bandit model by using the contextual bandit algorithm, an extreme gradient boosting algorithm, a regression algorithm, or a combination thereof, but the present disclosure is not limited thereto. For example, system 100 may operate a contextual bandit API that offers an accessible optimization tool based on the contextual bandit algorithm to build the bandit model, which is configured to determine the allocation of creatives to be included in emails sent to customers. Operations regarding the contextual bandit API will be discussed in detail below with an accompanying figure. Those skilled in the art can understand how to build various bandit models using an appropriate algorithm, and further explanation is omitted herein for the sake of brevity.
In applying the contextual bandit algorithm to email marketing herein, the performance of competing creatives is unclear before testing. By sending creatives and recording responses from customers, the contextual bandit algorithm can update the bandit model built by the contextual multi-armed algorithm to optimize the allocation of the creatives per email drop, i.e., each cycle of sending emails in the marketing campaign. Thus, system 100 may determine what creative and/or email content to be distributed to which segment of customers based on the updated model in a next email drop.
Accordingly, system 100 can effect a prompt adjustment to automatically re-allocate the distribution of creatives and reduce time and cost. For example, in a marketing campaign that sends emails on a daily basis, the bandit model may be updated in each email cycle, and the distribution of creatives may be re-allocated day-to-day. Thus, the bandit model may be trained and refined within a relatively short period, such as one week (e.g., 7 email cycle). Thus, comparing to A/B testing, system 100 provides a quick response and flexibility. In addition, system 100 also optimizes use of creatives by leveraging available customer data to match the email recipient with the creative likely to garner a favorable response, in order to maximize the advertising efficiency in the marketing campaign.
As shown in
System 100 further includes one or more storage device(s) 130. Storage device(s) 130 can be various computer-readable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Storage device(s) 130 can be communicatively coupled with processor(s) 120 via bus 110. In some embodiments, storage device(s) 130 may include a main memory, which can be used for storing temporary variables or other intermediate information during execution of instructions by processor(s) 120. Such instructions, after being stored in non-transitory storage media accessible to processor(s) 120, render system 100 into a special-purpose machine that is customized to perform operations specified in the instructions. The term “non-transitory media” as used herein refers to any non-transitory media storing data or instructions that cause a machine to operate in a specific fashion. Such non-transitory media can include non-volatile media and/or volatile media. Non-transitory media include, for example, optical or magnetic disks, dynamic memory, a floppy disk, a flexible disk, hard disk, solid state drive, magnetic cassettes, magnetic tape, or any other magnetic data storage medium, a CD-ROM, digital versatile disks (DVD) or any other optical data storage medium, any physical medium with patterns of holes, a Random Access Memory (RAM), a read-only memory (ROM), a Programmable Read-Only Memory (PROM), a EPROM, a FLASH-EPROM, NVRAM, flash memory, or other memory technology and/or any other storage medium with the same functionality that can be contemplated by persons of ordinary skill in the art to which this disclosure pertains.
In the embodiments consistent with system 100 shown in
In some embodiments, system 100 may also include other removable/non-removable, volatile/nonvolatile computer media. By way of example,
Hard disk 140 and storage device(s) 130 may further include client data, which may include information about particular customers of the financial service provider, such as client account information, debit or credit card information, history of purchase or payment transactions, financial statements, and one or more transaction rules and location information according to the disclosed embodiments. Client data may include a data record associating a client account with one or more other accounts according to the one or more transaction rules. Client data may further contain one or more user profiles, including information such as a credit score, an income value, an inquiry number, an annual application number corresponding to individual client accounts. In some embodiments, client data may be stored in a database, which may be an external storage device in communication with system 100 via a network 150 or any other suitable network.
System 100 can transmit data to and communicate with other servers, send emails, and receive response from client terminals through network 150. Network 150 may be a local network, an internet service provider, internet, or any combination thereof. A communication interface 160 of system 100 is connected to network 150. In addition, system 100 can be coupled via bus 110 to one or more peripheral devices 170, which includes displays (e.g., cathode ray tube (CRT), liquid crystal display (LCD), touch screen, etc.), input devices (e.g., keyboard, mouse, soft keypad, etc.), or other output devices (e.g., speaker, headphone, etc.).
In some other embodiments, system 100 can also be implemented using customized hard-wired logic, one or more ASICs or FPGAs, firmware, or program logic that in combination with the server causes system 100 to be a special-purpose machine.
System 100 may include or may access one or more storage device(s) 130 configured to store data and/or software instructions used by processor(s) 120 to perform operations consistent with the disclosed embodiments. For example, processor(s) 120 can be configured to execute a set of instructions stored in the storage device(s) 130 to cause system 100 to perform a method for targeted email marketing, in order to send emails including different creatives to different customer segments to achieve the creative optimization mentioned above.
Various forms of media can be involved in storing one or more sequences of one or more instructions for processor(s) 120 to execute. For example, the instructions can initially be stored on a magnetic disk or solid-state drive of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to system 100 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal. An infra-red detector can receive the data carried in the infra-red signal and appropriate circuitry can place the data on bus 110. Bus 110 carries the data to the main memory within storage device(s) 130, from which processor(s) 120 retrieves and executes the instructions.
The disclosed embodiments are not limited to separate programs or computers configured to perform dedicated tasks. For example, storage device(s) 130 may store a single program or multiple programs. Additionally, system 100 may execute one or more programs located remotely from system 100. For example, system 100 may access one or more remote programs stored in a remote component (such as a database) that, when executed, perform operations consistent with the disclosed embodiments. In addition, other components known to one of ordinary skill in the art may be included in system 100 to process, transmit, provide, and receive information consistent with the disclosed embodiments.
By executing the instructions and/or programs stated above, system 100 may divide targeted customers in a current cycle into different segments (e.g., segments S1, S2, S3, and S4) to receive emails 210, 220, 230, 240 respectively including creatives C1, C2, C3 and C4. Based on the feedback information from the email recipients in the current cycle, system 100 may accordingly update the allocation of creatives C1, C2, C3 and C4 to be distributed among targeted customers in a next email cycle. In some embodiments, system 100 may only send the email once to a specific customer during the whole email marketing campaign. That is, a customer will not be in recipient lists in two different cycles. In other embodiments, system 100 may target the same customer in different cycles, and send multiple emails to the same customers in respective cycles, and these email may include the same or different creatives. In addition, in some embodiments, when one or more new creatives are designed and added in the marketing campaign, system 100 may perform a similar process and determine a new allocation to distribute the existing creatives and the newly introduced creatives respectively to targeted customers. Moreover, in some embodiments, system 100 may be configured to design or suggest new creatives, or remove existing creatives from the marketing campaign.
At step 310, system 100 generates a plurality of email templates (e.g., including creatives C1, C2, C3 and C4 in
At step 320, in a first email cycle, system 100 sends first emails (e.g., emails 210, 220, 230, and 240 in
At step 330, system 100 determines a number of customers for receiving each corresponding email template in accordance with the allocation. For example, system 100 may determine to send 300,000 emails in total in the first email cycle. If the initial allocation is initialized as a uniform allocation, for each of the four creatives, the system 100 may select 75,000 customers in the mailing list as the recipients to receive corresponding advertising emails. In various embodiments, system 100 may determine the recipients in different ways. In some examples, system 100 may simply select recipients randomly. In some other examples, system 100 may select recipients based on customer characteristics (e.g., initial or preliminary segments), in order to ensure the diversity of recipients receiving the same creative.
After determining the number of customers, at step 340, system 100 sends emails to corresponding customers, also referred to herein as first customers based on receiving emails in the first email cycle. Thus, the first customers targeted in the first email cycle each receives an email which includes one of the four creatives. It is also noted that in some other embodiments, one email template may include two or more creatives, and the embodiments discussed here are merely examples and not meant to limit the disclosure.
At step 350, after the emails are sent out, system 100 compiles response information from the first customers who receives the emails and updates a bandit model in accordance with the response information. In some embodiments, the response information includes customer characteristics, content characteristics corresponding to the email templates, and performance criteria, which are discussed in detail below.
For further understanding of the operation of step 350, reference is made to
At step 351, system 100 obtains one or more customer characteristics corresponding to respective ones of the customers, and one or more content characteristics corresponding to the email templates of the emails received by the respective ones of the customers. More particularly, processor(s) 120 may retrieve customer characteristic(s) and/or content characteristic(s) from storage device(s) 130 or an external database containing such information via wired or wireless network, e.g., network 150. For example, the customer characteristics may include customer information related to financial activities such as a credit score (e.g., a FICO score), an income amount, a number of credit inquiries, an annual number of credit applications, or a combination thereof. In some embodiments, the customer characteristics may also include personal information such as gender, age, occupation, but the present disclosure is not limited thereto. In some cases, system 100 may obtain a large number of characteristics for a customer.
At step 352, system 100 determines, by the bandit model, a relationship between the one or more customer characteristics and the corresponding one or more content characteristics of the email templates. For instance, system 100 provides the customer characteristics and content characteristics to the contextual bandit model and applies the bandit algorithm to correlate the customers and a set of “arms” (e.g., email templates including creatives C1, C2, C3 and C4) with their corresponding feature vectors. The feature vector, also referred to as a context vector, is configured to summarize information of the customer and the creative received by the particular customer.
At step 353, system 100 updates the bandit model in accordance with the relationship, or correlation, between customer characteristic(s) and content characteristic(s) determined in step 352. More particularly, the bandit algorithm improves its arm-selection strategy based on correlations between the customer characteristics with the content characteristics and the performance criteria. For example, the performance criteria may include a click through rate (CTR) or a responding rate of the advertising content in the email. The performance criteria are referred to as the “payoff” in the bandit algorithm. In some embodiments, the expected payoff of the email templates is the click through rate. That is, choosing an email template with maximum CTR is equivalent to maximizing the expected number of clicks from customers, and maximizing the total expected payoff in the bandit formulation. Thus, by the above operations, system 100 can update the bandit model in accordance with the response information from the customers who receives the emails in this cycle.
Returning to
At step 361, system 100 obtains the one or more customer characteristics of respective ones of second customers who are targeted as recipients in a next, here second, email cycle. For example, system 100 may review characteristics, such as credit scores, incomes, ages, etc., of the second customers.
Then, at step 362, for respective ones of the second customers, system 100 estimates, by the bandit model, predicted responses corresponding to the email templates in accordance with the one or more customer characteristics. More particularly, in some embodiments, system 100 may provide the obtained customer characteristics associated with a specific customer to the bandit model. Accordingly, based on the customer characteristic(s), the bandit model makes an estimation regarding predicted responses of the specific customer to the email templates. Then, at step 363, system 100 selects one of the email templates for each of the second customers based on the predicted responses. For example, based on the observation in the previous cycle, the bandit algorithm may predict that customers having a high credit score and high income are more responsive to a specific creative (e.g., creative C1) compared to other creatives. Accordingly, if the characteristics of a customer targeted in the next cycle match the corresponding credit score and income criteria, system 100 selects the email template including the specific creative (e.g., creative C1) to include in the email sent to the customer.
In other words, system 100 can use a set of informative features to describe customers, email templates, and their relationship. Thus, system 100 may infer a target customer's interest based on his or her personal information, financial information, or any other information accessible, to choose a creative or an email template that is interesting or meets his or her need. By the above operations, system 100 can determine a revised allocation of the email templates. After determining the revised allocation applied in the second cycle, system 100 sends second emails respectively to second customers in accordance with the revised allocation. It is noted that, the allocation of the email templates may be further revised in following cycles based on the bandit model updated in each cycle. That is, new predictions about the customer preferences can be made in following cycle(s), e.g., a third or a fourth cycle, and not limited to the predictions made in previous cycle(s). Accordingly, before a “winner” is identified, system 100 performs exploration and exploitation at the same time.
In some embodiments, system 100 may further identify the “winner” from the candidates by the bandit algorithm. At step 370, system 100 determines if an optimized email template is identified corresponding to one or more customer characteristics among the email templates in accordance with the revised allocation and the response information. In other words. system 100 determines whether an optimized creative, also known as a “winner” choice with the best rewards in multi-armed bandit problem, is found. If the winner choice is not identified by the bandit algorithm, system 100 repeats steps 330-370, in order to collect more data and update the bandit model accordingly in following email cycles. Thus, the bandit model learns from the results in each round, and continually updates the customer segmentation and creative allocation in an automated manner. Customer segmentation refers to the process of dividing customers based on customer characteristics and/or customer responses to respective email templates. That is, by collecting feedback from customers, the bandit model can divide and/or reallocate customers into groups, i.e., “segments,” that can be targeted more accurately.
On the other hand, if the bandit algorithm identifies the optimized email template, step 380 is performed. At step 380, in a third email cycle, system 100 sends third emails with the optimized email template to third customers corresponding to the one or more customer characteristics after the optimized template is identified. For each customer segment, the optimized email template can be identified independently. Alternatively stated, a creative (e.g., creative C1) may be included in the optimized email template for one customer segment, and another creative (e.g., creative C2) may be included in the optimized email template for another customer segment.
In some embodiments, system 100 may identify the optimized email template in accordance with the response information after a predetermined number of iterations of sending emails, compiling response information, and revising allocation of the email templates. For example, system 100 may repeat steps 330-370 for four email cycles. After four email cycles, system 100 identifies the “winner,” i.e., the optimized email template for each customer segment by the bandit algorithm. Thus, in the following email cycles, system 100 may execute exploitation to maximize the click through rate. In some other embodiments, system 100 identifies the “winner,” i.e., the optimized email template if one creative outperforms other creatives for the customer segment by a threshold value. For example, if a difference between the click through rate of a leading creative (e.g., creative C1) and of other creatives is greater than 2%, the leading creative may be identified as the winner. However, other criteria can be applied to determine the “winner” based on requirements for, and relative performance of, creatives in a particular situation.
The above described processes may be implemented as a computer program or application or as a plugin module or sub component of another application. Some of the described processes may be executed by system 100 in
Accordingly, by performing method 300 described above, the bandit model learns and improves its understanding about how customer characteristics affect preference for different advertising elements, i.e., creatives, in email cycles. Thus, system 100 can predict the preferences according to customer characteristics, and send out emails including the creative related to the needs of each recipient, in order to achieve creative optimization and improve advertising efficiency in the email marketing campaign.
Assuming the collected data is large enough, the bandit model is able to provide accurate predictions. Size of the collected data may vary based on different scales of different email marketing campaigns. In one example, the bandit model may provide accurate predictions after about 70,000 emails are sent, but the present disclosure is not limited thereto. The number of emails sent in one email marketing campaign for updating the bandit model can vary from thousands to millions. Accordingly, based on the observation, new creatives can be generated and existing creatives with poor feedback can be deleted.
As shown in
After customers 714 receive emails 712 including the corresponding creatives, system 100 receives response information 716 to first emails 712 from first customers 714 and determines a revised allocation 718 of the email templates by a contextual bandit model 720 based on received response information 716.
More particularly, the response information 716 may be received by the email marketing platform in real-time and batched as input for contextual bandit model 720 in a next cycle. System 100 calls the contextual bandit API and feeds the response information 716 to contextual bandit model 720. Contextual bandit model 720 compiles the batched response information 716 about responses of first customers 714 to first emails 712 and updates the contextual bandit model 720 based on the compiled results accordingly, consistent with method 300 in
The updated contextual bandit model 720 is configured to apply the response information from the customers 714 in the bandit model to refine predictions and determine revised allocation 718. As shown in
In some embodiments, contextual bandit model 720 determines the revised allocations respectively corresponding to the segments of customers in accordance with the response information. Contextual bandit model 720 may apply the customer segment response information in the bandit model to determine customer response estimation values 730 corresponding to the email templates and the segments of customers, and then determine revised allocations 718 respectively corresponding to the customer segments in accordance with the customer response estimation values 730.
Reference is also made to
Reference is again made to
In addition, the performance of the contextual bandit model 720 can be monitored by a monitoring module 750 in system 100. Similar to the operation mentioned above, after system 100 applies the response information from customers 714 in the contextual bandit model 720 to determine customer response estimation values corresponding to the email templates, the customer response estimation values are sent to monitoring module 750. System 100 can compare, by monitoring module 750, the customer response estimation values with actual customer responses in historical data to rate a model performance of the bandit model. Thus, if the model performance decreases significantly or is abnormally poor after cycles of iterations, system 100 may identify the issue and take action to improve the poorly trained model. For example, system 100 may reset contextual bandit model 720 and train the model again if the model performance is below expectation for a number of consecutive cycles.
In some embodiments, system 100 may repeat the above processes, and send corresponding emails to customers, in accordance with the allocation, intermittently in multiple cycles. That is, in each of the cycles, the allocation of the email templates for a next cycle is updated by contextual bandit model 720 in accordance with responses of customers receiving the emails in a previous cycle.
In view of above, in various embodiments of the present disclosure, system 100 can implement an automated procedure for identifying user segments and a self-updated bandit model to improve the advertising efficiency by an iterative process. In addition, system 100 can achieve an appropriate exploration and exploitation using the contextual bandit framework, and improve the performance of targeted email marketing.
The various example embodiments herein are described in the general context of method steps or processes, which may be implemented in one aspect by a computer program product, embodied in a transitory or a non-transitory computer-readable medium, including computer-executable instructions, such as program code, executed by computers in networked environments. A computer-readable medium may include removeable and nonremovable storage device(s) including, but not limited to, Read Only Memory (ROM), Random Access Memory (RAM), compact discs (CDs), digital versatile discs (DVD), etc. Generally, program modules may include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Computer-executable instructions, associated data structures, and program modules represent examples of program code for executing steps of the methods disclosed herein. The particular sequence of such executable instructions or associated data structures represents examples of corresponding acts for implementing the functions described in such steps or processes.
In the foregoing specification, embodiments have been described with reference to numerous specific details that can vary from implementation to implementation. Certain adaptations and modifications of the described embodiments can be made. Other embodiments can be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims. It is also intended that the sequence of steps shown in figures are only for illustrative purposes and are not intended to be limited to any particular sequence of steps. As such, those skilled in the art can appreciate that these steps can be performed in a different order while implementing the same method.
As used herein, unless specifically stated otherwise, the term “or” encompasses all possible combinations, except where infeasible. For example, if it is stated that a database may include A or B, then, unless specifically stated otherwise or infeasible, the database may include A, or B, or A and B. As a second example, if it is stated that a database may include A, B, or C, then, unless specifically stated otherwise or infeasible, the database may include A, or B, or C, or A and B, or A and C, or B and C, or A and B and C.
In the drawings and specification, there have been disclosed exemplary embodiments. It will be apparent to those skilled in the art that various modifications and variations can be made to the disclosed system and related methods. Other embodiments will be apparent to those skilled in the art from consideration of the specification and practice of the disclosed system and related methods. It is intended that the specification and examples be considered as exemplary only, with a true scope being indicated by the following claims and their equivalents.
