Patent Application
20130080212
Embodiments are generally related to social network marketing. Embodiments are additionally related to engagement effectiveness measurement techniques. Embodiments are further related to the measurement of engagement effectiveness with respect to service agents in the context of a social media environment.
CRM (Customer Relationship Management) is a widely, implemented strategy for managing a company's interaction with respect to customers, clients, and sales prospects. CRM involves utilizing technology to organize, automate, and synchronize business processes such as sales activities, marketing, customer service, and technical support. The overall goals of CRM include the abilities to find, attract, and win a new client, nurture and retain the existing client, entice former clients back into the fold, and reduce the costs associated with marketing and client services.
Social media generally involves a large number of users who interact socially with one another in an electronic social network environment. In such a paradigm, users can freely express and share opinions with other users via an electronic social networking application. Social media encompasses online media such as, for example, collaborative projects (e.g. Wikipedia), blogs and microblogs (e.g. Twitter), content communities (e.g. YouTube), social networking sites (e.g. Facebook), virtual game worlds (e.g. World of Warcraft), and virtual social worlds (e.g. Second Life). Particularly, online social media can be harvested to share experience/issues with respect to a product, service, brand, and industry.
Customer care in the context of a social customer relationship management arrangement generally involves a product service agent who utilizes commercial social media listening and engagement tools (e.g. Radian6, Sysomos, InvisibleTech, etc.) to respond to a user complaint/issue and assist the user in tackling problems so as to improve the satisfaction of the customer/user. When conversation topics are identified, which relate to a product/service of interest, the product service agent enters the conversation to supply requested information and seeks to address the identified problems. Commercial social media tools in this context only provide aggregated levels of trends, patterns, and sentiment analysis based on keyword-centric brand relevant data, which afford little insights for answering one or more key questions in a social CRM system.
An engagement effectiveness measure quantifies the results and outcomes of the service agent in the social customer relationship management and assists in managing resources effectively. Conventional CRM effectiveness measurement techniques, especially in “Call Center” environments, measure the effectiveness based chiefly on time duration per call and/or the number of answered calls per day and often result in very poor performance metrics. Different from a “Call Center”, online social media involves a great deal of text-oriented interactions between customers and product service agents, which requires much more human labors and is more difficult to analyze. Therefore, the ability to ensure an effective engagement process in the scenario of electronic social media is a critical business requirement.
Based on the foregoing, it is believed that a need exists for improved methods and systems for measuring engagement effectiveness with respect to a service agent in a social media environment, as will be described in greater detail herein.
The following summary is provided to facilitate an understanding of some of the innovative features unique to the disclosed embodiments and is not intended to be a full description. A full appreciation of the various aspects of the embodiments disclosed herein can be gained by taking the entire specification, claims, drawings, and abstract as a whole.
It is, therefore, one aspect of the disclosed embodiments to provide an improved customer relationship management effectiveness measurement method and system.
It is another aspect of the disclosed embodiments to provide an improved method and system for measuring engagement effectiveness with respect to a service agent in a social media environment.
It is a further aspect of the disclosed embodiments to provide an improved method and system for computing a time-sensitive pyramid kernel function based on an implicit topical and sentimental correspondence with respect to daily conversations.
The aforementioned aspects and other objectives and advantages can now be achieved as described herein. Methods and systems for measuring engagement effectiveness with respect to a service agent by analyzing a conversation between the agent and a customer in a social media environment are disclosed herein. A conversation history between the agent and the customer can be mapped into a multi-resolution space based on different time frames via a mapping module. A polarized topical and sentimental distance between the continuous conversations can be calculated by applying a topic-sentiment mixture model and a divergence theorem (e.g., modified Kullback-Leibler (KL) divergence) onto the conversation history with respect to the time frame. Finally, the polarized topical distances can be aggregated in a time-sensitive way based on a time function and an effectiveness score can be calculated and represented as a weighted pyramid kernel of multiple levels. Such a time-sensitive pyramid kernel function based on the implicit topical and sentimental correspondences between daily conversations enables discriminative evaluation for the agent engagement in a customer care.
The uniform time frame can be defined and the conversation history can be separated into multiple pieces based on the time frame. The topic sentiment mixture model can be employed to obtain the probabilistic distribution of words (e.g., topic-based distribution, sentiment-based distribution) for each time frame. The modified Kullback-Leibler divergence of the probabilistic distribution of words can be computed by considering social influence of the customer having conservation with the agent for the continuous time frames and the divergences can then be aggregated in a weighted manner. The influence of the customer can be calculated by considering the scale of a personal social network to incorporate the social influence into the measurement of engagement effectiveness.
The modified KL-divergence of the topic-based distribution between the conversations having a small value represents the conversation topics have not changed much. The value of KL-divergence is normalized based on the entire distance space. Note that the sentiment distance is not captured by KL-divergence in the context of the disclosed embodiments. Instead, it is captured by modeling the sentiment changes utilizing the number of positive and negative works, as described in Equation (7) herein. The number of sentiment words whose probabilities are greater than a pre-defined threshold can be counted and then the sentiment change can be quantified. The topical KL-divergence and the sentimental change can then be aggregated in order to capture the topical and sentimental differences simultaneously.
The time function can be defined to determine appropriate time weights for each distance in the order that the recent conversation histories contribute more to the overall effectiveness when aggregating the distances of the multiple continuous time frames. The time function is a monotonic decreasing function, which deduces uniformly with time. The distance between the conversation histories can be hierarchically computed based on different time frames to render the kernel function more flexible. The proposed distance hierarchy provides effectiveness evidence in multiple time granularities in case that a manager tries to investigate and review the engagement effectiveness of the particular time frame. The greater engagement score represents the better engagement performance of the agent. The proposed invention describes a novel time-sensitive metric to measure the effectiveness of social engagement, and the metric comprehensively analyzes the content of conversations between the agent and the customer from topical, sentiment, and social impact perspectives.
The accompanying figures, in which like reference numerals refer to identical or functionally-similar elements throughout the separate views and which are incorporated in and form a part of the specification, further illustrate the present invention and, together with the detailed description of the invention, serve to explain the principles of the present invention.
The embodiments will now be described more fully hereinafter with reference to the accompanying drawings, in which illustrative embodiments of the invention are shown. The embodiments disclosed herein can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the scope of the invention to those skilled in the art. Like numbers refer to like elements throughout. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
As will be appreciated by one skilled in the art, the present invention can be embodied as a method, data processing system, or computer program product. Accordingly, the present invention may take the form of an entire hardware embodiment, an entire software embodiment or an embodiment combining software and hardware aspects all generally referred to herein as a “circuit” or “module.” Furthermore, the present invention may take the form of a computer program product on a computer-usable storage medium having computer-usable program code embodied in the medium. Any suitable computer readable medium may be utilized including hard disks, USB Flash Drives, DVDs, CD-ROMs, optical storage devices, magnetic storage devices, etc.
Computer program code for carrying out operations of the present invention may be written in an object oriented programming language (e.g., Java, C++, etc.). The computer program code, however, for carrying out operations of the present invention may also be written in conventional procedural programming languages such as the “C” programming language or in a visually oriented programming environment such as, for example, VisualBasic.
The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer. In the latter scenario, the remote computer may be connected to a user's computer through a local area network (LAN) or a wide area network (WAN), wireless data network (e.g., WiFi, Wimax, 802.xx, and cellular network) or the connection may be made to an external computer via most third party supported networks (for example, through the Internet using an Internet Service Provider).
The disclosed embodiments are described in part below with reference to flowchart illustrations and/or block diagrams of methods, systems, and computer program products and data structures according to embodiments of the invention. It will be understood that each block of the illustrations, and combinations of blocks, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function/act specified in the block or blocks.
The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions/acts specified in the block or blocks.
Although not required, the disclosed embodiments will be described in the general context of computer-executable instructions, such as program modules, being executed by a single computer. In most instances, a “module” constitutes a software application. Generally, program modules include, but are not limited to, routines, subroutines, software applications, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types and instructions. Moreover, those skilled in the art will appreciate that the disclosed method and system may be practiced with other computer system configurations such as, for example, hand-held devices, multi-processor systems, data networks, microprocessor-based or programmable consumer electronics, networked PCs, minicomputers, mainframe computers, servers, and the like.
Note that the term module as utilized herein may refer to a collection of routines and data structures that perform a particular task or implements a particular abstract data type. Modules may be composed of two parts: an interface, which lists the constants, data types, variable, and routines that can be accessed by, other modules or routines, and an implementation, which is typically private (accessible only to that module) and which includes source code that actually implements the routines in the module. The term module may also simply refer to an application such as a computer program designed to assist in the performance of a specific task such as word processing, accounting, inventory management, etc.
As illustrated in
The interface 153 is preferably a graphical user interface (GUI) and generally serves to display results, whereupon the user may supply additional inputs or terminate the session. In an embodiment, operating system 151 and interface 153 can be implemented in the context of a “Windows” system. It can be appreciated, of course, that other types of systems are possible. For example, rather than a traditional “Windows” system, other operation systems such as, for example, Linux may also be employed with respect to operating system 151 and interface 153. The software application 154 can include an engagement effectiveness identification module 152 for measuring engagement effectiveness with respect to a service agent by analyzing a conversation between the agent and a customer in social media environment. Software application 154, on the other hand, can include instructions such as the various operations described herein with respect to the various components and modules described herein such as, for example, the method 400 depicted in
The engagement effectiveness identification module 152 includes a mapping module 305, a polarized topical/sentimental distance computation module 355, and a time sensitive pyramid kernel function module 330. The mapping module 305 maps a conversation history between the agent 340 and one or more customers 345 into a multi-resolution space 310 based on a time frame. The polarized topical/sentimental distance computation module 355 determines a polarized topical and sentimental distance between the continuous conversations by applying a topic-sentiment mixture model 360 and a divergence theorem 315 onto the conversation history 325 with respect to the time frame. The topic-sentiment mixture model 360 proposes a probabilistic mixture model called topic-sentiment mixture to model and extract the multiple subtopics and sentiments in the conversation history.
Note that the divergence theorem 315 can be, for example, a Kullback-Leibler (KL) divergence which can be modified to determine the polarized topical and sentimental distance to incorporate the social influence into the measurement of engagement effectiveness. In probability theory and information theory, the Kullback-Leibler divergence (also information divergence, information gain, relative entropy, or KLIC) is a non-symmetric measure of the difference between two probability distributions P and Q. KL measures the expected number of extra bits required to code samples from P when using a code based on Q, rather than using a code based on P. Typically P represents the “true” distribution of data, observations, or a precisely calculated theoretical distribution. The measure Q typically represents a theory, model, description, or approximation of P. The Kullback-Leibler (KL) divergence is modified to determine the polarized topical and sentimental distance.
The polarized topical distances can be aggregated in a time-sensitive way based on a time function 320 and an effectiveness score 335 can be calculated and represented as a weighted pyramid kernel function 330 of multiple levels. The time-sensitive pyramid kernel function 330 based on the implicit topical and sentimental correspondences between daily conversations enables discriminative evaluation for the agent engagement in a customer care. Note that the time function 320 can be represented as an exponential form, which is able to describe the gradual decay of the importance of past conversations as time goes. It will be apparent to those skilled in the art that other forms of time function can be utilized as desired without departing from the scope of the invention.
D={D
t
,D
t
, . . . ,D
t
} (1)
wherein t0 denotes the current time frame. The probabilistic topic-based distribution and sentiment-based distribution for each time frame can be obtained using topic sentiment mixture model 360, as illustrated at block 420. For each time frame ti, i=0, 1, n, the topic sentiment mixture model 360 can be employed to obtain the probabilistic distribution of words in Dt, i.e., topic-based distribution: p(w|θ) and sentiment-based distribution: p(w|θP) and p(w|θN), where w represents a word, θ represents the topic hidden in texts, and θp, θN denote positive sentiment and negative sentiment, respectively.
The customer influence can be calculated by considering scale of personal social network 385 to incorporate social influence into measurement of engagement effectiveness, as shown at block 430. The divergence of probabilistic distribution of words can be computed by considering the social influence of the customers 345 having a conversation with the agent 340 for continuous time frames, as indicated at block 440. For continuous time frames, e.g., ti and ti+1, a modified Kullback-Leibler (KL) divergence of p(w|θ), p(w|θP) and p(w|θN) between Dt
The divergences can be aggregated in a weighted manner based on time function 320 to obtain polarized topical distances in a time-sensitive way with respect to recent conversation histories, as illustrated at block 450. For example, consider that the recent engagement of agents is more important than the historical engagement, and therefore, the time function ƒ(t) can be employed to aggregate the distances of multiple continuous time frames as the engagement score of the most fine-grained level as indicated in equation (2) as follows:
When aggregating the distances of multiple continuous time frames, the time function ƒ(t) is defined to determine appropriate time weights for each distance σ(Dt
θ(t)=e−λ·t (3)
wherein λ represents the profile decay rate and is the inverse of the time frame T that can be employed to divide the conversation history of agents 340 in each level of the multi-resolution space 310. The distance between conversation histories can be hierarchically computed based on different time frames to make the kernel function 330 more flexible. In general, a team manager might be interested in measuring the engagement effectiveness of the agents 340 from different time periods, e.g., per week or per month. The proposed distance hierarchy aims at providing effectiveness evidence in multiple time granularities, so that team managers can easily navigate and review engagement effectiveness in any particular time frame.
wherein
and κj(ψ(u)) represents the engagement score of level j in the multi-resolution space 310. κ(ψ(u)) is normalized by the number of conversation pairs that are compared by the DL divergence. Thus the engagement score does not favor the conversations within long time frames. The resulted multi-resolution distance space is flexible to navigate in case that the manager tries to investigate the engagement effectiveness of a particular time frame. The greater engagement score means the better engagement performance of the agent.
For example, consider the distance between two continuous time frames from both topical and sentimental perspective. The topics and sentiments of the conversations can be initially quantified in a probabilistic manner, and then the KL-Divergence of topic and sentiment distributions can be calculated. The polarized topical distance can be obtained by aggregating the two KL-Divergences. By applying a topic-sentiment mixture model 360 onto the conversation history of a particular time frame Dt
wherein θ represents the hidden topic, and wj denotes a word from the conversation text. In a social media scenario, it is imperative to pinpoint the social influence of customers 345 who have conversations with the agents 340, since the opinions of influential customers towards products/brands might diffuse within their personal social networks. To incorporate the social influence into the measurement of engagement effective, the social influence of the customer 345 is measured by considering the scale of the personal social network. Specifically, for a particular word wj, the messages that contain wj and the customers 345 who post these messages can be retrieved, and then the customers' social media profiles can be accessed to evaluate the scale of their social graphs. Hence the equation (5) is modified and can be written as shown in equation (6):
wherein |Uw
wherein WP,t represents the number of positive words in the time frame t, and WN,t denotes the number of negative words in t. By using the above equation, the sentimental transformation e.g., from “positive” to “negative”, or from “negative” to “positive”, or from “positive” to “more positive”, can be easily captured. The larger σS is the more effective the agent is. Note that the sentiment distance is not captured by KL-divergence in the context of the disclosed embodiments. Instead, it is captured by modeling the sentiment changes utilizing the aforementioned number of positive and negative works, as described in Equation (7) above. In order to capture the topical and sentimental differences simultaneously, the topical KL-divergence and the sentimental change can be aggregated as illustrated below in equation (8):
σ(Dt
As indicated in equation (8) above, the first component captures the topical distance, and the second one aims to investigate the sentimental difference from both positive and negative perspectives. Here, the parameter a controls the importance of topical and sentimental distance in the overall measurement. The discriminative evaluation for agent engagement in customer care can be enabled using time-sensitive pyramid kernel function 330, as indicated at block 470. The proposed invention describes a novel time-sensitive metric to measure the effectiveness of social engagement, where the metric comprehensively analyzes the content of conversations between the agent 340 and the customer 345 from topical, sentiment, and social impact perspectives.
Based on the foregoing, it can be appreciated that a variety of embodiments, preferred and alternative, are disclosed herein. For example, in an embodiment, a method for measuring engagement effectiveness is disclosed. Such a method can include the steps of mapping a conversation history between an agent and at least one customer into a multi-resolution space based on a time frame; determining a polarized topical and sentimental distance between continuous conversations by applying a topic-sentiment mixture model and a divergence theorem to the conversation history with respect to the time frame; and aggregating the polarized topical and sentimental distance in a time-sensitive manner based on a time function in order to thereafter calculate an effectiveness score and represent the effectiveness score as a weighted pyramid kernel function of multiple levels wherein the effectiveness score with a high value is indicative of an enhanced engagement performance by the agent and hence a measure of engagement effectiveness of the agent.
In other embodiments, the aforementioned mapping operation can be implemented as a step for mapping the conversation history between the agent and the at least one customer into the multi-resolution space based on the time frame via a mapping module. In another embodiment, steps can be implemented for defining the time frame and separating the conversation history into the multi-resolution space based on the time frame; obtaining a probabilistic distribution of word for each time frame via the topic sentiment mixture model; and computing a divergence of the probabilistic distribution of word utilizing the divergence theorem by considering a social influence of the customer having a conservation with the agent for the continuous time frame.
In yet another embodiment, the aforementioned probabilistic distribution can comprise a topic-based distribution. In still another embodiment, the aforementioned probabilistic distribution can comprise a sentiment-based distribution. In another embodiment, a step can be implemented for determining the social influence of the customer by considering a scale of a social network. In yet another embodiment, the aforementioned divergence theorem can comprise a modified Kullback-Leibler divergence.
In still other embodiments, a step can be implemented for indicating that the conversation topic remains unchanged for a period of time if the divergence with respect to the topic-based distribution between the conversations results in a small value. In another embodiment, a step can be implemented for indicating a miniscule changed positive sentiment and/or a miniscule change negative sentiment in the conversation if the divergence with respect to the sentiment-based distribution between the conversations results in a small value. In another embodiment, a step can be implemented for normalizing a value of the divergence theorem based on an entire distance space.
In other embodiments, a step can be implemented for counting a plurality of sentiment words if the probabilistic distribution of the plurality of sentiment words is greater than a pre-defined threshold in order to thereafter quantify a sentiment change. In still other embodiments, a step can be implemented for defining the time function to determine an appropriate time weight for each distance so that a recent conversation history contributes more to the engagement effectiveness when aggregating the polarized topical and sentimental distance of the continuous time frame.
In another embodiment, the aforementioned time function can comprise a monotonic decreasing function, which deduces uniformly with time. In other embodiments, a step can be implemented for providing the engagement effectiveness in a plurality of time frames in order to easily navigate and review the engagement effectiveness in any particular time frame among the plurality of time frames.
In another embodiment, a system for measuring engagement effectiveness can be implemented. Such a system can include, for example, a processor, a data bus coupled to the processor, and a computer-usable medium embodying computer code, the computer-usable medium coupled to and/or in communication with the data bus. The computer program code can comprise instructions executable by the processor and configured for: mapping a conversation history between an agent and at least one customer into a multi-resolution space based on a time frame; determining a polarized topical and sentimental distance between continuous conversations by applying a topic-sentiment mixture model and a divergence theorem to the conversation history with respect to the time frame; and aggregating the polarized topical and sentimental distance in a time-sensitive manner based on a time function in order to thereafter calculate an effectiveness score and represent the effectiveness score as a weighted pyramid kernel function of multiple levels wherein the effectiveness score with a high value is indicative of an enhanced engagement performance by the agent and hence a measure of engagement effectiveness of the agent.
In other embodiments, the aforementioned instructions for mapping the conversation history between the agent and the at least one customer into the multi-resolution space based on the time frame can be further configured for mapping the conversation history between the agent and the at least one customer into the multi-resolution space based on the time frame via a mapping module. In another embodiment, such instructions can be further configured for defining the time frame and separating the conversation history into the multi-resolution space based on the time frame; obtaining a probabilistic distribution of word for each time frame via the topic sentiment mixture model; and computing a divergence of the probabilistic distribution of word utilizing the divergence theorem by considering a social influence of the customer having a conservation with the agent for the continuous time frame.
In another embodiment, a processor-readable medium can be provided, which stores code representing instructions to cause a processor to perform a process to measure engagement effectiveness. Such code can be configured to, for example, map a conversation history between an agent and at least one customer into a multi-resolution space based on a time frame; determine a polarized topical and sentimental distance between continuous conversations by applying a topic-sentiment mixture model and a divergence theorem to the conversation history with respect to the time frame; and aggregate the polarized topical and sentimental distance in a time-sensitive manner based on a time function in order to thereafter calculate an effectiveness score and represent the effectiveness score as a weighted pyramid kernel function of multiple levels wherein the effectiveness score with a high value is indicative of an enhanced engagement performance by the agent and hence a measure of engagement effectiveness of the agent.
In other embodiments, for example, the aforementioned probabilistic distribution in the context of a process-readable medium can comprise one or more of the following: a topic-based distribution and a sentiment-based distribution. In still other embodiments, such code can further comprise code to determine the social influence of the customer by considering a scale of a social network and provide the engagement effectiveness in a plurality of time frames in order to easily navigate and review the engagement effectiveness in any particular time frame among the plurality of time frames.
It will be appreciated that variations of the above-disclosed and other features and functions, or alternatives thereof, may be desirably combined into many other different systems or applications. Also, that various presently unforeseen or unanticipated alternatives, modifications, variations or improvements therein may be subsequently made by those skilled in the art which are also intended to be encompassed by the following claims.
