Method and apparatus for detecting anomalies in communication data

Information

  • Patent Grant
  • 11971908
  • Patent Number
    11,971,908
  • Date Filed
    Friday, June 17, 2022
    2 years ago
  • Date Issued
    Tuesday, April 30, 2024
    a month ago
  • Inventors
    • Neves; Pedro Filipe Caldeira
    • Cardoso; Nuno André de Matos Lopes
  • Original Assignees
  • Examiners
    • Beausoliel, Jr.; Robert W
    • Shechtman; Cheryl M
    Agents
    • Rimon PC
    • Kaufman; Marc S.
  • CPC
    • G06F16/285
    • G06F16/2282
  • Field of Search
    • CPC
    • H04L63/1416
    • H04L63/1425
    • G06F16/285
    • G06F16/906
    • G06F21/552
    • G06F11/3476
    • G06F11/3466
  • International Classifications
    • G06F16/28
    • G06F16/22
Abstract
A method and system for determining anomalies in call center communications. Data relating to communications is streamed and processed to obtain baseline probability distributions over various domains of communications. Streams related to subsequent calls are compared to the baselines to determine anomalies.
Description
BACKGROUND

Contact centers, also referred to as “call centers”, in which agents handle communications with customers based on agent skills and customer requirements, are well known. The term “customer”, as used herein, can be any entity or individual contacting the contact center for information. FIG. 1 is an example system architecture of a cloud-based contact center system 100. Customers 110 interact with a contact center 150 using, for example, voice, email, text, and web interfaces to communicate with the agents 120 through a network 130 and at least one or more of text, voice or multimedia channels.


The agents 120 may be remote from the contact center 150 and handle communications (also referred to as “interactions” or “calls” herein) with customers 110 on behalf of an enterprise. The agents 120 may utilize devices, such as but not limited to, workstations, desktop computers, laptops, telephones, a mobile smartphone and/or a tablet. Similarly, customers 110 may communicate using a plurality of devices, including but not limited to, a telephone, a mobile smartphone, a tablet, a laptop, a desktop computer, or other. For example, telephone communication may traverse networks such as a public switched telephone networks (PSTN), Voice over Internet Protocol (VoIP) telephony (via the Internet), a Wide Area Network (WAN) or a Large Area Network (LAN). The network types are provided by way of example and are not intended to limit types of networks used for communications.


The agents 120 may be assigned to one or more queues representing call categories and/or agent skill levels. The agents 120 assigned to a queue may handle communications that are placed in the queue by the contact routing system 153. For example, there may be queues associated with a language (e.g., English or Chinese), topic (e.g., technical support or billing), or a particular country of origin. When a communication is received, the communication may be placed in a relevant queue, and eventually routed to one of the agents 120 associated with the relevant queue to handle the communication.


The contact center industry has been dealing with ever-more customer data on a daily basis and what was once a blind interaction with the customer at the other end of the line is now a data enriched experience that is very valuable to the call center and users of the call center (i.e., entities for which communications from customers are received). Moreover, within the past few years, there has been a trend to eliminate dedicated physical call centers in favor of virtual platforms in which call center services are provided to users in the form of Software as a Service (SaaS). In such platforms, agents can be employees or contractors and can be located centrally or in a distributed manner. For example, agents can works from their homes on flexible schedules. Such platforms reduce overhead for the user and scalable and convenient service.


Although this disaggregation has advantages, as it allows user companies to grow their business without the need to manage their call center or provide space for call center agents, it also poses some issues and challenges. Security issues are of a primary concern. The distributed nature of the systems provides multiple attach points for hackers. Also, it is more difficult to ensure that agents adhere to proper security protocols. Service level is also a concern as it is more difficult to train and supervise agents. It is known to increase service and security by detecting specific occurrences in call center communications. Known techniques apply filters that are looking for specific terms to trigger and action. For example, if a customer communication includes the words such as “angry” or “dissatisfied”, or phrases such as “cancel order”, the communication can be escalated to a manager or other agent equipped better to deal with unhappy customers. While sometimes referred to as “anomaly detection”, such techniques detect undesired activity but not necessarily anomalies. True anomaly detection of interactions in a call center requires a determination in substantially real time in view of a myriad of variables such as the subject matter of the call, times of day and year, the agent(s), call center user characteristics and domains, and the like. Current call center detection techniques do not provide the required speed and flexibility.


SUMMARY OF THE INVENTION

The disclosed implementations analyze agents' normal behavior and verify if there is any major change over time. This is often called anomaly detection and is very closely related to fraud detection. A first aspect of the invention is a method for creating a baseline database to be used to increase security in a call center implemented over a computing network by detecting anomalies in communication activities between call center agents and call center users, the method comprising: monitoring at least one stream of communication activity data indicating parameters of communication activities between call center agents and call center users; storing the communication activity data in a collected data database; aggregating the communication activity data into aggregated data; and creating, based on the aggregated data, at least one distribution of communication metrics over a period of time. A second aspect of the invention is a method for increasing security in a call center implemented over a computing network by detecting anomalies in communication activities between call center agents and call center users, the method comprising: monitoring event parameters of communication activities between call center agents and user; querying a baseline distribution database to determine that an event parameter represents a communication anomaly when the event parameter indicates an event that corresponds to a probability that is lower than a predetermined threshold probability and a calculated confidence of the event is higher than a predetermined confidence threshold, wherein the baseline distribution database is created by monitoring at least one stream of communication activity data indicating parameters of communication activities between call center agents and call center users, storing the communication activity data in a collected data database and aggregating the communication activity data to create at least one distribution of communication metrics over a period of time; and storing a record of the communication anomaly in and anomaly database.


A third aspect of the invention is a system for creating a baseline database to be used to increase security in a call center implemented over a computing network by detecting anomalies in communication activities between call center agents and call center users, the system comprising: at least one memory storing computer executable instructions; and at least one processor which, when executing the instructions accomplishes the method of: monitoring at least one stream of communication activity data indicating parameters of communication activities between call center agents and call center users; storing the communication activity data in a collected data database; aggregating the communication activity data into aggregated data; and creating based on the aggregated data, at least one distribution of communication metrics over a period of time.


A fourth aspect of the invention is a system for increasing security in a call center implemented over a computing network by detecting anomalies in communication activities between call center agents and call center users, the system comprising: at least one memory storing computer executable instructions; and at least one processor which, when executing the instructions accomplishes the method of: monitoring event parameters of communication activities between call center agents and user; querying a baseline distribution database to determine that an event parameter represents a communication anomaly when the event parameter indicates an event that corresponds to a probability that is lower than a predetermined threshold probability and a calculated confidence of the event is higher than a predetermined confidence threshold, wherein the baseline distribution database is created by monitoring at least one stream of communication activity data indicating parameters of communication activities between call center agents and call center users, storing the communication activity data in a collected data database and aggregating the communication activity data to create at least one distribution of communication metrics over a period of time; and storing a record of the communication anomaly in and anomaly database





BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing summary, as well as the following detailed description of the invention, will be better understood when read in conjunction with the appended drawings. For the purpose of illustrating the invention, there are shown in the appended drawings various illustrative embodiments. It should be understood, however, that the invention is not limited to the precise arrangements and instrumentalities shown. In the drawings:



FIG. 1 is an architectural diagram of a conventional cloud based contact center computing environment.



FIG. 2 is a block diagram an architecture and data flow of system for anomaly detection in a call center in accordance with disclosed implementations.



FIG. 3 illustrates data model that can be used as the basis for anomaly detection in accordance with disclosed implementations.



FIG. 4 illustrates an example of a data aggregation ontology in accordance with disclosed implementations.



FIG. 5 illustrates and example of a data workflow in accordance with disclosed implementations.



FIG. 6 is a flow chart of process for creating distributions for use in anomaly detection in accordance with disclosed implementations.



FIG. 7 is a flow chart of process for creating distributions for use in anomaly detection in accordance with disclosed implementations.





DETAILED DESCRIPTION


FIG. 2 illustrates an architecture of call center anomaly detection system 200 in accordance with disclosed implementations. System 200 is a hybrid model which consumes data as streams from database 202, a Kafka database in this example, and processes that data in both streaming and batch. Apache Kafka™ is an event streaming database platform capable of a high volume of events. In this example, Databricks was used provide flexibility in programming languages and connections to other systems. Databricks™ is a cloud-based data environment that is capable of processing and transforming large quantities of data. through, for example, Machine Learning models.


Communications, between agents and customers for example, are monitored and data streams representing the communications are stored in database 202. This data is then processed by data module 204. Data module 204 can aggregate and segregate the data in various manners as described in more detail below. Baseline module 206 then applies distribution algorithms to produces one or more baseline probability distributions. A probability distribution is a known statistical function that describes all the possible values and likelihoods that a random variable can take within a given range. Plotting of a baseline value (or multiple values) on the probability distribution can be based on a number of factors. These factors include the distribution's mean, standard deviation, skewness, and kurtosis. Data module 204 can create various distributions as needed. For example, distributions can correspond to specific agents, call centers, type of communication, and the like, or any combination thereof.


The anomaly detection system of disclosed embodiments takes raw data relating to communications and provides valuable insights through fast and reliable anomaly detection. FIG. 3 illustrates data model 300 that can be used as the basis for anomaly detection. Within the Databricks environment data can be consumed and stored into three different layers (referred to as “bronze”, “silver”, and “gold” layers herein). Bronze layer 302 holds the raw data, that may or may not be used immediately or at a later time for some other use cases. Silver layer 304 holds processed and filtered data that enables the creation of the baseline distributions which will be the used for anomaly detection. Gold layer 306 is focused on delivering valuable insights and consists of immutable or less mutable data while setting up support for advanced use cases related to anomaly detection. The data in each data layer is described in more detail below.


Data management layer 308 can include 6 modules define a set of policies or a way to trace data back to its origin:

    • Data Catalogue Module—A catalog on what data is being processed from the database and the data being generated to be consumed by other modules;
    • Data Dictionary Module—A detailed dictionary on every table and field within silver and gold data layers;
    • Data Lineage Module—Lineage is used to trace back results to origin and know exactly which processes originated the results, where it has consumed data and what data;
    • Data Archiving Module—Archiving policies define when, where, what and why data should be archived;
    • Data Retention Module—Data retention policies define how much time data needs to be stored in each table and S3 bucket discussed below;
    • Data Deletion Module—Data deletion policies define when, what and why data needs to be deleted, as well as it keeps record of what data was deleted, by whom and why.


The data management policies, catalog and dictionary can correspond to best practices and data engineering guidelines so that the data model can be scaled.


Any database management system can be used. However, in FIGS. 2 and 3 one Kafka database 202 illustrated. Data can be streamed to database 202 from various data sources, including:

    • Audit logs—provides information relating to agents' interactions, such as login and logouts, password changes or resets, authentication methods updates, and contact reads;
    • Calls—gives insights into the type of calls agents are performing, like inbound and outbound calls, missed calls, call initiated or finished;
    • Agents—provides information on updates done to agents' accounts, as when an agent account is created, deleted. activated or deactivated, as well as agent profile updates;
    • Accounts—shows when a client account was created, deleted, or updated;
    • Presence—indicates the agent's current status and when the status was altered (for example when the agent is online and becomes away, in a call, offline, or any of the other statuses that are available for the account the agent belongs to);
    • Call quality—gives an overview of the call quality of each agent;
    • Teams—incorporates updates, creation, and deletion of teams of agents;
    • Recordings—provides data on what call or screen recordings were accessed by whom;
    • Voice metrics—holds data concerning the quality of speech during calls, concerning the number of decibels and overall mood of the conversation.


The disclosed implementations for performing anomaly detection can be split into three main parts: (1) data ingestion into the data model; (2) creation of behavioral baselines; and (3) detecting anomalies on current data. Regardless of the original data source, data is ingested from database 202 (Kafka or another database) into bronze layer 302, which can be in the form of Amazon S3 buckets for example, for long-term storage. The data can be filtered and/or enriched for the events that need to be processed. This data can then be stored in delta tables within Databricks. All this can happen in streaming and data can be made available within the delta tables substantially in real-time, e.g., immediately after it is ingested by the Databricks processes.


Gold layer 306 is composed of processes that run in batches and fetch data from tables in silver layer 304 that pragmatically cannot be processed in streaming. One example of data in silver layer 304 is data related to sessions, where the session start event is processed long before the session end event, and therefore, the process cannot be waiting indefinitely. The baseline distributions can be created periodically (for example, once every day in batch at 00:05 UTC with 30 days of aggregated data from either silver and gold tables or other existing baselines). Data is aggregated by both agent and account per peer per day, in a predefined time period (for example, starting 31 days before the current UTC time and finishing 1 day before the current UTC time.


The baseline distributions can be composed of the four tables which are, for example:

    • Table 1—baseline per user per day;
    • Table 2—baseline per peers per day;
    • Table—baseline per details per user per day;
    • Table 4—baseline per details per peer per day.


The first table holds the aggregated metrics per agent, account, and day. The second, stores per account and day. So, these two tables have the aggregated baseline metrics per day, in either the agent or account level. The third and fourth tables have detailed information for either the agent or the account for each use case. Both baselines provide a different level/aspect of understanding from the agent perspective and the account perspective that can be used for calculating anomalies.


The following is an example situation illustrating a possible anomaly. An account is based in the US, so the agents usually log in from the US. However, the agent (who may be a remote contractor) has moved to France and is now logging in from there. The baseline for the agent will be the number of different countries that the login was made from. In this case, the number of different countries is 2 (U.S. and France) and will be stored in table 1 described above. In table 2, the baseline for the account, which is calculated considering all the agents in this account, will also store 2 as the number of countries from which the agents logged in (since all agents logged from the US and there was 1 agent that logged in from the US and then from France). Table 3 will store 2 records: (1) a record reflecting that a particular agent logged in from the US; (2) a record reflecting that this same particular agent has logged in from France. In table 4, there will be also 2 records: (1) a record that reflects the number of agents logged in from the US, which will be the total number of agents for the account; and (2) a record reflecting the number of agents logged in from France (in this example, 1).


As another example, a statistical distribution of a number of calls received from set of regions around the world per hour for a customer can be created and approximated as a Gaussian or some quasi-Gaussian distribution for instance. Then the probability of number of calls in each hour for each region is computed based on the corresponding distributions. A trigger can be actuated when the number of calls exceeds a threshold (computed as a function of the mean and first-order deviation from the distribution, for example). This technique can be used to identify unusual call volumes during hours where the expected numbers are within a range (as defined by the distribution).


Assume that the anomaly detection processes run every 10 minutes, aggregating data from that day and comparing it to the existing baselines for both users and peers. When a value is outside the norm or baseline distribution, an anomaly is detected and an anomaly message is triggered. For example, an anomaly message can be triggered specifying that the agent has made 20 outbound calls in a day when usually it only makes about 10 outbound calls in a day. However, the agents peers normally make about 19 outbound calls a day, so a particular agent making 20 outbound calls will not be completely out of the norm. Therefore, although an anomaly is detected for that particular agent, it is not an anomaly with respect to agents overall since the peers usually make about 19 outbound calls. Rules can be applied to determine an anomaly message based on which type(s) of anomalies have been detected.


Data in each layer can be processed and combined to cerate data streams for a subsequent layer in the workflow. The following table defines examples of the streams that can be used/generated in disclosed implementations:























Source
Source

Destination
Destination




ID
Origin data
system
Layer
Destination data
system
Layer
Description























First Level
1
event-splitter.audit_logs
kafka
Data
bronze_audit_logs
delta
bronze
Data comes from






source



Kafka, is then passed



2
event-splitter.calls
kafka
Data
bronze_calls
delta
bronze
and stored within






source



bronze tables



3
event-splitter.agents
kafka
Data
bronze_agents
delta
bronze







souce







4
event-splitter.teams
kafka
Data
bronze_teams
delta
bronze







souce







5
event-splitter.account
kafka
Data
bronze_accounts
delta
bronze







source







6
brokkr.historical-calls
kafka
Data
bronze_historical_calls
delta
bronze







source







7
broker.explore.agent-
kafka
Data
bronze_broker_agent_
delta
bronze





status-monthly-v3

source
status





Second Level
8
bronze_audit_logs
delta
bronze
silver_audit_logs_
delta
silver
Data from







contact_read


bronze_audit_logs



9



silver_audit_logs_create_
delta
silver
and is filtered into







interaction_recording


silver delta tables



10



silver_audit_logs_
delta
silver








delete_interaction_










recording






11



silver_audit_logs_list_
delta
silver








interaction_recording






12



silver_audit_logs_on_
delta
silver








recording_update_










event






13



silver_audit_logs_read_
delta
silver








call_recordings






14



silver_audit_logs_read_
delta
silver








recording_media_file






15



silver_audit_logs_
delta
silver








update_interaction_










recording






16



silver_audit_logs_user_
delta
silver








authentication_










settings_updated






17



silver_audit_logs_user_
delta
silver








login_attempt






18



silver_audit_logs_user_
delta
silver








password_changed






19



silver_audit_logs_user_
delta
silver








password_reset






20



silver_audit_logs_user_
delta
silver








session_created






21



silver_audit_logs_user_
delta
silver








session_revoked






22
bronze_calls
delta
bronze
silver_calls_agent_call_
delta
silver








answered






23



silver_calls_agent_
delta
silver








call_cancelled






24



silver_calls_agent_
delta
silver








call_finished






25



silver_calls_agent_
delta
silver








call_initiated






26



silver_calls_call_
delta
silver








agents_batch_dialed






27



silver_calls_call_
delta
silver








answered






28



silver_calls_call_
delta
silver








billed






29



silver_calls_call_
delta
silver








dialer_billed






30



silver_calls_call_
delta
silver








external_answered






31



silver_calls_call_
delta
silver








external_initiated






32



silver_calls_call_
delta
silver








finished






33



silver_calls_call_
delta
silver








initiated






34



silver_calls_call_
delta
silver








missed






35



silver_calls_csat_sent
delta
silver




36



silver_calls_outgoing_
delta
silver








call_answered






37



silver_calls_outgoing_
delta
silver








call_finished






38



silver_calls_outgoing_
delta
silver








call_initiated






39



silver_calls_outgoing_
delta
silver








call_missed






40
bronze_agents
delta
bronze
silver_agents_agent_
delta
silver








activated






41



silver_agents_agent_
delta
silver








created






42



silver_agents_agent_
delta
silver








deactivated






43



silver_agents_agent_
delta
silver








deleted






44



silver_agents_agent_
delta
silver








status_changed






45



silver_agents_agent_
delta
silver








updated






46



silver_agents_presence_
delta
silver








updated






47
bronze_teams
delta
bronze
silver_teams_system_
delta
silver








remove_members_










from_team






48



silver_teams_user_
delta
silver








add_member_to_team






49



silver_teams_user_
delta
silver








create_team






50



silver_teams_user_
delta
silver








delete_team






51



silver_teams_user_
delta
silver








remove_members_










from_team






52



silver_teams_user_
delta
silver








update_team






53
bronze_historical_calls
delta
bronze
silver_historical_calls_
delta
silver








call_finished





Third Level
54
user_session_created
delta
silver
gold_all_sessions
delta
gold
Data from delta



55
user_session_revoked
delta
silver



tables is combined



56
user_session_created
delta
silver
gold_closed_sessions
delta
gold
into gold delta tables.



57
user_session_revoked
delta
silver


gold
Extensive ETL



58
gold_users
delta
silver

delta
gold
processing will be in










place here










FIG. 4 illustrates a data aggregation ontology according to an example of disclosed implementations. As discussed above, data in accordance with the data model is organized in bronze layer 302, silver layer 304 and gold layer 306. The baselines in bronze layer 302 represent the basic aggregations that store the information processed from the data sources available, such as:

    • calls;
    • call quality;
    • logs;
    • agents;
    • accounts;
    • teams.


Silver layer 304 represents aggregations performed over bronze baselines, which means that the aggregations in silver layer 304 can be composed of:

    • calls—that combine calls and call quality;
    • agents—that combine logs and agents;
    • accounts—that combine accounts and teams.
    • The gold layer provides overall aggregations of the silver baselines.
    • Anomaly detection



FIG. 5 illustrates data workflow 500 in accordance with an example of disclosed implementations. Elements in FIG. 5 that are the same as, or similar to, those in FIG. 3 are labeled with like reference numerals. After, being collected and stored in database 202, data is sourced from database 202 and streamed into bronze layer 302, which is an S3 database in this example. The same data stream is parsed and filtered to tables in silver layer 304, which is a Delta Lake in this example. Amazon Simple Storage Service (Amazon S3)™ is an object storage service. Delta Lake is an open source storage layer. Tables in silver layer 304 are processed to agregrate/transform data from silver layer 304 into tables of gold layer 306. Anomalies are calculated within the Databricks environment and then synched into Al Kafka. This creates a messaging queue that is used to transport the output of anomaly detection to the client applications for various purposes such as inference/detection.


In Apache Kafka, categories used to organize messages are called “topics”. Each topic should have a name that is unique across the entire Kafka cluster. Messages can be sent to, and read from, specified topics. Kafka topics can have zero or more “consumers” subscribing to that topic and the data written to it. Topics can be partitioned and replicated throughout the implementation. As an example, the disclosed implementations can process the following topics:

    • event-splitter. audit_logs—all data related to logs (session create, session revoke, etc)
    • event-splitter.calls—all data related to calls (call started, call finished, call billed, etc)
    • event-splitter.agents—all data related to agents (agent created, agent removed, etc)
    • event-splitter.accounts—all data related to accounts (account created, account updated, etc)
    • brokkr.explore.agent-status-monthly-v3—all data related to agent status updated (agent online, etc.)


The anomaly detection process can include two main steps. The first step is to create the baseline of normal behavior and the second is to compare the current behavior to the baselines and check for anomalies. Thee baselines can be divided into several types, such as “session baselines” and “call baselines”. The sessions baselines can include four different tables calculated all within the same data pipeline.

    • silver_agents_sessions_baseline_user_details_day—contains the aggregation of each case per user
    • silver_agents_sessions_baseline_peer_details_day—contains the aggregation of each case account
    • silver_agents_sessions_baseline_user_day—pivots the cases for columns and aggregates data so each row is a unique combination of the cases, account, user and respective role
    • silver_agents_sessions_baseline_peer_day—pivots the cases for columns and aggregates data so each row is a unique combination of the cases and account


After creating these tables, the tables can be updated periodically, such as once per day, and used as the baselines of every account/user/use case for a predefined period of time, such as 30 days. The current day's data can then be run against, e.g., compared to, the baselines. A predetermined divergence form the baseline can be detected as an anomaly.


The calls baselines create the baselines for a user and peers and can include the following tables:

    • silver_agents_calls_baseline_user_details—This table can include the number of distinct countries per agent phone number or customer phone number per user in the last 30 days
    • silver_agents_calls_baseline_peer_details—This table can include the number of distinct countries per agent phone number or customer phone number per account in the last 30 days
    • silver_agents_calls_baseline_user_day—This table can include the call metrics per user and day
    • silver_agents_calls_baseline_peer_day—This table can include the call metrics per account and day


As noted above, the baselines process can run periodically to aggregates current data for the day and compare that data with the baselines. If the current data is different from the baselines in a predetermined manner, then an anomaly is detected, written to the nr_anomalies table and to Al Kafka into a topic, such as ai-guardian.nr_anomalies.


The disclosed implementations use baseline distributions as support for anomaly creation, which means that periodically a smaller baseline is created using the same query for the current day per agent and then compares the results to the matching baseline. The following categories can be used for aggregations of data and baselines:

    • Logs
      • Agent sessions—The number of sessions of this agent increased;
      • Agent IPs—The number of IP addresses used by this agent has grown;
      • Agent countries—Higher agent login related activity by country;
      • Agent browser—The number of browsers used by this agent has grown;
      • Agent operating systems—The number of operating systems used by this agent has grown;
    • Calls
      • Inbound calls—The user has a different number of inbound calls;
      • Outbound calls—User has a different number of outbound calls;
      • Calls for the same number—The user has made a different number of calls to the same number. As the number of calls performed or answered is often used as a metric to evaluate the agent, calling the same number many times is considered gaming the system which is something that we want to detect.


The anomaly detection algorithm is based on the calculation of the probability of a certain type of event to happen based on baselines. For each use case, the probability distributions of the event to occur is calculated in the baselines for both each agent or the agent's peers. A probability model is a mathematical representation of a random phenomenon. It is defined by its sample space, events within the sample space, and probabilities associated with each event. The sample space S for a probability model is the set of all possible outcomes. Various probability models can be used to determine the probability distributions. for example, binomial distribution, Poisson distribution, normal distribution, and/or bivariate normal distribution probability models can be used.


A low probability indicates that the event is unlikely to occur. Therefore, the anomaly is actually ranked higher. Together with the probability, it is helpful to also consider the confidence that the use case is actually an anomaly. So, the anomaly detection algorithm cand use a combination of probability and confidence thresholds to analyze the anomaly. For example, anomalies can be persisted (detected) only when the probability is low, less than, for example, 0.05, and the confidence is high, above, for example, 0.85. The stream processing and use of multiple probabilistic baselines, as disclosed herein allows the disclosed implementations to reliably detect call center anomalies in a meaningful manner is substantially real-time.



FIG. 6 is a flow chart of a process for creating distributions in accordance with disclosed implementations. Process 600 starts at 602 by monitoring at least one stream of communication activity data indicating parameters of communication activities between call center agents and call center users. At 604, the communication activity data is stored in a database. At 606, the data is aggregated and at 508 distributions are created based on the aggregated data. The distributions can include an agent baseline distribution of communication metrics for each of the agents per day and a peer baseline distribution of communication metrics for at least one group of the agents per day.



FIG. 7 is a flow chart of a process for detecting anomalies in accordance with disclosed implementations. Process 700 begins at 702 by monitoring event parameters of communication activities between call center agents and user. At 04, a baseline distribution database is queried to determine that an event parameter represents a communication anomaly when the event parameter indicates an event that corresponds to a probability that is lower than a predetermined threshold probability and a calculated confidence of the event is higher than a predetermined confidence threshold, wherein the baseline distribution database is created by monitoring at least one stream of communication activity data indicating parameters of communication activities between call center agents and call center users, storing the communication activity data in a collected data database and aggregating the communication activity data to create an agent baseline distribution of communication metrics for each of the agents per day and a peer baseline distribution of communication metrics for at least one group of the agents per day. At 706, a record of the communication anomaly is stored in and anomaly database.


The baselines can be refreshed on a schedule. Further personalized (e.g., customer-level or industry-level) baselines can be created to facilitate multi-level anomaly detection. For example, an observation could be an outlier at the customer-level but not for the industry. In such a case a trigger rule can be applied to detect (or not detect) and anomaly. Baselines can be multiple and dynamic, and continuously updated to accommodate holidays, supply chain disruptions, and the like. The disclosed implementations leverage distributional techniques to compute probability of an observation to be an outlier based on the computed baselines. Complex observations can be modeled using correlation based techniques using high-dimensional data.


A given computing platform may include one or more processors configured to execute computer program modules. The computer program modules associated with the computing platform allow the computing platform to provide the functionality disclosed herein. Computing platforms may include electronic storage, one or more processors, and/or other components, such as communication lines, or ports to enable the exchange of information with a network and/or other computing platforms. Electronic storage devices may comprise non-transitory storage media that electronically stores information. Electronic storage may include one or more of optically readable storage media (e.g., optical disks, etc.), magnetically readable storage media (e.g., magnetic tape, magnetic hard drive, floppy drive, etc.), electrical charge-based storage media (e.g., EEPROM, RAM, etc.), solid-state storage media (e.g., flash drive, etc.), and/or other electronically readable storage media. Electronic storage may include one or more virtual storage resources (e.g., cloud storage, a virtual private network, and/or other virtual storage resources). Electronic storage may store software algorithms, information determined by processor(s) and/or other information that enables server(s) 202 to function as described herein.


Processor(s) may be configured to provide information processing capabilities and may include one or more of a digital processor, an analog processor, a digital circuit designed to process information, an analog circuit designed to process information, a state machine, and/or other mechanisms for electronically processing information. As used herein, the term “module” may refer to any component or set of components that perform the functionality attributed to the module. This may include one or more physical processors during execution of processor readable instructions, the processor readable instructions, circuitry, hardware, storage media, or any other components.


It will be appreciated by those skilled in the art that changes could be made to the embodiments described above without departing from the broad inventive concept thereof. It is understood, therefore, that this invention is not limited to the particular implementations disclosed, but it is intended to cover modifications within the spirit and scope of the present invention as defined by the appended claims.

Claims
  • 1. A method for increasing security in a call center implemented over a computing network by detecting anomalies in communication activities between call center agents and call center users, the method comprising: monitoring event parameters of communication activities between call center agents and user;querying a baseline distribution database to determine that an event parameter represents a communication anomaly when the event parameter indicates an event that corresponds to a probability that is lower than a predetermined threshold probability and a calculated confidence of the event is higher than a predetermined confidence threshold, wherein the baseline distribution database is created by monitoring at least one stream of communication activity data indicating parameters of communication activities between call center agents and call center users, storing the communication activity data in a collected data database and aggregating the communication activity data to create at least one distribution of communication metrics over a period of time; andstoring a record of the communication anomaly in an anomaly database.
  • 2. The method of claim 1, wherein at least one stream of communication activity data includes at least one of event audit logs, communication events, account information, and agent status data.
  • 3. The method of claim 2, further comprising creating use case specific tables and analytics based on the communication activity data.
  • 4. The method of claim 1, wherein the at least one distribution includes an agent baseline distribution of communication metrics for each of the agents per day and a peer baseline distribution of communication metrics for at least one group of the agents per day.
  • 5. The method of claim 1, wherein the at least one stream of communication data includes direct data about communications and derived data about communications.
  • 6. A system for increasing security in a call center implemented over a computing network by detecting anomalies in communication activities between call center agents and call center users, the system comprising: at least one memory storing computer executable instructions; andat least one processor which, when executing the instructions accomplishes the method of: monitoring event parameters of communication activities between call center agents and user;querying a baseline distribution database to determine that an event parameter represents a communication anomaly when the event parameter indicates an event that corresponds to a probability that is lower than a predetermined threshold probability and a calculated confidence of the event is higher than a predetermined confidence threshold, wherein the baseline distribution database is created by monitoring at least one stream of communication activity data indicating parameters of communication activities between call center agents and call center users, storing the communication activity data in a collected data database and aggregating the communication activity data to create at least one distribution of communication metrics over a period of time; andstoring a record of the communication anomaly in an anomaly database.
  • 7. The system of claim 6, wherein at least one stream of communication activity data includes at least one of event audit logs, communication events, account information, and agent status data.
  • 8. The system of claim 7, wherein the method further comprises creating use case specific tables and analytics based on the communication activity data.
  • 9. The system of claim 6, wherein the at least one distribution includes an agent baseline distribution of communication metrics for each of the agents per day and a peer baseline distribution of communication metrics for at least one group of the agents per day.
  • 10. The system of claim 6, wherein the at least one stream of communication data includes direct data about communications and derived data about communications.
US Referenced Citations (678)
Number Name Date Kind
5343518 Kneipp Aug 1994 A
5570419 Cave et al. Oct 1996 A
5862203 Wulkan et al. Jan 1999 A
5897616 Kanevsky et al. Apr 1999 A
5966691 Kibre et al. Oct 1999 A
5970124 Csaszar et al. Oct 1999 A
6100891 Thorne Aug 2000 A
6128415 Hultgren et al. Oct 2000 A
6163607 Bogart et al. Dec 2000 A
6230197 Beck et al. May 2001 B1
6263057 Silverman Jul 2001 B1
6263065 Durinovic-Johri et al. Jul 2001 B1
6345093 Lee et al. Feb 2002 B1
6373938 Palacios et al. Apr 2002 B1
6377944 Busey et al. Apr 2002 B1
6385584 McAlister et al. May 2002 B1
6411687 Bohacek et al. Jun 2002 B1
6493695 Pickering et al. Dec 2002 B1
6560222 Pounds et al. May 2003 B1
6587831 O'Brien Jul 2003 B1
6639982 Stuart et al. Oct 2003 B1
6721416 Farrell Apr 2004 B1
6754333 Flockhart et al. Jun 2004 B1
6859776 Cohen et al. Feb 2005 B1
6970829 Leamon Nov 2005 B1
7023979 Wu et al. Apr 2006 B1
7076047 Brennan et al. Jul 2006 B1
7110525 Heller et al. Sep 2006 B1
7209475 Shaffer et al. Apr 2007 B1
7274787 Schoeneberger Sep 2007 B1
7292689 Odinak et al. Nov 2007 B2
7343406 Buonanno et al. Mar 2008 B1
7372952 Wu et al. May 2008 B1
7382773 Schoeneberger et al. Jun 2008 B2
7409336 Pak et al. Aug 2008 B2
7426268 Walker et al. Sep 2008 B2
7466334 Baba Dec 2008 B1
7478051 Nourbakhsh et al. Jan 2009 B2
7537154 Ramachandran May 2009 B2
7634422 Andre et al. Dec 2009 B1
7657263 Chahrouri Feb 2010 B1
7664641 Pettay et al. Feb 2010 B1
7672746 Hamilton et al. Mar 2010 B1
7672845 Beranek et al. Mar 2010 B2
7676034 Wu et al. Mar 2010 B1
7698163 Reed et al. Apr 2010 B2
7752159 Nelken et al. Jul 2010 B2
7774790 Jirman et al. Aug 2010 B1
7788286 Nourbakhsh et al. Aug 2010 B2
7853006 Fama et al. Dec 2010 B1
7864946 Fama et al. Jan 2011 B1
7869998 Di Fabbrizio et al. Jan 2011 B1
7949123 Flockhart et al. May 2011 B1
7953219 Freedman et al. May 2011 B2
7966187 Pettay et al. Jun 2011 B1
7966369 Briere et al. Jun 2011 B1
8060394 Woodings et al. Nov 2011 B2
8073129 Kalavar Dec 2011 B1
8116446 Kalavar Feb 2012 B1
8135125 Sidhu et al. Mar 2012 B2
8160233 Keren et al. Apr 2012 B2
8184782 Vatland et al. May 2012 B1
8223951 Edelhaus et al. Jul 2012 B1
8229761 Backhaus et al. Jul 2012 B2
8243896 Rae Aug 2012 B1
8300798 Wu et al. Oct 2012 B1
8335704 Trefler et al. Dec 2012 B2
8369338 Peng et al. Feb 2013 B1
8370155 Byrd et al. Feb 2013 B2
8391466 Noble, Jr. Mar 2013 B1
8447279 Peng et al. May 2013 B1
8488769 Noble et al. Jul 2013 B1
8526576 Deich et al. Sep 2013 B1
8535059 Noble, Jr. et al. Sep 2013 B1
8583466 Margulies et al. Nov 2013 B2
8594306 Laredo et al. Nov 2013 B2
8626137 Devitt et al. Jan 2014 B1
8635226 Chang et al. Jan 2014 B2
8644489 Noble et al. Feb 2014 B1
8671020 Morrison et al. Mar 2014 B1
8688557 Rose et al. Apr 2014 B2
8738739 Makar et al. May 2014 B2
8767948 Riahi et al. Jul 2014 B1
8811597 Hackbarth et al. Aug 2014 B1
8861691 De et al. Oct 2014 B1
8869245 Ranganathan et al. Oct 2014 B2
8898219 Ricci Nov 2014 B2
8898290 Siemsgluess Nov 2014 B2
8909693 Frissora et al. Dec 2014 B2
8935172 Noble, Jr. et al. Jan 2015 B1
8995648 Gibbs et al. Mar 2015 B1
8996509 Sundaram Mar 2015 B1
9020142 Kosiba et al. Apr 2015 B2
9026431 Moreno Mengibar et al. May 2015 B1
9060057 Danis Jun 2015 B1
9065915 Lillard et al. Jun 2015 B1
9082094 Etter et al. Jul 2015 B1
9100483 Snedden Aug 2015 B1
9117450 Cook et al. Aug 2015 B2
9123009 Etter et al. Sep 2015 B1
9137366 Medina et al. Sep 2015 B2
9152737 Micali et al. Oct 2015 B1
9160853 Daddi et al. Oct 2015 B1
9178999 Hegde et al. Nov 2015 B1
9185222 Govindarajan et al. Nov 2015 B1
9237232 Williams et al. Jan 2016 B1
9280754 Schwartz et al. Mar 2016 B1
9286413 Coates et al. Mar 2016 B1
9300801 Warford et al. Mar 2016 B1
9317825 Defusco et al. Apr 2016 B2
9319524 Webster Apr 2016 B1
9386152 Riahi et al. Jul 2016 B2
9397985 Seger et al. Jul 2016 B1
9426291 Ouimette et al. Aug 2016 B1
9473637 Venkatapathy et al. Oct 2016 B1
9514463 Grigg et al. Dec 2016 B2
9595049 Showers et al. Mar 2017 B2
9602665 Koster Mar 2017 B1
9609131 Placiakis et al. Mar 2017 B2
9674361 Ristock et al. Jun 2017 B2
9679265 Schwartz et al. Jun 2017 B1
9774731 Haltom et al. Sep 2017 B1
9787840 Neuer, III et al. Oct 2017 B1
9813495 Van et al. Nov 2017 B1
9813559 Noble et al. Nov 2017 B1
9823949 Ristock et al. Nov 2017 B2
9883037 Lewis et al. Jan 2018 B1
9894478 Deluca et al. Feb 2018 B1
9930181 Moran et al. Mar 2018 B1
9955021 Liu et al. Apr 2018 B1
RE46852 Petrovykh May 2018 E
9998596 Dunmire et al. Jun 2018 B1
10009465 Fang et al. Jun 2018 B1
10038788 Khalatian Jul 2018 B1
10044862 Cai et al. Aug 2018 B1
10079939 Bostick et al. Sep 2018 B1
10085073 Ray et al. Sep 2018 B2
10101974 Ristock et al. Oct 2018 B2
10115065 Fama et al. Oct 2018 B1
10135973 Algard et al. Nov 2018 B2
10154138 Te Booij et al. Dec 2018 B2
10194027 Daddi et al. Jan 2019 B1
10235999 Naughton et al. Mar 2019 B1
10241752 Lemay et al. Mar 2019 B2
10242019 Shan et al. Mar 2019 B1
10276170 Gruber et al. Apr 2019 B2
10277745 Araujo et al. Apr 2019 B1
10290017 Traasdahl et al. May 2019 B2
10331402 Spector et al. Jun 2019 B1
10354677 Mohamed et al. Jul 2019 B2
10380246 Clark et al. Aug 2019 B2
10440180 Jayapalan et al. Oct 2019 B1
10445742 Prendki et al. Oct 2019 B2
10460728 Anbazhagan et al. Oct 2019 B2
10497361 Rule et al. Dec 2019 B1
10554590 Cabrera-Cordon et al. Feb 2020 B2
10554817 Sullivan et al. Feb 2020 B1
10572879 Hunter et al. Feb 2020 B1
10574822 Sheshaiahgari et al. Feb 2020 B1
10601992 Dwyer et al. Mar 2020 B2
10623572 Copeland Apr 2020 B1
10635973 Dirac et al. Apr 2020 B1
10636425 Naughton et al. Apr 2020 B2
10699303 Ismail et al. Jun 2020 B2
10715648 Vashisht et al. Jul 2020 B1
10718031 Wu et al. Jul 2020 B1
10728384 Channakeshava et al. Jul 2020 B1
10735586 Johnston Aug 2020 B1
10742806 Kotak Aug 2020 B2
10750019 Petrovykh et al. Aug 2020 B1
10783568 Chandra et al. Sep 2020 B1
10789956 Dube Sep 2020 B1
10803865 Naughton et al. Oct 2020 B2
10812654 Wozniak Oct 2020 B2
10812655 Adibi et al. Oct 2020 B1
10827069 Paiva Nov 2020 B1
10827071 Adibi et al. Nov 2020 B1
10839432 Konig et al. Nov 2020 B1
10841425 Langley et al. Nov 2020 B1
10855844 Smith et al. Dec 2020 B1
10861031 Sullivan et al. Dec 2020 B2
10878479 Wu et al. Dec 2020 B2
10923127 Mckenzie et al. Feb 2021 B2
10929796 Stepanov Feb 2021 B1
10943589 Naughton et al. Mar 2021 B2
10970682 Aykin Apr 2021 B1
11017176 Ayers et al. May 2021 B2
11089158 Holland et al. Aug 2021 B1
11417343 Cohen et al. Aug 2022 B2
11425252 Martin et al. Aug 2022 B1
20010008999 Bull Jul 2001 A1
20010024497 Campbell Sep 2001 A1
20010054072 Discolo et al. Dec 2001 A1
20020019737 Stuart et al. Feb 2002 A1
20020029272 Weller Mar 2002 A1
20020034304 Yang Mar 2002 A1
20020038420 Collins et al. Mar 2002 A1
20020067823 Walker et al. Jun 2002 A1
20020143599 Nourbakhsh et al. Oct 2002 A1
20020169664 Walker et al. Nov 2002 A1
20020174182 Wilkinson et al. Nov 2002 A1
20020181689 Rupe et al. Dec 2002 A1
20030007621 Graves et al. Jan 2003 A1
20030009520 Nourbakhsh et al. Jan 2003 A1
20030032409 Hutcheson et al. Feb 2003 A1
20030061068 Curtis Mar 2003 A1
20030112927 Brown et al. Jun 2003 A1
20030126136 Omoigui Jul 2003 A1
20030154072 Young et al. Aug 2003 A1
20030167167 Gong Sep 2003 A1
20040044585 Franco Mar 2004 A1
20040044664 Cash et al. Mar 2004 A1
20040062364 Dezonno et al. Apr 2004 A1
20040078257 Schweitzer et al. Apr 2004 A1
20040098274 Dezonno et al. May 2004 A1
20040103051 Reed et al. May 2004 A1
20040141508 Schoeneberger et al. Jul 2004 A1
20040162724 Hill et al. Aug 2004 A1
20040162753 Vogel et al. Aug 2004 A1
20040174980 Knott et al. Sep 2004 A1
20040215451 MacLeod Oct 2004 A1
20050033957 Enokida Feb 2005 A1
20050043986 Mcconnell et al. Feb 2005 A1
20050063365 Mathew et al. Mar 2005 A1
20050065837 Kosiba et al. Mar 2005 A1
20050071178 Beckstrom et al. Mar 2005 A1
20050105712 Williams et al. May 2005 A1
20050177368 Odinak et al. Aug 2005 A1
20050226220 Kilkki et al. Oct 2005 A1
20050228774 Ronnewinkel Oct 2005 A1
20050246511 Willman et al. Nov 2005 A1
20050271198 Chin et al. Dec 2005 A1
20060095575 Sureka et al. May 2006 A1
20060126818 Berger et al. Jun 2006 A1
20060153357 Acharya et al. Jul 2006 A1
20060166669 Claussen Jul 2006 A1
20060173724 Trefler et al. Aug 2006 A1
20060188086 Busey et al. Aug 2006 A1
20060209797 Anisimov et al. Sep 2006 A1
20060215831 Knott et al. Sep 2006 A1
20060229931 Fligler et al. Oct 2006 A1
20060256953 Pulaski et al. Nov 2006 A1
20060271361 Vora et al. Nov 2006 A1
20060274856 Dun et al. Dec 2006 A1
20060277108 Altberg et al. Dec 2006 A1
20070011153 Pillai et al. Jan 2007 A1
20070016565 Evans et al. Jan 2007 A1
20070036334 Culbertson et al. Feb 2007 A1
20070038499 Margulies et al. Feb 2007 A1
20070041519 Erhart et al. Feb 2007 A1
20070061183 Seetharaman et al. Mar 2007 A1
20070078725 Koszewski et al. Apr 2007 A1
20070121894 Noble May 2007 A1
20070121902 Stoica et al. May 2007 A1
20070121903 Moore, Jr. et al. May 2007 A1
20070133760 Cotignola et al. Jun 2007 A1
20070136284 Cobb et al. Jun 2007 A1
20070155411 Morrison Jul 2007 A1
20070157021 Whitfield Jul 2007 A1
20070160188 Sharpe et al. Jul 2007 A1
20070162296 Altberg et al. Jul 2007 A1
20070198329 Lyerly et al. Aug 2007 A1
20070201636 Gilbert et al. Aug 2007 A1
20070211881 Parker-Stephen Sep 2007 A1
20070263810 Sterns Nov 2007 A1
20070265990 Sidhu et al. Nov 2007 A1
20070269031 Honig et al. Nov 2007 A1
20070280460 Harris et al. Dec 2007 A1
20070287430 Hosain et al. Dec 2007 A1
20080002823 Fama et al. Jan 2008 A1
20080004933 Gillespie Jan 2008 A1
20080043976 Maximo et al. Feb 2008 A1
20080065902 Spohrer et al. Mar 2008 A1
20080095355 Mahalaha et al. Apr 2008 A1
20080115213 Bhatt May 2008 A1
20080126957 Tysowski et al. May 2008 A1
20080205620 Odinak et al. Aug 2008 A1
20080225872 Collins et al. Sep 2008 A1
20080254774 Lee Oct 2008 A1
20080255944 Shah et al. Oct 2008 A1
20080260138 Chen et al. Oct 2008 A1
20080288770 Kline et al. Nov 2008 A1
20080300955 Hamilton et al. Dec 2008 A1
20090018996 Hunt et al. Jan 2009 A1
20090080411 Lyman Mar 2009 A1
20090086945 Buchanan et al. Apr 2009 A1
20090086949 Caspi et al. Apr 2009 A1
20090086953 Vendrow Apr 2009 A1
20090110182 Knight, Jr. et al. Apr 2009 A1
20090171164 Jung et al. Jul 2009 A1
20090222551 Neely et al. Sep 2009 A1
20090228264 Williams et al. Sep 2009 A1
20090234710 Belgaied et al. Sep 2009 A1
20090234732 Zorman et al. Sep 2009 A1
20090245479 Surendran Oct 2009 A1
20090285384 Pollock et al. Nov 2009 A1
20090306981 Cromack et al. Dec 2009 A1
20090307052 Mankani et al. Dec 2009 A1
20100106568 Grimes Apr 2010 A1
20100114645 Hamilton et al. May 2010 A1
20100114646 Mcilwain et al. May 2010 A1
20100165977 Mccord Jul 2010 A1
20100189249 Shah et al. Jul 2010 A1
20100189250 Williams et al. Jul 2010 A1
20100211515 Woodings et al. Aug 2010 A1
20100235341 Bennett Sep 2010 A1
20100250196 Lawler et al. Sep 2010 A1
20100262549 Kannan et al. Oct 2010 A1
20100266115 Fedorov et al. Oct 2010 A1
20100266116 Stolyar et al. Oct 2010 A1
20100274618 Byrd et al. Oct 2010 A1
20100287131 Church Nov 2010 A1
20100293033 Hall et al. Nov 2010 A1
20100299268 Guha et al. Nov 2010 A1
20100332287 Gates et al. Dec 2010 A1
20110014932 Estevez Jan 2011 A1
20110022461 Simeonov Jan 2011 A1
20110071870 Gong Mar 2011 A1
20110077994 Segev et al. Mar 2011 A1
20110082688 Kim et al. Apr 2011 A1
20110116618 Zyarko et al. May 2011 A1
20110125697 Erhart et al. May 2011 A1
20110143323 Cohen Jun 2011 A1
20110182283 Van et al. Jul 2011 A1
20110185293 Barnett et al. Jul 2011 A1
20110194684 Ristock et al. Aug 2011 A1
20110216897 Laredo et al. Sep 2011 A1
20110264581 Clyne Oct 2011 A1
20110267985 Wilkinson et al. Nov 2011 A1
20110286592 Nimmagadda Nov 2011 A1
20110288897 Erhart et al. Nov 2011 A1
20120046996 Shah et al. Feb 2012 A1
20120051537 Chishti et al. Mar 2012 A1
20120084217 Kohler et al. Apr 2012 A1
20120087486 Guerrero et al. Apr 2012 A1
20120095835 Makar et al. Apr 2012 A1
20120109830 Vogel May 2012 A1
20120257116 Hendrickson et al. Oct 2012 A1
20120265587 Kinkead Oct 2012 A1
20120290373 Ferzacca et al. Nov 2012 A1
20120321073 Flockhart et al. Dec 2012 A1
20130023235 Fan et al. Jan 2013 A1
20130060587 Bayrak et al. Mar 2013 A1
20130073361 Silver Mar 2013 A1
20130085785 Rogers et al. Apr 2013 A1
20130090963 Sharma et al. Apr 2013 A1
20130124361 Bryson May 2013 A1
20130136252 Kosiba et al. May 2013 A1
20130223608 Flockhart et al. Aug 2013 A1
20130223610 Kohler et al. Aug 2013 A1
20130236002 Jennings et al. Sep 2013 A1
20130257877 Davis Oct 2013 A1
20130304581 Soroca et al. Nov 2013 A1
20130325972 Boston et al. Dec 2013 A1
20140012603 Scanlon et al. Jan 2014 A1
20140016762 Mitchell et al. Jan 2014 A1
20140039944 Humbert et al. Feb 2014 A1
20140039962 Nudd et al. Feb 2014 A1
20140067375 Wooters Mar 2014 A1
20140079195 Srivastava et al. Mar 2014 A1
20140079207 Zhakov et al. Mar 2014 A1
20140099916 Mallikarjunan et al. Apr 2014 A1
20140101261 Wu et al. Apr 2014 A1
20140136346 Teso May 2014 A1
20140140494 Zhakov May 2014 A1
20140143018 Nies et al. May 2014 A1
20140143249 Cazzanti et al. May 2014 A1
20140161241 Baranovsky et al. Jun 2014 A1
20140164502 Khodorenko et al. Jun 2014 A1
20140177819 Vymenets et al. Jun 2014 A1
20140188477 Zhang Jul 2014 A1
20140200988 Kassko et al. Jul 2014 A1
20140219132 Delveaux et al. Aug 2014 A1
20140219438 Brown et al. Aug 2014 A1
20140233719 Vyemenets et al. Aug 2014 A1
20140244712 Walters et al. Aug 2014 A1
20140254790 Shaffer et al. Sep 2014 A1
20140257908 Steiner et al. Sep 2014 A1
20140270108 Riahi et al. Sep 2014 A1
20140270138 Uba et al. Sep 2014 A1
20140270142 Bischoff et al. Sep 2014 A1
20140270145 Erhart et al. Sep 2014 A1
20140278605 Borucki et al. Sep 2014 A1
20140278649 Guerinik et al. Sep 2014 A1
20140279045 Shottan et al. Sep 2014 A1
20140279050 Makar et al. Sep 2014 A1
20140314225 Riahi et al. Oct 2014 A1
20140335480 Asenjo et al. Nov 2014 A1
20140372171 Martin et al. Dec 2014 A1
20140379424 Shroff Dec 2014 A1
20150006400 Eng et al. Jan 2015 A1
20150010134 Erel et al. Jan 2015 A1
20150012278 Metcalf Jan 2015 A1
20150016600 Desai et al. Jan 2015 A1
20150023484 Ni et al. Jan 2015 A1
20150030151 Bellini et al. Jan 2015 A1
20150030152 Waxman et al. Jan 2015 A1
20150051957 Griebeler et al. Feb 2015 A1
20150066632 Gonzalez et al. Mar 2015 A1
20150071418 Shaffer et al. Mar 2015 A1
20150078538 Jain Mar 2015 A1
20150100473 Manoharan et al. Apr 2015 A1
20150127400 Chan et al. May 2015 A1
20150127441 Feldman May 2015 A1
20150127677 Wang et al. May 2015 A1
20150142704 London May 2015 A1
20150172463 Quast et al. Jun 2015 A1
20150178371 Seth et al. Jun 2015 A1
20150195406 Dwyer et al. Jul 2015 A1
20150213454 Vedula Jul 2015 A1
20150215464 Shaffer et al. Jul 2015 A1
20150222751 Odinak et al. Aug 2015 A1
20150256677 Konig et al. Sep 2015 A1
20150262188 Franco Sep 2015 A1
20150262208 Bjontegard et al. Sep 2015 A1
20150269377 Gaddipati Sep 2015 A1
20150271334 Wawrzynowicz Sep 2015 A1
20150281445 Kumar et al. Oct 2015 A1
20150281449 Milstein et al. Oct 2015 A1
20150281450 Shapiro et al. Oct 2015 A1
20150281454 Milstein et al. Oct 2015 A1
20150287410 Mengibar et al. Oct 2015 A1
20150295788 Witzman et al. Oct 2015 A1
20150296081 Jeong Oct 2015 A1
20150302301 Petersen Oct 2015 A1
20150334230 Volzke Nov 2015 A1
20150339446 Sperling et al. Nov 2015 A1
20150339620 Esposito et al. Nov 2015 A1
20150339769 Deoliveira et al. Nov 2015 A1
20150347900 Bell et al. Dec 2015 A1
20150350429 Kumar et al. Dec 2015 A1
20150350440 Steiner et al. Dec 2015 A1
20150350442 O'connor Dec 2015 A1
20150350443 Kumar et al. Dec 2015 A1
20150379562 Spievak et al. Dec 2015 A1
20160026629 Clifford et al. Jan 2016 A1
20160034260 Ristock et al. Feb 2016 A1
20160034995 Williams et al. Feb 2016 A1
20160036981 Hollenberg et al. Feb 2016 A1
20160036983 Korolev et al. Feb 2016 A1
20160042419 Singh Feb 2016 A1
20160042749 Hirose Feb 2016 A1
20160055499 Hawkins et al. Feb 2016 A1
20160057284 Nagpal et al. Feb 2016 A1
20160065739 Brimshan et al. Mar 2016 A1
20160080567 Hooshiari et al. Mar 2016 A1
20160085891 Ter et al. Mar 2016 A1
20160112867 Martinez Apr 2016 A1
20160124937 Elhaddad May 2016 A1
20160125456 Wu et al. May 2016 A1
20160134624 Jacobson et al. May 2016 A1
20160140627 Moreau et al. May 2016 A1
20160150086 Pickford May 2016 A1
20160155080 Gnanasambandam et al. Jun 2016 A1
20160162478 Blassin et al. Jun 2016 A1
20160171422 Wicaksono et al. Jun 2016 A1
20160173692 Wicaksono et al. Jun 2016 A1
20160180381 Kaiser et al. Jun 2016 A1
20160191699 Agrawal et al. Jun 2016 A1
20160191709 Pullamplavil et al. Jun 2016 A1
20160191712 Bouzid et al. Jun 2016 A1
20160234386 Wawrzynowicz Aug 2016 A1
20160247165 Ryabchun et al. Aug 2016 A1
20160261747 Thirugnanasundaram et al. Aug 2016 A1
20160295018 Loftus et al. Oct 2016 A1
20160295020 Shaffer et al. Oct 2016 A1
20160300573 Carbune et al. Oct 2016 A1
20160335576 Peng Nov 2016 A1
20160349960 Kumar et al. Dec 2016 A1
20160358611 Abel Dec 2016 A1
20160360033 Kocan Dec 2016 A1
20160360336 Gross et al. Dec 2016 A1
20160378569 Ristock et al. Dec 2016 A1
20160381222 Ristock et al. Dec 2016 A1
20170004178 Ponting et al. Jan 2017 A1
20170006135 Siebel et al. Jan 2017 A1
20170006161 Riahi et al. Jan 2017 A9
20170011311 Backer et al. Jan 2017 A1
20170024762 Swaminathan Jan 2017 A1
20170032436 Disalvo et al. Feb 2017 A1
20170034226 Bostick et al. Feb 2017 A1
20170068436 Auer et al. Mar 2017 A1
20170068854 Markiewicz et al. Mar 2017 A1
20170098197 Yu et al. Apr 2017 A1
20170104875 Im et al. Apr 2017 A1
20170111505 Mcgann et al. Apr 2017 A1
20170111509 McGann et al. Apr 2017 A1
20170116173 Lev-Tov et al. Apr 2017 A1
20170118336 Tapuhi et al. Apr 2017 A1
20170132536 Goldstein et al. May 2017 A1
20170148073 Nomula et al. May 2017 A1
20170155766 Kumar et al. Jun 2017 A1
20170161439 Raduchel et al. Jun 2017 A1
20170162197 Cohen Jun 2017 A1
20170169325 McCord et al. Jun 2017 A1
20170207916 Luce et al. Jul 2017 A1
20170214795 Charlson Jul 2017 A1
20170220966 Wang Aug 2017 A1
20170223070 Lin Aug 2017 A1
20170236512 Williams et al. Aug 2017 A1
20170286774 Gaidon Oct 2017 A1
20170288866 Vanek et al. Oct 2017 A1
20170308794 Fischerstrom Oct 2017 A1
20170316386 Joshi et al. Nov 2017 A1
20170323344 Nigul Nov 2017 A1
20170337578 Chittilappilly et al. Nov 2017 A1
20170344754 Kumar et al. Nov 2017 A1
20170344988 Cusden et al. Nov 2017 A1
20170359421 Stoops et al. Dec 2017 A1
20170372436 Dalal et al. Dec 2017 A1
20180018705 Tognetti Jan 2018 A1
20180032997 Gordon et al. Feb 2018 A1
20180052664 Zhang et al. Feb 2018 A1
20180053401 Martin et al. Feb 2018 A1
20180054464 Zhang et al. Feb 2018 A1
20180060830 Abramovici et al. Mar 2018 A1
20180061256 Elchik et al. Mar 2018 A1
20180077088 Cabrera-Cordon et al. Mar 2018 A1
20180077250 Prasad et al. Mar 2018 A1
20180083898 Pham Mar 2018 A1
20180097910 D'Agostino et al. Apr 2018 A1
20180114234 Fighel Apr 2018 A1
20180121766 Mccord et al. May 2018 A1
20180137472 Gorzela et al. May 2018 A1
20180137555 Clausse et al. May 2018 A1
20180146093 Kumar et al. May 2018 A1
20180150749 Wu et al. May 2018 A1
20180152558 Chan et al. May 2018 A1
20180164259 Liu et al. Jun 2018 A1
20180165062 Yoo et al. Jun 2018 A1
20180165691 Heater et al. Jun 2018 A1
20180165692 McCoy Jun 2018 A1
20180165723 Wright et al. Jun 2018 A1
20180174198 Wilkinson et al. Jun 2018 A1
20180189273 Campos et al. Jul 2018 A1
20180190144 Corelli et al. Jul 2018 A1
20180198917 Ristock et al. Jul 2018 A1
20180205825 Vymenets et al. Jul 2018 A1
20180248818 Zucker et al. Aug 2018 A1
20180248895 Watson Aug 2018 A1
20180260857 Kar et al. Sep 2018 A1
20180285423 Ciano et al. Oct 2018 A1
20180286000 Berry et al. Oct 2018 A1
20180293327 Miller et al. Oct 2018 A1
20180293532 Singh et al. Oct 2018 A1
20180300295 Maksak et al. Oct 2018 A1
20180300641 Donn et al. Oct 2018 A1
20180308072 Smith et al. Oct 2018 A1
20180309801 Rathod Oct 2018 A1
20180349858 Walker et al. Dec 2018 A1
20180361253 Grosso Dec 2018 A1
20180365651 Sreedhara et al. Dec 2018 A1
20180367672 Ristock et al. Dec 2018 A1
20180372486 Farniok et al. Dec 2018 A1
20180376002 Abraham Dec 2018 A1
20190013017 Kang et al. Jan 2019 A1
20190020757 Rao Jan 2019 A1
20190028587 Unitt et al. Jan 2019 A1
20190028588 Shinseki et al. Jan 2019 A1
20190037077 Konig et al. Jan 2019 A1
20190042988 Brown et al. Feb 2019 A1
20190043106 Talmor et al. Feb 2019 A1
20190058793 Konig et al. Feb 2019 A1
20190104092 Koohmarey et al. Apr 2019 A1
20190108834 Nelson et al. Apr 2019 A1
20190124202 Dubey et al. Apr 2019 A1
20190130329 Fama et al. May 2019 A1
20190132443 Munns et al. May 2019 A1
20190146647 Ramachandran et al. May 2019 A1
20190147045 Kim May 2019 A1
20190172291 Naseath Jun 2019 A1
20190180095 Ferguson et al. Jun 2019 A1
20190180747 Back et al. Jun 2019 A1
20190182383 Shaev et al. Jun 2019 A1
20190196676 Hillis et al. Jun 2019 A1
20190197568 Li et al. Jun 2019 A1
20190205389 Tripathi et al. Jul 2019 A1
20190236205 Jia et al. Aug 2019 A1
20190238680 Narayanan et al. Aug 2019 A1
20190253553 Chishti Aug 2019 A1
20190258825 Krishnamurthy Aug 2019 A1
20190287517 Green et al. Sep 2019 A1
20190295027 Dunne et al. Sep 2019 A1
20190306315 Portman et al. Oct 2019 A1
20190335038 Alonso Y Caloca et al. Oct 2019 A1
20190341030 Hammons et al. Nov 2019 A1
20190342450 Kulkarni et al. Nov 2019 A1
20190349477 Kotak Nov 2019 A1
20190377789 Jegannathan et al. Dec 2019 A1
20190378076 O'Gorman et al. Dec 2019 A1
20190385597 Katsamanis et al. Dec 2019 A1
20190386917 Malin Dec 2019 A1
20190392357 Surti et al. Dec 2019 A1
20190394333 Jiron et al. Dec 2019 A1
20200005375 Sharan et al. Jan 2020 A1
20200007680 Wozniak Jan 2020 A1
20200012697 Fan et al. Jan 2020 A1
20200012992 Chan et al. Jan 2020 A1
20200019893 Lu Jan 2020 A1
20200028968 Mendiratta et al. Jan 2020 A1
20200050788 Feuz et al. Feb 2020 A1
20200050996 Generes, Jr. et al. Feb 2020 A1
20200058299 Lee et al. Feb 2020 A1
20200076947 Deole Mar 2020 A1
20200097544 Alexander et al. Mar 2020 A1
20200104801 Kwon et al. Apr 2020 A1
20200118215 Rao et al. Apr 2020 A1
20200119936 Balasaygun et al. Apr 2020 A1
20200125919 Liu et al. Apr 2020 A1
20200126126 Briancon et al. Apr 2020 A1
20200128130 Geary Apr 2020 A1
20200134492 Copeland Apr 2020 A1
20200134648 Qi et al. Apr 2020 A1
20200137097 Zimmermann et al. Apr 2020 A1
20200154170 Wu et al. May 2020 A1
20200160870 Baughman et al. May 2020 A1
20200175478 Lee et al. Jun 2020 A1
20200193335 Sekhar et al. Jun 2020 A1
20200193983 Choi Jun 2020 A1
20200211120 Wang et al. Jul 2020 A1
20200218766 Yaseen et al. Jul 2020 A1
20200219500 Bender et al. Jul 2020 A1
20200242540 Rosati et al. Jul 2020 A1
20200250272 Kantor et al. Aug 2020 A1
20200250557 Kishimoto et al. Aug 2020 A1
20200257996 London Aug 2020 A1
20200280578 Hearty et al. Sep 2020 A1
20200280635 Barinov et al. Sep 2020 A1
20200285936 Sen Sep 2020 A1
20200329154 Baumann et al. Oct 2020 A1
20200336567 Dumaine Oct 2020 A1
20200342868 Lou et al. Oct 2020 A1
20200351375 Lepore et al. Nov 2020 A1
20200351405 Pace Nov 2020 A1
20200357026 Liu et al. Nov 2020 A1
20200364507 Berry Nov 2020 A1
20200365148 Ji et al. Nov 2020 A1
20200380451 Izadi Dec 2020 A1
20200395008 Cohen et al. Dec 2020 A1
20200410506 Jones et al. Dec 2020 A1
20210004536 Adibi et al. Jan 2021 A1
20210005206 Adibi et al. Jan 2021 A1
20210042839 Adamec Feb 2021 A1
20210056481 Wicaksono et al. Feb 2021 A1
20210067627 Delker et al. Mar 2021 A1
20210073819 Hernandez Mar 2021 A1
20210081869 Zeelig et al. Mar 2021 A1
20210081955 Zeelig et al. Mar 2021 A1
20210082417 Zeelig et al. Mar 2021 A1
20210082418 Zeelig et al. Mar 2021 A1
20210084149 Zeelig et al. Mar 2021 A1
20210089762 Rahimi et al. Mar 2021 A1
20210090570 Aharoni et al. Mar 2021 A1
20210091996 Mcconnell et al. Mar 2021 A1
20210105361 Bergher et al. Apr 2021 A1
20210124843 Vass et al. Apr 2021 A1
20210125275 Adibi Apr 2021 A1
20210133763 Adibi et al. May 2021 A1
20210133765 Adibi et al. May 2021 A1
20210134282 Adibi et al. May 2021 A1
20210134283 Adibi et al. May 2021 A1
20210134284 Adibi et al. May 2021 A1
20210136198 Leavitt et al. May 2021 A1
20210136204 Adibi et al. May 2021 A1
20210136205 Adibi et al. May 2021 A1
20210136206 Adibi et al. May 2021 A1
20210201244 Sella et al. Jul 2021 A1
20210201359 Sekar et al. Jul 2021 A1
20210295237 Taher et al. Sep 2021 A1
20210405897 Hansalia Dec 2021 A1
20220114593 Johnson Apr 2022 A1
20220114594 Nunes Apr 2022 A1
20220116415 Burgis Apr 2022 A1
20220122182 Marshall et al. Apr 2022 A1
20220129905 Sethumadhavan et al. Apr 2022 A1
20220398682 Tam et al. Dec 2022 A1
20230007123 Krucek et al. Jan 2023 A1
20230107335 Garyani Apr 2023 A1
Foreign Referenced Citations (7)
Number Date Country
1 418 519 May 2004 EP
5986065 Sep 2016 JP
1732352 May 1992 SU
2006037836 Apr 2006 WO
2012024316 Feb 2012 WO
2015099587 Jul 2015 WO
2019142743 Jul 2019 WO
Non-Patent Literature Citations (41)
Entry
Aksin et al., “The Modern Call Center: A Multi-Disciplinary Perspective on Operations Management Research”, Production and Operations Management, 2007, vol. 16, No. 6, pp. 665-688.
Aldor-Noiman, et al., “Workload forecasting for a call center: Methodology and a case study.” The Annals of Applied Statistics 3.4 (2009); 1403-1447.
Buesing et al., “Getting the Best Customer Service from your IVR: Fresh eyes on an old problem,” [online] McKinsey and Co., published on Feb. 1, 2019, available at: < https://www.nnckinsey.conn/business-functions/operations/our-insights/ getting-the-best-customer-service-from-your-ivr-fresh-eyes . . . (Year: 2019).
Chiu et al., “A multi-agent infrastructure for mobile workforce management in a service oriented enterprise”, Proceedings of the 38th annual Hawaii international conference on system sciences, IEEE, 2005, pp. 10.
Data Warehousing in the Age of Big Data, Krishnan, 2013, Morgan Kaufmann, Chapter 5.
Diimitrios et al., “An overview of workflow management: From process modeling to workflow automation infrastructure,” Distributed and parallel Databases, 1995, vol. 3, No. 2 pp. 119-153.
Ernst et al. “An Annotated Bibliography of Personnel Scheduling and Rostering”, CSIRO Mathematical and Information Sciences, 2003, 155 pages.
Ernst et al., “Staff scheduling and rostering: A review of applications, methods and models,” European Journal of Operational Research, 2004, vol. 153, pp. 3-27.
Federal Register, vol. 72, No. 195, Oct. 10, 2007, pp. 57526-57535.
Federal Register, vol. 75, No. 169, Sep. 1, 2010, pp. 53643-53660.
Federal register, vol. 79, No. 241 issued on Dec. 16, 2014, p. 74629, col. 2, Gottschalk v. Benson.
Federal Register, vol. 84, No. 4, Jan. 7, 2019, pp. 50-57.
Federal Register, vol. 84, No. 4, Jan. 7, 2019, p. 53-55.
Grefen et al., “A reference architecture for workflow management systems”, Data & Knowledge Engineering, 1998, vol. 27, No. 1, pp. 31-57.
Huang et al., “Agent-based workflow management in collaborative product development on the Internet”, Computer-Aided Design, 2000, vol. 32, No. 2, pp. 133-144.
Janarthanam, “Hands on Chatbots and conversational UI development: Build chatbots and voice user interfaces with Chatfuel, Dialogflow, Microsoft Bot Framework, Twilio, and Alexa Skills” Dec. 2017.
Koole, et al., “An overview of routing and staffing algorithms in multi-skill customer contact centers.” 2006.
Myers et al., “At the Boundary of Workflow and AI”, Proc. AAAI 1999 Workshop on Agent-Based Systems in The Business Context, 1999, 09 pages.
Niven, “Can music with prosocial lyrics heal the working world? A field intervention in a call center.” Journal of Applied Social Psychology, 2015; 45(3), 132-138. doi:10.1111/jasp.12282 ).
On Hold Marketing, “Growing Your Business with Customized on-Hold Messaging” (Published on Apr. 5, 2018 at https://adhq.com/about/ad-news/growing-your-business-with-customized-on-hold-messaging) (Year: 2018).
U.S. Appl. No. 16/668,214, NFOA mailed Nov. 10, 2021.
U.S. Appl. No. 16/668,215, NFOA mailed Dec. 7, 2021.
Van Den Bergh et al. “Personnel scheduling: A literature review”, European journal of operational research, 2013, vol. 226, No. 3 pp. 367-385.
United States Patent and Trademark Office, Non-Final Office Action for U.S. Appl. No. 16/550,961 mailed Mar. 2, 2020.
United States Patent and Trademark Office, Final Office Action for U.S. Appl. No. 16/550,961 mailed Jun. 17, 2020.
Gaietto, Molly., “What is Customer DNA?”,—NGDATA Product News, Oct. 27, 2015, 10 pages.
Fan et al., “Demystifying Big Data Analytics for Business Intelligence Through the Lens of Marketing Mix”, Big Data Research, vol. 2, Issue 1, Mar. 1, 2015, 16 pages.
Bean-Mellinger, Barbara., “What Is the Difference Between Marketing and Advertising?”, available on Feb. 12, 2019, retrieved from https://smallbusiness.chron .com/difference-between-marketing-advertising-2504 7 .html, Feb. 12, 2019, 6 pages.
Twin, Alexandra., “Marketing”, URL: https://www.investopedia.com/lerms/m/marketing.asp, Mar. 29, 2019, 5 pages.
dictionary.com, “Marketing”, URL: https://www.dictionary.com/browse/marketing, Apr. 6, 2019, 7 pages.
Ponn et al., “Correlational Analysis between Weather and 311 Service Request Volume”, eil.mie.utoronto.ca., Jan. 1, 2017, 16 pages.
Zhang et al., “A Bayesian approach for modeling and analysis of call center arrivals”, Jan. 1, 2013 Winter Simulations Conference (WSC), ieeexplore.ieee.org, pp. 713-723.
Mehrotra et al., “Call Center Simulation Modeling: Methods, Challenges, and Opportunities”, Proceedings of the 2003 Winter Simulation Conference, vol. 1, Jan. 1, 2003, pp. 135-143.
Mandelbaum et al., “Staffing Many-Server Queues with Impatient Customers: Constraint Satisfaction in Call Center”, Operations Research, Sep.-Oct. 2009, vol. 57, No. 5 (Sep. 1-Oct. 2009), pp. 1189-1205.
Fukunaga et al., “Staff Scheduling for Inbound Call Centers and Customer Contact Centers”, AI Magazine, Winter, vol. 23, No. 4, Jan. 1, 2002, pp. 30-40.
Feldman et al., “Staffing of Time-Varying Queues to Achieve Time-Stable Performance”, Management Science, Feb. 1, 2008, vol. 54, No. 2, Call Center Management, pp. 324-338.
Business Wire, “Rockwell SSD announces Call Center Simulator”, Feb. 4, 1997, 4 pages.
Nathan, Stearns., “Using skills-based routing to the advantage of your contact center”, Customer Inter@ction Solutions, Technology Marketing Corporation, May 1, 2001, vol. 19 No. 11, pp. 54-56.
An et al, “Towards Automatic Persona Generation Using Social Media”, Aug. 1, 2016, 2016 IEEE 4th International Conference on Future Internet of Things and Cloud Workshops (FiCloudW).
An, J., Kwak, H. and Jansen, B.J., ip.com, Nov. 2016. “Validating social media data for automatic persona generation”, Abstract, In 2016 IEEE/ACS 13th International Conference of Computer Systems and Applications (AICCSA), 2 pages.
European Search Report in corresponding European Application No. 22178124 dated Oct. 20, 2022.
Related Publications (1)
Number Date Country
20230409606 A1 Dec 2023 US