Patent Grant
8204884
1. Field of the Invention
The present invention relates to data analysis storage, retrieval and analysis, in general and to a method, apparatus and system for capturing and analyzing customer interactions including customer and business experience, intelligence and content, in particular.
2. Discussion of the Related Art
Many organizations are involved in generating interactions with customers or other businesses. Many organizations capture or collect such interactions, storing potentially vast volumes media. Examples of such organizations are call centers across many industries, financial trading floors, intelligence surveillance systems, and public safety, emergency and law enforcement entities.
To a limited extent, people, through playback of recordings and listening to interactions, perceive and sometimes document the content of such media. Nevertheless, the details passed in voice and other forms of interactions are largely lost simply due to the size of interaction volume, and the vast majority is not put to use, even when captured. Businesses are looking at their interactions with customers and other businesses as a major source for information and insight about customers and business operations. Increasingly, businesses are striving to keep a closer touch with the customers and “listen” to what customers have to say, believing this will provide a competitive advantage in the market place.
The overwhelming amounts of information collected by organizations require a structured approach if proper management is to be achieved, with the processes to develop a finely-honed content “distillery”, and the right tools to qualify, tag, sort reveal the relevant data. One example where large amounts of information are collected is the field of Customer Relationship Management (CRM). CRM is a business strategy whose outcomes optimize profitability; revenue and customer satisfaction by organizing around customer segments, fostering customer-satisfying behaviors and implementing customer centric processes. CRM should enable greater customer insight, increased customer access, effective customer interactions, and integration throughout all customer channels and back-office enterprise functions.
A substantial portion of CRM is Analytical CRM or Business Analytics (customer and business intelligence). Customer and business intelligence is the use of various data mining, databases, data warehouse and data-mart technologies on customer information and transactional data to create a better understanding of the customer. Such understanding is used to leverage a company's efforts to retain, up-sell and cross-sell a specific customer. It is also a major cornerstone for personalization of content and segmentation of customers leading to improved one-to-one marketing efforts and overall performance. A major portion of the interaction between a modern business and its customers are conducted via the Call Center or Contact Center. Interactions with the business' customers and prospects take the form of telephone and additional media such as e-mail, web chat, collaborative browsing, shared whiteboards, Voice over IP (VoIP) and the like. The additional media captured by the Call Center has transformed the Call Center into a Contact Center captured not only traditional phone calls, but also multimedia contacts.
The ability to capture digitized voice, screen and data is now available in Call Centers and Contact Centers. Such capturing abilities are typically used for compliance purposes, when such recording of the interactions is required by law or other means of regulation, risk management, limiting the businesses' legal exposure due to false allegations regarding the content of the interaction, or for quality assurance, using the re-creation of the interaction to evaluate an agent's performance. Other businesses areas where capturing digital data is becoming increasingly important are: betting and gambling, entertainment, dealing for personal accounts, frauds and money laundering, alternative dispute resolution, mobile telephones, tapping, front-running and the like. It should be emphasized that the call centers and the financial trading arenas are two distinct vertical markets.
Known analytical CRM focuses its analysis on the transactional data created by transaction processing systems such as the CRM platform or the Enterprise Resource Planning (ERP) system. Such analysis is not performed on the content of the interaction with the customer. Simply put, such systems fail to make use of all the information exchanged during the interaction. One example is a direct insurance service and a phone inquiry. Through advertisement, customers contact the insurance service business. Due to legal requirements the insurance service sends the insurance forms to the customer and have the customer sign them and mail them back to close the deal. Often customers call back to clarify contract details. When customers are handled, the type of call is classified and categorized in the data system, such as CRM and the like. Such call is categorized into one of a set of predefined criteria and a transactional piece of data is created. Such piece of data can include date and time, customer name or ID, agent name or ID, insurance policy number, other call related data such as duration, direction, and the call classification from a list of predefined categories. The call classification could be for example “contract clarification” or “contract inquiry”. In some cases the agent might add to the transactional data some free-form text that might or might not indicate the specific clause that the customer asked about. Current analytical solutions analyze transactional data, and as such would not yield information regarding the cause of inquiries regarding the contract. This means that while the system is recording such calls it is not using the information stored in connection with the call, which also includes the call content and the CRM record or screen event. Requesting the agents to provide deeper and more thorough “observations” of the interaction and its contents would interfere with their main task of responding to customer queries thus reducing their capability to handle calls and increasing the call centers' cost per call. In addition, the unpredictable nature of providing observations calls for improved judgmental skills, which incur sustained training and level adjustment costs. Screen events are the events identified by a system in response to one or more of the following: actions performed by the agent in association with the use of a system as viewed by the agent on the screen display including but not limited to keyboard press, mouse click, etc.; data entered into all or part (Region Of Interest) of the display or non-displayed window (window might not be in focus); operating system screen related events. Such as the Esc button pressed, etc; pre-defined multi-sequence events. Such as entering the amount in window application A can generate an update in certain reduction field in Application B. Only these dependant occurrences would yield either input or trigger for the analysis process.
In addition, current systems do not provide for analyzing interactions and at the same time analyze associated data or other interactions. Thus, for example, interactions made and recorded by traders who trade on financial floors are not fully analyzed. Similarly, interactions recorded by call center and contact center agents are not fully analyzed. Information received and logged is not fully understood because parts of such information is not processed and associated with actions of the agents. The result is a deficiency in exploitation of information and data recorded. The person skilled in the art will appreciate that there is therefore a need for a new and novel method and system for capturing and analyzing content.
It is an object of the present invention to provide a novel method, apparatus and system for capturing and analyzing content derived from customer interactions, which overcomes the disadvantages of the prior art.
In accordance with the present invention, there is thus provided an apparatus for capturing and analyzing customer interactions the apparatus comprising interaction information units, interaction meta-data information associated with each of the interaction information units, a rule based analysis engine component for receiving the interaction information, and an adaptive database. The apparatus further comprises an interaction capture and storage component for capturing interaction information. The rule based analysis engine component receives interaction meta-data information. The apparatus further comprises a customer relationship management application. The adaptive database can be a knowledge base component, a telephony integration component which may be accessed via a network. The interaction is a communication unit through which content is passed or exchanged. The interaction can be a telephone conversation, audio, video, voice over IP, data packets, screen events, e-mails, chat messages, text, surveys' results, quality management forms results, collaborative browsing results or sessions, e-mail messages or any coded data. The meta-data information is information related to the interaction information and passed over a media; each interaction has associated meta-data. The interaction and the associated meta-data may originate internal or external to the content analysis system and internal or external to the organization and is the primary input to the system. The adaptive database can be a customer relationship management database, or a computer telephony integration information database or a knowledge database or other databases in the organization or outside the organization. The rule based analysis engine component may be conditionally activated based on a predetermined rule or event. The apparatus can further comprise an intermediate storage area having an intermediate format wherein the results of the analysis made by the rule based analysis engine are stored on and used by or exported to the applications. The results of the analysis made by the rule based analysis engine are provided to and update the adaptive database. The results of the analysis made by the rule based analysis engine provide the user with selective operations based on the results of the analysis. The rule based analysis engine receives from an adaptive database predetermined rules used for analysis. The results of the analysis made by the rule based analysis engine update or create rules used by the rule based analysis engine. The interaction capture and storage component is also comprised of a computing device designed to log, capture and store information. The interaction capture and storage component also comprises a buffer area for intermediate storage of the interaction information. The interaction capture and storage component also provides the rule based analysis engine at least two interactions and at least one interaction meta-data associated with each of the at least two interactions stored in the interaction capture and storage component or stored in an adaptive database. The interaction capture and storage component also comprise an administrative database utilized for the setting up, initialization and operational follow up of the apparatus. The interaction capture and storage component can trigger recording of an interaction or a portion thereof in response to a predetermined event or rule. It is also comprised of an administrative database that operates according to rules base on the content of the interaction.
In accordance with the present invention, there is also provided an apparatus for capturing and analyzing customer interactions the apparatus comprising a multi segment interaction capture device, an initial set up and calibration device and a pre processing and content extraction device. The apparatus also comprises a rule based analysis engine and an interaction raw database for storing interactions captured by the multi segment interaction capture device and an interaction meta-data database wherein each interaction stored in the interaction raw database is associated with an interaction meta-data stored in the interaction meta-data database. Another database is the content data items database. In one preferred embodiment the rule based analysis engine is a software device operative to perform rule check on at least two data items stored in any of the following: the content data items database, the interaction raw database, the interaction meta-data database, the knowledge base, the CRM database. The results of the rule check are made available to applications. The apparatus is also comprised of an interpretation device for imposing rules on the rules based analysis engine.
In accordance with the present invention, there is also provided a method for capturing and analyzing customer interactions the method comprising pre-processing of interactions previously captured; the pre-processing stage comprising: identification; filtration; and classification of interactions; extracting selected content data items from the interactions. The identification is accomplished by examination of at least two interactions. The identification is accomplished by examination of meta-data associated with the interactions. The identification is accomplished by examination of at least one of the following: computer telephony interaction information or CRM information or knowledge base information or information extracted from an adaptive database.
In accordance with the present invention, there is also provided a method for capturing and analyzing customer interactions the method comprising a rule based analysis engine receiving at least one predetermined rule for the identification of at least two predetermined content data item; the rule based analysis engine sampling the at least two content data items from a database or interactions and associated data. The step of associating at least two or more interactions or content data items captured in compliance with at least one predetermined rule by the rule based analysis engine. The step of creating a content data item by the pre processing and content extraction device. The step of capturing interactions by a multi segment interaction capture device. The step of performing at least one adaptive operation on data by an initial set up and calibration device whereby the calibration of the appropriate configuration is customer or market segment tailored. The step of monitoring of an interaction or portion thereof in response to a predetermined event or rule. The step of activating the pre processing and content extraction device based on a predetermined rule or event. The step of updating any one of the following: an interaction raw database; an interaction meta-data database; a knowledge base, a CRM database, a computer telephony integration database with the results of the analysis. The step of providing an indication as to the result of the rule check. The step of imposing rules on the rules based analysis engine.
In accordance with the present invention, there is also provided in a customer service environment of an organization, a system for detecting and processing idea-related data, the system comprising: an interaction monitoring module for monitoring content of interactions; an subject-related managing module for detecting and processing subject-related data, the subject managing module comprising content analyzing tools for analyzing the interactions content. The system also comprises a quality management module for analyzing and evaluating the subject-related data. The idea managing module further comprises a module for sending a notification to an agent involved in an agent-customer interaction upon detecting an idea-related data in said interaction thereby assuring the agent inserts the subject-related data into customer service environment. The quality management module generates idea-related data customer surveys thereby providing further analysis to members of an organization.
The present invention will be understood and appreciated more fully from the following detailed description taken in conjunction with the drawings in which:
The present invention discloses a new method, apparatus and system for capturing and analyzing content derived from customer interactions. This present invention provides for a coherent, integrative analysis process for the contents of all forms of customer communications.
Various environments use the capturing of information and data from agents. Such may include call centers, contact centers, trading floors, money foreign exchange centers or trade centers, and other institutions such as banks, back and front offices in various centers. Two distinct environments are the call centers and the trading floors.
Call centers, also known as the factory floor of the 21st century, are centers where customer and other telephone calls are handled by an organization. Typically, a call center has the ability to handle a considerable volume of calls at the same time, to screen calls and forward them to someone qualified to handle them, and to log calls. Telemarketing companies, computer product help desks, and any large organization that uses the telephone to sell or service products and services may use call centers. Agents supervised by managers and supervisors often man such centers of floors.
Trading floors are the call centers of the financial world. Typically, a trade floor has the same ability as a call center, with the exception that regulatory requirements mandate that calls are always logged and traders are constantly supervised by compliance officers and chief traders. Traders man trading floors. The government is increasingly regulating the operation of traders and trading floors. Various legal requirements are placed on the traders to deal fairly and to avoid irregularities in their dealings.
The person skilled in the art will appreciate that while various market and regulatory conditions may affect and apply to agents or traders, the present invention may be implemented in connection with both environments and any like environment. To enable a better understanding of the present invention the term agent shall also refer to traders in the reminder of the text below.
It is the business concern that agents work efficiently and avoid misconduct, misuse of the system or clients or irregularities in their work abilities and output. Information while the agent performs his duties may assist the manager or supervisor to determine that the agent or traders perform adequately and that the business avoids legal liability due to malpractice or regulation violation. The present invention provides a system for the analysis of at least two interactions captured as a result of the agent's interaction with the client. Analyzing more then one interaction enables system according to the present invention to effectively monitor the interactions between the agent and the client. Such interactions may take place between a business and a customer or between businesses. The interactions captured can be associated there with each other and with other information already present in the organization, such as the organization knowledge base. The interactions may also be associated with data received about the capturing of the interaction such as Computer Telephony Integration (CTI) information or various other data pertaining to the manner of recording and logging of the interaction. One non-limiting example is the information provided as to the length of a call a chat session, the source of the call or the chat session (telephone number or IP address or e-mail identifier) to be associated with what was said (through voice or otherwise) by an agent or a customer.
Recent dynamic changes in the environments mentioned above for a system to be able to capture, analyze and identify inefficiencies, malpractices, misconduct, pattern and customer or agent behavior, quality issues, causes of dispute, regulatory violations in real time and the like. For example, because agents may become vulnerable to third party inducement to accept gifts in exchange for conducting actions that are not in the best interest of the organization, monitoring particular irregularities in the agent activities are paramount to the business. In this non limiting example the voice of the agent can be analyzed to determine patterns of over friendliness or to identify particular words and at the same time screen events or content from the agent's screen may be analyzed to determine if particular favors or reductions or tips are offered to the client. Moreover, recent research has shown that abuse of illicit or restricted substances among agents is on the rise. Analyzing the agent's voice in association with the speed at which the agent is operating his CRM application (which is captured directly or indirectly) can indicate a problem and alert the management. Businesses operating call centers and contact centers face the same concerns and problems. Another non-limiting example, in places where dealing for personal account is permitted, management should control, monitor and detect cases such as “front running”, where an agent could execute a personal trade in advance of a client's or institutional order to benefit from an anticipated movement in the market. The agent's screen activity together with the order for execution of the trade, are captured such that behavior of the agent is verified along with the sequence of execution. Any indications of irregularity will alert the management that bad practice occurred. Moreover, businesses are constantly anxious to gain a competitive edge over their competitors by having better agents, which perform best. The performance of agents may be analyzed effectively through the capture and analysis of various data associated with the interaction with the client. The present invention provides for such a system.
Referring now to
Referring now to
Referring now to
Still referring to
The proposed system and method provide advanced analysis capabilities. In order to demonstrate the concept of the invention, the following exemplary scenario will be assumed via which the functionality of the various devices and components of the apparatus will be described. In the exemplary scenario a customer using the proposed system and method desires to find out the reason for the unusual success of a specific human agent. First, an audio classification module 264 of the pre-processing and content extraction sub-device 246 extracts words and sentences 240 that the agent uses, then the agent's recurrent behavioral patterns are detected. Reference is made to Banter RME from Banter, Inc. located in San Francisco, Calif., which provide a tool for word extraction from text. The agent's screen activities 292 are captured as well during the interactions and the inner conversational emotional level 242 is identified. All the above-identified interactions content information is first captured by the multi segment interaction capture device 280 and then saved to the interaction raw data database 272 and interaction meta-data database 274. It is then processed by the pre processing and content extraction device 246 and saved as a content data item 230 later to be further analyzed by the content-analysis rule base engine 218 to produce a result.
Each of the interactions may be linked with another type of interaction and the relationship matched and analyzed. Exemplary agent-specific results that were derived could include agent-specific behavioral characteristics, such as courtesy, conversational manner, cooperation, and operating methods such as collaborative web browsing and proper use of the CRM application. The above scenario is a particular case of automatically analyzing an agent's conduct regarding behavioral characteristics while handling customers for purposes of Quality Management. The CRM database 278 serves as a source for supplying Transactional Information required during the analysis process. The results of the analysis could be fed back to the CRM database 278. Another source of vital information used by the analysis process is the enterprise knowledge database 276. The database 276 is commonly used for retrieving organization related information, such as products information, agent QM information, agent profiles, multi-media parameters, and the like. Notably, CTI information 258 is used during the analysis process to allow real time content analysis. Call information is received either from the Automatic Call Distributor (ACD) or from the ACD through the CTI. Call information coming from the ACD can be used in monitoring agent activity while the agent is engaged in interaction with a customer. Call information can also arrive from a Turret system, also known as a Dealer Board or from a PBX system. An exemplary benefit of the above option was described in detail in the referenced co-pending U.S. provisional patent application Ser. No. 60/350,345 titled IDEA MANAGEMENT BASED ON CONTENT OF INTERACTION, filed 24 Jan. 2002, the contents of which is incorporated herein by reference and in association with the description of
Referring now to
Now referring to
Still referring to
The input to “speaker segmentation function” is a summed audio signal. Unsummed recorded audio can be summed or compressed or processed prior to being archived or used. In addition, and optionally, signal processing can be performed prior to recording of the audio signal, thus refraining from audio signal degradation that may occur during the recording session. Output includes the following signals or segments marked by a time index: a) signal 1 is a sequence of segments each of which belongs to speaker 1, b) signal N is a sequence of segments each of which belong to speaker N, c) non-voice, d) silence, and e) talk over. The function is both text independent and speaker independent. Integration of the speakers and an inherent acoustic model of the site significantly improve the segmentation performance. The same integration of the speakers could provide the use of the system for real-time applications, such as speaker-based trigger start recording, monitoring of speaker-based trigger starts, and the like. The system is configured to analyze specific parts of the call based on information from other applications and from other pre-processing functions, such as information from CTI events, speech detection and classification functions, and the like. The outputted results of the system are cross-referenced with the output of other systems in order to improve overall system performance. The person skilled in the art will appreciate that the above-described function is an example and that other variations to analyze and examine the audio or other types of interactions can be implemented as well in connection with the present invention.
Reference is made now to
Referring back now to
The full specification of the speech noise reduction 92 function is described next. The noise reduction algorithm package comprises of three algorithms, each designed to cope with different noise features that might be expected. The three algorithms can be independently turned on and off, so that the expected noise(s) may be reduced while minimizing the damage to speech intelligibility—by disabling the algorithm(s) that may be irrelevant to the encountered noise features. Per each algorithm invoked, an operator-based level of operation (either Low, Medium or High) may be set, to realistically meet the noise's severity. This way, the trade-off of SNR and quality improvement vs. degradation in intelligibility may be set to near optimum, according to the encountered input SNR. The functions of the speech noise reduction algorithm are described next. A tone elimination algorithm is a part of the noise reduction function. The tone elimination algorithm is capable of eliminating, or reducing, noises that comprise of several (up to five) nearly “pure” tones over each 500 mS intervals, almost independently. The elimination is based on adaptive spectral detection of the tones' frequencies, and consequent notch filtering. The detection algorithm is based both on spectral observation of the processed block and on the past history of occurrence of the suspected tone in preceding blocks. In addition to “pure” continuous tones, the algorithm can detect short bursts of tones, usually typical to Morse or slow FSK background signals. The adaptive notch-filtering is implemented in the vicinity of the detected frequency, using a single (double-sided) zero and a single (double-sided) pole, thus implementing an ARMA (2, 2) linear filter. The filters are cascaded to sequentially operate on all the detected tones per 500 mS data block. The use of such filters enables local tracking of the eliminated tone, so as to prevent artificial generation of a tone where the disturbing tone is locally absent. Provision is made for the cascaded notch-filters to retain inter-segment signal continuity when the same tone frequencies are repeated. The humming sounds elimination algorithm is also a part of the noise reduction function. “Humming” sounds usually resemble time-domain impulse trains, reflected in frequency-domain impulse trains (possibly widened) that are stationary over relatively long periods (500 mS). Such noises are usually typical to HF environments, or to acoustic environments that are subjected to mechanical periodic sources such as low RPM engines, propellers etc. They are frequently accompanied by white or slightly colored noises. The detection of humming sounds is implemented using spectral detection of such trains that may be comprised of up to 400 elements in the spectrogram reflected in the FFT of the processed data block. Consequently, these trains are eliminated from the spectrogram, and the refined time-domain block is reconstructed from the modified spectrogram using an inverse FFT. The white noise elimination algorithm is also a part of the noise reduction function. Additive white (or slightly colored) noises are typically encountered in VHF environments, or remain as residues after the elimination of “humming” or tone-like noises. The well-known “spectral subtraction” technique is used with several modifications in order to reduce noises of this nature. The basic analysis is based on shorter (64 mS) data blocks than the previously discussed algorithms; however, considerable block overlapping and averaging efforts are made in both signal analysis and synthesis, to retain long-term continuity and consistency. The short-period analysis is necessary for relying on the expected short-term stationary of the desired speech signals. The spectral noise level is estimated using non-linear order-statistics approaches that minimize the effect of desired speech-like signals on the estimation error. The estimated level is then spectrally subtracted in a way that compromises the subtraction in an attempt to preserve speech information where it is apparently present. The detection and subtraction of the spectral noise-level is performed separately on four spectral sub-bands, thus allowing for slight variations in the noise's whiteness (at the expense of statistical accuracy), and increasing the algorithm's robustness. Each sub-band is processed using different processing parameters, to accommodate sub-band dependent trade-offs between quality and intelligibility. The main well-known drawback of the spectral subtraction method is the so-called “musical noise” artifact. Operator selection of operation level (Low, Medium, High) sets the processing parameters so as to meet the operators' preferred trade-offs between the original noise subtraction and the musical noise artifact.
Still referring to
The system and method proposed by the present invention includes a specifically designed performance measurement tool for the word spotting function 116. This automatic tool is analyzing the effects of the software updates, parameter optimization and setting different words to spot. The function 166 input consists of two kinds of files: a) a searched word list, and b) a manually transcribed text file for each voice file. These files contain time stamps every pre-defined number of seconds for timing information resolution. The output of the function 116 is the results of the word spotting in terms of detections and false alarms. The results would include details of the software version, parameters checked and file ID for comparison and analysis purposes. The word spotting function 116 creates an estimated “real” word location (timing) list. Due to the timing information limitation of the transcribed files the list entries are in the following format: WORD FOUND-→ LAG NUMBER (leg 0:0-x sec, leg 1:x-20), and the like. The list may contain more information regarding the found words. Once the “real locations” list is created, the word spotting function is executed. Each word supposedly detected by the word spotting function is compared to the “real location” list. If an instance of the word exists in the relevant x-second leg then a hit is indicated. If the word does not exists in the relevant x-second leg then a False Alarm (FA) is indicated. The HIT/FALSE ALARM statistics are essentially the output of the word spotting function. The output is stored into a designated database in the following format: DOCUMENT ID, such as file identification, VERSION ID, such as a software type and software version number, WORD LIST, such as a vocabulary looked for, NUMBER OF HITS, such as the number of detections, FALSE ALARMS, such as the number of false alarms, OUT OF, such as the total number of words looked for. The designated database enables the analyzing of the results using a method that is similar to the manual one currently used. Consequent to the introduction of the results to the database querying and mining of the results is possible in a variety of ways. The call flow function 119 analyzes the dynamics of the call. The function 119 attempts to provide an indication of the call-flow parameters of the call. The calculated parameters include the percentages of the call's length, complete silence; talk over, agent speaking and customer speaking. The function 119 counts also the number of times the agents interrupts the speech of the customer and vice versa. It also gives details about the silence, talk over, and activity sections during the call. The function 119 is fed with a variety of streams where each stream represents a specific participant of the call. The function 119 is based on calculating energy levels within the digital speech of each participant of the call. Each of the analyzed interactions can be analyzed independently or in association with another type of interaction captured at the same time. Such can be a video interaction, a chat interaction, a screen event captured from the screen of the agent and the like. Similarly, associations between various interactions may be analyzed as well. So for example, audio and video interactions or audio and CRM data associated with the same call may be analyzed to identify various predetermined combinations of events or elements relating to the handling of the call or query (or offer for goods or services), the response by the agent, the appropriate response to a client or entry of data into the CRM at any given time during a call or an interaction between the business and the customer. The person skilled in the art will appreciate the various types of interactions, which may be associated together and analyzed to obtain like result and enhance the ability to analyze and respond to various events.
Referring now to
Referring back to
Referring now to
Referring back now to
Next, several exemplary rules associated with the rules engine 112 will be described. It would be easily understood by one with ordinary skills in the art that these examples are not meant to be limiting as diverse other rules with associated required actions and indications could be contemplated or could be implemented when practicing the present invention. The exemplary rule could include: a) the user of the system may wish to define an “angry” conversation by defining “angry” such that the conversation should contain certain words, a relatively high percent of talk over (when two or more persons talk at the same time on the same line) and/or negative excitement detection, b). the user of the system detects an unprofessional behavior of an agent by the detection of negative excitement on the agent side followed by a negative excitement on the customer side. The detection of the negative excitement patterns suggests that the agent was angry during the call and as a result the customer became aggravated. The indication data can further be cross linked to CRM information indicating unhappiness of the customer concerning the service, c) a user desires to identify patterns behavioral misconduct of speech manner by either a customer or an agent in order to better understand the reasons for “bad interaction” and furthermore to update the profiles of the agent and the customer accordingly, such as updating the CRM inherent customer profile categorized as a “hostile” customer such as an I-rate customer, d) a user wishes to handle a VIP customer in a careful, sensitive manner. For example, a VIP customer suffering from speech deficiency could be identified as such by the system following detection of certain speech deficiencies (stuttering, word repetitions, syllable repetitions). Consequently the user may chose to demonstrate high customer sensitivity by updating data in the organization's databases, such as the CRM database, leading to assigning a “sensitive” well qualified agent to handle such speech disabled VIP by selectively skill routing the call, e) a user detects impolite agent behavior by the identification of specific events during a call session, such as the agent interrupting the customer, agent is non-responsive to the needs of the customer, agent responds to the repeated requests of a customer by repeating the same sequence of words in his answer. The above agent behavioral pattern shows that the agent is not aware of the customer's difficulties in clarifying his/her requests, f) a combination of at least two rules such as shown above could be chosen to be a new rule. Thus, only when the two selected rules are met the combination rule is also met and a proper indication is provided, g) the use of specific words combined with screen events and/or CRM entries made at the time of use of the words. This rule will require the examination of CTI information as well as screen events captured and the voice interaction analyzed to find the word or words selected. In addition, and at the same time the organization's knowledge base may be queried to identify additional information required to perform the rule.
The person skilled in the art will appreciate that the rules provide enhanced simplicity for the introduction of any additional desired rules and the “calibration” of the rules during the operation of the system would be evident. The user is further provided with the liberty and flexibility to decide and to select the phenomena to look for and the manner for looking. One or more rules embodying one or more interactions and one or more associations may be easily captured, analyzed and an according response or event generated. By providing access to all types of extracted information, CRM data, the definition of time and event sequences and the combination of the above, a diversity of scenarios is operative in enhancing detection of specific characteristics, such as for example a search for impolite words followed by a high tone in the conversation or a particular screen event or a particular CRM entry or operation. The results of the rule analysis are easily implemented in the classification component thus enabling faster and more efficient future analysis.
Using the rule engine a plurality of phenomena included in but not-limited to a session can be sensed, recognized, identified, organized and optionally handled: a) multiple occurrences of events in a certain time frame, b) sequenced or concurrent occurrences of events, c) logical relations between events, the timing of the events and the extracted information, such as when an agent did not open a suitable application screen for at least 10 seconds after the customer asked to purchase shares in over $10,000, or where an agent was offered $10,000 worth of options if he can secure a particular limit on a particular share, d) customer-agent interaction analysis based on a combination of different sources, such as spotted words, simultaneous talking, silence periods, excitement type, excitement level, screen events, CTI information and the like.
The recognized phenomena could include the following non-limiting exemplary conclusions: a) total number of bursts in conversation, b) negative excitement in at least one side of the conversation, c) large percentage of talk over during the conversation, d) the average percent of the agent's talking time, e) the number of bursts the agent made into the customer's speech, f) the negative agent excitement prior to or consequent to customer excitement, g) agent tends to make a relatively high percentage of customers angry, h) long or frequent hold periods or long and frequent silence periods, which imply that the interaction of the agent with the system is inefficient, I) recurrent repetitions of the same answer by the agent. Additional recognized phenomena may include the association of each of the above phenomena with interactions or data or information extracted from CTI or other sources such as CRM or other interactions. Such phenomena may further be analyzed in connection with various other events such as screen events and CRM records, entries and free text. The actions generated by the rule engine may preferably drive high-level real-time status reports to the applications that will facilitate real-time alerts and real-time responses while simultaneously enhancing the information storage with the results. For example, long or frequent hold periods or long and frequent silence periods with out screen events or CRM activity may indicate a particular agent is ineffective. In another non-limiting example, the average percent of the agent's talking time is more then a predetermined threshold and various CRM entries are left empty may suggest the agent at the contact center has not been attentive or failed to properly conduct the call or interaction with a customer. In another example, a compliance officer or chief trader observes in real time the performance of the trader and receives notifications as to various content analysis results, such as that the agent has greeted the client properly or that the agent has used the word “bet” in the conversation while making a substantial transaction with another business. The supervisor may immediately call up the relevant session (whether it is a call or a chat session or e-mail or otherwise) and view at the same time the agent's screen captures. Other indications which may be available to the supervisor are whether the agent followed a specific procedure, whether the tone of the conversation is within acceptable parameters, items of need of investigation, call evaluation, use of client's name or other pleasantries, surveys performed, abusive behavior indication and the like.
Analysis processing may require intensive processing and can be implemented in any of the following fashions: a) as software processes running in an operating system environment of dedicated standard servers using the entire server data processing resources for the software. The processes could be run on one or more computing devices in the organization, such as for example the call center agent computing devices. Suitable load distributing utilities could be implemented to the handling of the large loads. As DSP processing boards with firmware, such as an array of DSP boards running the analysis function. The board could be used inside a voice-recording server, such as the NiceLog Voice Logger by Nice Systems of Ra{grave over ( )}anana, Israel. The board could be further used in dedicated servers where each server integrates a plurality of such boards, or installed on a plurality of COMPUTING DEVICEs in the organization, such as every agent's COMPUTING DEVICE, localizing and distributing the processing load with little or no effect on the COMPUTING DEVICEs performance, c) for performance enhancement some of the processing that can be done in real time might be performed prior to the recording in such a manner as not to be affected by degradation of the voice signal associated with the recording process, d) the control and data infrastructure for this entire process can be implemented as software on one single standard server platform.
The content analysis process as proposed by the invention possesses several additional respects: a) Configurable Processing Power—During the system setup or during a call session an authorized user using a dedicated Man-Machine Interface (MMI) can intelligently control and manage the CPU resource allocation in accordance with the priorities and the performances. Thus, for example, a user could allocate about 30% of the CPU resources for word spotting, about 15% for excitement extraction/emotion detection and about 10% for speaker identification and verification. b) Utilization of Users Workstation Processing Power—When only insufficient processing power is available (due, for example, to server bottle-necks, malfunctions, insufficient bandwidth or the like) the agent's workstations are being used in order to enhance the processing power capacities, exploiting the agent's workstations particularly during periods when the machines are in logged off state. c) Customized Adaptive Database: c1) Vertical Market (e.g. vocabulary in trading floors)—The characteristics of a particular environment in terms of inherent vocabulary is identified and stored in the system database to be used on the analysis stage. For example, the word “shares” is used frequently in Trading Floors therefore it will be stored in a Trading Floor vocabulary. Various models can be created to keep track of the adaptive databases based on previous analysis so as to continuously update the databases and the rules of the system. c2) Acoustic Environment Modeling—The particular acoustic surrounding of a business environment is identified and stored in a database to be used by the audio classification module of the pre-processing stage. Different business environments are dominated by different acoustic elements. For example, the acoustic environment characteristics of a Trading Floor could include loud cross talk, commotion, slamming down of telephone receivers, and the like, in contrast with Call Centers where the ambient acoustics is quieter but other types of noise sounds dominate, such as keyboard clicks. c3) Multi-Media Adaptive CA Resource Allocation—The system's content analysis resources could be manually adapted in accordance with the preferences of a customer and/or in accordance with the environmental characteristics. A user manipulating a dedicated MMI could individually allocate CA resources to each multi-media type interaction. For example, about 5% of the analysis processing power could be assigned could be allocated to e-mail, about 5% to chat channels, about 40% to audio information and about 50% for video data. In the same manner about 50% of the processing power could be allocated to word spotting regarding e-mail, about 40% for emotion detection regarding video information, and the like. d) Controlled Real-Time and Off-Line Processing—The real-time processing of signals is performed via firmware utilizing powerful DSP arrays as this type of processing requires adequate processing power. In contrast, off-line processing requires mainly substantially large amount of memory and therefore could be performed by utilizing a plurality of computing devices substantially simultaneously. e) Coupling with other system platform inherent capabilities, such as retention, migration, and the like—The capability of retaining information on the platform is useful in avoiding situations where a word is spotted in real-time and when off-line evaluation starts the call session is no longer exists as it was automatically deleted by a inherent logger mechanism. Retention is also a valuable option in association with the migration feature. Under certain circumstances it is important to keep a call in the on-line storage device for quick access even when a call is migrated to an off-line storage device. f) Time Adaptive Resource Allocation—Most of the time there is a backlog of calls within specific data structure queues pending for the performance of analysis, such as for word spotting. The backlog is generated due to a substantially large amount of calls selected for content analysis processing and the inherent constraints of the user site, such as the amount of processing power available, and dynamically changing bandwidth limitations. The decision required from the system regarding “which call to analyze next?” is not a trivial task as there is a plurality of calls to choose from. The required solution has to serve the user's requirements in an optimal manner. The solution (preferable but not limiting) proposed is designed to operate as follows: Off-peak periods are typically non-random and usually fixed in time and known in advance as they typically occur at night, on weekends and on holidays. During the off-peak periods the most-recent-call method, such as FIFO, should not be used as typically it will distort the number of calls processed and will favor later day calls on earlier calls. Similarly on weekend it will create a plurality of analyzed calls towards the last-days-of-the week while discriminating the start-of-the-week days. Thus the proposed solution is to use different techniques under the following circumstances: a) When there is no backlog the system should always handle each required call or interaction within about 5 minutes after the call was completed or even sooner. At off-peak periods the system is idle. b1) When the backlog is small in such a manner that the analysis process could be typically closed completely within a short period of time (up to about 24 hours) when utilizing only the off-peaks hours during the night, the system should take high-priority calls, going from the most recent back and only following the completion of all high-priority calls should the low-priority calls handled. At night the system should select randomly dispersed high-priority calls from the day and then select the lower-priority calls in descending order. At weekends the system is idle, b2) When the backlog is medium in such a manner that the analysis process could be typically closed with a period of about 1 week (using week-end off-hours) the system should perform in similar manner as the small backlog conditions with calls remaining each day and then at the week-end the system should select (within each priority class) randomly-dispersed across the entire previous week. If a day's calls or a week's call are completely processed then the system should proceed to the previous day or previous week respectively, b3) When the backlog is large and/or growing and can not be closed (the system can not “clean” the queue) the system should finish the calls of the current day and should continue to process backward in time. Activity and manner of operations on nights, on weekends and on holidays should be preferably automatically determined in accordance with the call volume and the point in time. However, alternatively a system administrator could define the activity dynamically in accordance with the site's profile and its typical business activity. Backlog can be further handled by choosing in advance to analyze only the “interesting” portions of a call, in a predetermined manner according to the non-limiting important criteria, such as the different vertical market characteristics, user preferences and the like. Note should be taken that the above described manner of operations, timetables, activities and call handlings may be changed and that like techniques may be used as well in the context of the present invention. The underlying backlog-handling-related concept of the invention is the adoption/selection of appropriate functions for the analyzing process according and with respect to the requirements, preferences and needs of the user. g) Surveillance/Security Related Benefits—The system and method proposed by the present invention provide a capability that contributes both to the actual performance of the analysis functions and simultaneously could be used for security-related purpose, such as the identification of suspicious signs. For example the capability of detecting a foreign accent or a specific language dialect will contribute to the operators and users of in at least two useful benefits, g1) The technology of voice recognition today relies on examining how people pronounce phonemes. Pronunciation varies with accents and dialects. The closer the found pronunciation matches the expected one, the better the detection accuracy. Currently, different packages are provided per language variants, allowing focusing on one type of dialect and this increasing accuracy. Therefore, when an accent or a dialect is known in advance, the voice recognition function can use the phonetic distinction of this accent or dialect to increase the efficiency of the performance. The inherent functions are enhanced due to pre-known automatically detected accent, 2g) Once an accent is detected in real-time security key personnel can be notified and the profile of the subject is updated. For example, after the events of September 11 any video or audio detection that can enhance the real-time detection of suspicious signs is welcomed by the security forces. One of the input sources of the content analysis system of the present invention is video. Examples of the capabilities, usages and applications that a video content analysis system can provide are presented co-pending U.S. patent application Ser. No. 60/259,158 titled CONTENT-BASED ANALYSIS AND STORAGE MANAGEMENT, filed 3 Jan. 2001, and to co-pending U.S. provisional patent application Ser. No. 60/354,209 titled ALARM SYSTEM BASED ON VIDEO ANALYSIS, filed 6 Feb. 2002 and U.S. patent application Ser. No. 10/056,049 titled VIDEO AND AUDIO CONTENT ANALYSIS filed 30 Jan. 2002.
h) Automatic Classification into Customer segments—This option is used to improve the handling, the up-selling and the cross-selling. The technique uses a speech detection function to identify gender, age, and area of residence, demographical background, and the like. Such classification information will substantially assist an agent during a call session vis-a-vis a potential customer. For example, subsequent to the identification of the gender of the customer as a woman products suitable only for women will be offered. Selective information stored in external databases such as a CRM database is being used both in real-time and off-line to collect a priori information on the customers, i) Audio Splitting and Summing—To reduce the overhead of the system and the implied cost of ownership in terms of storage a non-limiting technique is proposed. The solution involves audio streams that are recorded un-summed, such as being split into two speaking sides, are consequently summed and compressed after processed and prior to being moved to long term storage. The solution affects a considerable reduction of storage space and network load. Typically, the storage space taken by split recording is about 50% more then that of a summed recording. Compression methods currently achieve about 12-fold reduction in the volume of information. When combined the two methods can achieve about 18-fold saving, j) Agent Auto-Coaching—Using real-time content analysis combined with a set of rules that take into account specific content elements of all types, organizations could define criteria that evaluate agent performance and customer behavior “on-the-fly”. The conclusions could be presented to the agents during or after the performance of the call. The application will use the rule to continuously look for specific keywords, emotion levels, talk behavior and other content. When a pre-defined combination is found it will pop-up a matching coaching statement on the agent computing device screen. When working after the call the application will display a list of tips and statements as a summary for the agent to study the list and act on it for later improvement. k) Extraction of predetermined parts of the Interaction—The system of the present invention is also configurable to save computer power and computing resources by pre processing and/or analyzing certain predetermined parts of an interaction. For example, the pre processing and capture device shall only extract the portion of agent A to talking to customer B rather than extracting the full conversation.
Referring now back to
During the classification stage the system utilizes all relevant information such as meta-data and customer history files in order to improve the analysis of an individual interaction. Typically, the more attributes are provided for an interaction the better the resulting categorization.
Referring now back to
A) Analytical CRM applications: The entire set of original and processed information described above can be exported and used by Analytical CRM applications in conjunction with any other information in an enterprise data warehouse or in a smaller scale data-mart. These solutions use diverse data analysis functions for customer segmentation, customer behavior analysis, predictive module building, and the like. The information revealed in the above-discussed dimensions is directly related to customer information used in data warehouses. However, this information does not include the aspects of customer interaction content, which is a critical authentic element of the problem. For example, a telephone customer attrition predictive model is typically built against CRM databases and billing databases. But, the analysis of conversation topics may expose that the optimal predictor for customer attrition are requests for competitive rates. The visualization tools of the Analytical CRM tools could also display analyzed content; Content analysis output is applicable in the following major dimensions for analytical purposes:
1a) Propagated data that is data analyzed in bulk to create knowledge relating to the entire customer base, or extensive sub-groups of the same. The number of interactions matched to pre-defined categories and the new categories identified expose a large number of propensities. For example, showing the terms customers use to refer to a new campaign or a product advertised by the business or seeing patterns of certain customer behavior, such as the stages leading up to a customer discontinuing a relationship with the business.
2a) Customer specific data that is all data attributed to a specific customer. Such data is analyzed and related to the customer in order to expose knowledge specific to the customer behavior pattern, language and preferences.
3a) Segment specific data that is data analyzed and related to a specific category, such as a certain product, to produce information regarding the relation to the product in the content of interactions. For example, the distribution of emotional interactions and correlation with release of new products/versions could suggest that specific products are being marketed before being ready.
B) Customer Experience Management (CEM) applications: All the applications focused on the customer's experience and on the agent's quality of service will be particularly enhanced consequent to the utilization of the content analysis results. In addition, new applications are made possible:
1b) Enhanced Playback: Typically, the playback of calls is a time consuming and highly complex task. It takes just about the duration of the entire original recording to play it back and when complex segments of the recording are needed to be replayed, the duration of the playback process could be even longer than that of the original recording. For example, when a large trade transaction is made in a busy and noisy environment, such as a trade floor, via a call session having a significant amount of cross talk regarding a customer/agent dispute, in order to faithfully restore the details of the trade the recorded passages containing the vital details will need to be played back several times, while all other parts will also need to be played back to provide the suitable context. Thus, a considerable waste of time and resources will be affected. Although known playback mechanisms allow pause/resume playback functions, random access to a specific point in the recording, acceleration and deceleration control, skipping over silence, loop repeat, and even noise-reduction processing, none of the methods are particularly efficient when unclear, crucial details are scattered throughout the call. All existing tools are lacking the direct support for achieving optimal playback audio acoustic cleanness while decreasing the duration of the listening.
Referring back to
2b) Scheduling of recording can be defined in association with specific conditions. The conditions could include diverse content classification entities such as the identification of excitement in the voice of the participants, the appearance of a word or a certain topic, the combination of more then one condition such as the appearance of a particular word in an interaction combined with a particular action by the agent, and the like. Thus, a recording could be initiated following the emergence of a severe debate in a call session or consequent to the mentioning of specific negotiation-related elements, such as commodity price, supply date, or when an agent has used words relating to presents and received an e-mail containing words affecting a promise in exchange for favors, and the like. Recording can also be started even after the call has began from a particular time frame after the call started or from the beginning of the call.
3b) The monitoring of the interaction performed in real-time is advantageous as it is substantially enhanced by the utilization of advanced content-based mechanisms described above. The content analysis system based upon the content of the interaction will perform specific real-time actions. For example, upon detecting specific pre-defined verbal expressions within the customer's speech stream, such as “I have a suggestion”, “I have a complaint”, or the like, the agent is alerted by the reception of a real-time notification. Thus, the system ensures that the agent will “stay alert” and maintain a set of suitable memory aids (notes, memos) for recording the customers comments, ideas, complaints, and requests. This feature will provide future follow up and the distribution of the customer's ideas to the appropriate organizational units. The real time monitoring may also examine more than one interaction at the same time. For example, the speech stream monitored may be associated with collaborative web sessions performed by the client and if the client errs on how to use the web application offered by the organization and the agent fails to notify or correct the client the content analysis system may alert the agent and/or a supervisor or a manager.
4b) Real time alert/notification, such as alerting an agent, a customer, compliance officers, supervisors, and the like is utilized for the purposes of fraud detection and other operational activities within an organization which require the taking of immediate action following specific indications detected via the analysis of the interaction data. These actions could be operative in the lowering of the operating costs of the business and the timely prevention of potential legal and liability issues.
5b) Improved querying capability and searching capability within multi-media databases of interactions relaying on content parameters as well as meta-data or extrinsic data will provide more accessible interaction-related information to additional functions and to persons within the organization.
6b) Reports: The reports are generated using a specifically designed and developed web based software product referred to as the Reporter. The scalabilities, multi-site and multi-database characteristics of the product substantially contribute to the straightforward manner and ease of adding content analysis based reports. Content analysis reports include statistics, direct comparison results, follow-ups and the like. All the reports are addressing appearances of certain content commonly used in regard to other interaction/transactional information. The following are non-limiting examples of groups of reports: Word Spotting and CTI reports where CTI information is used in order to retrieve an agent user ID, the call time, and the like, Emotion/Excitement, CTI and User Information reports, Word Spotting, CTI and QA Information reports, Agent-Customer Interaction Talk Analysis reports, and the like.
Referring now to
7b) E-learning content based sessions: Based on specific evaluation results the system is triggered to send an e-learning tutorials to specific agents in order to improve their skills in the identification and description of the customer-supplied ideas provided during the interaction. For example, an e-learning session is sent to an agent in association with a sample of a recorded interaction, such as an AVI file, that includes a customer-supplied idea. The agent is required to identify the idea and fill up a pre-defined form in order to describe the idea.
8b) Customer Surveys Content Analysis: The surveys that reside in the organization database are analyzed using text extraction methods. Based on the results derived from the analysis specific actions are initiated. For example, a Call Center manager detects that a certain campaign group is not achieving the predicted profit. Consequently the manager utilizes the IVR post-call surveys to obtain customer reactions. Analyzing the content of the customer's surveys producing reports could provide the reasons for the lack of profits, such as product is unsatisfactory, lack of experience of the handling agents and the like.
9b) Automatic quality monitoring: Based on pre-defined criteria regarding an agent's use of conversational and negotiation guidelines, such as form of greetings, call termination, and operational skills and the like, the system will notify a supervising function in instances where the guidelines are not followed. In addition, appropriate evaluation forms will be created according to the results. For example, the content analysis engine could identify that the proper greeting is missing in a call. Thus, in the QM evaluation form the sub-section scoring the agent's courtesy is automatically filled by the value “0”. In another example, the content analysis system could identify that the agent did not ask a particular question and that the CRM application was not updated for the answer of that particular question. The use of more than one condition will enable the system to be more efficient targeting on the proper events for review.
10b) Data Visualization presents the information and knowledge created in the entire analysis process in a visual form, which is adjustable and controllable by the user. Visualization provides an intuitive and flexible display of various dimensions of the information. Beginning at a high-level view, the user could browse the information to examine areas of interest, to enlarge and sharpen the display resolution of one segment of a more general field of view, change the dimensions displayed (category popularity versus cohesion versus growth trend) and the like. Populations of interactions can be zoomed in on allowing the pinpointing of individual interactions by placement, and the color of similar visual attributes. Further zooming in could display segments of the interaction with diverse attributes of interest. The visualization tool can draw the analyst attention based on a set of pre-defined rules regarding specific subject matter.
11b) Content based knowledge management enables access to information that is part of the interaction stored in a scattered manner across the organization's knowledge database, CTI database, CRM database, Screen Event database, Administrator database and the like.
12b) Customer interaction analytics: Using the entire customer interaction database created as describe above, various data mining and analytical modeling techniques can be applied, enabling a deep research of the information, finding correlations, hidden patterns, trend and the like.
Further examples of e-learning content based sessions generated following the recognition of specific content of an interaction and further description of the Automatic Quality Management form and further examples of real-time events generated following the recognition of specific content of an interaction can be seen in association with
Referring now to
Still referring to
The person skilled in the art will appreciate that what has been shown is not limited to the description above. The person skilled in the art will appreciate that examples shown here above are in no way limiting and serve to better and adequately describe the present invention. Those skilled in the art to which this invention pertains will appreciate the many modifications and other embodiments of the invention. It will be apparent that the present invention is not limited to the specific embodiments disclosed and those modifications and other embodiments are intended to be included within the scope of the invention. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation. Persons skilled in the art will appreciate that the present invention is not limited to what has been particularly shown and described hereinabove. Rather the scope of the present invention is defined only by the claims, which follow.
The current application is a divisional filing of application Ser. No. 10/484,107 dated Jul. 14, 2004 the disclosure of which is incorporated herein by reference. The present invention relates and claims priority from U.S. provisional patent application Ser. No. 60/350,345 titled IDEA MANAGEMENT BASED ON CONTENT OF INTERACTION, filed 24 Jan. 2002 and from U.S. provisional patent application Ser. No. 60/306,142 titled CUSTOMER INTERACTION CONTENT BASED APPLICATIONS, filed 19 Jul. 2001. The present invention relates to U.S. patent application Ser. No. 60/259,158 titled CONTENT-BASED ANALYSIS AND STORAGE MANAGEMENT, filed 3 Jan. 2001, and to U.S. provisional patent application Ser. No. 60/354,209 titled ALARM SYSTEM BASED ON VIDEO ANALYSIS, filed 6 Feb. 2002 and to U.S. provisional patent application Ser. No. 60/274,658 titled A METHOD FOR CAPTURING, ANALYZING AND RECORDING THE CUSTOMER SERVICE REPRESENTATIVE ACTIVITIES filed 12 Mar. 2001 and to PCT patent application serial number PCT/IL02/00197 titled A METHOD FOR CAPTURING, ANALYZING AND RECORDING THE CUSTOMER SERVICE REPRESENTATIVE ACTIVITIES filed 12 Mar. 2002 and to PCT patent application titled CONTENT-BASED STORAGE MANAGEMENT filed 3 Jan. 2002, and to U.S. provisional patent application Ser. No. 60/227,478 titled SYSTEM AND METHOD FOR CAPTURING, ANALYZING AND RECORDING SCREEN EVENTS filed 24 Aug. 2000 and to PCT patent application titled SYSTEM AND METHOD FOR CAPTURING BROWSER SESSIONS AND USER ACTIONS filed 24 Aug. 2001, and U.S. patent application Ser. No. 10/056,049 titled VIDEO AND AUDIO CONTENT ANALYSIS filed 30 Jan. 2001, and US provisional patent application titled RECORDING OF FACE TO FACE CLIENT-AGENT MEETING, filed 6 Sep. 2001, the content of which is hereby incorporated by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4145715 | Clever | Mar 1979 | A |
| 4527151 | Byrne | Jul 1985 | A |
| 4821118 | Lafreniere | Apr 1989 | A |
| 5051827 | Fairhurst | Sep 1991 | A |
| 5091780 | Pomerleau | Feb 1992 | A |
| 5303045 | Richards et al. | Apr 1994 | A |
| 5307170 | Itsumi et al. | Apr 1994 | A |
| 5353618 | Roper et al. | Oct 1994 | A |
| 5404170 | Keating | Apr 1995 | A |
| 5491511 | Odle | Feb 1996 | A |
| 5519446 | Lee | May 1996 | A |
| 5734441 | Kondo et al. | Mar 1998 | A |
| 5742349 | Choi et al. | Apr 1998 | A |
| 5751346 | Dozier et al. | May 1998 | A |
| 5790096 | Hill, Jr. | Aug 1998 | A |
| 5790798 | Beckett et al. | Aug 1998 | A |
| 5796439 | Hewett et al. | Aug 1998 | A |
| 5847755 | Wixson et al. | Dec 1998 | A |
| 5895453 | Cook | Apr 1999 | A |
| 5920338 | Katz | Jul 1999 | A |
| 6014647 | Nizzari et al. | Jan 2000 | A |
| 6028626 | Aviv | Feb 2000 | A |
| 6031573 | MacCormack et al. | Feb 2000 | A |
| 6037991 | Thro et al. | Mar 2000 | A |
| 6055540 | Snow et al. | Apr 2000 | A |
| 6058163 | Pattison et al. | May 2000 | A |
| 6067357 | Kishinsky et al. | May 2000 | A |
| 6067517 | Bahl et al. | May 2000 | A |
| 6070142 | McDonough et al. | May 2000 | A |
| 6081606 | Hansen et al. | Jun 2000 | A |
| 6092197 | Coueignoux | Jul 2000 | A |
| 6094227 | Guimier | Jul 2000 | A |
| 6097429 | Seeley et al. | Aug 2000 | A |
| 6098066 | Snow et al. | Aug 2000 | A |
| 6111610 | Faroudja | Aug 2000 | A |
| 6134530 | Bunting et al. | Oct 2000 | A |
| 6138139 | Beck et al. | Oct 2000 | A |
| 6167395 | Beck et al. | Dec 2000 | A |
| 6170011 | Macleod Beck et al. | Jan 2001 | B1 |
| 6178239 | Kishinsky et al. | Jan 2001 | B1 |
| 6185550 | Snow et al. | Feb 2001 | B1 |
| 6212178 | Beck et al. | Apr 2001 | B1 |
| 6230197 | Beck et al. | May 2001 | B1 |
| 6246752 | Bscheider et al. | Jun 2001 | B1 |
| 6249570 | Glowny et al. | Jun 2001 | B1 |
| 6252946 | Glowny et al. | Jun 2001 | B1 |
| 6252947 | Diamond et al. | Jun 2001 | B1 |
| 6295367 | Crabtree et al. | Sep 2001 | B1 |
| 6327343 | Epstein et al. | Dec 2001 | B1 |
| 6330025 | Arazi et al. | Dec 2001 | B1 |
| 6345305 | Beck et al. | Feb 2002 | B1 |
| 6370574 | House et al. | Apr 2002 | B1 |
| 6404857 | Blair et al. | Jun 2002 | B1 |
| 6427137 | Petrushin | Jul 2002 | B2 |
| 6441734 | Gutta et al. | Aug 2002 | B1 |
| 6477533 | Schiff et al. | Nov 2002 | B2 |
| 6510220 | Beckett et al. | Jan 2003 | B1 |
| 6526397 | Chee et al. | Feb 2003 | B2 |
| 6549613 | Dikmen | Apr 2003 | B1 |
| 6559769 | Anthony et al. | May 2003 | B2 |
| 6563920 | Flockhart et al. | May 2003 | B1 |
| 6570608 | Tserng | May 2003 | B1 |
| 6604108 | Nitahara | Aug 2003 | B1 |
| 6614903 | Flockhart et al. | Sep 2003 | B1 |
| 6628835 | Brill et al. | Sep 2003 | B1 |
| 6633640 | Cohen et al. | Oct 2003 | B1 |
| 6651042 | Field et al. | Nov 2003 | B1 |
| 6661889 | Flockhart et al. | Dec 2003 | B1 |
| 6704409 | Dilip et al. | Mar 2004 | B1 |
| 6711585 | Copperman et al. | Mar 2004 | B1 |
| 6728345 | Glowny et al. | Apr 2004 | B2 |
| 6757361 | Blair et al. | Jun 2004 | B2 |
| 6785369 | Diamond et al. | Aug 2004 | B2 |
| 6785370 | Glowny et al. | Aug 2004 | B2 |
| 6898277 | Meteer et al. | May 2005 | B1 |
| 6937706 | Bscheider et al. | Aug 2005 | B2 |
| 7015945 | Sullivan | Mar 2006 | B1 |
| 7076427 | Scarano et al. | Jul 2006 | B2 |
| 7103806 | Horvitz | Sep 2006 | B1 |
| 7203285 | Blair | Apr 2007 | B2 |
| 7336779 | Boyer et al. | Feb 2008 | B2 |
| 7401087 | Copperman et al. | Jul 2008 | B2 |
| 7415417 | Boyer et al. | Aug 2008 | B2 |
| 20010040942 | Glowny et al. | Nov 2001 | A1 |
| 20010043685 | Bscheider et al. | Nov 2001 | A1 |
| 20010043697 | Cox et al. | Nov 2001 | A1 |
| 20010052081 | McKibben et al. | Dec 2001 | A1 |
| 20010054064 | Kannan | Dec 2001 | A1 |
| 20010055372 | Glowny et al. | Dec 2001 | A1 |
| 20020005898 | Kawada et al. | Jan 2002 | A1 |
| 20020010705 | Park et al. | Jan 2002 | A1 |
| 20020022977 | Schiff et al. | Feb 2002 | A1 |
| 20020035616 | Diamond et al. | Mar 2002 | A1 |
| 20020059283 | Shapiro et al. | May 2002 | A1 |
| 20020087385 | Vincent | Jul 2002 | A1 |
| 20020143609 | Magouirk et al. | Oct 2002 | A1 |
| 20020198775 | Ryan | Dec 2002 | A1 |
| 20030059016 | Lieberman et al. | Mar 2003 | A1 |
| 20030078782 | Blair | Apr 2003 | A1 |
| 20030128099 | Cockerham | Jul 2003 | A1 |
| 20030129986 | Blair et al. | Jul 2003 | A1 |
| 20030163360 | Galvin | Aug 2003 | A1 |
| 20040012540 | Treibitz et al. | Jan 2004 | A1 |
| 20040024739 | Copperman et al. | Feb 2004 | A1 |
| 20040098295 | Sarlay et al. | May 2004 | A1 |
| 20040141508 | Schoeneberger et al. | Jul 2004 | A1 |
| 20040161133 | Elazar et al. | Aug 2004 | A1 |
| 20040249650 | Freedman et al. | Dec 2004 | A1 |
| 20050015286 | Rudnik et al. | Jan 2005 | A1 |
| 20050030374 | Goldenberg et al. | Feb 2005 | A1 |
| 20050125437 | Cardno | Jun 2005 | A1 |
| 20060089837 | Adar et al. | Apr 2006 | A1 |
| 20060093135 | Fiatal et al. | May 2006 | A1 |
| 20060262922 | Margulies et al. | Nov 2006 | A1 |
| 20070033221 | Copperman et al. | Feb 2007 | A1 |
| 20070055694 | Ruge et al. | Mar 2007 | A1 |
| 20070217576 | Blair | Sep 2007 | A1 |
| 20080107255 | Geva et al. | May 2008 | A1 |
| 20080152122 | Idan et al. | Jun 2008 | A1 |
| 20080181417 | Pereg et al. | Jul 2008 | A1 |
| 20080240374 | Conway et al. | Oct 2008 | A1 |
| 20080240404 | Conway et al. | Oct 2008 | A1 |
| 20080240405 | Conway et al. | Oct 2008 | A1 |
| Number | Date | Country |
|---|---|---|
| 0833489 | Jan 1998 | EP |
| Number | Date | Country | |
|---|---|---|---|
| 20110206198 A1 | Aug 2011 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 10484107 | Jul 2004 | US |
| Child | 13098504 | US |
