Patent Application
20110218798
This description relates to techniques for obfuscating sensitive content in audio sources.
A contact center provides a communication channel through which business entities can manage their customer contacts and handle customer requests. Audio recordings or captures of spoken interactions between contact center agents and contact center callers are often used, for example, for later confirmation of content of the interaction, verification of compliance to required protocols, searching and analysis. However, recording or capturing may result in the storing of a host of sensitive information associated with contact center callers, including social security numbers, credit card numbers and authorization codes, and personal identification and authorization numbers. Storing such sensitive content may increase the possibility of compromising the privacy of the callers and may violate applicable privacy policies, regulations, or laws.
In general, in one aspect, the invention features a method for obfuscating sensitive content in an audio source representative of an interaction between a contact center caller and a contact center agent. The method includes performing, by an analysis engine of a contact center system, a context-sensitive content analysis of the audio source to identify each audio source segment that includes content determined by the analysis engine to be sensitive content based on its context; and processing, by an obfuscation engine of the contact center system, one or more identified audio source segments to generate corresponding altered audio source segments each including obfuscated sensitive content.
Embodiments of the invention include one or more of the following features.
The method of further includes preprocessing the audio source to generate a phonetic representation of the audio source. The method of performing the context-sensitive content analysis includes searching audio data according to a search query to identify putative occurrences of the search query in the audio source, wherein the search query defines a context pattern for sensitive content; and for each identified putative occurrence of the search query in the audio source, examining content of an audio source segment that excludes at least some portion of an audio source segment corresponding to the identified putative occurrence of the search query to determine whether linguistic units corresponding to a content pattern for sensitive content are present in the examined content. Searching the audio data according to the search query may include determining a quantity related to a probability that the search query occurred in the audio source. The search query may further define the content pattern for sensitive content. The method may further include accepting the search query, wherein the accepted search query is specified using Boolean logic, the search query including terms and one or more connectors. At least one of the connectors may specifiy a time-based relationship between terms. Search query may be accepted via a text-based interface, an audio-based interface, or some combination of both. The search query may be one of a plurality of predefined search strings for which the audio data is searched to identify putative occurrences of the respective search strings in the audio source. The search query may include a search lattice formed by a plurality of predefined search strings for which the audio data is searched to identify putative occurrences of the respective search strings in the audio source. The method of performing the context-sensitive content analysis may include determining a start time and an end time of each audio source segment. The method start time, the end time, or both may be determined based at least in part on one of the following: a speaker change detection, a speaking rate detection, an elapsing of a fixed duration of time, an elapsing of a variable duration of time, a contextual pattern of content in a subsequent audio source segment, and voice activity information. The method of processing one or more identified audio source segments to generate corresponding altered audio source segments may include substantially reducing a volume of at least a first of the one or more audio source segments to render its corresponding sensitive content inaudible. The method of processing one or more identified audio source segments to generate corresponding altered audio source segments may include substantially masking at least a first of the one or more audio source segments to render its corresponding sensitive content unintelligible. The method of processing one or more identified audio source segments to generate corresponding altered audio source segments may include redacting at least a portion of a first of the one or more audio source segments to render its corresponding sensitive content unintelligible. The method of processing one or more identified audio source segments to generate corresponding altered audio source segments may further include storing the portion of the first of the one or more audio source segments that is redacted as supplemental information metadata. The method may further include permanently removing the one or more identified audio source segments prior to storing a modified version of the audio source representative of the interaction between the contact center caller and the contact center agent. The method may further include combining the altered audio source segments with unaltered segments of the audio source prior to storing a result of the combination as a modified version of the audio source representative of the interaction between the contact center caller and the contact center agent. The method may further include storing the altered audio source segments and unaltered segments of the audio source in association with a value that uniquely identifies the interaction between the contact center caller and the contact center agent. The sensitive content may include one or more of the following: a credit card number, a credit card expiration date, a credit card security code, a personal identification number, and a personal authorization code. The context-sensitive content analysis of the audio source, the generation of corresponding altered audio source segments, or both may occur substantially in real-time or offline.
Other general aspects include other combinations of the aspects and features described above and other aspects and features expressed as methods, apparatus, systems, computer program products, and in other ways.
Other features and advantages of the invention are apparent from the following description, and from the claims.
Referring to
Very generally, a caller contacts a contact center by placing telephone calls through a telecommunication network, for example, via the public switched telephone network (PSTN). In some implementations, the caller may also contact the contact center by initiating data-based communications through a data network (not shown), for example, via the Internet by using voice over internet protocol (VoIP) technology.
Upon receiving an incoming request, a control module of the system 100 uses a switch to route the customer call to a contact center agent. The connection of an agent's telephone to a particular call causes a Voice Response Unit (“VRU”) module in the system 100 to notify the caller that the call may be recorded for quality assurance or other purposes, and signal an audio acquisition module 102 of the system 100 to start acquiring signals that are being transmitted over audio channels associated with the caller and the agent. In the depicted two-channel example of
In some implementations of the contact center service system 100 in which a stored audio record of the telephone call is desired, rather than directly storing the audio signals in a permanent archive, the audio input signals are stored as raw media files (e.g., raw caller media file 104 and raw agent media file 106) in a temporary data store (not shown) only for the period of time needed to process the media files and obfuscate any sensitive content that is identified within. Once the sensitive content is obfuscated, the raw media files 104, 106 are permanently deleted from the temporary data store.
During a pre-processing phase, a wordspotting engine 108 of the system 100 takes as input the raw media files 104, 106, and executes one or more queries to detect any occurrences of sensitive content. In some implementations, the wordspotting engine first performs an indexing process on each media file 104, 106. In the depicted example, the results of the indexing process are two phonetic audio track (PAT) files. The first PAT file (PATCaller file 110) is a searchable phonetic representation of the audio track corresponding to the caller audio input signal, and the second PAT file (PATAgent file 112) is a searchable phonetic representation of the audio track corresponding to the agent audio input signal.
During a search phase, the wordspotting engine 108 performs phonetic-based query searching on the PATAgent file 112 to locate putative occurrences (also referred to as “putative hits” or simply “Put. Hits 114”) of one or more queries (e.g., search term or phrase) in the PATAgent file 112. Details of implementations of the wordspotting engine 102 are described in U.S. Pat. No. 7,263,484, titled “Phonetic Searching,” issued Aug. 28, 2007, and U.S. patent application Ser. No. 10/565,570, titled “Spoken Word Spotting Queries,” filed Jul. 21, 2006, U.S. Pat. No. 7,650,282, titled “Word Spotting Score Normalization,” issued Jan. 19, 2010, and U.S. Pat. No. 7,640,161, titled “Wordspotting System,” issued Dec. 29, 2009. The content of these patents and patent applications are incorporated herein by reference in their entirety.
One example of such phonetic-based query searching is described below in the context of an application (referred to herein as “CCV application”) that detects and obfuscates of all digit sequences representative of credit card verification codes. First, a context-based analysis includes searching the PATAgent file 112 to identify contextual patterns of words that occur within PATAgent file 112 is performed. Such contextual patterns of words (referred to generally as “query 116”) may include some combination of the following words: “credit card number,” “verification code,” “validation code,” “verification value,” “card verification value,” “card code verification,” “card code verification,” “security code,” “three-digit,” “four-digit,” “sixteen-digit,” “unique card code,” “got it,” “thank you”). The query 116 may be specified using Boolean logic, where connectors may represent distances between query terms. In one example, the query 116 may specify searching for the term “verification code” within the same sentence, or within five seconds of the terms “three-digit” or “four-digit.” In another example, the query 116 may specify searching for the term “verification code” within two seconds of the term terms “three-digit” or “four-digit” and within fifteen seconds of the term (“got it” OR “thank you”). Search results (Put. Hits 114) are a list of time offsets into the raw agent media file 106 storing the agent audio input signals, with an accompanying score giving the likelihood that a match to the query happened at this time.
Next, the context-based analysis includes passing the Put. Hits 114 to an obfuscation engine 118 of the system 100, which uses the Put. Hits 114 to locate likely sensitive time intervals (at times also referred to herein as “context-based caller intervals of interest”) in the raw caller media file 104 that should be obfuscated. Contextual patterns of words detected in the PATAgent file 112 effectively serve as a hint (i.e., increasing the likelihood) that part of the raw caller media file 104 in close time proximity may include content to be obfuscated. The obfuscation engine 118 can implement obfuscation logic 120 that, amongst other things, identifies the time of the raw caller media file 104 that corresponds to a speaker change (e.g., from agent to caller) following a putative hit. This time represents a start time of an interval of interest. The end time of the context-based caller interval of interest may correspond to a point in time after: (1) some fixed duration of time has elapsed (e.g., 10 seconds after the start time); or (2) some variable duration of time has elapsed (e.g., based in part on a determined speaking rate of the caller). The obfuscation engine 118 can also implement obfuscation logic 120 that identifies the time interval of the raw caller media file 104 that is straddled by multiple putative hits that satisfies a single query. One such example is the designation of the time of the raw caller media file 104 that occurs after the term “verification code” is located within two seconds of the term “three-digit” in the PATAgent file 112 as the start time of the context-based caller interval of interest, and the time of the raw caller media file 104 that precedes the detection of the term “got it” in the PATAgent file 112 as the end time of the context-based caller interval of interest.
Finally, in some implementations, the context-based analysis includes use of the obfuscation logic 120 to process each context-based caller interval of interest in the raw caller media file 104 and obfuscate its content. Such processing may include the generation of altered voice segments of the caller audio input signal corresponding to the specified interval of interest in the raw caller media file 104. In the depicted example, a voice segment may be altered by substantially masking its content through the overwriting of the content by a “bleeper” 122 with an auditory tone, such as a “bleep.” In other examples, a voice segment may be altered by substantially reducing its volume to render its content inaudible to a human listener or otherwise processed in the audio domain. In some examples, the processing effectively encrypts the voice segment. In some examples, an indication (e.g., an audio message) of why the voice segment was altered may be appended to or otherwise stored in association with the voice segment. In some examples, in lieu of altering the voice segment, the voice segment corresponding to the time interval of interest in the raw caller media file 104 is removed from the raw caller media file 104 prior to the commitment of the raw caller media file 104 to a permanent or semi-permanent storage module as a final caller media file 124.
In some implementations, the results of the context-based analysis are validated prior to obfuscating the content in the context-based caller intervals of interest. In one example, the PATCaller file 110 is examined to determine whether any portion of the PATCaller file 110 satisfies a grammar specification (e.g., three consecutive digits representative of a three-digit verification code) for sensitive content. Such grammar specifications for sensitive content may be specified using a predefined set of queries 128. The wordspotting engine 108 performs phonetic-based query searching on the PATCaller file 110 to locate putative occurrences (also referred to as “putative hits” or simply “Put. Hits 130”) of one or more the queries 128 in the PATCaller file 110, and passes the Put. Hits 130 to the obfuscation engine 118. The bleeping logic 120 can be implemented to examine each of the Put. Hits 130 to determine whether the Put. Hit 130 falls within a context-based caller interval of interest. A positive result validates the result of the context-based analysis and the content within the context-based caller interval of interest is obfuscated by the bleeper 122. In some implementations, the entirety of the content within the context-based caller interval of interest is obfuscated. In other implementations, only the portion of the context-based caller interval of interest that corresponds to its Put. Hit 130 is obfuscated. In those instances in which the examination yields a negative result, no action is taken by the bleeper 122 with respect to the context-based caller interval of interest.
In some implementations, the obfuscation engine 118 of the system 100 uses the Put. Hits 114 to locate interesting time intervals (at times also referred to herein as “context-based agent intervals of interest”) in the raw agent media file 104 that should be obfuscated. Contextual patterns of words detected in the PATAgent file 112 serve as a hint that part of the raw agent media file 104 in close time proximity may include content to be obfuscated. Suppose, for example, the query 116 specifies searching for the terms “did you say” or “I'm going to repeat” within the same sentence or within ten words of the terms “verification code” and “three-digit.” The obfuscation engine 118 can implement obfuscation logic 120 that, amongst other things, determines whether any portion of the PATAgent file 112 satisfies a grammar specification (e.g., three consecutive digits representative of a three-digit verification code) for sensitive content, and obfuscates the sensitive content if the examination yields a positive result. In this manner, the sensitive content representative of the three-digit verification code is not only obfuscated in the final caller media file 106 but also the final agent media file 126.
In the depicted example of
Although one implementation of the present invention is described above in a batch mode context, the techniques of the present invention are also applicable in a real-time context, in which the raw media files 104, 106 are processed at about the time the speech is uttered by the speakers and the final media files 124, 126 are made available to a listener in real-time shortly thereafter. For example, in a near real-time monitoring application, a person monitoring the telephone conversation may hear a beep in place of sensitive information.
Referring now to
The audio mining module 252 processes the audio input signals on a per-channel basis to generate information (referred to in this description as “supplemental information” 256) that is representative of characteristics of the audio signal(s) being processed. Some of the supplemental information 256 may be representative of characteristics of a single audio input signal, while others of the supplemental information 256 may be representative of characteristics of multiple audio input signals relative to one another. Referring also to
The process of generating the monaural recording 254 may be performed by the audio aggregation module 250 concurrent with, or within close temporal proximity of, the processing of the audio input signals by the audio mining module 252.
Referring again to
Referring also to
In the depicted examples of
During a pre-processing phase, a wordspotting engine 208 of the system 200 takes as input the raw media files 204, 206, and performs an indexing process on each media file 204, 206 to generate a PATCaller file and a PATAgent file. During a search phase, the wordspotting engine 208 performs phonetic-based query searching on the PATAgent file to locate putative occurrences “Put. Hits 214” of one or more queries (e.g., search term or phrase) in the PATAgent file. The Put. Hits 214 are passed to an obfuscation engine 218 of the system which performs a context-based analysis and optionally performs a content-based validation as described above with respect to
The foregoing approaches may be implemented in software, in hardware, or in a combination of the two. In some examples, a distributed architecture is used in which the techniques implemented by the audio acquisition module are performed at a different location of the architecture than those implemented by the audio aggregation module and/or the audio mining module. In some examples, a distributed architecture is used in which the wordspotting stage is performed at a different location of the architecture than the automated speech recognition. For example, the wordspotting may be performed in a module that is associated with a particular conversation or audio source, for example, associate with a telephone for a particular agent in a call center, while the automated speech recognition may be performed in a more centralized computing resource, which may have greater computational power. In examples in which some or all of the approach is implemented in software, instructions for controlling or data imparting functionality on a general or special purpose computer processor or other hardware is stored on a computer readable medium (e.g., a disk) or transferred as a propagating signal on a medium (e.g., a physical communication link).
It is to be understood that the foregoing description is intended to illustrate and not to limit the scope of the invention, which is defined by the scope of the appended claims. Other embodiments are within the scope of the following claims.
This application is related to U.S. patent application Ser. No. ______, titled “Channel Compression,” (Attorney Docket No.: 30004-032001) filed concurrently with the present application. The content of this application is incorporated herein by reference in its entirety.
