Patent Grant
10951763
Customer Relationship Management (‘CRM’) is an approach to managing a company's interaction with current and potential customers. CRM implements data analysis of customers' history with a company to improve business relationships with customers, specifically focusing on customer retention and sales growth. CRM systems compile data from a range of communication channels, including telephone, email, live chat, text messaging, marketing materials, websites, and social media. Through the CRM approach and the systems used to facilitate it, businesses learn more about their target audiences and how to best address their needs.
Enterprise CRM systems can be huge. Such systems can include data warehouse technology, used to aggregate transaction information, to merge the information with information regarding CRM products and services, and to provide key performance indicators. CRM systems aid managing volatile growth and demand and implement forecasting models that integrate sales history with sales projections. CRM systems track and measure marketing campaigns over multiple networks, tracking customer analysis by customer clicks and sales. Some CRM software is available through cloud systems, software as a service (SaaS), delivered via network and accessed via a browser instead of installed on a local computer. Businesses using cloud-based CRM SaaS typically subscribe to such CRM systems, paying a recurring subscription fee, rather than purchasing the system outright.
Despite their sheer size, many CRM systems today lack the infrastructure to make full use of the information they can access. Customer contacts alone, for example can be difficult to track. A tele-agent today does not limit customer contacts merely to phone calls from a desk in a call center. Such contacts are often administered through contact centers that can administer multiple modes of contact, phone, text, email, and so on, among multiple agents across multiple locations. When a particular tele-agent representing a contact center on behalf of a marketing client accepts a phone call from a customer representative, it is entirely possible that there can have been multiple intervening contacts with that customer representative, including text messages and emails or even automated messages from artificial intelligence agents, very difficult for the current tele-agent to know or sort.
Example methods and apparatus for customer relationship management (“CRM”) implemented in a computer system are described with reference to the accompanying drawings, beginning with
A lead engine (134) is automated computing machinery that gathers leads from various resources and provides them to a tele-agent through a GUI for use with customers and prospective customers. A lead is structured data representing a customer or potential customer typically including a lead ID, lead name, company, role of the lead, address of the lead or company, phone number of the lead and other relevant information as will occur to those of skill in the art. Such a lead may be implemented as a record, message, object, or other data structure useful to automated computing machinery for automatic lead generation and presentation through a GUI to a tele-agent.
A script engine (206) is automated computing machinery that creates in real-time a dynamic script for a tele-agent to use in communicating with a customer. A dynamic script is a script that changes in real-time in dependence upon various factors such as current industry trend data and often the specific products that the tele-agents supports and the products that the customer does or does not have. That is, the dynamic script is dynamic in the sense that it changes in real-time based on industry trends. The sentences in the script are dynamically reordered, added, or deleted in real-time for the benefit of the customer. Such sentences may be dynamically altered in the script by being dynamically reordered or created in real-time, retrieved from a repository of relevant industry and software descriptions, provided by other tele-agents, or in other ways as will occur to those of skill in the art.
Automated computing machinery, as that phrase is used in this specification, means a module, segment, or portion of code or other automated computing logic, hardware, software, firmware, or the like, as well as a combination of any of the aforementioned, local or remote. Automated computing machinery is often implemented as executable instructions, physical units, or other computing logic for implementing specified logical functions.
A tele-agent (128) is a person, an agent of a contact center, responsible for selling or supporting commercial products and services. A customer representative (129) is a person who represents a customer, a company or other enterprise that is a current or prospective purchaser of goods or services of a contact center.
A CRM contact center (305) is an organization of personnel and computer resources that provide CRM according to embodiments of the present invention. In the example of
A semantic graph database (152) is a database that uses graph structures for semantic queries with nodes, edges and properties to represent and store data. A key concept of this database system is the graph (or edge or relationship), which directly relates data items in a data store. The relationships allow data in the store to be linked together directly, and in many cases retrieved with one operation. Such a graph database contrasts with conventional relational databases, where links among data are mere indirect metadata, and queries search for data within the store using joins to collect related data. Graph databases, by design, make explicit relations among data and allow simple and fast retrieval of complex hierarchical structures that are difficult to model in relational systems.
The underlying storage mechanism of graph databases can vary. Some depend on a relational engine and store the graph data in a table. Others use a key-value store or document-oriented database for storage, making them inherently NoSQL structures. Retrieving data from a graph database often requires a query language other than SQL, which was designed for relational databases and does not elegantly handle traversing a graph. There are a number of systems, most often tightly tied to one product and there are some multi-vendor query languages like Gremlin, SPARQL, and Cypher. In addition to having query language interfaces, some, if not most, graph databases are accessed through application programming interfaces (APIs).
Graph databases are based on graph theory, and employ nodes, edges, and properties. Nodes represent entities such as people, businesses, accounts, or any other item to be tracked. They are roughly the equivalent of the record, relation, or row in a relational database, or the document in a document database. Edges, also termed graphs or relationships, are the lines that connect nodes to other nodes; they represent the relationship between them. Meaningful patterns emerge when examining the connections and interconnections of nodes, properties, and edges. Edges are the key concept in graph databases, representing an abstraction that is not directly implemented in other systems. Properties are germane information that relate to nodes. For example, if N3 were one of the nodes, it might be tied to properties such as web-services support, cloud-computing, or a word that starts with the letter N, depending on which aspects of N3 are germane to a given database.
The graph database of
The enterprise knowledge graph (154) of
Computer memory (169) can include cache, random access memory (“RAM”), disk storage, and so on, most forms of computer memory. Computer memory so configured typically resides upon speech-enabled devices, or, as shown here (169), upon one or more triple servers (157).
Words (509) of text, spoken or typed, among other things such as images, digital text, and the like, typically compose at least some of the content (509) of sessions and contacts. Such words can be words typed into a text box, words from email or text messages, or words of digitized speech for recognition (315) from a conversation (313). The speech for recognition can be the entire conversation, where, for example, both persons speaking are in the same room, and the entire conversation is picked up by a microphone on a speech-enabled device. The scope of speech for recognition can be reduced by providing to a speech-enabled device only one side of the conversation, as only through a microphone on a headset (105). The scope of speech for recognition can be further reduced by providing for recognition only speech that responds to a prompt from a VoiceXML dialogue executing on a client computer. As the scope of speech for recognition is reduced, data processing burdens are reduced across the system as a whole, although it remains an option, in some embodiments at least, to recognize the entire conversation and stream across a display (110) a flow of all words in the conversation.
Speech from the conversation (313) is recognized into digitized words by operation of a natural language processing speech recognition (“NLP-SR”) engine (153), shown here disposed upon a voice server (151), but also amenable to installation on speech-enabled client computers. In addition to digitizing the word (509) by speech recognition functions of a voice server (151), for a further example, the word (509) can be digitized by operation of widgets of a graphical user interface (110). Or, for an even further example, the word (509) can be digitized by a user's (128) typing it into a text entry box (366) of a graphical user interface (110).
The AIA is an implementation of artificial intelligence, automation that seeks to present behavior that resembles human behavior in communications and learning. The AIA operates with a voice server to present human-like speech. The AIA operates a natural language processor to present human-like text in emails. And the AIA implements machine learning by expanding an enterprise knowledge graph (154) with newly parsed triples (752) and newly inferred triples (754). The AIA is said to be “of” the CRM application, optionally because in some embodiments it is an actual component of the CRM application, or optionally because the AIA effectively operates the CRM through an application programming interface, so that the AIA as ‘agent’ is, to some extent at least, an agent of the CRM application in particular and the contact center generally. In administering, effecting, or carrying out communications with both the tele-agent and the customer representative the AIA functions as a kind of wrapper or container of the CRM application and the computer system in general. That is, in embodiments, the tele-agent and the customer representative often encounter the entire contact center primarily through the AIA.
In many embodiments, the AIA (196) establishes, upon a first communications contact (142) between a tele-agent (128) and a customer representative (129), as structure of computer memory (169) of the computer system, a communications session (140). The AIA then administers through the communications session across communications platforms (101) a sequence of communications contacts, including the first contact (142) and also subsequent communications contacts (144), between the tele-agent and the customer representative. Examples of communications platforms include email platforms, chatbots, telephones, and text messaging platforms. Examples of text messaging platforms that can be adapted as CRM communications platforms according to embodiments include SMS messaging over telephone systems, web chat inline within a web application, and web-oriented messaging applications such as WhatsApp, Facebook Messenger, Slack, Snapchat, Cryptocat, Kik, Google Hangouts, Line, MeowChat, Ethan, China's WeChat and QQ Messenger, Viber, Korea's KakaoTalk, Vietnam's Zalo, as well as direct messaging functions in Instagram and Twitter.
In many embodiments, the AIA establishes, as structure of computer memory (169) of the computer system, a session (140) as an object-oriented module of automated computing machinery whose structure and contents (509) are also stored as semantic triples in an enterprise knowledge graph (154). That is, a session in such embodiments is established, initially at least, as an instance of an object-oriented session class. Establishing such a session can include storing in computer memory as member data elements of the session a subject code, a timestamp, identification of the tele-agent, identification of the customer representative, and optionally as well other information regarding the session.
The AIA in this example administers a sequence of contacts (142, 144) by establishing each contact as an object-oriented module of automated computing machinery that structures computer memory (169) of the computer system and whose structure and contents are also stored as semantic triples in an enterprise knowledge graph (154). That is, a contact in this example is established initially at least as an instance of an object-oriented contact class. Administering a sequence of contacts includes recording in computer memory as member data elements of each contact a timestamp beginning the contact, duration of the contact, a session identifier for the contact, platform type, contact status, any communications content of the contact, and optionally as well other information regarding the contact.
Administering contacts across platforms includes administering contacts asynchronously across a same platform or across different platforms, and administering across platforms includes administering across platform types. A first contact in a session can be by telephone with a subsequent contact by the same telephone or a different telephone. A first contact can be by email with subsequent contacts by telephone or text message.
Administering contacts across platforms includes administering contacts across physical locations of platforms. A first contact of a session can be by landline telephone in a call center with a subsequent contact by cell phone in a restaurant. A first contact can be by email from a desk in a call center with subsequent contacts by cell phone or text message from a supermarket.
The example computer system of
A parsed triple (752) is a semantic triple composed of, for example, a subject, a predicate, and an object, that is ‘parsed’ in the sense that it is composed from elements derived from a session or a contact, including, for example, recognized speech, selections from GUI widgets, text typed through a GUI, or the like. That is, “parsing” means taking component elements of sessions and contacts from various sources and forming them into semantic triples. Parsing can include forming into semantic triples raw text from data entry or conversational speech that is processed by an NLP-SR engine into parts of speech. Such parts of speech can be formed into triples by placing each part appropriately as subject-predicate-object in a triple.
An inferred triple (754) is inferred from a parsed triple (752) according to inference rules (376) that govern relations among data elements in a knowledge graph, here, an enterprise knowledge graph (154). Inferencing is a process by which new triples are systematically added to a graph based on patterns in existing triples. Information integration, inclusion of newly inferred triples, can be achieved by invoking inferencing before or during a query process. A query executed with inference rules in an engine designed not only for queries but also for inferencing, returns not only asserted data but also inferred information. Readers will recognize that all such references to “data” or “information” or “knowledge” are in fact references to semantic triples.
Here is an example of an inference rule:
In plain language, this rule says that if class A is a subclass of class B, anything of type A is also of type B. We refer to this rule as a ‘type propagation rule,’ A is a subclass of B, x is an A, x is also a B. Or in a SPARQL CONSTRUCT query, this type propagation rule can be expressed:
For further explanation, assume these example triples are previously asserted:
Applying the type propagation rule results in the following inferences:
Two of the inferred triples were also previously asserted:
And here, for an inference engine that simply inserts the inferred triples back into the asserted graph, is the new graph, with inferred triples in bold italics:
The underlying purpose of such inferencing is to create data that are more connected, better integrated, and in which the consistency constraints on the data are expressed in the data itself. The data itself describes how the data can be used. For purposes of making data more integrated and consistent, simple inferences like those just described are often more useful than elaborate ones, not very exciting, but very useful. It is this kind of workaday consistency completion of CRM data that can be done with inferencing according to embodiments. Although inferencing of this kind may seem at first glance to have little significance, in fact, this kind of inferencing is exactly the kind of correlation that effects consistency in large data stores.
In this paper, “parsed triples” are triples parsed more or less in real time from structure and content of contacts and sessions as or promptly after a contact occurs. Possibly many of the triples in the enterprise knowledge graph were parsed at some point, but, nevertheless, for clarity of description in this paper, triples already in the enterprise knowledge graph are “asserted triples,” and triples just now parsed from structure and content of a session and a contact are “parsed triples.” “Inferred triples” are triples inferred more or less in real time from parsed triples as a contact occurs or promptly afterwards. Possibly many of the triples in the enterprise knowledge graph were inferred at some point, but, nevertheless, for clarity of description in this paper, triples already in the enterprise knowledge graph are “asserted triples,” and triples just now inferred from triples parsed from a session or contact are “inferred triples.”
There is no structural, functional, or logical difference among these triples. Asserted triples, parsed triples, inferred triples, they are all exactly the same kind of entity. The labels “asserted,” “parsed,” and “inferred” are used here merely for ease of explanation, not for identifications of differences in kind. All of these triples are three-part expressions of information conforming to some form of defined logic of a knowledge graph, in some embodiments a description logic, in some embodiments a logic that supports decidability.
In some embodiments, the enterprise knowledge graph (154) may include all or most information describing or pertinent to or useful in an entire corporate enterprise, financials, business entities and structures, employee data, incorporation data, transactions, contracts, sales history, product descriptions, and so on, and so on. Thus, CRM information is a subset of overall corporate information, and asserted, parsed, and inferred triples are kinds of CRM information. The present description of triples as subgraphs of an overall enterprise knowledge graph is for explanation rather than limitation. In some embodiments at least, for various reasons, CRM data, communications session data, contact data, customer data, information related to customer representatives, or call note data can be implemented in separate graphs rather than subgraphs. In the example of
A triple (752, 754) is a three-part statement expressed in a form of logic. Depending on context, different terminologies are used to refer to effectively the same three parts of a statement in a logic. In first order logic, the parts are called constant, unary predicate, and binary predicate. In the Web Ontology Language (“OWL”) the parts are individual, class, and property. In some description logics the parts are called individual, concept, and role. In this paper, the elements of a triple are referred to as subject, predicate, and object—and expressed like this: <subject> <predicate> <object>—or like this: (subject predicate object)—or in other abstract forms intended to be read by a human. There are many modes of expression for triples. Elements of triples can be represented as Uniform Resource Locaters (“URLs”), Uniform Resource Identifiers (“URIs”), or International Resource Identifiers (“IRIs”). Triples can be expressed in N-Quads, Turtle syntax, TriG, Javascript Object Notation or “JSON,” the list goes on and on. The expression used here, subject-predicate-object in angle brackets or parenthesis, is a form of abstract syntax, optimized for human readability rather than machine processing, although its substantive content is correct for expression of triples. Using this abstract syntax, here are examples of triples:
The same item can be referenced in multiple triples. In this example, Bob is the subject of four triples, and the Mona Lisa is the subject of one triple and the object of two. This ability to have the same item be the subject of one triple and the object of another makes it possible to effect connections among triples, and connected triples form graphs.
For further explanation of relations among triples and graphs,
In systems of knowledge representation, knowledge is represented in graphs of triples, including, for example, knowledge representations implemented in Gremlin, in Cypher, in Prolog databases, in Lisp data structures, or in RDF-oriented ontologies in RDFS, OWL, and other ontology languages. Search and inference are effected against such graphs by search engines configured to execute semantic queries and semantic inference in, for example, Gremlin, Cypher, Prolog, Lisp, or SPARQL.
Gremlin is a query language provided through the TinkerPop graph computing framework from the Apache Foundation. Cypher is an AI program that generates SPARQL queries from natural language input, allowing users to speak plain language to update and query databases; Cypher brings its own grammar and lexicon to natural language processing. SPARQL is a recursive acronym for “SPARQL Protocol and RDF Query Language.” Lisp is a reliable, flexible programming language that is widely used in artificial intelligence, knowledge representation, and semantic applications. Prolog is a general-purpose logic programming language. Prolog supports queries against connected triples expressed as statements and rules in a Prolog database. SPARQL supports queries against ontologies expressed in RDFS, OWL, or other RDF-oriented ontologies. Regarding Prolog, SPARQL, Cypher, Gremlin, Lisp, and so on, these are examples of technologies explanatory of example embodiments of the present invention. Thus, such are not limitations of the present invention. Knowledge representations useful according to embodiments can take many forms in the art, now or in the future, and all such are now and will continue to be well within the scope of the present invention.
A description logic is a member of a family of formal knowledge representation languages. Some description logics are more expressive than propositional logic but less expressive than first-order logic. In contrast to first-order logics, reasoning problems for description logics are usually decidable. Efficient decision procedures therefore can be implemented for problem of search and inference in description logics. There are general, spatial, temporal, spatiotemporal, and fuzzy descriptions logics, and each description logic features a different balance between expressivity and reasoning complexity by supporting different sets of mathematical constructors.
Search queries are disposed along a scale of semantics. A traditional web search, for example, is disposed upon a zero point of that scale, no semantics, no structure. A traditional web search against the keyword “derivative” returns thousands of HTML documents discussing the literary concept of derivative works as well as calculus procedures. A traditional web search against the keyword “differential” returns many web pages describing automobile parts and many web pages discussing calculus functions.
Other queries are disposed along mid-points of the scale, some semantics, some structure, not entirely complete. This is actually a current trend in web search. Such systems may be termed executable rather than decidable. From some points of view, decidability is not a primary concern. In many Web applications, for example, data sets are huge, and they simply do not require a 100 percent correct model to analyze data that may have been spidered, scraped, and converted into structure by some heuristic program that itself is imperfect. People use Google because it can find good answers a lot of the time, even if it cannot find perfect answers all the time. In such rough-and-tumble search environments, provable correctness is not a key goal.
Other classes of queries are disposed where correctness of results is important, and decidability enters. A user who is a tele-agent in a data center speaking by phone with an automotive customer discussing a front differential is concerned not to be required to sort through calculus results to find correct terminology. Such a user needs correct definitions of automotive terms, and the user needs query results in conversational real time, that is, for example, within seconds.
In formal logic, a system is decidable if there exists a method such that, for every assertion that can be expressed in terms of the system, the method is capable of deciding whether or not the assertion is valid within the system. In practical terms, a query against a decidable description logic will not loop indefinitely, crash, fail to return an answer, or return a wrong answer. A decidable description logic supports data models or ontologies that are clear, unambiguous, and machine-processable. Undecidable systems do not. A decidable description logic supports algorithms by which a computer system can determine equivalence of classes defined in the logic. Undecidable systems do not. Decidable description logics can be implemented in C, C++, SQL, Lisp, RDF/RDFS/OWL, and so on. In the RDF space, subdivisions of OWL vary in decidability. Full OWL does not support decidability. OWL DL does.
For further explanation,
The overall example computer system illustrated in
The example computer system illustrated in
The AIA also operates the system of
The example system of
In the example of
In this example, the inference engine (298) operates in the context of a semantic reasoner (378). A semantic reasoner is a module of automation that expands an inference engine with enriched functionality. In this paper, it is generally assumed that query execution and inferencing are carried out by the same engine, a query and inference engine, often referred to simply as an inference engine. It is possible to configure a query engine separate from an inference engine, but for ease of explanation, in this paper, the query function and the inference function are combined in an “inference engine.” With a reasoner, the functionality of the inference engine can be limited to query execution and inferencing. Additional functionality associated with queries and inferencing is provided by the semantic reasoner. Examples of such additional functionality include inserting parsed triples and inferred triples into an enterprise knowledge base, storing parsed or inferred triples as a separate graph for further processing, storing parsed and inferred triples into a new separate dataset such as for publication, serializing parsed and inferred triples for saving to a file, displaying parsed and inferred triples for administration by a user, and so on.
For further explanation,
The dashboard implements digital controls called widgets. A widget is a GUI element of interaction, such as a button or a scroll bar. Widgets are automated elements with which a user interacts with through direct manipulation to read or edit information or administer control. Each widget appears as a digital part of a GUI and facilitates a specific type of user-computer interaction. Some widgets support interaction with the user, for example, labels, buttons, and check boxes. Others act as containers that group widgets added to them, for example, windows, panels, and tabs. Examples of widgets optionally useful in CRM controls according to embodiment include buttons, radio buttons, check boxes, sliders, list boxes, spinners, drop-down lists, menus, menu bars, scroll bars, text boxes, dialog boxes, and so on.
In the example of
In the example of
The dashboard in the example of
In a second alternative way of providing a digitized content to the parsing engine, the dashboard gathers into a contact the content of call notes from a text box widget (366). The tele-agent, rather than speaking words as content of a contact, types call notes into a text box widget (366), and the text so typed is provided by the dashboard as digitized content of a contact to a natural language processing engine (155). The natural language processing engine sees no difference between typed words from the text box (366) and the words in a stream of digitized speech from a spoken conversation. Thus, this second alternative is similar to the first alternative with the exception that there is no need for speech recognition, because when a stream of digitized text arrives in the speech engine (153), the words in the stream are already digitized by typing them in through the text box (366). The natural language processing engine (155) works the same way as in the first alternative, processing the digitized text from the text box (366) into sentences and parts of speech and passing the words so processed (509) to the parsing engine (298) where they are parsed into triples (752). This is a second alternative way in which the dashboard provides content for inclusion in CRM contacts.
In a third alternative way of providing a digitized word to the parsing engine, the dashboard passes to the parsing engine words designated as elements of triples through widgets (360, 362, 364) of the dashboard. These widgets are pull-down menu listings of Subjects for triples (360), Predicates for triples (362), and Objects (364) for triples. The Predicates (362) and Objects (364) usually are triple elements already defined in an ontology supportive of a semantic CRM triple store portion of an enterprise knowledge graph. The Subjects (360) are a stream of text word candidates for inclusion in triples. Text in the Subjects pull-down menu (360) is provided by the speech engine (153) from call notes text (366) or from words recognized from a conversation (313) or otherwise. The tele-agent (128) designates content (509) for inclusion in a contact by selecting text from the Subjects pull-down menu (360), for example, selecting by keyboard or mouse (181). The tele-agent can select text from the pull-down (360) by double-click or by drag-and-drop onto the assembly box (368) for triples. The tele-agent can optionally also select a Predicate (362) or an Object (364) for inclusion in the same triple with the selected Subject, also by double-click or drag-and-drop. The tele-agent's selections of predicates and objects in some embodiments can be binding upon the parsing engine. In other embodiments, the tele-agent's selections are treated by the parsing engine merely as recommendations. The parsing engine (298) optionally accepts the tele-agent's selections of predicates and objects, or the parsing process makes its own selections of predicates and triples for inclusion with a word in at least one parsed triple (752). This is a third alternative way in which the dashboard provides content for inclusion in CRM contacts.
For further explanation,
In the particular example of
The example apparatus of
VOIP stands for ‘Voice Over Internet Protocol,’ a generic term for routing speech over an IP-based data communications network. The speech data flows over a general-purpose packet-switched data communications network, instead of traditional dedicated, circuit-switched voice transmission lines. Protocols used to carry voice signals over the IP data communications network are commonly referred to as ‘Voice over IP’ or ‘VOIP’ protocols. VOIP traffic may be deployed on any IP data communications network, including data communications networks lacking a connection to the rest of the Internet, for instance on a private building-wide local area data communications network or ‘LAN.’
Many protocols may be used to effect VOIP, including, for example, types of VOIP effected with the IETF's Session Initiation Protocol (‘SIP’) and the ITU's protocol known as ‘H.323.’ SIP clients use TCP and UDP port 5060 to connect to SIP servers. SIP itself is used to set up and tear down calls for speech transmission. VOIP with SIP then uses RTP for transmitting the actual encoded speech. Similarly, H.323 is an umbrella recommendation from the standards branch of the International Telecommunications Union that defines protocols to provide audio-visual communication sessions on any packet data communications network.
The AIA (196) and the CRM application (195) in this example are user-level, speech-enabled, client-side computer programs that presents a voice interface to user (128), provide audio prompts and responses (314) and accept input speech for recognition (315). The AIA and the CRM application (195) provide a speech interface through which a user may provide oral speech for recognition through microphone (176) and have the speech digitized through an audio amplifier (185) and a coder/decoder (‘codec’) (183) of a sound card (174) and provided for recognition to a voice server (151). Speech-enabled application (195) packages digitized speech in recognition request messages according to a VOIP protocol, and transmits the speech to voice server (151) through the VOIP connection (216) on the network (100).
The voice server (151) provides voice recognition services for speech-enabled devices by accepting dialog instructions, VoiceXML segments, or the like, and returning speech recognition results, including text representing recognized speech, text for use as variable values in dialogs, and output from execution of semantic interpretation scripts as well as voice prompts. Voice server (151) includes computer programs that provide text-to-speech (‘TTS’) conversion for voice prompts and voice responses to user input in speech-enabled applications such as, for example, X+V applications, SALT applications, Java Speech applications, or the like.
The computer (152) in the example of
Example formulations of semantic queries are had in C, C++, Java, Prolog, Lisp, and so on. The semantic web technology stack of the W3C is an example formulation that offers SPARQL to effect semantic queries in a syntax similar to SQL. Semantic queries are used against data structured in triple stores, graph databases, semantic wikis, natural language, and artificial intelligence systems. As mentioned, semantic queries work on structured data, and in the particular examples of the present case, the structured data is words, including literal phrases and names identified in URIs, described and defined in semantic triples connected in ways that conform to a formal logic, sometimes a predicate logic, often a description logic. In many embodiments of the present invention, semantic queries are asserted against data structured according to a description logic that implements decidability.
In the example apparatus of
The overall example computer system illustrated in
The computer system (152) also functions generally by administering through the communications session (140) across communications platforms a sequence of communications contacts (144), including the first contact and also subsequent communications contacts, between the tele-agent and a customer representative. The computer system administers a sequence of contacts by establishing each contact as an object-oriented module of automated computing machinery that structures computer memory (168) of the computer system and whose structure and contents (509) are also stored as semantic triples in an enterprise knowledge graph (154). That is, a contact (144) in this example is established initially at least as an instance of an object-oriented contact class. Administering a sequence of contacts includes recording in computer memory as member data elements of each contact a timestamp beginning the contact, duration of the contact, a session identifier for the contact, platform type, contact status, any communications content of the contact, and optionally as well other information regarding the contact.
Effecting CRM with semantic triples according to embodiments of the present invention, particularly in a thin-client architecture, may be implemented with one or more voice servers. A voice server is a computer, that is, automated computing machinery, that provides speech recognition and speech synthesis. For further explanation,
Stored in RAM (168) is a voice server application (188), a module of automation capable of operating a voice server in a system that is configured for use in CRM according to embodiments of the present invention. Voice server application (188) provides voice recognition services for speech-enabled client devices by accepting requests for speech recognition and returning speech recognition results, including text representing recognized speech, text for use as variable values in dialogs, and text as string representations of scripts for semantic interpretation. Voice server application (188) also includes the capability to provide text-to-speech (‘TTS’) conversion for voice prompts and voice responses to user input in speech-enabled client-side applications such as, for example, speech-enabled browsers, X+V applications, SALT applications, Java Speech applications, and the like.
Voice server application (188) may be implemented as a web server, implemented in Java, C++, Python, Perl, or any language that supports X+V, SALT, VoiceXML, or other speech-enabled languages, by providing responses to HTTP requests from X+V clients, SALT clients, Java Speech clients, or other speech-enabled client devices. Voice server application (188) may, for a further example, be implemented as a Java server that runs on a Java Virtual Machine (102) and supports a Java voice framework by providing responses to HTTP requests from Java client applications running on speech-enabled devices. And voice server applications that support embodiments of the present invention may be implemented in various other ways, and all such ways are well within the scope of the present invention.
The voice server (151) in this example includes a natural language processing speech recognition (“NLP-SR”) engine (153). An NLP-SR engine is sometimes referred to in this paper simply as a ‘speech engine.’ A speech engine is a functional module, typically a software module, although it may include specialized hardware also, that does the work of recognizing and generating human speech. In this example, the speech engine (153) is a natural language processing speech engine that includes a natural language processing (“NLP”) engine (155). The NLP engine accepts recognized speech from an automated speech recognition (‘ASR’) engine, processes the recognized speech into parts of speech, subjects, predicates, objects, and so on, and then makes such parts available to parsing engines for conversion into semantic triples for further inference and for inclusion in triple stores.
The speech engine (153) includes an automated speech recognition (‘ASR’) engine for speech recognition and a text-to-speech (‘TTS’) engine for generating speech. The speech engine also includes a grammar (104), a lexicon (106), and a language-specific acoustic model (108). The language-specific acoustic model (108) is a data structure, a table or database, for example, that associates speech feature vectors (‘SFVs’) with phonemes representing pronunciations of words in a human language. The lexicon (106) is an association of words in text form with phonemes representing pronunciations of each word; the lexicon effectively identifies words that are capable of recognition by an ASR engine. Also stored in RAM (168) is a Text-To-Speech (‘TTS’) Engine (194), a module of computer program instructions that accepts text as input and returns the same text in the form of digitally encoded speech, for use in providing speech as prompts for and responses to users of speech-enabled systems.
The grammar (104) communicates to the ASR engine (150) the words and sequences of words that currently may be recognized. For further explanation, distinguish the purpose of the grammar and the purpose of the lexicon. The lexicon associates with phonemes all the words that the ASR engine can recognize. The grammar communicates the words currently eligible for recognition. The two sets at any particular time typically are not the same.
Grammars may be expressed in a number of formats supported by ASR engines, including, for example, the Java Speech Grammar Format (‘JSGF’), the format of the W3C Speech Recognition Grammar Specification (‘SRGS’), the Augmented Backus-Naur Format (‘ABNF’) from the IETF's RFC2234, in the form of a stochastic grammar as described in the W3C's Stochastic Language Models (N-Gram) Specification, and in other grammar formats as may occur to those of skill in the art. Grammars typically operate as elements of dialogs, such as, for example, a VoiceXML <menu> or an X+V <form>. A grammar's definition may be expressed in-line in a dialog. Or the grammar may be implemented externally in a separate grammar document and referenced from with a dialog with a URI. Here is an example of a grammar expressed in JSFG:
In this example, the elements named <command>, <name>, and <when> are rules of the grammar. Rules are a combination of a rulename and an expansion of a rule that advises an ASR engine or a voice interpreter which words presently can be recognized. In this example, expansion includes conjunction and disjunction, and the vertical bars ‘|’ mean ‘or.’ An ASR engine or a voice interpreter processes the rules in sequence, first <command>, then <name>, then <when>. The <command> rule accepts for recognition ‘call’ or ‘phone’ or ‘telephone’ plus, that is, in conjunction with, whatever is returned from the <name> rule and the <when> rule. The <name> rule accepts ‘bob’ or ‘martha’ or ‘joe’ or ‘pete’ or ‘chris’ or ‘john’ or ‘artoush’, and the <when> rule accepts ‘today’ or ‘this afternoon’ or ‘tomorrow’ or ‘next week.’ The command grammar as a whole matches utterances like these:
The voice server application (188) in this example is configured to receive, from a speech-enabled client device located remotely across a network from the voice server, digitized speech for recognition from a user and pass the speech along to the ASR engine (150) for recognition. ASR engine (150) is a module of computer program instructions, also stored in RAM in this example. In carrying out automated speech recognition, the ASR engine receives speech for recognition in the form of at least one digitized word and uses frequency components of the digitized word to derive a speech feature vector or SFV. An SFV may be defined, for example, by the first twelve or thirteen Fourier or frequency domain components of a sample of digitized speech. The ASR engine can use the SFV to infer phonemes for the word from the language-specific acoustic model (108). The ASR engine then uses the phonemes to find the word in the lexicon (106).
Also stored in RAM is a VoiceXML interpreter (192), a module of computer program instructions that processes VoiceXML grammars. VoiceXML input to VoiceXML interpreter (192) may originate, for example, from VoiceXML clients running remotely on speech-enabled devices, from X+V clients running remotely on speech-enabled devices, from SALT clients running on speech-enabled devices, from Java client applications running remotely on multimedia devices, and so on. In this example, VoiceXML interpreter (192) interprets and executes VoiceXML segments representing voice dialog instructions received from remote speech-enabled devices and provided to VoiceXML interpreter (192) through voice server application (188).
A speech-enabled application like a client-side CRM application (195 on
As mentioned earlier, a Form Interpretation Algorithm (‘FIA’) drives the interaction between a user and a speech-enabled application. The FIA is generally responsible for selecting and playing one or more speech prompts, collecting a user input, either a response that fills in one or more input items, or a throwing of some event, and interpreting actions that pertained to the newly filled-in input items. The FIA also handles speech-enabled application initialization, grammar activation and deactivation, entering and leaving forms with matching utterances and many other tasks. The FIA also maintains an internal prompt counter that is increased with each attempt to provoke a response from a user. That is, with each failed attempt to prompt a matching speech response from a user an internal prompt counter is incremented.
Also stored in RAM (168) is an operating system (154). Operating systems useful in voice servers according to embodiments of the present invention include UNIX™, Linux™, Microsoft NT™, AIX™, IBM's i5/OS™, and others as will occur to those of skill in the art. Operating system (154), voice server application (188), VoiceXML interpreter (192), ASR engine (150), JVM (102), and TTS Engine (194) in the example of
The example voice server (151) of
Voice server (151) of
The example voice server of
The example voice server (151) of
For further explanation,
Also disposed in RAM are a triple server application program (297), data communications sessions (140), data communications contacts (144), structure and content (509) of sessions and contacts, inference rules (376), a natural language processing (“NLP”) engine (155), a parsing engine (380), an inference engine (298), and a semantic reasoner (378). Also disposed in RAM are a triple serializer (294), a triple converter (292), and one or more triple files (290).
The triple server application program (297) accepts, through network (100) from client-side devices such as laptop (126), instances of sessions (140) and contacts (14) from which it extracts structure and content (509) for parsing. The NLP engine optionally eases the parsing process by first processing at least some content into sentences and parts of speech before providing the content so processed to the parsing engine. The parsing engine parses the structure and content into parsed triples (752). The inference engine (298) takes as inputs the parsed triples (752) and the inference rules (376) and infers inferred triples (754). The triple server application (297) then returns copies of the parsed triples and inferred triples to the client side for further processing and stores the parsed triples and inferred triples in the enterprise knowledge graph (154). In this way, the triple server offloads from the client device (126) most of the triple-related processing and storage data processing workload.
The serializer (294) administers the transfer of triples between triple stores (750, 752, 754) in RAM and various forms of disk storage or other non-volatile storage. The serializer (294) accepts as inputs the contents of triple stores and serializes them for output as triple files (290), tables, relational database records, spreadsheets, text documents, or the like, for long-term storage in non-volatile memory, such as, for example, on a hard disk (170). The serializer (294) accepts triple files (290) as inputs and outputs parsed triples into triple stores in RAM. In many embodiments, when the serializer (294) accepts triple files (290) as inputs and outputs parsed triples into triple stores in RAM, the serializer stores the output triple stores into contiguous segments of memory. Contiguous storage can be effected in the C programming language by a call to the malloc( ) function. Contiguous storage can be effected by the Python buffer protocol. Contiguous storage can be effected in other ways as will occur to those of skill in the art, and all such ways are within the scope of the present invention. In many embodiments, triple stores (750, 752, and 754) would be stored in segments of contiguous memory.
Contiguous memory is explained in more detail with reference to
The use of contiguous memory deals with a particular challenge in memory management for CRM. In typical embodiments, cache access takes 10 nanoseconds. RAM access takes 100 nanoseconds. Disk access takes 10,000,000 nanoseconds. Those numbers are not intuitive. People do not experience time in nanoseconds. Therefore let us look at this challenge in more familiar terms. If cache access is viewed as taking one minute, then RAM access takes 10 minutes, and disk access for the same data takes two years. Triples scattered across virtual memory addresses risk being stored in multiple page frames. Triples stored near one another in a contiguous memory segment are much more likely to be stored in a small number of page frames.
Suppose a pertinent set of triples, a subgraph of triples, for example, parsed, inferred, or previously asserts and stored, is composed of 10 kilobytes of storage. Some computer systems today support memory page sizes of a megabyte or more. Such a set of triples (704) can be stored in a single memory page (710), and, once that page is in RAM (168), operation of the triple set for CRM can proceed with no risk of page faults at all. Even if contiguous storage for such a set falls across a page boundary, the entire set of triples can still be loaded with only two page faults, and, after it is loaded into RAM, it can be operated with zero page faults going forward. Cache misses would still be required to load the contents into cache, but, except for the first one or two misses, none of the others would risk a page fault. The inventors estimate that after a short period of operation, the cache miss rate would be less than one percent for operation of such a set of triples in the context of CRM. That is, when a set of triples is disposed in contiguous memory in support of CRM according to embodiments of the present invention, memory access times generally will approximate cache access times, just a few nanoseconds, for more than 99% of memory access.
For further explanation,
Disposed in RAM are an AIA (196) that operates through an API (198) a CRM application program (195) that exposes and operates a GUI dashboard (110), communications sessions (140) and communications contacts (144) established and administered by the CRM application and the AIA running on the computer (152) and, sessions and contacts structure and content (509) to be parsed into triples (752), inference rules (376), a natural language processing (“NLP”) engine (155), a parsing engine (380), an inference engine (298), and a semantic reasoner (378). Also disposed in RAM are a triple serializer (294), a triple converter (292), and one or more triple files (290). The NLP engine optionally eases the parsing process by first processing the structure and content (509) into sentences and parts of speech before providing the words so processed to the parsing engine. The parsing engine parses the structure and content into parsed triples (752). The inference engine (298) takes as inputs the parsed triples (752) and the inference rules (376) and infers inferred triples (754).
The serializer (294) administers the transfer of triples between triple stores (752, 754) in RAM and disk storage or other non-volatile storage. The serializer (294) accepts as inputs the contents of triple stores and serializes them for output as triple files (290), tables, relational database records, spreadsheets, text documents, or the like, for long-term storage in non-volatile memory, such as, for example, on a hard disk (170). The serializer (294) accepts triple files (290) as inputs and outputs parsed triples into triple stores in RAM. In many embodiments, when the serializer (294) accepts triple files (290) as inputs and outputs parsed triples into triple stores in RAM, the serializer stores the output triple stores into contiguous segments of memory. Contiguous storage can be effected in the C programming language by calls to the malloc( ) function. Contiguous storage can be effected by the Python buffer protocol. Contiguous storage can be effected in other ways as will occur to those of skill in the art, and all such ways are within the scope of the present invention. In many embodiments, triple stores (752, 754) would be stored in segments of contiguous memory.
The speech-engine (153) is a full-service NLP-SR engine that includes natural language processing (155), speech recognition (150), a grammar (104), a lexicon (106), a model (108), and text-to-speech processing (194), all as described in more detail above with regard to
The overall example computer system illustrated in
The AIA and CRM application also administer through the communications session (140) across communications platforms a sequence of communications contacts (144), including the first contact and also subsequent communications contacts, between the tele-agent and a customer representative. The computer system administers a sequence of contacts by establishing each contact as an object-oriented module of automated computing machinery that structures computer memory (168) of the computer system and whose structure and contents (509) are also stored as semantic triples in an enterprise knowledge graph (154). That is, a contact (144) in this example is established initially at least as an instance of an object-oriented contact class. Administering a sequence of contacts includes recording in computer memory as member data elements of each contact a timestamp beginning the contact, duration of the contact, a session identifier for the contact, platform type, contact status, any communications content of the contact, and optionally as well other information regarding the contact.
For further explanation,
The method of
The AIA (106) can determine (215) whether a new contact is a first contact of a session by searching session memory for a session for a customer and subject of the new contact. Finding no match (217), the AIA calls the establishing function (381) for a new session (140), records the new session ID (393) in both the new session (140) and the first contact (142), and hands off the new session and the new contact to the administration function (382). Finding a match (219), the AIA records the found session ID in the new contact and hands off the found session ID and the new contact to the administration function (382). The session ID (393) functions as a foreign key linking the session to all contacts associated with the session. The session thus functions as a wrapper or container for all contacts linked by the session ID. In this example, identification codes for a tele-agent and a customer representative (205, 207), recorded in the contact (142), are not recorded in the session. This example session (140) therefore can serve as a wrapper, for contacts related to a customer and having a particular subject matter, for multiple contacts among multiple agents and multiple customer representatives.
The session's member data elements include a timestamp (386) for when the session (140) in created. A session can also include a time limit (221), represented here as a time-to-live or ‘TTL,’ after which the session is either terminated or a user is notified of a timeout and prompted whether to terminate the session. A session can also include a status code (219) to indicate whether the session is active or terminated. Terminated sessions retained in storage optionally can be configured to be reactivated.
The session member data elements in this example also include a content element (389). A typical use for the content element (389) is to elaborate on the subject matter (385) of contacts of the session. The content (213) of communications in the contacts in this example are stored in association with the contacts themselves (142, 144) rather than in a session object. In many embodiments, the structure and content of the session (140), typically including all session data elements and optionally others as well, after parsing (304) into semantic triples, are stored as semantic triples in an enterprise knowledge graph (154).
The method of
The method of
In the method of
In the method of
Communications contacts can be implemented through speech alone without GUI operations, including determining platform types. A tele-agent can issue oral instructions to the AIA, “Computer, email Bob and ask to schedule a call.” “Computer, text Bob and ask him to reschedule our call.” “Computer, ring Bob for me and if he doesn't answer, leave a message asking for a call back.” Each of these examples expressly identifies a platform type, respectively, email, text message, and telephone.
In the method of
In the method of
In the method of
In the method of
In the example of
The status code (211) can be used to indicate success or failure of a contact. The AIA with voice enablement can indicate in response to a status query from a tele-agent, “You tried him last Wednesday and left a message, but we have had no further contact.” The contact structure in this example includes identification codes for the agent (205) who initiated the contact and the customer representative (207) who was sought to be contacted. Because the agent ID and the customer representative ID are recorded in the contact rather than the session, this example session structure (140) can serve as a wrapper, for contacts related to a customer and having a particular subject matter, for multiple contacts among multiple agents and multiple customer representatives.
To the extent that the content (213) of a communications contact is speech, the speech is optionally either recorded raw with a URL locating it in a stored triple—or recognized into text and then stored as an element of a contact of a session. For a transcript, to automate speaker recognition, an example computer system, CRM applications, and AIA according to embodiments implements a speech recognition function running in a loop with current voice print extraction and comparison with previously stored voice prints of known speakers. For a new speaker with no voice print on record, a computer according to embodiments can take speaker identification from context, e.g., a tele-agent telephoned a particular customer representative selected from an address list. Alternatively for a new speaker with no voice print on record, an AIA according to embodiments can prompt for identification, “Excuse me, I did not catch your name.” Other ways to associate a new speaker with a voice print may occur to implementers, and all such ways are well within the scope of the present invention.
In the method of
In the method of
The method of
The parsing process (304) functions by forming words from the structure and content (509) into semantic triples (752). The parsing process (304) can function by forming into semantic triples (752) words designated through GUI widgets (368) as element of triples, subject, predicate, object. The parsing process (304) can function by forming into semantic triples (752) words designated through voice commands in a VoiceXML dialog (522) as elements of triples, subject, predicate, object. The parsing process (304) can function by forming into semantic triples (752) words designated by a natural language processing engine (155) as parts of speech, subject, predicate, object. The parsing process hands off the parsed triples (752) to the inference engine (298) and the storage function (322) for further processing. In many embodiments, the parsing process hands off the parsed triples (752) to the inference engine (298) and the storage function (322) by disposing the parsed triples in segments of contiguous memory and providing to the inference engine (298) and the storage function (322) memory addresses for the segments.
The method of
The method of
For further explanation,
The root of the graph (154) is a class object named Enterprise Knowledge (199). The nodes Customer Data (200) and Cust Rep Data (226) are subclasses of Enterprise Knowledge, whereas the session node (140) is an example instance of a particular session parsed and stored. The contact objects (142, 144) of the session are each linked to the session (140) through a session ID object (393). The example of
Queries against a triple store in a graph like graph (154), such as, for example, queries used by semantic reasoners to extract prior content for use in present contacts, can include a clause specifying a subgraph:
For further explanation,
For further explanation,
In addition to those similarities to the method of
In a second alternative way of providing digitized words to the parsing process, the method of
In a third alternative way of providing digitized words to the parsing process, the method of
For further explanation,
In addition to the above-mentioned similarities to the method of
In the method of
In the method of
In the method of
In the method of
The method of
The method of
Query (341) does not ask merely whether data is in a store; query (341) asks for the data itself to be returned. Thus, again with reference to the graph regarding Bob and the Mona Lisa, this query, a further example for explanation, which requests predicates and objects from all triples in which Bob is the subject:
This query:
And this query:
It will be understood from the foregoing description that modifications and changes may be made in various embodiments of the present invention without departing from its true spirit. The descriptions in this specification are for purposes of illustration only and are not to be construed in a limiting sense. The scope of the present invention is limited only by the language of the following claims.
This application is a continuation application of U.S. patent application Ser. No. 16/183,736, filed on Nov. 8, 2018, the priority of which is claimed.
| Number | Name | Date | Kind |
|---|---|---|---|
| 6785380 | Ribera | Aug 2004 | B2 |
| 6829603 | Chai et al. | Dec 2004 | B1 |
| 7275083 | Seibel et al. | Sep 2007 | B1 |
| 7486785 | Flores | Feb 2009 | B2 |
| 8108237 | Bourne et al. | Jan 2012 | B2 |
| 8332279 | Woolston | Dec 2012 | B2 |
| 8411843 | Cyriac | Apr 2013 | B1 |
| 9049295 | Cooper | Jun 2015 | B1 |
| 9165556 | Sugar | Oct 2015 | B1 |
| 9848082 | Lilland | Dec 2017 | B1 |
| 9860391 | Wu | Jan 2018 | B1 |
| 9948783 | Farrell | Apr 2018 | B1 |
| 10026092 | Heater et al. | Jul 2018 | B2 |
| 10101976 | Cavalcante | Oct 2018 | B2 |
| 10303466 | Karman | May 2019 | B1 |
| 10482384 | Stoilos | Nov 2019 | B1 |
| 10623572 | Copeland | Apr 2020 | B1 |
| 20030126136 | Omoigui | Jul 2003 | A1 |
| 20040143473 | Tivey et al. | Jul 2004 | A1 |
| 20050005266 | Datig | Jan 2005 | A1 |
| 20050044357 | Fano | Feb 2005 | A1 |
| 20050105712 | Williams et al. | May 2005 | A1 |
| 20060095273 | Montvay et al. | May 2006 | A1 |
| 20060098625 | King | May 2006 | A1 |
| 20060239439 | Blackwood | Oct 2006 | A1 |
| 20070019618 | Shaffer | Jan 2007 | A1 |
| 20070064913 | Shaffer | Mar 2007 | A1 |
| 20070094183 | Paek et al. | Apr 2007 | A1 |
| 20070219933 | Datig | Sep 2007 | A1 |
| 20070233561 | Golec | Oct 2007 | A1 |
| 20080162498 | Omoigui | Jul 2008 | A1 |
| 20080275744 | MacIntyre et al. | Nov 2008 | A1 |
| 20090070322 | Salvetti | Mar 2009 | A1 |
| 20090245500 | Wampler | Oct 2009 | A1 |
| 20090271192 | Marquette | Oct 2009 | A1 |
| 20100036788 | Wu | Feb 2010 | A1 |
| 20100063799 | Jamieson | Mar 2010 | A1 |
| 20100114563 | Choi | May 2010 | A1 |
| 20110077999 | Becker et al. | Mar 2011 | A1 |
| 20110082829 | Kolovski | Apr 2011 | A1 |
| 20110113094 | Chunilal | May 2011 | A1 |
| 20110206198 | Freedman | Aug 2011 | A1 |
| 20110264451 | Hoepfinger | Oct 2011 | A1 |
| 20120059776 | Estes | Mar 2012 | A1 |
| 20120078636 | Ferrucci | Mar 2012 | A1 |
| 20120233558 | Naim | Sep 2012 | A1 |
| 20120275642 | Aller | Sep 2012 | A1 |
| 20130091090 | Spivack et al. | Apr 2013 | A1 |
| 20130006916 | McBride | Jun 2013 | A1 |
| 20130163731 | Yan | Jun 2013 | A1 |
| 20130204663 | Kahlow | Aug 2013 | A1 |
| 20140022328 | Gechter et al. | Jan 2014 | A1 |
| 20140081934 | Mizell | Mar 2014 | A1 |
| 20140122535 | Gerard | May 2014 | A1 |
| 20140164502 | Khodorenko | Jun 2014 | A1 |
| 20140189680 | Kripalani | Jul 2014 | A1 |
| 20140201234 | Lee et al. | Jul 2014 | A1 |
| 20140270108 | Riahi et al. | Sep 2014 | A1 |
| 20140278343 | Tran | Sep 2014 | A1 |
| 20140314225 | Riahi | Oct 2014 | A1 |
| 20140372630 | Bostick | Dec 2014 | A1 |
| 20140379755 | Kuriakose | Dec 2014 | A1 |
| 20150012350 | Li et al. | Jan 2015 | A1 |
| 20150066479 | Pasupalak | Mar 2015 | A1 |
| 20150189085 | Riahi et al. | Jul 2015 | A1 |
| 20150201077 | Konig et al. | Jul 2015 | A1 |
| 20150242410 | Pattabhiraman et al. | Aug 2015 | A1 |
| 20150261743 | Sengupta | Sep 2015 | A1 |
| 20150294405 | Hanson | Oct 2015 | A1 |
| 20150348551 | Gruber | Dec 2015 | A1 |
| 20150379603 | Gupta | Dec 2015 | A1 |
| 20160019882 | Matula | Jan 2016 | A1 |
| 20160021181 | Ianakiev et al. | Jan 2016 | A1 |
| 20160034457 | Bradley | Feb 2016 | A1 |
| 20160036981 | Hollenberg | Feb 2016 | A1 |
| 20160036982 | Ristock | Feb 2016 | A1 |
| 20160036983 | Korolev | Feb 2016 | A1 |
| 20160117593 | London | Apr 2016 | A1 |
| 20160162474 | Agarwal | Jun 2016 | A1 |
| 20160162913 | Linden et al. | Jun 2016 | A1 |
| 20160171099 | Lorge et al. | Jun 2016 | A1 |
| 20160188686 | Hopkins | Jun 2016 | A1 |
| 20160189028 | Hu et al. | Jun 2016 | A1 |
| 20160217479 | Kashyap et al. | Jul 2016 | A1 |
| 20160239851 | Tanner | Aug 2016 | A1 |
| 20160321748 | Mahatm | Nov 2016 | A1 |
| 20170017694 | Roytman et al. | Jan 2017 | A1 |
| 20170091390 | Joul | Mar 2017 | A1 |
| 20170124193 | Li | May 2017 | A1 |
| 20170147635 | McAteer et al. | May 2017 | A1 |
| 20170154108 | Li et al. | Jun 2017 | A1 |
| 20170177715 | Chang | Jun 2017 | A1 |
| 20170200220 | Nicholson | Jul 2017 | A1 |
| 20170195488 | Pendyaia | Aug 2017 | A1 |
| 20170262429 | Harper | Sep 2017 | A1 |
| 20170262530 | Okura | Sep 2017 | A1 |
| 20170293610 | Tran | Oct 2017 | A1 |
| 20180082183 | Hertz et al. | Mar 2018 | A1 |
| 20180115644 | Al-Khaja | Apr 2018 | A1 |
| 20180144250 | Kwon | May 2018 | A1 |
| 20180150459 | Farid | May 2018 | A1 |
| 20180288098 | Wang | Oct 2018 | A1 |
| 20180300310 | Shinn | Oct 2018 | A1 |
| 20180315000 | Kulkarni | Nov 2018 | A1 |
| 20180315001 | Garner | Nov 2018 | A1 |
| 20180338040 | Carly | Nov 2018 | A1 |
| 20180365772 | Thompson | Dec 2018 | A1 |
| 20180376002 | Abraham | Dec 2018 | A1 |
| 20190042988 | Brown | Feb 2019 | A1 |
| 20190080370 | Copeland | Mar 2019 | A1 |
| 20190188617 | Copeland | Jun 2019 | A1 |
| 20190206400 | Cui | Jul 2019 | A1 |
| 20190220794 | Kulkarni | Jul 2019 | A1 |
| 20190340294 | Spangler | Nov 2019 | A1 |
| 20200042642 | Bakis | Feb 2020 | A1 |
| 20200097814 | Devesa | Mar 2020 | A1 |
| 20200110835 | Zhao | Apr 2020 | A1 |
| Number | Date | Country |
|---|---|---|
| 1020180058877 | Jul 2018 | KR |
| 2016139666 | Sep 2016 | WO |
| 20160139666 | Sep 2016 | WO |
| Entry |
|---|
| International Search Report and Written Opinion dated Nov. 28, 2018 for PCT/US2018/049813. |
| International Search Report and Written Opinion dated Dec. 30, 2019 for related PCT/US2019/062629. |
| International Search Report and Written Opinion dated Jan. 9, 2020 for related PCT/US2019/055488. |
| International Search Report and Written Opinion dated Jan. 14, 2020 for related PCT/US2019/060174. |
| International Search Report and Written Opinion dated Jan. 17, 2020 for related PCT/US2019/058997. |
| International Search Report and Written Opinion dated Mar. 9, 2020 for related PCT/US2019/059949. |
| International Search Report and Written Opinion dated Apr. 1, 2020 for related PCT/US2019/055483. |
| Liew. “Strategic integration of knowledge management and customer relationship 1-20 management.” In: Journal of Knowledge Management. Jul. 18, 2008 (Jul. 18, 2008) Retrieved on Dec. 25, 2019 (Dec. 25, 2019) from <http://student.bms.lk/GDM/49/Slides/MarManaSampleAssi/MMAsuportingJouArti/13673270810884309.pdf> entire document. |
| Tung. “Google's human-sounding AI to answer calls at contact centers.” In: ZDNet. Jul. 25, 2018 (Jul. 25, 2018) Retrieved on Dec. 25, 2019 (Dec. 12, 2019) from <https://www.zdnet.com/article/googles-human-sounding-ai-to-answer-calls-at-contact-centers/> entire document. |
| Jan. 25, 2020 Office Action for related U.S. Appl. No. 15/844,512. |
| Jan. 10, 2020 Office Action for related U.S. Appl. No. 15/700,210. |
| International Search Report and Written Opinion dated Feb. 22, 2019 for PCT/US2018/065584. |
| May 4, 2020 Office Action for related U.S. Appl. No. 16/154,718. |
| Final Office Action dated Jul. 7, 2020 for corresponding U.S. Appl. No. 15/700,210. |
| Final Office Action dated Jul. 27, 2020 for corresponding U.S. Appl. No. 15/844,512. |
| Final Office Action dated Jul. 30, 2020 for corresponding U.S. Appl. No. 16/154,718. |
| Non-Final Office Action dated Sep. 29, 2020 for corresponding U.S. Appl. No. 16/157,075. |
| Number | Date | Country | |
|---|---|---|---|
| 20200329143 A1 | Oct 2020 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16183736 | Nov 2018 | US |
| Child | 16911717 | US |
