Patent Grant
10176827
Creating and maintaining knowledge bases that are used for various projects can be a time consuming task. In the past, a great deal of human effort has been required to create and maintain knowledge bases in terms of developing, debugging and maintaining such knowledge bases. Errors that present themselves in a knowledge base can often cascade or grow when such knowledge bases are deployed and used by various applications. Thus, an important consideration in knowledge base development concerns accurately developing and deploying the knowledge base.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
Various embodiments provide a tool, referred to herein as “Active Lab” that can be used to develop, debug, and maintain knowledge bases. These knowledge bases (KBs) can then engage various applications, technology, and communications protocols for the purpose of task automation, real time alerting, system integration, knowledge acquisition, and various forms of peer influence.
In at least some embodiments, a KB is used as a virtual assistant that any real person can interact with using their own natural language. The KB can then respond and react however the user wants: answering questions, activating applications, or responding to actions on a web page.
The same numbers are used throughout the drawings to reference like features.
Overview
Various embodiments provide a tool, referred to herein as “Active Lab” that can be used to develop, debug, and maintain knowledge bases. These knowledge bases (KBs) can then engage various applications, technology, and communications protocols for the purpose of task automation, real time alerting, system integration, knowledge acquisition, and various forms of peer influence.
In at least some embodiments, a KB is used as a virtual assistant that any real person can interact with using their own natural language. The KB can then respond and react however the user wants: answering questions, activating applications, or responding to actions on a web page.
In the discussion that follows, various terminology will be used. The following glossary is provided to assist the reader in the discussion that follows:
Agent: Agents are entities programmed to respond to inputs in a pre-determined fashion. They are often used to mimic a human consciousness.
AppCalls: AppCalls are Unit responses that perform application operations instead of simply generating a text response.
Building Blocks: Building Blocks are sets of vocabulary or Concepts, often used to group together terms and Concepts that can be associated with one another, but are not always synonymous.
Cascade: Cascades are graphical representations of the load order and match type of vocabulary, Concepts, and Units within the database.
Concepts: Concepts serve as the bricks of Units. As such, they are composed of Vocab Terms, Helper Terms, and Wild Cards arranged in specific input patterns that trigger the Concept's activation.
Context IQ Engine, the: The Context IQ Engine is essentially the algorithm that tracks and processes user inputs.
Data Mining: Data-mining is the targeted recursion of a set of FPML. After the results for a particular input have been achieved, the remainder of the input is run through a specific cascade, which is a self-sufficient FPML set in its own right.
Dependencies: Dependencies for a Concept or Vocab Term are all those objects which depend upon the aforementioned term or Concept for operation.
Dockable Window: Dockable windows are objects made visible by means of the View menu. These windows can be dragged and dropped at any location on the screen, and may be docked either on the right-hand side of the screen, or the top. When docked, multiple windows can be accessed via tabs.
FPML: A programming language, FPML, short for Functional Presence Mark-up Language is used to govern the behavior of Active Lab projects.
Helper Terms: Helper Terms are words or phrases that are commonly used, but have only one meaning. As such, they are not of sufficient importance to devote separate Vocab Terms and Concepts to them. Helper Terms may be used in both Concepts and Units.
Input Files: Input Files are *.txt files that contain words and/or phrases. They are used in Record Lists and Concepts.
Input Patterns: Input patterns determine the triggering of Concepts and Units. In Concepts, input patterns are composed of Vocab Terms, Wild Cards, and Helper Terms. In Units, input patterns are in turn composed of Concepts and Helper Terms. In both Concepts and Units, input patterns may be configured to trigger only if the input is in a specific order, or by proximity. All Concepts and Units must have at least one input pattern, and many have more than one.
Projects: Projects are the knowledge base storage configuration within the lab.
Situations: Situations are groups of related Units. For instance, a group of Units that all deal with the personality of a project may be grouped together under a personality situation.
Synonyms: Synonyms are words that share the same definition with one another and may be used interchangeably. Linguist's note: This is not entirely true. Many words may have the same definition, but carry different connotations. While they could technically be substituted, doing so would alter the reader's understanding of the sentence.
Test Questions: Test questions are sample inputs that are intended to trigger Units and Concepts. They are generally archetypal in nature, and are used as a guide for input patterns.
Units: Units are composed of Concepts and Helper Terms arranged in input patterns. When those input patterns are triggered, the Unit activates a response, in the form of text or an AppCall.
Units, Nested: Like Units, except that their response is contingent upon another Unit. A nested Unit may only be triggered immediately after its parent Unit has been triggered. This allows for continuity in conversation with the project.
User: The user is the individual human being who operates the software.
Vocab: Short for “Vocabulary,” Vocab Terms form the basis of Concepts, and consist themselves of synonyms.
Wild Cards: Wild Cards function as placeholders within Concepts for any random word or words.
In the discussion that follows, a section entitled
Principles of Active Lab
Knowledge bases (KB) cover the entire scope of a project, including all the information that ActiveAgent uses, along with all of its capabilities. The files themselves are written in a programming language known as FPML (Functional Presence Markup Language), a language similar to XML. This includes master FPML files, optional FPML files, and lex files. For additional information on FPML, the reader is referred to the following U.S. patent applications, the disclosures of which are incorporated by reference herein: Ser. Nos. 10/839,425 and 11/169,142.
Active Lab is a visual tool (GUI) that allows the user to easily create and maintain a knowledge base without the necessity of knowing the actual FPML code.
The discussion that follows provides some of the principles behind what ActiveAgent can do, and how it works. Instructions for physically putting these Concepts into the lab will come later, along with more advanced uses of each of these principles.
Principles of a Knowledge Base
To begin understanding how to create a knowledge base, we will start with the idea of an input and a response by ActiveAgent. Consider the following input and response:
Units
First, let's examine the response. The response is a known answer in the knowledge base. When a response is paired together with patterns that will trigger it, it is called a Unit. A group of Units can be categorized into Situation Groups which are similar to folders in Windows®, and are used mainly for organizational purposes due to the large number of responses a KB may have.
Next, we must determine what questions should solicit that response. It turns out there are many different ways to phrase questions, all of which should be taken into account:
These questions are then listed with the Unit for the purposes of testing to make sure that when they are asked, this particular response is given.
Concepts
A Concept is a breakdown of the critical ideas in a test question. In the above list of possible questions there seems to be three main themes: Cost, Savings Account, and Minimum Balance. Notice that not all of these Concepts exist in every possible question. Some questions have the idea of Cost, some Minimum Balance, and all have the Concept of Savings Account. To make a successful Unit, these Concepts are listed within the Unit and paired in such a way that this combination of Concepts is unique within the KB. This is called a Pattern. A Unit may have multiple patterns featuring different combinations of Concepts. For example:
Note that a pattern for a Unit includes at least one Concept, however it is not recommended to have more than four Concepts in a pattern. A Concept is made up mostly of Vocab, Helper Terms, and Building Blocks, which are described individually below.
Vocab
A Vocab is a grouping of unambiguous synonyms and misspellings. The name of a particular grouping of synonyms is known as a Vocab Term. Vocab Terms are usually end with the suffix “vocab.”
In the example of PriceVocab, the word cost is included because for this KB the user wants the words price and cost to be synonymous.
Helper Terms
A Helper Term is a word that has no unambiguous synonyms, however functions almost the same way as a Vocab, and mostly consists of conjunctions. Here are some examples of common Helper Terms:
A Building Block is a list of either Vocab/Helper Terms or a list of Concepts that may be useful categorized together. As an example, consider the following:
Anatomy (Vocab Building Block):
Now that these Vocab Terms are bundled together, they can be used in a Concept pattern. Here is an example use of a Vocab Building Block:
In this example, because the Anatomy Building Block contained five Vocab Terms, it turned what would have been ten patterns into two. Also notice that Vocab Terms, which include all the synonyms and misspellings, are also included in this example. As another example, consider the following:
Types of Accounts (Concept Building Block)
This Building Block would then be used to reduce the number of patterns necessary.
Creating Concepts
A Concept is made of patterns of Vocab, Helper Terms, and Building Blocks listed within the Concept. Here are the patterns that would make up the Savings Account Concept:
Notice that the word “for” is a Helper Term and does not end with the suffix “vocab.” In Concepts, two things that are important are Order and Proximity, both of which are optional when creating any given pattern.
If Order is selected, a pattern will only trigger if it is received in that order. For example, a person would have to type in “savings account” to trigger the above example. If a person typed in “account savings” this pattern would not trigger because it is not in the correct order.
If Proximity is selected, a pattern will only trigger if the inputs are received next to each other. For example, a person would have to type in “savings account” to trigger the above example. If a person typed in “savings in my account” the pattern would not trigger because “savings” and “account” were not typed in next to each other.
Note that for most input patterns, it is important to have both Order and Proximity selected. In the above example, “Savings Account” has a very different meaning than “Account Savings” and “Savings in my Account.” Concepts also have their own associated test questions similar to Units for the purposes of testing.
Examples of Savings Account test questions can include
The relationship between these principles can be stated simply:
The biggest benefit of this system is its flexibility. Once a Vocab has been created, it can be used in any number of Concepts, and once a Concept is created, it can be used in any number of Units. Conversely, any Unit or Concept can have any number of patterns, and the same Vocab or Concept can be used multiple times within each Unit or Concept.
In
In
Context IQ Engine
The Context IQ Engine (CIE) is a term used to describe the part of ActiveAgent that actually does the processing of the FPML code written using Active Lab. The CIE has different functions relative to FPML code, and part of Active Lab is determining which functions to use, and in what order.
Cascades
Completing a single process through use of the CIE is known as a cascade. Any numbers of cascades can, and usually do, run every time the FPML is activated.
Here is an example of the simplest set of cascades necessary for a virtual agent:
In this example, each cascade is necessary for a single person's input to output a response, and can do very different things. The Vocab cascade only does substitution; it inputs words and outputs Vocab. The Concept cascade inputs Vocab and Helper Terms, and then outputs all Concepts that have patterns that include those terms. This is known as Match All because it outputs all matches, not just the most accurate one. The Unit cascade inputs Concepts, but only outputs one response. This is known as a Best Match and is a mathematical calculation done by the Context IQ Engine, with the Unit with the highest scoring pattern giving the response.
Static Files
Static Files are individual FPML files that are written by hand without the help of Active Lab's user interface. The most common files include setups and teardowns, as well as a lex file that handles special cases that Active Lab is currently not set up to handle. These are most commonly established during the initial creation of the project, and usually don't need to be modified during general use.
Common Terms
Within this document, the following terms are important to the proper understanding of Active Lab:
Cascades—A cascade is a layer of FPML processing and is essentially a “mini” fpml runtime contained within a larger runtime. It contains most, if not all, the processing features available to a global runtime, including preprocessing, a specialized lexer, setups and teardowns, chunkers, unit nodes, functions, etc.
Static Files—Static files are .fpml files that remain unchanged, such as lex files.
Input Files—Input files are text files that list various user inputs that can be easily categorized, such as a list of names.
Group—A group is a term that refers to separating similar items into smaller lists for organizational purposes. For example: a Concept Group is a group of similar Concepts.
The following describes some aspects of an Active Lab Family and Concepts:
Basics of Active Lab
The following describes the basic components of Active Lab in accordance with one or more embodiments. It also provides directions for getting started with Active Lab.
Components of Active Lab
The main application window has several components that allow access to Active Lab features. The following table describes the main features of the illustrated main application window.
Menu Bar
The menu bar contains four dropdown menus: File, View, Admin, and Help, which include the sub-menus illustrated in
All activity begins with the File menu.
Dockable Windows
The dockable windows are the set of windows normally positioned on the right hand side of Active Lab, with each window differentiated by its own tab at the bottom. All dockable windows are opened and closed via the View menu. They may be docked on either the right side of the screen or the top, or allowed to float freely. When docked, multiple windows can be accessed through their corresponding tabs. The following table describes the components available through the dockable windows feature.
Searches
Searches enable users to find Concepts and Units by words occurring within their input patterns and/or responses (as appropriate). This sub-section discusses the components of Search and Unit Search. With respect to
Search contains the following components:
With respect to performing a search, consider the following. Searches in dockable windows only find text that appears in the display names of individual items. Performing a search in a normal dockable window uses the following procedure:
Searching for text that occurs within the makeup of a Concept or Unit requires the use of Search and Unit Search, respectively.
To run a search for a Concept, use the following procedure:
To run a search for a Unit in the Unit Search dockable window, use the following procedure:
To run a search for a Unit in the Unit Search window, use the following procedure:
Active Lab's main window contains five main tabs, in addition to the dockable windows described above. A brief description of each can be found in the following table.
Status Bar
The status bar, located at the bottom of the main window, shows the operations currently being undertaken by the program, and the current progress. An example of this is shown in
The full utility of Active Lab comes when handling knowledge bases (KBs). There are many things to take into consideration when creating a KB, including defining the scope, domain, and terminology surrounding the KB. Conversation directions should also be laid out before creating the KB in Active Lab. Other preparations vary depending on the specific KB. The section below entitled “Project Management” provides additional information in this regard. For information on modifying KBs directly, reference should be made to the sections below entitled “Vocabulary”, “Concepts” and “Units”.
Project Management
Each project represents an FPML knowledge base (KB) storage configuration, which tells lab what databases to use for the KB components, which comprise Units, Concepts, and Vocabulary. Active Lab features many means by which the user can create, edit, view, and manipulate aspects of a project. This section details the configuration of KBs through the file menu options, the file menu, searches, administrating a project, and cascades.
Projects
Projects serve as a collection of configuration information for a KB, enabling the user to manipulate the KB through Active Lab. Each project is connected to a database, where all of its constituent components are stored. Multiple projects can work from the same database.
Create New Project is accessed through the File menu by selecting New Project. This window allows the user to specify the foundational specifications of the project. Create New Project contains two tabs: Project Info, and Chat Review. As an example, consider
Project Info allows you to set the basic parameters of the project, including the name of the project, its local file directory, the project's database connection settings, user settings, and web server and subversion servers. Project Info contains the following components:
The Chat Review tab of Create New Project stores the location of Agent Chat Log databases. Chat Review contains the following components:
Project Files
The database that the Knowledge Base is created from can have multiple projects associated with it. A project file is a file that can point to either part of or all of the database. When multiple users use identical project files, all those users will be working on the same part of the database (also known as the same project) at the same time.
The Project Files window, accessed from the dropdown→Admin→Project Files, can be used to access all the different projects that have been saved in the database. When a user has already has a project open, this window will be filled out when opened. Clicking <Connect> will display all project files associated with the database. These project files can then be downloaded by highlighting them and clicking <Download>.
Currently, most projects are stored on the same server with identical passwords. This means that if a user has a project open, they can view the project files of a different database simply by changing Database Name field and clicking <Connect>. With respect to the Project Files window, consider the following in connection with
Application Data
Application data can be viewed by selecting Application Data from the Admin menu. This selection opens up Application Editor, which displays the project name, the version of the project, and the lex file associated with it. Once a project has been opened, you can view and change parts of the configuration through Application Editor, as well as copy projects to and from the database. AS an example, consider the following in connection with
Manage Users
Selecting Manage Users from the Admin menu opens the Manage Users window, which allows a user to view and edit user information, specifically display name, login name, password, and user type. With respect to Manage Users, consider the following in connection with
Deployment
Deployment is the process of moving all the FPML files associated with the project from a user's local copy to another location, usually to a test or production server. To configure the deployment, select Deployment→Configure Deployment from the Admin menu. This will open the Deployment Configuration window. The user can select a means by which the test environment manager can be notified of changes to the FPML. The files may be transferred either by direct e-mail to the test environment manager, or by copying the files to a shared directory. With respect to a Deploy FPML Window, consider the following in connection with
A Deployment Configuration Window allows the user to set up the deployment configuration. This includes the destination folder, email settings, and whether to include copies or Text-to-Speech. With respect to this window, consider the following in connection with
FPML Reports
Active Lab has functionality that creates documents that may be useful to the user. In one or more embodiments, there are two documents that Active Lab can create: FPML Statistics which is an Excel® spreadsheet displaying the percentage statistics of Unit test questions and Print Units which is an Excel® spreadsheet displaying all the Units along with their respective test questions.
To create the FPML Statistics spreadsheet, follow these procedures in connection with the user interface shown in
FPML Statistics contains the following components:
To create the FPML Units spreadsheet, follow these procedures in connection with the user interface shown in
Print Units contains the following components:
Text To Speech Management
Text-To-Speech (TTS) refers to the ability to translate on-screen text into audio speech. TTS files can be generated automatically, or can be pre-recorded voice files. The Text-To-Speech File Management window allows the user to easily manage importing large sets of TTS files.
Text-To-Speech File Management is accessed by selecting Text-To-Speech from Admin as indicated in the user interface of
Cascades
A cascade represents a single function of the Context IQ Engine (CIE). Each cascade takes an input, processes that input through the CIE, and then gives an output.
The order that the cascades are listed is very important because many cascades only take the output of previous cascades. When an input is received, it is processed through all cascades in the order listed on the View/Edit Load Order window. There are many different types of cascades, some of which are unique to a project.
When it comes to Concepts and Units in particular, there are two types of outputs that are very important: One is called “Match All” and the other is called “Best Match.” A cascade that is set to “Match All” will output all matches to the input pattern that is received. A cascade that is set to “Best Match” will only have one output, which is calculated by the CIE based on the highest scoring pattern. If there is a tie in a “Best Match” cascade, then the output will be the last item that was calculated.
Concepts Groups can be set to either “Match All” or “Best Match” depending on the user. The majority of Concepts are usually set into “Match All” cascades, because Units take the input of multiple Concepts. Concepts are usually set into “Best Match” cascades when their patterns are so similar that they almost always triggered at the same time.
As an example, consider the following two Concepts—Home Equity and Home Equity Loan. The first Concept will always trigger because it will contain patterns that are also in the second Concept, therefore they should be put in their own “Best Match” cascade so only one can be outputted.
Units must always be assigned to “Best Match” cascades. The reason for this is because the output of a Unit is a response. This way, when an input is received, only one response is outputted.
Vocab is processed through the Context IQ Engine using cascades that only do substitution: it takes in a synonym replaces it with the name of its corresponding Vocab Term. This usually needs to be done three times for full effectiveness: the first time called the Pre-Vocab cascade where contractions are separated into separate words; the second time called the Vocab cascade where Vocab Terms are substituted for words; and a third time called the Post-Vocab cascade where multiple Vocab Term phrases are combined into single Vocab Terms, mainly for use in compound words.
The Cascades dockable window, shown in
Cascades are organized by type. In Cascade Group Type, shown in
Cascade Groups represent the primary grouping of all components of a KB. Cascade Group, as shown in the user interface of
Load Order
Selecting View/Edit Load Order from the Admin menu opens the Application Load Order window as shown in
Unused Items
An Unused Item is an item contained in the database, but not used in the current project. This can occur in many ways: newly created projects that point to an existing database will show all the items in that database as unused; and groups of Vocab, Units, and Concepts can be removed from a project, but not from the database, so they can be available for other projects.
In the Unused Items dockable window, as shown in
Resource Files
The Resource Files dockable window, as shown in
Managing Projects
This section provides detailed instructions for creating projects and interacting with the project management tools. To create a new project, use the following procedure in connection with the user interfaces shown in
To opening an existing project, using the user interface of
Within Active Lab, configuration settings that may be changed are found in Application Editor. To change these configurations, use the following procedure:
Project deployment involves sending the project files from one location to another, usually from the user's local hard drive to a test or production environment. There are two methods of deployment: by email or by copying the project files to a shared directory. You may select one or both of these methods as the situation warrants.
Before deploying a project, the following steps can be used to ensure a stable set.
Once the above steps have been completed, it is very important not to do any other activity in Active Lab before you deploy, or it may interfere with the deployment. This means that after opening the project, you continue directly to the following steps, without doing any other activity.
When the Unit responses have been finalized, one can generate a spreadsheet of the ones requiring new audio files. If you have previously imported voice files, this will be a two step process. First, generate a list of new requests; and secondly, generate a list of responses changed since the last recording.
When updated with the names of the audio files, these spreadsheets can be used as TTS import files.
Vocabulary
Active Lab uses vocabulary to allow the user a way to define important terms in a project as well as group synonyms together in order to streamline the Concept, Unit building, and maintenance process. This section contains a description of the details and principles surrounding vocabulary as well as instructions on creating and maintaining vocabulary within Active Lab.
Vocabulary is at the core of defining a Knowledge Base. Concepts and Units build on the foundation of vocabulary. The purpose of vocabulary is to group synonymous words together to be used in formulating input patterns for Concepts and Units. Vocabulary also vastly streamlines the maintenance involved in managing a KB.
The way that vocabulary is streamlined in the KB is by grouping synonyms together under a single label, called a Vocab Term. Synonyms can include any unambiguous synonyms, misspellings, tenses, pluralities, and/or possessive forms. Together, a list of Synonyms and its corresponding Vocab Term are known as a Vocab.
Vocab functions as a substitution algorithm: when a word is entered, it is substituted for its corresponding Vocab Term. This way, only the Vocab Term needs to be used when creating patterns later on. Vocab Terms are organized into Vocab Groups, mainly for organizational purposes. Vocab Groups can then be assigned to different cascades. Vocab is primarily viewed in the Vocab dockable window (See
The Vocab dockable window contains the following components:
The Vocab dockable window has three different right-click menus (shown in
Vocab Groups are sets of Vocab Terms that are categorically similar. For instance, the Vocab Terms “dog,” “cat,” and “parrot” might be placed in a Vocab Group called “animals.” Vocab Groups are used to simplify Vocab management within cascades and projects. Individual groups can be added and removed from projects, and can be moved around, in, and between cascades. Each Vocab Term can only belong to one Vocab Group.
The Add New Vocab Group window (
Vocab Terms are used to simplify maintenance of Concepts and Units by converting all synonymous words into a single term. Thus, Vocab Terms act as a form of substitution. In cases of ambiguous synonyms, create two separate Vocab Terms to be placed in a Building Block together later on. Vocab Terms are created via the Vocab Editor (See
Helper Terms are commonly used words and phrases that are used to build sentences, and have no synonyms or common misspellings, such as “is,” “or,” or “but.” Helper Terms are also used for words that should not be made into Vocab, especially those that are only necessary for a single Concept or Unit. Some names/proper nouns that you want to reuse in response may be used as Helper Terms. Helper Terms can be used anywhere Vocab is used, and also directly in Units. The Helper Terms dockable window is shown in
The Add New Helper Term and Edit Helper Term windows (see
Building Blocks serve as a means to group related Vocab and Helper Terms and Concepts together. There are two types of Building Blocks: Standard and Phrase.
Standard Building Blocks provide a useful means for associating terms or Concepts that are commonly though not always related to one another, specifically ambiguous Vocab, or words with multiple meanings based on context, such as “arm,” which can mean a part of the body or to ready a weapon.
Phrase Building Blocks are used to group short patterns of Vocab and/or Helper Terms together in a particular order. When a phrase Building Block is used in an input pattern, that pattern will only trigger when an input is received that exactly matches the order, from top to bottom, that is listed in the Building Block.
Building Blocks composed of Concepts are represented by a first type of icon, while those composed of Vocab Terms are represented by a second type of icon. Vocab and Helper Term Building Blocks can be used in Concept input patterns, while Concept Building Blocks can be used in Unit input patterns.
Building Blocks can be nested. This means that Building Blocks can be used to make other Building Blocks. Building Blocks has a dockable window shown in
Building Block Editor (
Managing Vocabulary
This section describes how to create, edit, and delete Vocab Groups, Vocab Terms, Helper Terms, and Building Blocks. Creating a Vocab Group can be performed in the following way.
Because Vocab Groups are shared across projects that share the same database, deleting a Vocab Group from the database affects the functionality of all projects attached to it. For this reason, it is best to remove the group from the project rather than deleting it from the database. Note: Vocab Groups may only be removed if they have no dependencies. To remove a Vocab Group from a project, use the following procedure:
As noted above, it is generally not recommended to delete Vocab Groups. However, should it be necessary to delete a Vocab Group from the database altogether, use the following procedure:
To create Vocab terms, consider the following in connection with
Adding new synonyms can be accomplished as follows:
Editing a Vocab Term can take the form of adding or deleting synonyms, changing the title of the term, or changing the Vocab Group to which it belongs. To edit an existing Vocab Term use the following procedure:
Editing a synonym in Vocab Editor uses the following procedure:
To delete synonyms, use the following procedure:
To remove a Vocab Term from a project:
To create a Helper Term, consider the following in connection with
With respect to editing Helper Terms, consider the following in connection with
With respect to deleting Helper Terms, use the following procedure:
With respect to creating, editing and deleting Building Blocks, consider the following in connection with
To create a Building Block, use the following procedure:
To edit a Building Block, use the following procedure:
To delete a Building Block, use the following procedure:
Concepts can serve as the middle step in the Vocab, Concept, and Unit process. It is an important step in the process that defines the meanings behind groups of words and phrases. This section details what a Concept is, as well as the processes involved in creating and editing Concepts.
In the English language, groups of words and phrases can imply a particular meaning. This meaning could be a goal, as in the phrase, “tell me about”, or it can be an idea, such as “a refreshing drink”. A group of all the possible phrasings that can define a meaning is known as a Concept. These Concepts are then used to create input patterns for any number of Units later on.
The Basics of Concepts
Concepts are composed of input patterns of Vocab Terms, Building Blocks, Wild Cards, and Helper Terms. A Concept is considered triggered (or “hit”) when it receives an input pattern that completely matches at least one of its patterns.
Concept Groups
Concepts are organized into groups by subject matter. The placement of Concept Groups in a cascade determines load order, and whether they are “Best Match,” or “Match All.” If a Concept Group is in a cascade labeled “Best Match,” only the constituent Concept with the best matching input pattern is triggered. If, on the other hand, the Concept Group is in a cascade labeled “Match All,” any Concept in this cascade having an input pattern matching the user's input will trigger.
Material Concepts
Knowledge Bases created by Active Lab can be used for many different things, even the use of functions not currently supported by the Lab. One of these uses is to help create Information Retrieval (IR) databases, which are used for other advance types of searching.
For use with these IR searches, Active Lab has the functionality to mark a Concept as a Material Concept, which will have a specific functionality within the IR search.
Generic Concept
For every Concept Group, one Concept can be marked as the Generic Concept. The Generic Concept will trigger any time any Concept triggers from that Concept Group.
Concepts Dockable Window
The Concepts dockable window, as shown in
Concepts also has two right-click menu options, as shown in
Concepts Editor Tab
Concepts are comprised of two principle components: tests and patterns. Patterns are made up of Vocab Terms, Helper Terms, Building Blocks, and Wild Cards and determine what phrases trigger the Concept. Tests are ways to make sure that the Concept is working as intended.
Tests Tab
Tests allows the user to create and edit test questions. Test questions represent user inputs that should cause the Concept to trigger. When looking at the set of test questions, it is best to consider the ideas involved. Ideal test questions should represent complete user inputs. In this manner, each Concept can be tested to see how it interacts with other Concepts. Test questions serve as a useful guide for the creation of input patterns. The Tests tab, as shown in
Tests also includes a right-click menu, as shown in
Patterns, as shown in
The tabs shown in
As discussed earlier, the Concepts Builder function is accessed either by means of the Concepts Editor tab, or by right clicking in the Concepts dockable window.
Creating a New Concepts Group
New Concept Groups are created using the following procedure in connection with the user interface shown in
New Concepts can be added using the following procedure in connection with the user interface shown in
Once a Concept has been created, the user can add test questions on the Tests tab using the following procedure in connection with the user interface shown in
To add Vocab Terms to a Concept pattern, use the following procedure in connection with the user interface shown in
Once patterns for either Concepts or Units have been changed, it is necessary for the FPML files themselves to be generated. There are currently two options:
Once test questions and patterns have been added to the Concept, it is necessary to run a test to ensure correct patterns have been used.
When the test is completed, the results will show in the Test Questions display next to each question. As discussed above, a green circle indicates success, a yellow circle indicates the Concept was untested or has been marked as InActive, and a red circle indicates failure.
Editing Concepts
Editing a Concept works as follows:
To delete a Concept, use the following procedure:
Because Concept Groups are shared by every project that accesses a given database, they may only be removed from the given project using the following procedure:
Units are the primary form of response to user input. A response is what results from user input. Units ensure that the appropriate responses appear for the each user input. The response may be as simple as a plain text output, or as complex as a series of Application Calls (AppCalls) that trigger specific actions, or any combination thereof. When preparing a Unit, it is important to consider the kinds of questions to which it will respond. What are the key phrases or words that are likely to show up in questions that should trigger the Unit? What question is the Unit supposed to answer? The answers to these questions will influence the user's choice of Concepts and the structure of the patterns.
In order to trigger the Unit, an input must match all the Concepts and Helper Terms in at least one of the input patterns of the Unit. It is often helpful to determine the Unit's response first, as that will in turn define which questions it will answer.
Units are organized into sets called Situation Groups according to their subject matter. Like Concept Groups, Situation Groups are placed under cascades, except they are set to “Best Match.” The placement of these cascades in the load order is usually last, because the Unit cascade outputs the response.
The Units Dockable Window
The Units dockable window, shown in
Unit contains two right-click menus as shown in
The Unit Editor Tab
The Unit builder function can be accessed through Unit Editor as shown in
Tests, as shown in
The Tests tab also contains the right-click menu shown in
The Patterns tab, as shown in
The Patterns tab also has the following right-click menu shown in
Response Tab
The response tab, as shown in
The right click menu shown in
Urls
One of the most powerful tools of ActiveAgent is the ability to direct a response to a particular web page. This is done by creating a Url in the Urls dockable window, shown in
Clicking the <Add> button or right-clicking→Add will open the Edit Url window shown in
Web Links
Web Links allow the user to include links directly to other Units and/or Urls along with a response. Web Links are created and edited from the Web Links dockable window shown in
The Web Links dockable window is separated into two folders: Links Containing Urls, and Links Containing Units. This is done for mainly organizational purposes. The Web Links dockable window contains the following components:
Web Links are created and edited by selecting <Add> or by selecting right-click→Add to open the Web Links window, which is shown in
Advanced Tabs
These tabs may only be viewed when <Advanced View> is selected.
Response Blocks are responses shared by multiple Units in addition to the unique Unit-specific responses. Response Blocks can be manipulated via the Response Blocks dockable window, and the Response Blocks advanced tab found under the Units tab.
The Text-To-Speech tab allows the user to manually attach a voice file to a Unit and has the following features:
The following discussion describes various ways to create, test, and edit Units.
Creating a New Situation
To create a new situation, use the following procedure:
Adding New Units
Adding Questions
Once the Unit has been created, test questions may be added under the Tests tab of the Unit Editor.
Adding Responses
Each Unit has a response that occurs as a result of an input that matches any of its input patterns. As discussed above, this may be as simple as a single line of text or as complicated as an application action. Though responses do not require URLs to function, they can benefit from the versatility associated with web pages. To add a URL to a response:
To create the response itself:
Web Links allows the user to add links to additional web pages and/or Units in the response. To add a Web Link to a response:
Generating the Patterns
In order to generate a pattern for a Unit, use the following procedure:
Use steps 1-7 to create as many patterns as desired. Note: You must double click on the pattern in order to later make any changes to the Position or <Order Matters> and <Proximity Matters> check boxes. Click <Save> to save these changes.
Testing the Patterns
In order to test the input patterns, use the following procedure similar to testing Concepts:
To debug a failed question:
Editing a Unit
To edit a unit, perform the following procedure:
To delete a pattern, perform the following procedure:
To deactivate a pattern, perform the following procedure:
In the same way that patterns may be deactivated, the same may be done to entire Units.
Test questions may be edited using the following procedure:
Responses may be edited using the following procedure:
Deleting a Unit
To delete a unit, use the following procedure:
Directives are the way conversation scenarios are created for ActiveAgent. They can take the form of a directed conversation, or a series of steps the user needs a person to follow. This section discusses in detail what directives are, along with the different windows that are associated with it, as well as how to implement them.
Definition of a Directive
A Directive can be thought of as a series of Units that are linked together by specific responses, forming conversations. These conversations can be triggered by events on a web page, or by a person's input to ActiveAgent.
Take this simple example:
In this example, there are actually two Units being triggered. The first is the response of “I'm fine, thanks! How are you?” and the second Unit has the response “I'm glad to hear that.” The work of the Directive is to make the second Unit only accessible directly after the first Unit has triggered. The reason for this is to prevent ActiveAgent from responding with “I'm glad to hear that” anytime a person types is “Good.”
A single Directive is not limited to one conversation stream; it can have many branches, based on the variety of things a person could say to the question of “How are you?”
In this example, the same Unit response was hit, of “I'm fine, thanks! How are you?” but a different response is given.
Directive Trees
Directives are displayed in a Directive Tree, which is a series of responses that can branch out as many times and with as much complexity as the user wants.
In the following example, a single Unit, called the Parent Unit, is triggered. The conversation then flows into Child Units, or Units that can be accessed through Directives.
Initial input: “I want to buy a watch”
As seen in this example, specific responses to a Unit that is set up as a Directive will produce other Units. If a person typed “small” and wasn't in a Directive, no Unit would be triggered. However, if “small” is typed in response to a particular Unit, a specific Unit is triggered. In this manner, conversations can be directed.
Another feature of Directives is that a Child Unit still has all the qualities of a regular Unit. This means that a Child Unit can also be accessed from outside the Directive. For example, Unit 3 from the above example can be set to always respond directly to the question, “Can I buy a gold watch?” even if its Parent Unit has never been triggered.
Directive Dockable Window
Since Directives are simply Units that have Child Units, the Directives dockable window, as shown in
Directive Builder
Directives are created and edited in the Directive Builder window. This window consists of two main Tabs:
Builder Tab
The Builder tab of the Directive Builder window, as shown in
Advanced Tab
The Advanced tab of the Directive Builder window, as shown in
The Advanced tab contains three main sections:
The Directive Tree is a tree view of the entire Directive. Each indent on the tree implies one level further into the tree. Branches on the same level of the tree are linked together with vertical lines.
To access many of the features of the Directives field, access the right click menu. The Directives field has the following components:
The Edit Info tab, as shown in
Test questions and patterns listed under the Edits Tab are Directive specific. This means that they only apply within the Directive, and cannot be accessed unless the previous point in the Directive has been hit. The Edit Info tab has the following components:
In the Tests tab, shown in
Except for the fact that the tests are in the form of a tree, the functionality of Tests is very similar to the Tests tab from the Unit Editor. The Tests Tab has the following components:
Variables
This section describes the creation and editing of Directives using the Directive Builder, and Directives dockable window.
The Builder Tab
When the Directive Builder opens, it will default to the Builder tab. The Parent Unit will be displayed in the upper left corner. From this window, both the Parent Unit and the Child Unit can be edited, and an entire Directive Tree can be easily built. The Builder tab creates an easy way to build a Directive Tree, however patterns and test questions are added using the Advanced tab discussed below.
The left hand side of the Builder tab is a slide show view of the entire Directive. The uppermost slide will always be the Parent Unit of the entire Directive. Below it on smaller slides are all of the Child Units that are directly below it in the Directive Tree.
If a Child Unit is selected, it will increase in size and any of its Child Units will appear in as small slides below it. This can be repeated as long as there are more Child Units below each subsequent Child Unit.
Clicking on any Parent Unit will change the slide show view to show all Child Units of that Unit.
A Directive can be created in a variety of ways from a variety of places. The reason for this is because a Directive is simply multiple Units linked together in a series. This makes creating Directives very flexible.
There are three different ways to begin the process of creating a Directive:
When the Directive Builder opens, it will default to the Builder tab. The Parent Unit will be displayed in the upper left corner. All Child Units will be displayed below.
To add a Child Unit:
To add a Child Unit to an existing Child Unit:
To delete a Child Unit:
Once a Parent Unit or Child Unit has been created its User Input display can be added or changed:
Once a Parent Unit or Child Unit has been created its Response can be added or changed:
Once a Parent Unit or Child Unit has been selected, its Web Links can be added and removed, and the order of the Web Links modified.
Unused Answer Links, listed below Web Links, are the Child Units of the selected Unit. They can be added as Web Links to the Unit by clicking and dragging them from Unused Answer Links to the Web Links field.
Web Links can be removed from the Unit by clicking the <Remove> button.
Urls can be added and removed from the Unit response in a Directive using the same steps as a regular Unit.
The Advanced Tab
When the Directive Builder window opens, it will default to the Builder tab, as shown in
The Advanced tab allows an entire directive to be built and maintained. It includes all the functionality of the Builder tab, just in a different form. It also has the ability to add test questions, add patterns, and actually test the Directive. This section describes the process.
There are two ways to create a Parent Unit. Using Existing Units:
To create a new Parent Unit from the Directive Builder:
Once a Parent Unit is set in the Directive Tree, you can now add possible user responses. Each response will be considered a Child Unit, therefore only add one response per Child Unit desired. There are two ways to add a Child Unit:
Note: These processes can be used on any branch of the Directive Tree to create Child Units for any branch.
Using Existing Units:
Creating a new Child Unit:
Units that are created in the Directive Builder do not technically exist until they are officially created. This is done by right-clicking on the selected Directive and selecting→Create Unit. The Unit Builder window will open. You may edit the Unit as necessary, or close the window.
Once the Unit is created, it can be found in its corresponding Situation Group in the Units window, or by selecting→Edit Unit from the right-click menu. Patterns and test questions created from the Unit Editor can be accessed from outside the Directive, while patterns and test question created inside the Directive Builder will only operate when in the Directive.
There are three ways to edit both the User Input Display and Response Display:
To delete any branch of the Directive Tree, including any Child Units of that branch:
Adding Test Questions within a Directive
To add test questions that will only apply within the Directive, follow these steps:
Type test question into the text field and press <Enter> or <Add>. Repeat this step as many times as desired.
To add test questions that will only apply within the Directive, follow these steps:
To edit test questions within a Directive, follow these steps:
Adding Patterns to Child Units
Adding a pattern to a Child Unit within the Directive will limit that pattern to work only in the Directive. Adding these patterns is done in the exact same way as it is done in the Patterns tab of the Unit Editor.
To edit patterns within a directive, perform the following procedure:
To delete patterns within a directive perform the following procedure:
To deactivate patterns within a directive, perform the following procedure. When set inactive, a pattern will not trigger the Unit when matched.
Additional Variables within a Directive
To add a Variable to a Directive, drag and drop any chosen Variable from the Components dockable window into the Additional Variables tab under Edit Info.
Adding a Url within a Directive
Though responses do not require URLs to function, they can benefit from the versatility associated with web pages. To add a URL to a response:
To remove a URL from a response:
Testing Directives
To test only the test questions associated with the Directive, follow these steps:
The status of each test question is displayed as a colored ball next to the question. A green circle indicates success, a yellow ball indicates the question was not tested, and a red ball indicates failure. Questions that fail feature a brief summary of the error(s) encountered.
To debug a failed question:
Variables are means by which the KB may store information captured in a conversation for later use. This section discusses Variables and Record Lists as they fit into Units and Concepts.
Components Dockable Window
Variables and Record Lists are accessed via Components, along with AppCalls. The Components dockable window is shown in
The above components only apply to Variables and Record Lists.
Variables
Variables are components used for profiling and contextualization. When used for profiling, they log a site user's activities on the user's site, and any information they might divulge, such as name, IP address, e-mail address, or credit card number.
Variables are means by which a user may allow certain Units to trigger only in specific situations. Much like Directives, they allow for greater specificity in a Unit's functioning. The user may specify the type of Variable, the Variable's name, and the Variable's initialization value (be it true/false, empty string, or other value). Furthermore, the user may specify whether Variable information is saved in the chat logs. Note: If the Save In Chat Logs selection is not checked, all information gained from the Variable will be lost when the site-user leaves the chat. Save In Chat must be selected to view the information at a later time.
Variables are grouped together by type. This is done mostly for organizational purposes. There are three categories of Variables:
By clicking on Variables, and selecting <Add> or selecting a Variable Type and selecting <Edit>, the Variable Type window shown in
Variable Editor, shown in
View Variable Initialization and Setup
Selecting→Admin→View Variable Initialization And Setup will open the Variables window. The Variables window is a list of all Variables associated with the project, as well as what value the Variables are set to at either initialization or setup (before each User Input).
The Variables window, shown in
The Variables window also contains the following right-click menu:
Record Lists
Record Lists are lists of Variables that work in conjunction with one another. They are used when information should be grouped together and when an unspecified number of values is possible.
The Record List Editor, shown in
Record List configuration appears whenever a Record List is added to a Concept, Unit; or Response Block. The Configure Record List window, shown in
The following section discusses the procedures involved in creating, maintaining, and using Variables and Record Lists.
Managing Variables and Record Lists
This sub-section deals with creating, editing and deleting Variables and Record Lists. These operations are performed via Components. To create variable types, perform the following procedure in connection with
To edit an existing variable type:
To create variables perform the following procedure in connection with
To edit variables, perform the following procedure in connection with
To delete variables:
To creating record lists, perform the following procedure in connection with
To edit record lists, perform the following procedure in connection with
To deleting record lists, perform the following procedure:
Using Variables and Record Lists
When Variables and Record Lists are placed in a Concept or Unit, they are assigned a value. This sub-section concerns the assigning of value for Variables and Record Lists.
To set variables in concepts and units, perform the following procedure:
7.
To remove variables from concepts and units, perform the following procedure:
To set record lists in concepts and units, perform the following procedure in connection with
To removing a record list from a concept or unit, perform the following procedure:
Advanced vocabulary refers to items that may be used to add greater flexibility to Concepts and Units, as well as discern the differences between various types of inputs. This section describes Variables, lex types, Wild Cards, and input files.
Lex Types
Lex types classify user inputs to further specify what is to be matched. Lex types function as placeholders that determine the type of input that is recognized. For instance, lex types are used to differentiate between numerical inputs and word inputs, or between units of weight and units of distance.
Lex Types includes a dockable window, shown in
Lex files are static files that are edited manually using a text editor or the Editor tab. They may be added to a project through the lab by means of a Lex Builder component. A Lex Builder window is shown in
Wild Cards are place-holders that allow the user to create more flexible input patterns for Concepts. A Wild Card is placed between two Vocab Terms to indicate that there may be any words in between the Vocab Terms. There are three Wild Cards, and they are applicable to all projects:
Wild cards includes a dockable window shown in
The Wild Card Builder, shown in
Input Files are lists of words in a text file (*.txt). They are used as substitutes for large groups of words, such as names and locations. An Input File can be placed in the pattern for a Concept just like a single Vocab Term. The pattern for the Concept will then trigger if any word in the Input File is hit, much like a Building Block.
Once an Input File has been created, it can be compared against all Vocab in the KB. Any overlapping words are converted to Vocab in the Input File. In this way, common misspellings can be included if a word has already been assigned to a Vocab Term. Input files are accessed via a dockable window of the same name, illustrated in
Input File Builder, shown in
The next section deals with the procedures involved in managing lex types and input files. As mentioned above, lex types are created as part of the overall lex file, which is created manually by the administrator.
To add a lex type to a project, use the following procedure. Note that corresponding FPML must be manually entered into the static file *.lex before this will have any affect.
To edit lex types, use the following procedure in connection with
Lex types may be added to the input patterns of Concepts just like any other form of vocabulary. To do so, use the following procedure:
To delete a lex type from a project, use the following procedure:
Managing Wild Cards
This sub-section deals with creating, editing, and adding Wild Cards to Concepts.
In one or more embodiments, Wild Cards are created by the administrator only. To add a Wild Card to a Concept input pattern, use the following procedure:
Input files are manipulated primarily through Input Files. This sub-section outlines the procedures involved in creating, editing, and deleting input files.
Creating Input Files
Editing Input Files
Adding Input Files to Concepts
Deleting Input Files
This section deals with advanced response items, specifically Dynamic Functions, AppCalls, Conditional Statements and Response Blocks. All of these items are used primarily, though not exclusively, in response-related capacities.
This section examines the principles behind Conditional Statements, AppCalls, Response Blocks, and Dynamic Functions.
Conditional Statements
Conditional statements represent the necessary prerequisites for a specific action being taken. They are used in Response Blocks and often for data-mining operations. In such situations, the conditional statement is a pre-established Variable value. The value of this Variable will then determine the way in which a Unit that contains the given Response Block functions. For instance, a Unit might respond differently based upon whether the user registers as male or female. The user's gender thus functions as the pre-condition of the Response Block.
Components Window
AppCalls are accessed via Components (
AppCalls
AppCalls, short for “Application Calls,” are a form of response wherein an external application is used. For example, a Unit might have an AppCall that opens a new program or fills out a form when triggered. The AppCall consists of two parts. The first is the call to the outside application.
The second part of an AppCall is to define what parameters (ex: the value of a Variable) needs to be passed along with the AppCall.
AppCall Editor (
Configure AppCall Window
The Configure AppCall window (
Search AppCall Usage
The Search AppCall Usage window (
Response Blocks
Response Blocks are responses that are shared by multiple Units, which may or may not have other additional response features, such as AppCalls, Dynamic Functions, or Record Lists, which coincide. Response Blocks are also used when more control over the print order of the Unit is desired. They also provide a means of data-mining when combined with dynamic functions. Response Blocks may be added to both Concepts and Units by means of dragging and dropping them into the Response Blocks tab.
Response Blocks Dockable Window, shown in
The Response Block Editor window, shown in
The following discussion describes aspects of the tabs of the Response Block Editor window.
Call Items are Record Lists, AppCalls, Variables, and dynamic functions that are used when the Response Block is called. Variables, AppCalls, and Record lists are set to the value listed here, similar to the Components tab in the Unit Editor. Call Items contains the following components:
Response allows the user to set a plain text and/or tagged response associated with the Response Block and includes the following components.
Conditional Statement allows the user to associate specific conditional statements with the Response Block and includes the following components.
Text-To-Speech allows the user to select a specific Text-To-Speech File for the Response Block. Text-To-Speech includes the following components.
Dynamic Functions
Dynamic functions serve as a means for profiling site users and running searches based upon ranges (such as “cameras between $100 and $200”). They are used in Call Items for Response Blocks. Each dynamic function must reside in its own cascade, which must bear the same name. This cascade must have leveling enabled. Dynamic functions has a dockable window as shown in
A Dynamic Function Editor, shown in
Managing Advanced Response Items
This section deals with managing the AppCalls, Response Blocks, and dynamic functions. AppCalls are found in Components. This sub-section deals with the procedures involved in creating, editing, and using AppCalls. Creating new app calls can be performed using the AppCall Editor shown in
To edit AppCalls, perform the following procedure using the AppCall editor of
To delete AppCalls, perform the following procedure using the editor of
To create new response blocks, perform the following procedure:
To edit response blocks, perform the following procedure:
To deleting response blocks, perform the following procedure:
To create new dynamic functions, perform the following procedure:
To edit dynamic functions, perform the following procedure:
To delete dynamic functions, perform the following procedure:
Until they are placed in a Concept or Unit, advanced response items are just empty templates that do not actually contain any information, just the form. Like Variables, they must be assigned values in order to function properly.
To call AppCalls from a Concept or Unit, perform the following procedure:
AppCalls may also be manually input into the response of a Unit in the Response tab.
To use response blocks in a concept or unit, perform the following procedure:
Response Blocks can also be incorporated into the Concept or Unit by adding it to Wild Cards within any input patterns.
Dynamic Functions are added to Concepts and Units as part of Response Blocks, which are in turn added to Concepts and Units as described above. To add dynamic functions to a concept or unit, perform the following procedure:
Testing and Debugging
Testing and its sister function, debugging, are important elements of creating a successful project. To this end, this section will examine both the testing and debugging features of Active Lab.
Details of Tests
While tests for each Unit and Concept may be found in the respective tabs, it is also possible to view the tests for the entire project in a Tests Tab shown in
Unit tests are organized by Situation Group, while Concept tests are organized by Concept Group. Each Concept Group and Situation Group may be expanded to view its constituent parts, and each of these in turn may be expanded to view the individual test questions.
As with the tests in individual Units and Concepts, green circles represent success, yellow circles represent a test that did not run, and red circles indicate a failed test. If one test for one Unit or Concept shows up red, the status of the Unit or situation itself will show red, and will in turn lead to a red status for the Situation Group or Concept Group in which it appears. This status is also reflected on the project as a whole. Thus, if so much as one test question in one Concept in the entire project shows red, the entire project will be red. Put simply, the status of any group is only as good as that of its worst constituent.
The Tests Tab contains the following components.
TestEditor, shown in
Details of Debugger
The Debugger enables you to validate the FPML to make sure it is syntactically correct. Syntax is only one part of creating FPML. The other is ensuring the right response and action is taken for the right input. The Debugger also allows you to see what response will be given for a given input. To further aid in testing and validating FPML, the Debugger allows entire input scripts to be run in order to verify comprehensive response results.
The Debugger is designed to allow users great flexibility in their testing and debugging processes. After a KB has been thoroughly debugged in Active Lab, it is recommended to also test it in the Active Agent application before using it live. The Debugger environment does not replicate the AppCalls, as they are defined in the application using the FPML.
Chat Control
The Chat Control dockable window, usually residing on the left hand side of the Debugger tab, is used to directly ask questions to the Agent.
Anything entered into the Chat Here window will be processed by Active Lab. The response will be displayed in the Chat History window directly above it. The processes that occur to receive that response are displayed in the variety of tabs that appear in the Debugger.
Note: Any time that “Send To Debugger” is selected elsewhere in Active Lab, the Debugger tab is opened and the selected test question in entered into the Chat Here field and processed.
The Match Data Tab
Match Data, as described above, displays the procedure used by the project to trigger the Unit. More specifically, it shows how the pre-processed string is converted into Vocab and Helper Terms, which Concepts are triggered by the Vocab and Helper Terms, then which Units are then triggered by those Concepts.
Here is a breakdown of the pieces of the Match Data:
Note: In case of a tie, the Unit that was triggered last will be listed as the Highest Scoring Unit. In case of a tie between Concepts that are in a Best Match cascade, the Concept that was triggered last will be listed as a Concept Hit.
The App Calls Tab
The App Calls tab, shown in
The Brain Variables Tab
The Brain Variables tab, shown in
Right-click on any Variable and select Add Watch to add selected Variable to the Watch Variables window.
The Web Page Tab
The Web Page tab, shown in
The Fpml Error Messages Tab
The Fpml Error Messages tab, shown in
The Load Data Tab
The Load Data tab, shown in
Each entry in the Load Data tab displays consists of an action and a file path. Common actions include: Loading, Learning, and IRDLL. Each *.FPML file that is loaded correctly will be listed in black, and errors will be listed in red.
The Raw Match Data Tab
The Raw Match Data tab, shown in
Watch Variables Dockable Window
The Watch Variables window, shown in
Variables can be easily added or removed from Watch Variables, using the available right-click menu.
Viewing Dependencies
One of the most powerful tools to track down problems is the Dependencies window shown in
The Dependency window can be opened by selecting a particular item and choosing <Show Dependencies> from many of the Button Bars. It is also available on many different right-click menus.
Displayed in
The Dependencies window has the following components:
Testing Conditions
The Test Conditions dockable window, shown in
There are two Test Conditions that can be set to Variables and Record
Lists: Pre-Conditions and Post-Conditions. These conditions are then assigned to any number of test questions.
When a Pre-Condition and inserted into a test question, the value of the condition is set just prior to the test question being run. When a Post-Condition is set into a test question, the value of the condition is set just after the test question is run.
Test Conditions contains the following components:
Create/Edit New Pre/Post-Condition
Creating and editing pre and post conditions is completed using three different windows, an example of which is shown in
Edit contains the following components:
Managing Tests and Debugger
This section deals with the procedures involved in testing and debugging.
Managing Tests
This sub-section deals with adding new test questions, editing existing test questions, and running tests.
Adding New Test Questions
In addition, new test questions may be added in Tests. To add a new test question, use the following procedure in connection with the window shown in
Editing Test Questions
Test questions may be edited in Test Editor or in Concept Editor or Unit Editor, respectively. The following is the procedure for editing a test found in Tests in its appropriate tab:
To edit test questions in Test Editor, use the following procedure:
Deleting Test Questions
To delete a test question in Tests, use the following procedure:
To delete a test question in Test Editor, use the following procedure:
Running Tests
To run tests in Tests, use the following procedure:
Debugging Questions
After examining the test results, it is important to examine the causes of any failures that might have cropped up. To debug a question (particularly a failed question), use the following procedure:
Debugging
This sub-section deals specifically with the procedures involved in debugging Units and Concepts.
Manual Debugging Procedures
Manual debugging is useful for running scripts through the FPML. To manually debug a project, use the following procedure:
Debugging from Outside the Debugger
To debug any question automatically from outside the Debugger, simply right-click on the test question and select →Debug Question. This will have the same effect as typing the question into the Debugger directly.
Dependencies
This sub-section shows the different uses of the Dependencies window.
Find Dependency
There are two ways to find the dependencies on an item:
Replace Usage
To replace either one item or a large group of a selected item with another, follow this procedure.
Test Conditions
This sub-section will describe the creation, editing, and use of Test Conditions.
Creating Test Conditions
The steps for creating both Pre and Post Condition are the same:
Editing Test Conditions
The steps for creating both Pre and Post Condition are the same:
Deleting Test Conditions
The steps for deleting both Pre and Post Condition are the same:
Adding a Test Condition to a Test Question
Once a Test Condition has been created, it can be linked with any number of test questions:
Removing a Test Condition from a Test Question
To remove a Test Condition from a test question, follow this procedure:
The Editor tab allows the user to view and interact with the raw FPML code of static files and lex files that forms the basis of all of Active Lab's operations.
The Editor tab, shown in
Using Editor
The Editor can be used in the same manner as other text editors. There are also customizable shortcuts which can be configured in the shortcuts.xml file located in the directory with Active Lab.
Static Files
Static Files are FPML files created manually using either a standard text editor or Active Lab's Editor tab. They allow the user to create custom FPML functions beyond Active Lab's capabilities should such a need arise. Static files must be created manually outside of the lab and then imported into the database using the Static Files dockable window.
Lex Files
Lex Files are instrumental to the proper function of a project, and as such should generally be left alone by the user. The user may view and edit Lex files through the Editor tab, though this is not generally recommended for most users.
Change Request
While change requests do not directly affect the functioning of a project, they represent a way for the user to view feedback from other users regarding the construction of the project. In this regard, they can be quite useful in the process of fine-tuning the components of a project.
Because all of these change requests pertain to Units, they may be accessed through their respective Units or through Change Requests. This dockable window contains the components illustrated in
Change request management occurs primarily within the dockable window of the same name. The right-click menu allows the user to go directly to the Unit in question and make any changes. Change requests may not be deleted, as they serve as a record of the changes made to the project.
There are six types of change requests:
Create New Unit Request
New Unit request can be edited, changed, and implemented in the Create New Unit Requests window. This window is broken up into two tabs: the Create Unit Request tab, which shows all the parameters for the Unit; and the Request Information tab, which shows history, comments, and the option to ignore the request.
The Request Information tab is shown in
The Response Change Request window, illustrated in
Url Change Request
The Url Change Request window, illustrated in
Add Question to Unit Request
The Add Question To Unit Request window, illustrated in
Wrong Response/Wrong Url
The Request Details window, illustrated in
Example System
In one or more embodiments, computer-readable instructions reside on the computer-readable media and implement the functionality described above. Such functionality can reside in the form of an application referred to as Active Lab 1366. Active Lab 1366 can present the various user interfaces described above and can implement the described functionality.
Various embodiments provide a tool, referred to herein as “Active Lab” that can be used to develop, debug, and maintain knowledge bases. These knowledge bases (KBs) can then engage various applications, technology, and communications protocols for the purpose of task automation, real time alerting, system integration, knowledge acquisition, and various forms of peer influence.
In at least some embodiments, a KB is used as a virtual assistant that any real person can interact with using their own natural language. The KB can then respond and react however the user wants: answering questions, activating applications, or responding to actions on a web page.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5278980 | Pedersen et al. | Jan 1994 | A |
| 5339391 | Wroblewski et al. | Aug 1994 | A |
| 5418948 | Turtle | May 1995 | A |
| 5535120 | Chong et al. | Jul 1996 | A |
| 5615112 | Liu Sheng et al. | Mar 1997 | A |
| 5677835 | Carbonell et al. | Oct 1997 | A |
| 5682539 | Conrad et al. | Oct 1997 | A |
| 5727174 | Aparicio, IV et al. | Mar 1998 | A |
| 6012053 | Pant et al. | Jan 2000 | A |
| 6112177 | Cosatto et al. | Aug 2000 | A |
| 6144938 | Surace et al. | Nov 2000 | A |
| 6175829 | Li et al. | Jan 2001 | B1 |
| 6282507 | Horiguchi et al. | Aug 2001 | B1 |
| 6285978 | Bernth et al. | Sep 2001 | B1 |
| 6353817 | Jacobs et al. | Mar 2002 | B1 |
| 6388665 | Linnett et al. | May 2002 | B1 |
| 6396951 | Grefenstette | May 2002 | B1 |
| 6401061 | Zieman | Jun 2002 | B1 |
| 6658627 | Gallup et al. | Dec 2003 | B1 |
| 6661418 | McMillan et al. | Dec 2003 | B1 |
| 6757362 | Cooper et al. | Jun 2004 | B1 |
| 6826540 | Plantec et al. | Nov 2004 | B1 |
| 6829603 | Chai et al. | Dec 2004 | B1 |
| 6834120 | LeClerc et al. | Dec 2004 | B1 |
| 6987514 | Beresin et al. | Jan 2006 | B1 |
| 6999932 | Zhou | Feb 2006 | B1 |
| 7058902 | Iwema et al. | Jun 2006 | B2 |
| 7076430 | Cosatto et al. | Jul 2006 | B1 |
| 7194483 | Mohan et al. | Mar 2007 | B1 |
| 7263493 | Provost et al. | Aug 2007 | B1 |
| 7337158 | Fratkina et al. | Feb 2008 | B2 |
| 7483829 | Murakami et al. | Jan 2009 | B2 |
| 7536413 | Mohan et al. | May 2009 | B1 |
| 7539656 | Fratkina et al. | May 2009 | B2 |
| 7548899 | Del Favero, Jr. et al. | Jun 2009 | B1 |
| 7558792 | Bier | Jul 2009 | B2 |
| 7599831 | Ford | Oct 2009 | B2 |
| 7610382 | Siegel | Oct 2009 | B1 |
| 7711547 | Abir | May 2010 | B2 |
| 7739604 | Lyons et al. | Jun 2010 | B1 |
| 7797146 | Harless et al. | Sep 2010 | B2 |
| 7818183 | Schoenberg | Oct 2010 | B2 |
| 7912701 | Gray et al. | Mar 2011 | B1 |
| 7970663 | Ganz et al. | Jun 2011 | B2 |
| 8160979 | Evans et al. | Apr 2012 | B1 |
| 8346563 | Hjelm et al. | Jan 2013 | B1 |
| 8352266 | Farmaner et al. | Jan 2013 | B2 |
| 8401842 | Ginzburg et al. | Mar 2013 | B1 |
| 8433556 | Fraser et al. | Apr 2013 | B2 |
| 8473420 | Bohus et al. | Jun 2013 | B2 |
| 8510276 | Haiby et al. | Aug 2013 | B2 |
| 8519963 | Kocienda et al. | Aug 2013 | B2 |
| 8670979 | Gruber et al. | Mar 2014 | B2 |
| 8677377 | Cheyer et al. | Mar 2014 | B2 |
| 8731929 | Kennewick et al. | May 2014 | B2 |
| 8756326 | Elberse et al. | Jun 2014 | B1 |
| 8762152 | Bennett et al. | Jun 2014 | B2 |
| 8930191 | Gruber et al. | Jan 2015 | B2 |
| 8942986 | Cheyer et al. | Jan 2015 | B2 |
| 8943094 | Miller et al. | Jan 2015 | B2 |
| 9117447 | Gruber et al. | Aug 2015 | B2 |
| 9202171 | Kuhn | Dec 2015 | B2 |
| 9501741 | Cheyer et al. | Nov 2016 | B2 |
| 20010000356 | Woods | Apr 2001 | A1 |
| 20010033298 | Slotznick | Oct 2001 | A1 |
| 20010044751 | Pugliese, III et al. | Nov 2001 | A1 |
| 20010049688 | Fratkina et al. | Dec 2001 | A1 |
| 20020008716 | Colburn et al. | Jan 2002 | A1 |
| 20020032591 | Mahaffy et al. | Mar 2002 | A1 |
| 20020123994 | Schabes et al. | Sep 2002 | A1 |
| 20020129031 | Lau et al. | Sep 2002 | A1 |
| 20020198885 | Streepy, Jr. | Dec 2002 | A1 |
| 20030004908 | Linthicum et al. | Jan 2003 | A1 |
| 20030041307 | Park | Feb 2003 | A1 |
| 20030061029 | Shaket | Mar 2003 | A1 |
| 20030088547 | Hammond | May 2003 | A1 |
| 20030126089 | Fukuoka et al. | Jul 2003 | A1 |
| 20030126090 | Fukuoka et al. | Jul 2003 | A1 |
| 20030142829 | Avigni | Jul 2003 | A1 |
| 20030212544 | Acero et al. | Nov 2003 | A1 |
| 20040107088 | Budzinski | Jun 2004 | A1 |
| 20040141013 | Alcazar et al. | Jul 2004 | A1 |
| 20040186705 | Morgan et al. | Sep 2004 | A1 |
| 20050027694 | Sauermann | Feb 2005 | A1 |
| 20050054381 | Lee et al. | Mar 2005 | A1 |
| 20060004826 | Zartler et al. | Jan 2006 | A1 |
| 20060020466 | Cousineau et al. | Jan 2006 | A1 |
| 20060036430 | Hu | Feb 2006 | A1 |
| 20060037076 | Roy | Feb 2006 | A1 |
| 20060047632 | Zhang | Mar 2006 | A1 |
| 20060067352 | John et al. | Mar 2006 | A1 |
| 20060074689 | Cosatto et al. | Apr 2006 | A1 |
| 20060080107 | Hill | Apr 2006 | A1 |
| 20060092978 | John et al. | May 2006 | A1 |
| 20060161414 | Carignano et al. | Jul 2006 | A1 |
| 20060206483 | Knepper et al. | Sep 2006 | A1 |
| 20060253427 | Wu et al. | Nov 2006 | A1 |
| 20070043687 | Bodart et al. | Feb 2007 | A1 |
| 20070100790 | Cheyer et al. | May 2007 | A1 |
| 20070106670 | Yoakum et al. | May 2007 | A1 |
| 20070130112 | Lin | Jun 2007 | A1 |
| 20070134631 | Hardy et al. | Jun 2007 | A1 |
| 20070156677 | Szabo | Jul 2007 | A1 |
| 20070185702 | Harney et al. | Aug 2007 | A1 |
| 20070197296 | Lee | Aug 2007 | A1 |
| 20070242656 | Klassen et al. | Oct 2007 | A1 |
| 20070265533 | Tran | Nov 2007 | A1 |
| 20070294229 | Au | Dec 2007 | A1 |
| 20080005158 | Zartler et al. | Jan 2008 | A1 |
| 20080010268 | Liao et al. | Jan 2008 | A1 |
| 20080016040 | Jones et al. | Jan 2008 | A1 |
| 20080036756 | Gaos et al. | Feb 2008 | A1 |
| 20080091406 | Baldwin et al. | Apr 2008 | A1 |
| 20080096533 | Manfredi et al. | Apr 2008 | A1 |
| 20080162498 | Omoigui | Jul 2008 | A1 |
| 20080222734 | Redlich et al. | Sep 2008 | A1 |
| 20080235604 | Ebert | Sep 2008 | A1 |
| 20080305815 | McDonough | Dec 2008 | A1 |
| 20090006525 | Moore | Jan 2009 | A1 |
| 20090030800 | Grois | Jan 2009 | A1 |
| 20090063427 | Zuta et al. | Mar 2009 | A1 |
| 20090070103 | Beggelman et al. | Mar 2009 | A1 |
| 20090077488 | Ording | Mar 2009 | A1 |
| 20090089100 | Nenov et al. | Apr 2009 | A1 |
| 20090119095 | Beggelman et al. | May 2009 | A1 |
| 20090119587 | Allen et al. | May 2009 | A1 |
| 20090157386 | Zhou | Jun 2009 | A1 |
| 20090171923 | Nash et al. | Jul 2009 | A1 |
| 20090182702 | Miller | Jul 2009 | A1 |
| 20090204677 | Michaelis et al. | Aug 2009 | A1 |
| 20090216691 | Borzestowski et al. | Aug 2009 | A1 |
| 20090225041 | Kida et al. | Sep 2009 | A1 |
| 20090227223 | Jenkins | Sep 2009 | A1 |
| 20090228264 | Williams et al. | Sep 2009 | A1 |
| 20090235356 | Jensen et al. | Sep 2009 | A1 |
| 20090248399 | Au | Oct 2009 | A1 |
| 20090271205 | Finn et al. | Oct 2009 | A1 |
| 20100005122 | Jackson | Jan 2010 | A1 |
| 20100030549 | Lee et al. | Feb 2010 | A1 |
| 20100050237 | Bokor et al. | Feb 2010 | A1 |
| 20100070448 | Omoigui | Mar 2010 | A1 |
| 20100070871 | Liesche et al. | Mar 2010 | A1 |
| 20100153398 | Miller et al. | Jun 2010 | A1 |
| 20100169336 | Eckhoff-Hornback et al. | Jul 2010 | A1 |
| 20100218113 | White et al. | Aug 2010 | A1 |
| 20100226490 | Schultz et al. | Sep 2010 | A1 |
| 20100235808 | Dayan et al. | Sep 2010 | A1 |
| 20100281012 | Imig et al. | Nov 2010 | A1 |
| 20100312547 | Van Os et al. | Dec 2010 | A1 |
| 20110004841 | Gildred et al. | Jan 2011 | A1 |
| 20110071819 | Miller et al. | Mar 2011 | A1 |
| 20110078105 | Wallace | Mar 2011 | A1 |
| 20110119196 | Ventura et al. | May 2011 | A1 |
| 20110179126 | Wetherell et al. | Jul 2011 | A1 |
| 20110213642 | Makar et al. | Sep 2011 | A1 |
| 20110282664 | Tanioka et al. | Nov 2011 | A1 |
| 20110288947 | Biran | Nov 2011 | A1 |
| 20110301982 | Green, Jr. et al. | Dec 2011 | A1 |
| 20110307245 | Hanneman et al. | Dec 2011 | A1 |
| 20120016678 | Gruber et al. | Jan 2012 | A1 |
| 20120022872 | Gruber et al. | Jan 2012 | A1 |
| 20120030553 | Delpha et al. | Feb 2012 | A1 |
| 20120041903 | Beilby et al. | Feb 2012 | A1 |
| 20120078891 | Brown et al. | Mar 2012 | A1 |
| 20120110473 | Tseng | May 2012 | A1 |
| 20120117005 | Spivack | May 2012 | A1 |
| 20120221502 | Jerram et al. | Aug 2012 | A1 |
| 20120245926 | Montyne et al. | Sep 2012 | A1 |
| 20120253825 | Di Fabbrizio et al. | Oct 2012 | A1 |
| 20120265528 | Gruber et al. | Oct 2012 | A1 |
| 20120284040 | Dupin | Nov 2012 | A1 |
| 20120311541 | Bullard et al. | Dec 2012 | A1 |
| 20130017523 | Barborak et al. | Jan 2013 | A1 |
| 20130031476 | Coin et al. | Jan 2013 | A1 |
| 20130046149 | Gettelman et al. | Feb 2013 | A1 |
| 20130152092 | Yadgar | Jun 2013 | A1 |
| 20130204813 | Master et al. | Aug 2013 | A1 |
| 20130254139 | Lei | Sep 2013 | A1 |
| 20130258040 | Kaytaz et al. | Oct 2013 | A1 |
| 20130262467 | Zhang et al. | Oct 2013 | A1 |
| 20130275875 | Gruber et al. | Oct 2013 | A1 |
| 20130283168 | Brown et al. | Oct 2013 | A1 |
| 20140029734 | Kim et al. | Jan 2014 | A1 |
| 20140040748 | Lemay et al. | Feb 2014 | A1 |
| 20140047001 | Phillips et al. | Feb 2014 | A1 |
| 20140053102 | Lee et al. | Feb 2014 | A1 |
| 20140074454 | Brown et al. | Mar 2014 | A1 |
| 20140098948 | Kulkarni et al. | Apr 2014 | A1 |
| 20140115456 | White et al. | Apr 2014 | A1 |
| 20140164476 | Thomson | Jun 2014 | A1 |
| 20140164508 | Lynch et al. | Jun 2014 | A1 |
| 20140181741 | Apacible et al. | Jun 2014 | A1 |
| 20140201675 | Joo et al. | Jul 2014 | A1 |
| 20140244266 | Brown et al. | Aug 2014 | A1 |
| 20140244712 | Walters et al. | Aug 2014 | A1 |
| 20140245140 | Brown et al. | Aug 2014 | A1 |
| 20140282109 | Wenger et al. | Sep 2014 | A1 |
| 20140297284 | Gruber et al. | Oct 2014 | A1 |
| 20140317502 | Brown et al. | Oct 2014 | A1 |
| 20140337048 | Brown et al. | Nov 2014 | A1 |
| 20140343924 | Brown et al. | Nov 2014 | A1 |
| 20140343928 | Brown et al. | Nov 2014 | A1 |
| 20140365223 | Brown et al. | Dec 2014 | A1 |
| 20140365407 | Brown et al. | Dec 2014 | A1 |
| 20150066817 | Slayton et al. | Mar 2015 | A1 |
| 20150185996 | Brown et al. | Jul 2015 | A1 |
| 20150186154 | Brown et al. | Jul 2015 | A1 |
| 20150186155 | Brown et al. | Jul 2015 | A1 |
| 20150186156 | Brown et al. | Jul 2015 | A1 |
| 20150363697 | Spivack | Dec 2015 | A1 |
| 20160012186 | Zasowski et al. | Jan 2016 | A1 |
| 20160110071 | Brown et al. | Apr 2016 | A1 |
| 20170132220 | Brown et al. | May 2017 | A1 |
| 20170277993 | Beaver et al. | Sep 2017 | A1 |
| Number | Date | Country |
|---|---|---|
| 103051669 | Apr 2013 | CN |
| WO2011088053 | Jul 2011 | WO |
| Entry |
|---|
| Non-Final Office Action for U.S. Appl. No. 12/564,546, dated Dec. 21, 2011, Tanya Miller et al., “Apparatus, System, and Method for Natural Language Processing”, 12 pages. |
| Final Office Action for U.S. Appl. No. 12/564,546, dated Feb. 26, 2013, Tanya Miller et al., “Apparatus, System, and Method for Natural Language Processing”, 15 pages. |
| Non-Final Office Action for U.S. Appl. No. 12/636,571, dated Apr. 12, 2013, Tanya Miller et al., “Leveraging Concepts With Information Retrieval Techniques and Knowledge Bases”, 31 pages. |
| Non-Final Office Action for U.S. Appl. No. 12/564,546, dated Jun. 12, 2013, Tanya Miller et al., “Apparatus, System, and Method for Natural Language Processing ”, 18 pages. |
| U.S. Appl. No. 13/774,381, filed Feb. 22, 2013, Fred A. Brown et al., “Interaction with a Portion of a Content Item through a Virtual Assistant,” 68 pages. |
| U.S. Appl. No. 13/774,519, filed Feb. 22, 2013, Fred A. Brown et al, “Virtual Assistant Transfer Smart between Devices,” 65 pages. |
| Final Office Action for U.S. Appl. No. 12/636,571, dated Nov. 7, 2013, Tanya Miller, “Leveraging Concepts With Information Retrieval Techniques and Knowledge Bases”, 31 pages. |
| Final Office Action for U.S. Appl. No. 13/341,261, dated Feb. 27, 2014, Fred A. Brown, “Providing Variable Responses in a Virtual-Assistant Environment”, 32 pages. |
| Office action for U.S. Appl. No. 13/341,261, dated Aug. 14, 2013, Brown et al., “Providing Variable Responses in a Virtual-Assistant Environment”, 22 pages. |
| Office action for U.S. Appl. No. 14/451,009, dated Jul. 15, 2016, Brown et al., “Wearable-Based Virtual Agents”, 7 pages. |
| Office action for U.S. Appl. No. 14/446,153, dated Aug. 25, 2016, Brown et al., “Conversational Virtual Healthcare Assistant”, 13 pages. |
| Office action for U.S. Appl. No. 14/293,529, dated Aug. 31, 2016, Brown et al., “Virtual Assistant Team Identification”, 18 pages. |
| Office action for U.S. Appl. No. 14/302,096, dated Oct. 8, 2014, Brown, “Active Lab”, 27 pages. |
| Office action for U.S. Appl. No. 12/636,571, dated Aug. 14, 2014, Miller et al., “Leveraging Concepts With Information Retrieval Techniques and Knowledge Bases”, 35 pages. |
| Office action for U.S. Appl. No. 14/467,221, dated Oct. 9, 2014, Brown, “Context-Based Virtual Assistant Conversations”, 24 pages. |
| Office Action for U.S. Appl. No. 14/293,529, dated Oct. 1, 2015, Fred A. Brown, “Virtual Assistant Team Identification”, 18 pages. |
| Office Action for U.S. Appl. No. 14/293,619, dated Aug. 13, 2015, Fred A. Brown, “Virtual Assistant Acquisitions and Training”, 17 pages. |
| Office Action for U.S. Appl. No. 13/341,261, dated Sep. 23, 2015, Fred A. Brown, “Providing Variable Responses in a Virtual-Assistant Environment”, 26 pages. |
| Office Action for U.S. Appl. No. 14/315,852, dated Sep. 24, 2015, Fred Brown, “Virtual Assistant Conversations for Ambiguous User Input and Goals”, 6 pages. |
| PCT Search Report and Written Opinion dated Sep. 2, 2015 for PCT Application No. PCT/US15/33594, 9 pages. |
| “Case Study With Alme, Alaska Airlines soars”, retrieved on Apr. 10, 2015 at <<http://www.nextit.com/media/downloads/Case-study-Alaska-Air.pdf>>, 3 pages. |
| “Frost & Sullivan Commends Next IT for Leading the Virtual Agent Applications Industry in Competitive Strategy Innovation”, Frost & Sullivan, Dec. 18, 2014, 5 pages. |
| “Meet Jenn, Your Virtual Assistant at alaskaair.com”, retrieved on Apr. 13, 2015 at <<http://www.alaskaair.com/content/about-us/site-info/ask-jenn.aspx>>, 1 page. |
| Office action for U.S. Appl. No. 14/293,586, dated Feb. 17, 2015, Brown et al., “Virtual Assistant Team Customization”, 11 pages. |
| Office action for U.S. Appl. No. 14/467,221, dated Feb. 17, 2015, Brown et al., “Context-Based Virtual Assistant Conversations”, 5 pages. |
| Final Office Action for U.S. Appl. No. 14/293,529, dated Feb. 23, 2015, Fred A. Brown, “Virtual Assistant Team Identification”, 17 pages. |
| Final Office Action for U.S. Appl. No. 14/315,852, dated Apr. 10, 2015, Fred Brown, “Virtual Assistant Conversations for Ambiguous User Input and Goals”, 18 pages. |
| Final Office Action for U.S. Appl. No. 14/293,619, dated Apr. 13, 2015, Fred A. Brown, “Virtual Assistant Acquisitions and Training”, 17 pages. |
| Final Office Action for U.S. Appl. No. 14/467,715, dated Apr. 16, 2015, Fred Brown, “Virtual Assistant Conversations”, 5 pages. |
| Office action for U.S. Appl. No. 13/341,261, dated May 21, 2015, Brown et al., “Providing Variable Responses in a Virtual-Assistant Environment”, 30 pages. |
| Final Office Action for U.S. Appl. No. 14/451,009, dated May 21, 2015, Fred Brown, “Wearable-Based Virtual Agents”, 10 pages. |
| Final Office Action for U.S. Appl. No. 12/636,571, dated Jun. 12, 2015, Tanya Miller, “Leveraging Concepts With Information Retrieval Techniques and Knowledge Bases”, 37 pages. |
| Final Office Action for U.S. Appl. No. 14/293,529, dated Jun. 15, 2015, Fred A. Brown, “Virtual Assistant Team Identification”, 16 pages. |
| Final Office Action for U.S. Appl. No. 14/293,586, dated Jul. 24, 2015, Fred A. Brown, “Virtual Assistant Team Customization”, 14 pages. |
| Office Action for U.S. Appl. No. 14/293,673, dated Jul. 24, 2015, Fred A. Brown, “Virtual Assistant Conversations”, 25 pages. |
| Final Office Action for U.S. Appl. No. 14/302,096, dated Jul. 29, 2015, Fred Brown, “Active Lab”, 7 pages. |
| Office Action for U.S. Appl. No. 14/302,096, dated Jan. 12, 2015, Fred Brown, “Active Lab”, 4 pages. |
| “SGT STAR Wins Intelligent Assistant Award”, San Francisco, Calif (PRWEB) Sep. 24, 2014, PRWEB Online Visibility from Vocus, 2 pages. |
| “TAM”, Case Study Meet Juli—TAM Airlines' most famous new hire, Next IT Corporation, 2 pages. |
| “The Armys Robot Recruiter”, Transcript from New York Public Radio, Aug. 8, 2014, 3 pages. |
| “AskJennMediaCoverage”, retrieved on Nov. 12, 2014, 76 pages. |
| Cassell, et al., “Embodied Conversational Agents”, MIT Press, 2000, pp. 272 and 275. |
| Office Action for U.S. Appl. No. 14/467,715, dated Oct. 1, 2014, Fred Brown, “Virtual Assistant Conversations”, 14 pages. |
| Office Action for U.S. Appl. No. 13/341,261, dated Nov. 6, 2014, Fred A. Brown, “Providing Variable Responses in a Virtual-Assistant Environment”, 26 pages. |
| Final Office Action for U.S. Appl. No. 14/293,673, dated Dec. 4, 2014, Fred A. Brown, “Virtual Assistant Conversations”, 22 pages. |
| Office Action for U.S. Appl. No. 14/315,852, dated Dec. 4, 2014, Fred Brown, “Virtual Assistant Conversations for Ambiguous User Input and Goals”, 15 pages. |
| Office Action for U.S. Appl. No. 14/451,009, dated Dec. 4, 2014, Fred Brown, “Wearable-Based Virtual Agents”, 9 pages. |
| Office action for U.S. Appl. No. 14/293,529, dated Sep. 10, 2014, Brown et al., “Virtual Assistant Team Identification”, 13 pages. |
| Office action for U.S. Appl. No. 14/293,619, dated Sep. 8, 2014, Brown et al., “Virtual Assistant Acquisitions and Training”, 15 pages. |
| Office action for U.S. Appl. No. 14/293,673, dated Sep. 8, 2014, Riegler et al., “Virtual Assistant Conversations”, 22 pages. |
| PCT Search Report and Written Opinion dated Nov. 12, 2014 for PCT Application No. PCT/US14/31047, 14 Pages. |
| “Undercover Virtual Agent Article”, KOMO News, retrieved Nov. 12, 2014, 2 pages. |
| Office action for U.S. Appl. No. 14/293,619, dated Feb. 26, 2016, Brown et al., “Virtual Assistant Acquisitions and Training”, 16 pages. |
| Office action for U.S. Appl. No. 14/293,529, dated Mar. 17, 2016, Brown et al., “Virtual Assistant Team Identification”, 19 pages. |
| Office action for U.S. Appl. No. 14/293,586, dated Mar. 17, 2016, Brown et al., “Virtual Assistant Team Customization”, 13 pages. |
| Office action for U.S. Appl. No. 12/636,571, dated Mar. 24, 2016, Miller et al., “Leveraging Concepts With Information Retrieval Techniques and Knowledge Bases”, 31 pages. |
| Office action for U.S. Appl. No. 13/341,261, dated Mar. 24, 2016, Brown et al., “Providing Variable Responses in a Virtual-Assistant Environment”, 30 pages. |
| Office action for U.S. Appl. No. 14/446,153, dated Mar. 25, 2016 Brown et al., “Conversational Virtual Healthcare Assistant”, 7 pages. |
| Final Office Action for U.S. Appl. No. 14/446,153, dated Apr. 29, 2015, Fred A. Brown, “Conversational Virtual Healthcare Assistant”, 9 pages. |
| Final Office Action for U.S. Appl. No. 13/449,927, dated Apr. 9, 2015, Fred A. Brown, “Conversation User Interface”, 35 pages. |
| Office action for U.S. Appl. No. 14/467,715, dated May 18, 2016, Brown et al., “Virtual Assistant Conversations”, 14 pages. |
| Office Action for U.S. Appl. No. 13/607,414, dated Jul. 21, 2015, Fred A. Brown, “Conversational Virtual Healthcare Assistant”, 25 pages. |
| Office Action for U.S. Appl. No. 13/449,927, dated Aug. 15, 2014, Fred A. Brown, “Conversation User Interface”, 29 pages. |
| Office Action for U.S. Appl. No. 14/446,153, dated Sep. 18, 2015, Fred A. Brown, “Conversational Virtual Healthcare Assistant”, 11 pages. |
| Office Action for U.S. Appl. No. 14/446,153, dated Sep. 26, 2014, Fred A. Brown, “Conversational Virtual Healthcare Assistant”, 7 pages. |
| Office Action for U.S. Appl. No. 13/341,261, dated Sep. 15, 2016, Fred A. Brown, “Providing Variable Responses in a Virtual-Assistant Environment”, 29 pages. |
| Office action for U.S. Appl. No. 14/293,673, dated Jan. 15, 2016, Brown et al., “Virtual Assistant Conversations”, 29 pages. |
| Office action for U.S. Appl. No. 14/451,009, dated Jan. 5, 2016, Brown et al., “Wearable-Based Virtual Agents”, 10 pages. |
| Office action for U.S. Appl. No. 14/467,221, dated Feb. 18, 2016, Brown et al., “Context-Based Virtual Assistant Conversations”, 14 pages. |
| Pandorabots Inc., “AIML Targeting: Supervised Learning for Bots”, uploaded on Oct. 29, 2009, at https://www.youtube.com/watch?v=aGe30NTVDOk. |
| Langkilde, Irene et al., “Automatic Prediction of Problematic Human-Computer Dialogues in ‘How May I Help You?’”, AT&T Labs Research, 1999, 5 pages. |
| Wikipedia page “CALO”, retrieved on Nov. 15, 2017 at <<https://en.wikipedia.org/wiki/CALO>>, 5 pages. |
| Supplementary European Search Report dated Oct. 31, 2016 for European Patent Application No. 14785575.3, 10 pages. |
| Extended European Search Report dated Mar. 17, 2017 for European patent application No. 14785575.3, 16 pages. |
| Guzzoni, Didier, et al., “Modeling Human-Agent Interaction with Active Ontologies” Spring 2007 AAAI Symposium, 8 pages. |
| Walker, Marilyn et al., “Learning to Predict Problematic Situations in a Spoken Dialogue System: Experiments with How May I Help You?”, AT&T Labs Research, NAACL 2000 Proceedings of the 1st North American chapter of the Association for Computational Linguistics conference, Seattle, Washington, Apr. 29-May 4, 2000, 8 pages. |
| Office Action for U.S. Appl. No. 14/293,529, dated Jan. 31, 2017, Fred A. Brown, “Virtual Assistant Team Identification”, 20 pages. |
| Office Action for U.S. Appl. No. 14/837,282, dated Jan. 20, 2017, Spivack, “System and Method for Providing Distributed Intelligent Assistance”, 16 pages. |
| Office Action for U.S. Appl. No. 14/293,619, dated Oct. 5, 2017, Fred A. Brown, “Virtual Assistant Acquisitions and Training”, 21 pages. |
| Office action for U.S. Appl. No. 14/293,619, dated Oct. 6, 2016, Brown et al., “Virtual Assistant Acquisitions and Training”, 17 pages. |
| Office Action for U.S. Appl. No. 13/271,175, dated Oct. 7, 2014, Nova T. Spivack, “System and Method for Providing Distributed Intelligent Assistance”, 14 pages. |
| Office action for U.S. Appl. No. 14/293,673, dated Nov. 1, 2016, Brown et al., “Virtual Assistant Conversations”, 34 pages. |
| Office Action for U.S. Appl. No. 14/980,388, dated Nov. 27, 2017, Brown, “Conversation User Interface”, 27 pages. |
| Office action for U.S. Appl. No. 14/467,221, dated Nov. 3, 2016, Brown et al., “Context-Based Virtual Assistant Conversations”, 15 pages. |
| Office action for U.S. Appl. No. 13/865,789, dated Nov. 7, 2016, Brown et al., “Virtual Assistant Focused User Interfaces”, 20 pages. |
| Office action for U.S. Appl. No. 14/467,715, dated Dec. 1, 2016, Brown et al., “Virtual Assistant Conversations”, 10 pages. |
| Office action for U.S. Appl. No. 12/636,571, dated Feb. 15, 2017, Miller et al., “Leveraging Concepts With Information Retrieval Techniques and Knowledge Bases”, 35 pages. |
| Office Action for U.S. Appl. No. 13/341,261, dated Feb. 7, 2017, Fred A. Brown, “Providing Variable Responses in a Virtual-Assistant Environment”, 34 pages. |
| Final Office Action for U.S. Appl. No. 14/293,673, dated Apr. 25, 2017, Fred A. Brown, “Virtual Assistant Conversations”, 32 pages. |
| Office action for U.S. Appl. No. 13/865,789, dated May 19, 2017, Brown et al., “Virtual Assistant Focused User Interfaces”, 22 pages. |
| Final Office Action for U.S. Appl. No. 13/271,175, dated May 30, 2014, Nova T. Spivack, “System and Method for Providing Distributed Intelligent Assistance”, 14 pages. |
| Office action for U.S. Appl. No. 14/293,619, dated May 4, 20017, Brown et al., “Virtual Assistant Acquisitions and Training”, 18 pages. |
| Non-Final Office Action for U.S. Appl. No. 13/271,175, dated Jul. 19, 2013, Nova T Spivack, “System and Method for Providing Distributed Intelligent Assistance”, 13 pages. |
| Office action for U.S. Appl. No. 14/467,221, dated Jul. 25, 2017, Brown et al., “Context-Based Virtual Assistant Conversations”, 15 pages. |
| Office Action for U.S. Appl. No. 14/467,715, dated Jul. 25, 2017, Brown et al., “Virtual Assistant Conversations”, 12 pages. |
| Office action for U.S. Appl. No. 14/293,586, dated Sep. 23, 2016, Brown et al., “Virtual Assistant Team Customization”, 9 pages. |
| Office Action for U.S. Appl. No. 14/302,096, dated Sep. 27, 2016, Brown et al., “Regression Testing”, 6 pages. |
| Office Action for U.S. Appl. No. 14/837,282, dated Oct. 20, 2015, Nova T. Spivack, “System and Method for Providing Distributed Intelligent Assistance”, 8 pages. |
| Krahmer, Emiel et al., “Problem Spotting in Human-Machine Interaction”, IPO, Center for Research on User-System Interaction, Sixth European Conference on Speech Communication and Technology, Sep. 5-9, 1999, Budapest, Hungary, 4 pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20090182702 A1 | Jul 2009 | US |
