Patent Application
20160188150
The present application relates to speech technology, specifically the area of Natural Language Understanding (herein after “NLU”) and text interpretation. Currently, the design and implementation of an NLU engine is reserved for the highly-trained and specialized. Thus, the cost and time for implementing an NLU engine in a device (such as a simple consumer device) is prohibitively high.
A graphical user interface of a web-based toolkit application for a cloud-based NLU engine is a drag-and-drop toolkit application for building NLU contextual recognition models. Using the graphical user interface, users who do not have expertise in NLU and text interpretation can focus on building customized NLU contextual recognition models to enable human (end-user) interaction with electronics and software applications (herein after the “associated client applications”) without requiring expert programming skills or prior in-depth knowledge of NLU. The NLU models provide an intelligent, natural conversational speech and/or text interface for end-users of electronic devices and software applications.
The toolkit's graphical user interface features a series of interactive development, live-testing, and deployment windows including a sandbox for building a hierarchical framework, a set of configuration menus, and a set of development, live testing, and deployment command buttons. Through this intuitive graphical user interface, a user who is a novice in the areas of NLU and text interpretation, can easily configure the three main components of an NLU engine, namely the Parser, the Dialog Manager, and the Prompt Generator, without knowing what they are, what each does or the interactions among these components, to yield a working, customized NLU model. The functions of the three main components are as follows:
The Parser spots and collects data in the form of keyword values identified by the user for each transaction and passes them to the Dialog Manager.
The Dialog Manager analyses the data collected by the Parser by checking it against the NLU model definition. The user can define conditions through the toolkit's user interface such that Dialog Manager and the Prompt Generator can determine the appropriate static or context-sensitive response message to the end-user of the associated client application. The Dialog Manager transfers the prompt messages and the end-user choices that comply with the NLU model definition to the associated client application for processing and storage.
The Prompt Generator receives data from the Dialog Manager and uses it to generate the system response message containing static and context-sensitive content for the end-user of the associated client application. Upon generating the prompt messages, the Prompt Generator returns the prompt messages to the user of the client application.
The present invention is in the field of Natural Language Understanding (NLU) speech technology. The invention is the graphical user interface of a web-based drag-and drop toolkit application for building NLU contextual recognition models; these NLU models provide an intelligent, natural conversational speech and/or text interface for end-users of electronic devices and software applications.
The toolkit's graphical user interface is a series of interactive windows featuring a drag-and-drop sandbox for constructing the framework of the NLU model, a set of configuration menus, and a set of command buttons for development, live testing, and deployment functions. The design of the toolkit's graphical user interface integrates the underlying NLU engine such that a novice user without a background in NLU and text interpretation, can easily and quickly build NLU contextual recognition models that are customized for the associated application. The NLU Engine's main components are the Parser, the Dialog Manager, and the Prompt Generator.
In order to use the toolkit's graphical user interface to build a customized NLU contextual recognition model, the user is only required to determine what end-user transactions in the associated client application need to be interpreted, either as menu navigation or form data capture transactions, plus the governing conditions, including the keyword values that activate each transaction. Unlike many other NLU engines, keyword values are used rather than training data. With this information in hand, the user is ready to use the web-based toolkit's user interface.
The user then uses drag-and-drop actions in the graphical user interface's sandbox to model these transactions as a hierarchical framework of nodes; each path through the framework defines a transaction choice available to the associated client application's end-user. Upon defining each node, the user uses click or click-and-add actions in a set of menus provided in the graphical user interface to configure conditions and keyword values. When modeling is complete, the user can live test and deploy the NLU model to the cloud repository by clicking buttons provided in a toolbar. An API enables two-way communication between the deployed NLU model and the associated client application.
These and other features of the invention would be better understood from the following specifications and drawings, the following of which is a brief description.
The drawings that accompany the detailed description can be briefly described as follows:
If the NLU model 20 definition includes prompt messages, the dialog manager 14 communicates with the prompt generator 16 and provides the prompt generator 16 with data to build the NLU system message. The prompt generator 16 returns an appropriate prompt message to the dialog manager 14. The dialog manager 14 returns the appropriate prompt message to the user in an NLU engine response 18. In one example, the NLU engine response 18 is a voice utterance through a text-to-speech engine. In another example, the NLU engine response 18 is a text response in a text display. The appropriate prompts guide the user of the cloud-client application 30 towards providing data required by the cloud-client application 30. The appropriate prompts also instruct courtesy messages, such as greeting messages. In one example, the appropriate prompts can be configured for the NLU model 20 for general transactions. In another example, the appropriate prompts can be configured for the NLU model 20 for selected transactions.
A layout selection button 62, wherein the layout button refreshes and re-arranges a graph of nodes 91 located in the sandbox 88.
A center selection button 64, wherein the center selection button re-centers the graph of nodes 91.
A zoom in selection button 66 and a zoom out selection button 68, wherein the zoom in button 66 increases the zoom level for viewing the graph of nodes 91 and the zoom out button 68 decreases the zoom level for viewing the graph of nodes 91.
An export image selection button 70, wherein the export image selection button 70 exports the graph of nodes 91 as a separate file.
A save selection button 72, wherein the save selection button saves the current NLU model 20.
An undo selection button 74, wherein the undo selection button 74 un-does the last action performed.
A redo selection button 76, wherein the undo selection button 76 re-does the last action performed.
A test selection button 78, wherein the test selection button 78 live-tests the NLU model 20.
A deploy selection button 80, wherein the deploy selection button 80 deploys the NLU model 20 to the NLU cloud 28.
An insert project selection button 82, wherein the insert project selection button 82 inserts the graph of nodes 91 from one NLU model 20 into a current NLU model 20.
A commands selection button 84, wherein the commands selection button 84 displays an edit commands configuration window (not shown). The edit commands configuration window allows the user to control actions which can be configured by the cloud-client application's user. In one example, the user may want to enable a universal command allowing the user to discard previous selections chosen and start anew from the beginning of a client application. The user would use the edit commands configuration window to enable a restart command. In the same example, the user may also add words to trigger a restart command.
A quick guide selection button 86, wherein the quick guide selection button 86 displays a user help information. In one example, the quick guide selection button 86 displays user help information directly on the graphical user interface display 49.
A set of configuration menus 93 is placed adjacent the sandbox 88, as shown in
Node menu 90 contains configurable properties of a node N selected in the sandbox 88.
button 106 to add a keyword value.
button 108 to add a file containing a list
entities showing a defined value in
Descendants menu 92 containing the configurable properties of the descendants of the node N selected in the sandbox 88.
Advanced menu 94 containing advanced configuration options for the node N selected in the sandbox 88.
In the prompts menu 96, the user can specify the NLU system message content of either a static or context-sensitive prompt for the node N selected in the sandbox 88.
A user must perform preparatory tasks before building an NLU model 20 in the graphical user interface display 49. The preparatory tasks determine information requirements, structuring, and presentation. In one example, the information requirements, structuring, and presentation include menu navigation, logical node structures, conditions, and selection values. In the same example, the preparatory tasks include:
Reviewing the client application to decide what transactions need to be interpreted by the NLU engine 20.
Analyzing the transactions to identify and classify information needed from the client application's user.
Identifying the various options, the selection values for each option, and the conditions to be placed on each option for each “information chunk” from the identification and classification task.
Decide, if any, which NLU response messages the users will receive to facilitate the completion of the transactions (prompts to inform the users if they have made incorrect or incomplete selections, to confirm end-user selection actions or selection choices, courtesy messages, etc.)
Once the user completes the preparatory tasks, the user may then create the NLU model 20 within the graphical user interface 49. The user being with the start node 89, and creates a graph of nodes 91. In one example, each node in the graph of nodes 91 represents a menu navigation node. In another example, each node in the graph of nodes 91 represents a form data capture function node corresponding to the cloud-client applications requirements. In one example, the menu navigation nodes indicate the status of the user decisions. In another example, the form data capture nodes serve as information storage. In the same example, menu navigation nodes specify a unique path to accomplish a transaction. In still the same example, the form data capture nodes provide an optional path selection for the user. In one example, the menu nodes are represented by an ellipse shape. In the same example, the form data capture nodes are represented by an oval shape.
The user then builds the NLU model at step 22 directly in the sandbox 88. The user begins with the start node 89. A mode of the start node 89 can be specified upon project initiation by the user. The mode of any node can be changed at any time in the node menu 90. To add one or more descendant nodes from the descendant menu 92 to the start node 89 and related connectors, the user selects the start node 89 and clicks a plus button downstream of the start node 89.
All other descendant nodes are created in this manner to represent the client application's transactions. The user can connect any two nodes A and B to represent a relationship by clicking and dragging one of the four ports in node A to node B. To delete an unwanted node, the user selects it and clicks the − (minus) button on its upper edge. To delete a connector between two nodes to remove an unwanted relationship, select the unwanted connector and click Delete.
The NLU model application assigns defaults for various properties which can be over-ridden using the four configuration menus (
The user live-tests the NLU model with the NLU Engine in the cloud using test phrases, either through voice (requires a connected microphone) or text input. A list box 118 contains a selection of speech recognizers. A clickable microphone icon 120 is provided to indicate the start and end of the test phrase spoken into the microphone connected to the computer. The user can type the test phrase into a text entry box 112.
Once the spoken or typed test phrase is entered, the user clicks the Understand button 126 to ask the NLU engine to interpret and process the test phrase. The GUI provides a Start Over button 124 for users who choose to close the current test session and then initiate a new test session rather than continue testing phrases in the current session.
The example GUI in
The NLU Engine Response (also as 18 in
The graph of nodes in the sandbox 88 displays one of three possible responses:
When the user is satisfied with the responses (e.g. all of the nodes are successfully tested), the user can implement the NLU model in the user's application.
In accordance with the provisions of the patent statutes and jurisprudence, exemplary configurations described above are considered to represent a preferred embodiment of the invention. However, it should be noted that the invention can be practiced otherwise than as specifically illustrated and described without departing from its spirit or scope.
| Number | Date | Country | |
|---|---|---|---|
| 62020780 | Jul 2014 | US |
