A conventional process for developing a speech user interface (“SUI”) includes three basic steps. First, a SUI designer creates a human readable specification describing the desired SUI using drawings, flowcharts, writings, or other human-readable formats. Second, the SUI designer gives the specification to a code developer, who programs the application using an existing markup language, usually Voice eXtensible Markup Language (“VoiceXML” or “VXML”) or Speech Application Language Tags (“SALT”). The code developer simultaneously incorporates business logic that retrieves information from an outside database and brings it into the markup language file. Third, the coded application is tested by quality assurance (“QA”) to screen for errors. When QA finds errors, it reports back to the code developer, who either debugs the code and returns it to QA for further analysis, or, if the error lies within the SUI design, gives the specification back to the SUI designer for revision. The SUI designer then revises the specification and returns it to the code developer, who re-implements the SUI code simultaneously with the business logic and again returns the coded application to QA for further analysis. This process is repeated until QA determines that the product is suitable for final release.
Thus, in many commercial frameworks that use existing markup languages like VXML or SALT, SUI logic is mixed with business logic in speech applications. Mixing SUI logic with business logic increases the chances of bugs in speech applications, which may diminish the quality of the speech applications through inconsistent speech user interface behavior and lengthened development, QA and release cycles.
One problem with this approach is that it leads to inefficient development. After the code developer implements the application, the design goals produced by the designer and the final implemented SUI are rarely the same thing. To illustrate why this happens, consider the following: when humans play the game of telephone, one person speaks a message to another, who repeats the message to a third, and so on, until the message is invariably altered due to the imperfection of human verbal interaction. Such is also the case in the above described software development cycle. The SUI designer, ideally a non-programmer who specializes in human interactions, has one idea for the project which he or she communicates to development in the form of human-readable text, flowcharts, figures or other methods. The code developer attempts to implement precisely the designer's ideal from the technical perspective of a programmer, who must simultaneously implement the required business logic underlying the application. Thus, the code developer's output is inevitably altered from the idea of the designer.
Another problem with the existing approach is that the mixture of business logic and SUI logic makes transparent SUI design impossible. In existing commercial frameworks, the SUI designer's SUI often takes the form of either: (1) a rough prototype for the SUI consisting of human-readable text, flowcharts, or other types of diagrams, such as those created with software such as Visio® by Microsoft Corporation, which a developer must then implement; or (2) an exact design for which code is automatically generated. The first approach creates the telephone-game problems of inefficient development outlined above. The second approach inevitably includes business logic, which means the SUI designer must have some idea of how the underlying business logic will work, making the SUI non-transparent for a non-programmer.
Yet another inefficiency arises if either the SUI design goals change or if QA finds an error in the SUI design at any time during the life cycle of the speech application. Either scenario requires that the code developers pass control of the project back to the designer for SUI revision. Once the designer corrects the error, he or she must return the revised SUI design to the code developer, who must once again re-implement it along with the business logic, which may have not even had errors by itself. Thus, in the prior art, if there is an error, regardless of whether it is found in business logic or SUI logic, the code developer must debug the entire application, not just the SUI or the business logic, in order to find and eliminate the bug. This repeated implementation and integration of the business logic with the SUI logic drives up labor and development costs and lengthens development and release cycles.
Still another problem with the traditional three-step development process is that it leads to inefficient testing. It is virtually impossible to machine-read existing markup languages like VXML or SALT and derive any knowledge about the SUI, because it is mingled with business logic which cannot be easily parsed out. In most applications, the markup language is not static but rather is generated on the fly as the application is running. This makes it virtually impossible to automate SUI testing in all the existing markup languages. QA testing of the final release thus involves simultaneous testing of business and SUI logic, a process incapable of automation. This means that more labor is required to perform comprehensive quality assurance, which increases the cost of QA labor and again lengthens the overall release cycle.
Accordingly, there is a need for an improved method of SUI design, testing and deployment which is built around a fundamental separation of the logic behind the SUI and any other business logic.
In accordance with one aspect of the present invention, a method for developing a speech application is provided. The first step of the method is creating a speech user interface description devoid of business logic in the form of a machine readable markup language directly executable by a runtime environment based on business requirements. The second step of the method is creating separately at least one business logic component for the speech user interface, the at least one business logic component being accessible by the runtime environment.
In accordance with another aspect of the present invention, a system for developing a speech application is provided. The system includes a runtime environment and a speech user interface description devoid of business logic in the form of a machine readable markup language directly executable by the runtime environment based on business requirements. The system further includes at least one business logic component for the speech user interface, the at least one business logic component being accessible by the runtime environment.
The present invention separates SUI logic from business logic by utilizing a markup language and markup language interpreter combination that, aside from a few exceptions, completely controls the SUI logic. SUI logic is defined in the present invention as any logic directing the interaction between the caller and the interactive voice response (“IVR”) system, subject to possible limited business logic overrides. It includes dialogs, grammars, prompts, retries, confirmations, transitions, overrides, and any other logical tool directing the human-machine interaction. Business logic is defined as any logic outside the realm of SUI logic. It includes data pre-processing actions (e.g., checking the validity of the phone number entered), the actual database query formation and data retrieval, any possible post-processing of the data returned, and any other logic not directing the human-machine interaction. Markup language should be understood to mean any machine-readable language that abstracts the presentation or layout of the document; in other words, a markup language will separate the structure and appearance of a file as experienced by a user from its content.
The present invention utilizes a markup language that abstracts out or automates all of these different actions. The result is a separation of SUI logic from business logic. The markup language interpreter is capable of fully controlling the SUI, aside from possible overrides from the business logic. The interpreter sends requests for data or user commands to the business logic, and the business logic returns either the requested data, or error messages giving one of a plurality of reasons for the failure. The user commands may be given in the form of speech commands, DTMF or touch-tone commands, touchpad or mouse commands, keyboard or keypad commands, or through drag-and-drop commands of a graphical user interface (“GUI”). The interpreter similarly interacts with speech recognition (or DTMF recognition) engines by sending user inputs, and the interpreter receives in return values indicating either a match, no match, or no response.
By making this fundamental separation, the present invention seeks to accomplish the following objectives: (1) solve the “telephone” problem inherent in traditional IVR application design; (2) make SUI transparent for a non-programmer; (3) provide for separate debugging and revision of SUI logic and business logic, which allows each respective team to solely focus on their areas of expertise; and (4) allow for automated testing of the SUI logic.
The first objective of the invention is to eliminate the “telephone” problem by allowing the SUI designer complete control of the SUI, from start to finish. The SUI's output does not need to be coded but rather is ready to execute, eliminating the inevitable confusion created in the prior art where the SUI design was implemented by code developers. This means that the code developers will not be required to implement the SUI designer's idea of the SUI, an area in which the code developers are not likely trained.
The second objective of the present invention is to make the SUI transparent to a non-programmer. The markup language used in the present invention makes it possible for the SUI designer to create the SUI without any knowledge of programming or the underlying business logic whatsoever. It allows the designer to include simple placeholders which should be “filled” with caller-requested information, instead of requiring the designer to include server-side scripts or other types of business logic.
Because of the static nature of the present invention's markup language, design of such interfaces using a toolkit is simple and intuitive. In one embodiment of the invention, this toolkit is an application consisting of a GUI and underlying logic, which allows non-technically trained personnel to drag and drop various dialog elements into a what-you-see-is-what-you-hear environment. The designer will be able to specify placeholders, transitions, prompts, overrides and possible commands available with each dialog. When the designer saves his or her work, the output is in the form of a markup language that, unlike VXML or other similar voice markup languages, is a self-contained static flowchart description. The preferred output format is in Web Speech Dialog Markup Language (“WSDML”), an XML-based language developed by Parus Interactive and owned by Parus Holdings, Inc.
Although the prior art contains numerous similar-appearing development toolkits, e.g., U.S. Pat. No. 5,913,195 to Weeren et al., the disclosed toolkit is the only development toolkit capable of producing markup language describing a completely autonomous SUI independent from any business logic. The toolkits contained in the prior art similarly implement intuitive GUIs, but they require programming knowledge on the part of the designer because he or she must indicate exactly how the SUI will interact with the business logic. These are complex tools, which require a trained programmer to use. The present invention avoids this problem by eliminating any requirement of business logic knowledge on the part of the SUI designer. Thus, it is possible for a non-technical person to create and revise SUIs. Conversely, code developers (i.e., business logic programmers) are able to focus almost exclusively on business logic development and are not required to have any knowledge of SUI design.
The third objective of the invention is to improve the development process by way of separation of business logic error correction and SUI logic error correction. Whatever the designer creates will be machine-readable, which means that it can immediately be tested and/or run by a machine, eliminating the delay of waiting for the code developers to implement the SUI, as required by the prior art. Further, if an error is detected in the SUI design, or if QA simply decides that there is a better way to design the interface, feedback goes straight to the designer, who can immediately fix any problems and resubmit the SUI. This separate development of the SUI will not interfere with the code developers, who only need to know the position of placeholders within the SUI design. Likewise, if QA finds a business logic error, QA only needs to tell the code developers, who will fix any problems without touching the SUI.
The fourth objective of the present invention is to automate testing. QA personnel have the ability to completely automate testing of the SUI. Once they create the test cases (or the test cases are defined for them), QA personnel only need to initiate the automated testing, and then they are free to test other aspects of the project, such as the business logic. QA also benefits in that they only need to communicate SUI problems to the SUI designer, and business logic problems to the code developers.
The present invention will now be described more fully with reference to the Figures in which the preferred embodiment of the present invention is shown. The subject matter of this disclosure may, however, be embodied in many different forms and should not be construed as being limited to the embodiment set forth herein.
Referring now to the drawings, wherein like reference numerals designate identical or corresponding parts throughout the several views,
On the business logic side, the code developers 18 receive the same markup language code 20 containing the placeholders and possible user commands. The job of the code developers 18 is to return the appropriate values for the placeholders. This one-way communication from the SUID 12 to the code developers 18 is illustrated further in
Once the SUID 12 and the code developers 18 complete their respective pieces of the final product, they give their work to quality assurance (QA) 24 for testing and debugging. The SUI toolkit 14 outputs machine-readable markup language 16, such as WSDML as shown in
This is one place where the present invention diverges greatly from the prior art. While any markup language that entirely separates the business logic from the SUI logic as described thus far would suffice, the preferred embodiment utilizes Web Speech Dialog Markup Language (“WSDML”), an XML-based language developed by Parus Interactive and owned by Parus Holdings, Inc. WSDML introduces elaborate dialog automation and dialog inheritance at the WSDML interpreter level. One of the main differences between WSDML and VXML (or any other existing speech markup languages, such as SALT) is that WSDML describes both individual dialogs and transitions between them. For instance, VXML does not provide for robust dialog transitions beyond simple form filling.
VXML was developed as a very web-centric markup language. In that way VXML is very similar to the Hyper Text Markup Language (“HTML”) in that HTML applications consist of several individual web pages, each page analogous to a single VXML dialog. To illustrate this difference, VXML will first be analogized to HTML, and then WSDML will be contrasted to both VXML and HTML.
In a typical HTML/web-based scenario, if a user wanted to log onto a bank's website, the user first is presented with a simple page requesting a bank account number, which provides a field or space for the user to input that information. After entering the account number, the user's input is submitted to the web server. With this information, the web server executes a common gateway interface (“CGI”) program, which uses the user input to determine what information, typically in the form of a user interface coupled with the desired information, should be presented to the user next. For instance, if the user gives an invalid account number, the CGI program will discover the error when it references the input to the bank's database. At this point, based on a negative response from the bank's database, the CGI program produces output which, through the web server, presents the user with a webpage, such as a page displaying the message “Invalid Account Number. Please try again.”
VXML operates in a similar fashion. The same user, this time using a telephone to access the bank's automated system, is presented with an audio prompt asking, “Please enter or speak your account number.” After speaking or inputting via DTML the account information, the VXML browser interacts with separate speech (or DTML) recognition software to determine whether the input satisfies the present grammar, or a finite set of speech patterns expected from the user. Then, the VXML browser sends this input to the VXML server or a web server capable of serving VXML pages. The VXML server tests the input against the bank's remote database using CGI and determines what information, in the form of a user interface coupled with the desired information, should be presented to the user next. For instance, if the user gives an invalid account number this time, the CGI program again receives a negative response from the bank's database. The CGI program then sends a VXML page to the web server, which transmits this page to the VXML browser and, in turn, prompts the user with an audio response, such as: “The account number was invalid. Please try again.”
WSDML operates differently. The SUI is static; it does not depend on the information returned from the CGI. Instead, the SUI is automated at the WSDML interpreter (or WSDML browser) level. The WSDML interpreter does not need to run any CGI scripts (or have an adjunct script interpreter run subscripts) to determine what to do next, as the typical CGI setup does. In that sense, the WSDML file serves as a comprehensive static flowchart of the conversation.
To illustrate, the same user once again calls a bank to access an automated telephone system, which this time utilizes a WSDML-based IVR system. The system may prompt the caller with an audio message, such as: “Please enter or speak your account number.” The caller speaks (or dials) his account number, which is processed by a separate speech (or DTMF) recognition engine against a grammar. Provided the spoken input satisfies the grammar, the WSDML interpreter makes a simple request to the business logic containing the account number instead of running a CGI program with the account number as input. And, instead of a CGI program determining whether the account number matches the bank's database or what the caller will be presented with next, the business logic simply returns values to the WSDML interpreter indicating whether the command was valid and, if so, the requested information. Based on this return value, the WSDML interpreter decides what to present to the user next. Using the same example, if the account number given to the business logic is invalid despite satisfying the grammar, the business logic returns an error indicating that the input was invalid, and a separate reason for the error. The WSDML outcome, which up to this point has been “MATCH” as a result of the satisfaction of the grammar, is converted to “NOMATCH,” and the WSDML interpreter continues to another dialog depending on the reason for the invalidity.
Additionally, in an embodiment of the present invention, the markup language allows for dialog inheritance or templates, meaning that the user may create top level dialogs that operate similarly to high-level objects in object-oriented programming. Lower level objects inherit common properties from the top level objects. In this way the top level dialogs operate as “templates” for the lower dialogs, allowing for global actions, variables, and other dialog properties.
In another embodiment of the invention, SUI testing is automated as shown in
In another embodiment of the present invention as shown in
In yet another embodiment of the present invention, the interpreter consists of a computer program server that takes as input WSDML files and, using that information, conducts “conversations” with the caller. The interpreter may obtain the WSDML files from a local storage medium, such as a local hard drive. The interpreter also may obtain the WSDML files from a remote application server, such as a web server capable of serving XML-style pages.
In still another embodiment of the present invention, the interpreter has built-in individual business logic functions for each possible user request. For example, the code developers program “black box” functions that simply take as input the user's account number, and return the information the user requests, such as the user's account balance or the user's credit limit. These functions reside in entirely separate locations from the interpreter code that interprets and serves WSDML dialog to the user.
In still yet another embodiment of the present invention, the interpreter is implemented as a library for an application. In this scenario, the application provides the WSDML server with “hooks,” or callback functions, which allow the interpreter to call the given business logic function when necessary. The application server similarly provides “hooks” for when the caller instigates an event, such as a user command.
WSDML Dialog Concept:
WSDML is, to at least some extent, an expression of the WSDML Dialog concept. WSDML Dialog (“dialog”) describes a certain set of interactions between the caller and the voice application over the telephone. A dialog ends when one of defined outcomes is detected based on the caller's input; at that point it is ready to proceed to the next dialog. A dialog may pass a certain number of intermediate states based on a preset counter, before it arrives to an acceptable outcome. The main dialog outcomes explicitly defined in WSDML are: “No Input,” “No Match,” and “Match.” Dialog error outcomes caused by various system failures are handled by the corresponding event handlers, and any related error announcements may or may not be explicitly defined in WSDML. A single dialog interaction normally is accomplished by a single Play-Listen act when the application plays a prompt and listens to the caller's input. This general case of interaction also covers various specific interaction cases: play-then-listen (with no barge-in), pure play, and pure listen. The notion of “listen” relates to both speech and touch-tone modes of interaction.
Following are the steps of the dialog process, which are shown in
(a) A dialog starts with the initial prompt presented to the caller.
(b) The caller's input is collected and the result is processed.
(c) The outcome is determined and the next prompt is set accordingly.
(d) Depending on the outcome and the preset maximum number of iterations, the next caller interaction (perhaps playing a different prompt) within the same dialog is initiated, or control is passed to the next dialog.
(e) The dialog may include a confirmation interaction. In this case, if low confidence is returned as part of the interaction result, then the outcome is determined by the result of the confirmation dialog. Irrespective of the confirmation result and the subsequent outcome, control is always passed to the next dialog after the confirmation.
Table 1 below describes possible dialog outcomes:
*“ALL” is not necessarily a dialog outcome, but may be used to initiate an action based on all possible outcomes. For example, when “ALL” is specified, the same action is taken for a dialog outcome of “MATCH,” “NO MATCH,” or “NO INPUT.”
WSDML Structure:
A WSDML document is organized via <wsdml> element:
Structurally, a WSDML document includes the following major groups:
Other currently defined elements are used within the WSDML groups defined above.
Dialog element: <dialog>.
Prompt elements: <prompts>, <prompt>, <audio>.
Input elements: <grammar-source>, <slots>, <slot>, <commands>, <command>, <dtmf-formats>, <dtmf-format>.
Transitional elements: <actions>, <action>, <goto>, <target>, <return>.
Logic elements: <if>/<elseif>/<else>, <vars>, <var>.
WSDML Elements
This section provides detailed information about each WSDML element including:
(a) Syntax: how the element is used.
(b) Description of attributes and other details.
(c) Usage: information about parent/child elements.
(d) Examples: short example to illustrate element usage.
The present application claims the benefit of priority to U.S. Provisional Patent Application No. 60/664,025 filed Mar. 22, 2005, U.S. Provisional Patent Application No. 60/697,178 filed Jul. 7, 2005, and U.S. Provisional Patent Application No. 60/703,596 filed Jul. 29, 2005, all of which are hereby incorporated by reference in their entirety.
Number | Date | Country | |
---|---|---|---|
60664025 | Mar 2005 | US | |
60697178 | Jul 2005 | US | |
60703596 | Jul 2005 | US |