Interactive voice response (IVR) systems are increasingly being used in a variety of applications, for example in telephony based applications such as call answering systems, voice navigated systems, automated call generation systems, etc. Testing of IVR products used in IVR systems is common for purposes of evaluating, debugging and otherwise improving the systems. Automation of this testing is also finding increasing use.
The challenges associated with automation of IVR system testing are significant and encompass a variety of different tasks. For example, simulation of incoming and outgoing telephone calls (a.k.a. call control), including dialing, disconnecting, holding, forwarding, etc., presents one set of challenges. Another set of challenges relates to controlling the interactive stimuli and response of the “end user” once the call has been connected. This includes, but is not limited to, recognition of Fax calls, DTMF tones, and human voice.
Addressing these tasks in a fashion that controls/utilizes the network topology in which the two endpoints (the IVR system and the “end user”) are connected presents another set of challenges. The possibilities for this are vast. Typically, the IVR side of the communication is well understood, but the ‘end user’ side may be difficult to automate (e.g., a piece of OEM hardware that has a telephone connector as it's only input interface). It is a significant challenge to create deterministic test cases, in environments in which the matrix of network topology configurations can very quickly become inapproachable, without having to create test case code that is specific to each configuration.
The discussion above is merely provided for general background information and is not intended to be used as an aid in determining the scope of the claimed subject matter.
To address some challenges associated with testing interactive voice response (IVR) systems in environments where the network topology configurations can be vast, a network independent computer-implemented method of executing test cases on IVR systems is provided. A test driver and a call controller residing on a test driver virtual machine are used to configure one or more IVR virtual machines to participate in a test case. User simulators on each IVR virtual machine are configured and loaded by the IVR product under test, which also resides on the IVR virtual machine. Under control of the call controller, the IVR virtual machines establish a call connection between their respective IVR products over a telephony network. The call controller then synchronously sends commands to the user simulators on each IVR virtual machine to control execution of the test case using the call connection. Execution of the test case is monitored by the test driver on the test driver virtual machine to evaluate IVR product performance.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. The claimed subject matter is not limited to implementations that solve any or all disadvantages noted in the background.
As noted, interactive voice response (IVR) systems are finding increasing use in a variety of applications. Automating tests of IVR systems in a manner that does not require that test code be written specifically for each possible configuration is challenging. To address this issue, disclosed embodiments include methods and systems which automate IVR products and the network topology that makes up the end-to-end system, simultaneously. The disclosed embodiments provide the ability to create an automated test case that interacts with the IVR system, and also allow the same test to be repurposed to an entirely different network topology.
The disclosed IVR test case execution methods, apparatus and systems can be embodied in a variety of computing environments, including personal computers, hand held computers, laptop computers, notebook computers, server computers, etc. Before describing the embodiments in greater detail, a discussion of an example computing environment in which the embodiments can be implemented may be useful.
The illustrated embodiments are operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, and/or configurations that may be suitable for use with the illustrated embodiments include, but are not limited to, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, telephony systems, distributed computing environments that include any of the above systems or devices, and the like.
The illustrated embodiments may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The illustrated embodiments may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communication network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices. Tasks performed by the programs and modules are described below and with the aid of figures. Those skilled in the art can implement the description and figures provided herein as processor executable instructions, which can be written on any form of a computer readable medium.
With reference to
Computer 110 typically includes a variety of computer readable media. Computer readable media can be any available media that can be accessed by computer 110 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer readable media may comprise computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computer 110. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of any of the above should also be included within the scope of computer readable media.
The system memory 130 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 131 and random access memory (RAM) 132. A basic input/output system 133 (BIOS), containing the basic routines that help to transfer information between elements within computer 110, such as during start-up, is typically stored in ROM 131. RAM 132 typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 120. By way of example, and not limitation,
The computer 110 may also include other removable/non-removable volatile/nonvolatile computer storage media. By way of example only,
The drives and their associated computer storage media discussed above and illustrated in
A user may enter commands and information into the computer 110 through input devices such as a keyboard 162, a microphone 163, and a pointing device 161, such as a mouse, trackball or touch pad. Other input devices (not shown) may include a joystick, game pad, satellite dish, scanner, or the like. These and other input devices are often connected to the processing unit 120 through a user input interface 160 that is coupled to the system bus, but may be connected by other interface and bus structures, such as a parallel port, game port or a universal serial bus (USB). A monitor 191 or other type of display device is also connected to the system bus 121 via an interface, such as a video interface 190. In addition to the monitor, computers may also include other peripheral output devices such as speakers 197 and printer 196, which may be connected through an output peripheral interface 195.
The computer 110 is operated in a networked environment using logical connections to one or more remote computers, such as a remote computer 180. The remote computer 180 may be a personal computer, a hand-held device, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer 110. The logical connections depicted in
When used in a LAN networking environment, the computer 110 is connected to the LAN 171 through a network interface or adapter 170. When used in a WAN networking environment, the computer 110 typically includes a modem 172 or other means for establishing communications over the WAN 173, such as the Internet. The modem 172, which may be internal or external, may be connected to the system bus 121 via the user input interface 160, or other appropriate mechanism. In a networked environment, program modules depicted relative to the computer 110, or portions thereof, may be stored in the remote memory storage device. By way of example, and not limitation,
Referring now to
As shown in
The test driver virtual machine 205 includes a test case definition file 225, a test driver or process 230, and a call controller 235. Test case definition file 225 contains the configuration information (phones numbers, physical machine names, etc) of the participants in the call to be simulated as well as the set of actions defining what is to occur during the call. The definition file can in some embodiments specify one or more calls to occur either in parallel or serially. Each call may utilize the same set of test case actions or each call may have differing sets of actions. The test driver process 230 is responsible for hosting the call controller 235 and parsing the test case definition file 225, then performing the activity specified in the file. Call controller 235 is configured to be the synchronization point between the test driver 230 and the user simulators 240-1 and 240-2 (described below) participating in the call. Call controller 235 also distributes the configuration information for the run to the virtual machines 210 and 215.
Although IVR virtual machines 210 and 215 (designated IVR Virtual Machine #1 and IVR Virtual Machine #2) will typically be configured differently during the test case execution, they share common components which are described below. When discussing the components of virtual machines 210 and 215 generically, the common reference numbers are used to describe the components which reside in each system (e.g., 240), but when differences are being described, the more specific designations are used (e.g., 240-1 and 240-2).
At least one IVR virtual machine is required in embodiments of the system 200. In some cases, only one is useful when an actual user or an IVR application interacting with the IVR product and the test driver 230 is desired. An IVR application is any application built upon an IVR system that can accept or make phone calls (e.g., via Voice over Internet Protocol or VOIP, via Public Switched Telephone Network or PSTN, or via other related technologies) which interacts with users by voice. In other embodiments, two IVR virtual machines are employed to implement a fully automated end-to-end test case. In various embodiments, a “User simulator” can be 1) an actual user, 2) a test hook library (as diagramed in 240-1) or 3) an IVR application. The design of these embodiments handles controlling either one party or both parties in a call. In other words, at least one of the parties needs to be the test hook library.
Each IVR virtual machine includes a user simulator 240. The user simulator can be implemented as an application for the IVR system or as a shim on top of the IVR work engine or product 255. The user simulator 240 of each IVR virtual machine is coupled to the call controller 235 of the test driver virtual machine, and is configured with the functionality to call into the action library 250. A configuration information file 245 is used to store physical machine locations for the call controller 235 and for the user simulators 240. The above-mentioned action library 250 is a library of atomic synchronous methods or functions that automate the IVR product 255 which is to be tested.
Telephony network 220 represents the physical connection(s) that the IVR product 255 uses to connect to the real-world. This topology can be vast and complicated. Disclosed embodiments require only that each of IVR virtual machines 210 and 215 participating in the test execution be connected to the same telephony network.
The following discussion illustrates an example of how the components described above and illustrated in
Then, the test driver 230 establishes a call connection between the two user simulators 240-1 and 240-2. It does this by first triggering the mechanism in IVR product 255-1 for placing an outbound call and indicating that the user simulator 240-1 is the application that the IVR product should load. When the outbound user simulator (i.e., user simulator 240-1) loads, it first loads the configuration files from the call controller (stored in configuration files 245-1) and establishes a connection to the call controller at the previously specified physical machine location, and then continues to place the indicated call via the telephony network 220. Upon placing the call, the outbound simulator 240-1 goes to a “wait” state.
On the inbound IVR machine 215, another instance of the user simulator 240-2 is registered as the application for IVR product 255-2 to load for incoming calls. When IVR virtual machine 215 receives the incoming call (via IVR product 255-2), and when IVR product 255-2 loads user simulator 240-2, the user simulator 240-2 also establishes a connection to the call controller and enters a wait state. The use of configuration information 245-2 in this respect is similar to its use in IVR virtual machine 210.
When call controller 235 determines that a pairing has occurred, it then indicates as such to the test driver 230. Test driver 230 now (via the call controller 235) has a direct channel to both sides of the call and starts to send commands to one side, the other or both in order to play out the test case activity specified in the test case definition file 225. These commands are implemented as synchronous activity in order to ensure the test case remains deterministic and may return values to be validated by the test driver. When the activities either complete or fail to complete, the test driver 230 tells the user simulators 240-1 and 240-2 to disconnect the call and to then disconnect from the call controller, thus completing the test case loop.
Referring now to
Referring now to
Referring now to
Then, at step 610, the call controller 235 is used to configure one or more IVR virtual machines (e.g., virtual machines 210; 215) to participate in a test case. As described above, each IVR virtual machine implements a user simulator 240 and an IVR product 255 to be tested by the test case. Configuring each IVR virtual machine includes, as described above, transferring call configuration information and a physical location of the call controller 235 from the test driver virtual machine 205 to the IVR virtual machine(s) 210 and 215.
At step 615, the method embodiment includes establishing a call connection between IVR products 255-1 and 255-2, on the IVR virtual machines, over a telephony network 220. Then, the method includes step 620 of using the call controller to synchronously send commands to the user simulators on each IVR virtual machine to control execution of the test case using the call connection. The method also includes the step 625 of monitoring execution of the test case, between the virtual machines, using the test driver 230. Such monitoring allows IVR product performance to be evaluated and improved.
Referring now to
In the alternative represented in
Referring now to
Now referring to
Referring now to
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.