Patent Application
20130219368
The present Invention generally relates to development of software applications and more particularly to a mechanism for testing web applications.
A big issue of the web applications is to find out an effective approach for the test automation of the web interfaces. The problem is more relevant with web interfaces built with the new technologies of the web 2.0® (such as AJAX or JavaScript™ frameworks, e.g. Dojo®). A web page of the web 2.0® applications is not simply dynamic but is a very complex application that is the result of the composition of many object oriented language DOM elements (DOM is the Document Object Model that Is a hardware and language independent model for representing HMTL pages and managing objects in HMTL). Each DOM element could be built dynamically through JavaScript (JavaScript™ is a trademark or registered trademark of Oracle Corporation or its subsidiaries in the United States and other countries) code interacting with a backend and implementing the AJAX® (asynchronous JavaScript™ and XML) technology to create client side web applications and even applying cascading style sheets to allow change the appearance from the client side. Also, the code for creating the DOM elements is scattered in many points and could also be generated by widgets (created with AJAX® frameworks) written separately to implement a single functionality.
The main problem is not only to implement a system for a unit test suite to test the single functionalities, the single widget classes, but also to test the real interaction among DOM elements generated by the widgets and the other parties producing the DOM elements.
Tools of prior art for testing web applications based only on the “external” appearance of a web page or application, or even if they are aware of the DOM elements, will completely miss the internal of the application. Conversely, the application has no way to force a test of this important area, because it has no means to access and drive the “outside” behavior of itself, such software having strong dependencies on the hosting environment (e.g. these applications run in the browser). Automatically testing such web applications is a challenge.
The US patent application US2004/0025083A1 describes the general idea of embedding test code into a program, so that the program can test its functionality. However, despite the broad scope of the invention, it is not possible to use that method to solve the problem of fully testing the internal of the applications in the context of the browser and the operating system hosting it. Some code portions of the applications that depend on differences in the environment cannot be tested; furthermore, some code portions of the applications that depend not only on such differences but also on the different code paths that such differences may lead to execute will not be tested as well. For example, a web application must be separately tested on each of the supported browsers, e.g. Internet Explorer® (Internet Explorer® is a trademark of Microsoft Corporation in the United States, other countries, or both), Firefox® (Firefox® is a registered trademark of the Mozilla Foundation), Opera™ (Opera™ is either trademark or registered trademark of Opera Software ASA), etc. However, for testing purpose, it is not reliable having a test code that ignores the differences that each browser may introduce and only takes into account direct calls to the software functions.
There is a need for a method of instrumenting web interface code with code simulating a user input in a page of the web interface for testing the internal of the application in the context of the browser and the operating system hosting it.
It is therefore an object of the present invention to have an automatic testing mechanism simulating a user input in a page of the web interface and validating the web interface internal data structures of the web application in the context of the browser and the operating system hosting it.
The object is achieved, according to one illustrative embodiment, with method for automatically testing a web application running inside a web browser of a testing computer, the web application being instrumented with test code. The method comprises sending from the test code executing in the web application, through an API defined in the web application, a message to a server asking the server to simulate a user input, the user input being In relation with the test code running in the web application. The method further comprises the server receiving the message, simulating the user input by at least one system event according to the message and providing it to the web browser. The method further comprises the web browser processing (430) the simulated at least one system event which itself triggers execution of some code in the web application. The method further comprises the test code in the web application checking validity of result of triggering of some code in the web application by the at least one system event, by inspecting the web application internal data structure. The internal data structure of a web application comprises the elements that are visible from web pages and internal counters or other possible variables and objects.
In the web application running in the Testing Host, the JavaScript™ application programming interfaces (APIs) enable the Self-Piloting of the JavaScript™ test code. The Self-piloting commands for Javascript™ API (235) allow the communication between the Javascript™ code from the browser (the web application) and the test service. A Browser starter (230) is optional and is for launching the browser on the URL of the Testing Host. When used, the browser starter receives the event from the Remote Event Dispatcher (225) which has itself received the event from the System Event Generator (215), and transmits the event to the browser. When there is no browser launcher, the browser is able to receive directly the simulated system events because the browser is started before the test code in the web application is executing. The browser (110) receives and processes the same event on the Testing Host. Inside the web application, the Test Suite code (web pages containing html/JavaScript™/Dojo® code) contains the code to test the objects and uses the Self-piloting commands for Javascript™ API (235) to check verification points and to trigger actions towards the HTTP Server (210) providing the test service.
It is noted that the entire Test System comprising the components of the Test Automation System and the Testing server could reside on a single computer. In this case where the Test Automation System and the Testing server reside on a single computer, there is no need to have the Remote event Dispatcher (225) and the Repository is local to the web application.
The JavaScript™ API establishes a communication with the HTTP server to allow the web application to send the event commands. The Service gets the commands and through the System Event Generator generates keyboard or mouse events that the browser (that runs always as on top window) catches, the events and the web pages changes occurring as if they were created by the user himself. At this point the test automation code continues executing itself inside the web application.
Example of Verification points based on web page object checking, for instance DOM checks:
Assuming that the web interface provides a button, when the user clicks on the button, a new panel is displayed. In this example, the JavaScript™ code of the test automation should be the following: the JavaScript™ code requests a click of the mouse on the element of the button above. Then, the service receives the command through the JavaScript™ API and produces the related system events. Once the event is received, a new container structure (<DIV>) is generated in the DOM to contain the new panel (a sub DOM). The JavaScript™ code can then check if the DOM element has been inserted as expected.
Example Verification points based on object status checks:
Another way to implement the verification points is to save the objects (linearized for example as JSON strings) before to request simulated system events, then once the simulated system events are received, and the changes have been applied on the DOM, the JSON objects saved before can be loaded and compared to the changed objects. Assuming the web application includes in a web page a widget that changes the background-color when the mouse is over it. Suppose having on the master repository the widget object serialized as master value (with the background color expected with the color when the mouse is over the widget). In this case, the JavaScript™ test consists of requesting a JavaScript™ command event “mouse over” the widget then, when the events arrives to the browser and the mouse is moved over the widget automatically, the JavaScript™ test requests to the Service to compare the actual widget status width the one persisted into the repository. Each of these test suites generates the following actions: makes HTTP requests on dynamic web pages, save JSON objects as current status, generates event commands, (the Browser) catches the event and the web page changes, verify DOM changes comparing results with master values (templates), compare JSON objects with master values. Each test suite generates any kind of interactions between the Browser and the system.
The following exemplary tests can be implemented:
“Look Ahead,” (generating the events to type the first chars of a string to search).
“Field Validation” (generating the all keyboard chars).
“Save pages” (generating the event to access to the browser menu and save the page).
“File Upload” (generating the events to type and open a file on a browse popup).
“Localization” (generating the event to access to the browser menu and change the browser language).
“Browser-dependent,” generating the same keyboard and mouse events to test the different behavior of the events on the different browsers.
“Accessibility test cases,” using the sequences of (tab, enter) keyboard events.
Self-Piloting commands for JavaScript™ API are asynchronously sent to the HTTP server on a port, for example:
http://Servicehostname:port/receiverhost-hostname&command=<name >&arg1=value1&... ...&argN=valueN
Some of the base API commands may be as follows:
click(x,y)
doubleclick(x,y)
key(k) ‘special keys are (tab, enter, left, right, up, down)
pause(delay)
fileload(filename)
filesave(filename)
Some API commands for the verification points based on the JSON comparing may be as follows:
saveAsJSON (object, name), saves the object as JSON string,
object loadJSON (name), converts the JSON string into an object.
checkJSON (object, object), compares two JSON objects.
Beginning in
In the case where the web application is only able to test its response to a kind of “abstract” event, one that is unknown and does not affect the environment and the web application container, a full code coverage is not possible in this case. On the contrary, and this is particularly the case for web applications, every container (e.g. web browser) is different from any other, and often even from other versions of itself, Because of this, it is necessary to test the web application in every supported environment and again this is certainly not possible if the environment is fully integrated into the test scenario.
Some specific steps of the method are related to the example of verification points based on HTML object checks. For instance, assuming the mouse clicking of the user in a web panel, which is a part of a web page, generates a new web panel to be displayed by the application, step 410 will consist in requiring mouse clicking simulation, and step 430 will further include the web application in its normal code execution (not included test code execution) generates a new container structure corresponding to one HTML object (DOM object) instantiation corresponding to the new web page to be displayed. Step 440 will include test code checking that the object corresponding to this new web page has been well instantiated in the web application.
Other specific steps of the method are related to the example of verification points based on object status checks. For instance, checking that the use of a widget by the user has changed the color of the background of an HTML page, A request from the test code of the web application (410) could consist in asking that the initial status of the widget object be saved (for instance a widget to be saved as JSON string in the Repository) and asking through a JavaScript™ command to simulate a system event, such as a mouse move over a widget in the web page displayed to the user and containing this widget. Then, at reception of the simulated system event, when the browser executes the simulated mouse move, the web application executes the corresponding function and the test code (440) capturing the web page changes, sending a new request to the test service (410) to retrieve the saved object (JSON string) from the Repository, receiving the saved object through the web application API, and the test code reading the current widget status and verifying DOM changes, comparing JSON objects with saved object in the Repository. Each test suite generates any kind of interactions between the Browser and the system.
In summary, the mechanism for automatically testing the web application is complete as it tests both the internal structures (440) of the web application and the outside context (430) of the web application.
One example illustrating testing of the internal structure is to assume that the web application keeps an internal counter that keeps track of the number of widgets in error so that if the number is greater than zero then the whole panel is invalid and cannot be processed. A test that checks a correct response of the web application to an invalid input from the user could include three tests:
1) that the corresponding widget changes background color in order to show the error;
2) that a message appears to inform the user; and
3) that the internal counter is incremented by one unit.
Point 3) refers to an internal data structure of the application. The internal counter exists only in the web application, it cannot be tested by testing the external inputs as no HTML element or external representation corresponds to this counter. The code can analyze itself. An HTML, DOM, or JavaScript™ object has a data structure which is external as visible through the web interface of the web application and internal when it comprises data internal to the web application as the counter above.
One example illustrating testing of the outside context of the web application: assuming something happens out of the web application, in the operating system of the computer hosting the web application. The testing mechanism can test how the application reacts responding to the system events. Different behavior occurs for changing the browser or for changing the operating system in the host on which the web application is executed. Consequently, while the internal structure can be tested in response to an external event, as in the embedded test code of the prior art methods, the internal structure of the application can be tested only in response to an internal input simulated by the test code itself not by the system context.
The mechanisms of the illustrative embodiments allow testing data structures of the application by inspection of the data structure after a simulated user input. The mechanisms allow also to check application internal data structures by comparing object status after a simulated user input has occurred with an model object status which has been saved.
The objects may be HTML, DOM or JavaScript™ objects. An automatic testing system will comprise the HTTP server and the code to generate simulated system events corresponding to the tested system inputs.
To fully test a web application, the mechanisms of the illustrative embodiments not only comprise writing some test code and embedding it into the application but also give web applications access to the “outside” environment, including the ability of reproducing the real interaction of the user with a web interface. This method allows to simulate external inputs so that it is possible to receive it as if it comes from a human user or some external event source. In this way, the hosting environment will be able to also process the event and update its own data structure that also have a direct influence on the application behavior. Without this system in place, no test will be able to cover the entire software functionality.
The web application has the ability to interact with itself from the “outside” by means of an external service. The service controls the browser and the external systems (disk, etc.) and provides services to the web application by means of an HTTP connection with specially encoded URL's. In turn the web application uses the provided services to manipulate and control its internal state from the “outside”, which is not normally possible with current test automation systems.
The following method and system overcomes the limitations of the current test methodologies by reproducing and simulating the human user input and use the simulation to validate its internal data structures and algorithms. The method invokes the system events generated by a user input from the JavaScript code inside the web pages itself and produces changes on the web interface (thus on the object model elements DOM elements for instance). Once the changes are reflected on the DOM elements (by the user event requests) the JavaScript code can check the status of the internal data structures comparing with the expected values.
The mechanism of the illustrative embodiment provides several advantages. The mechanism can test anything of DOM elements reproducing the customer scenario by testing all the DOM elements in the application web pages. The solution allows covering the interesting test cases on the advanced examples sections above which cannot be covered by other type of products. A web page can be tested inside and outside of the page itself, The way to check the verification points based on the DOM checks and/or through JSON (for JavaScript™ Object Notation, is a simple format for representing objects with strings) objects comparing, The solution does not only apply to the DOJO(r) framework but can be leveraged on all technologies used for generating the DOM elements of the web pages. The mechanism can be implemented while the language used for the test suite and the application is different from JavaScript™ scripting language. The language needs only to be understandable by the browser.
With respect to the prior art automatic testing products and methodologies, in particular for web applications, the current state of the art provides a lot of features but ultimately the test result is obtained by comparing the “output” of the application to some template that is known to be correct. For a web application, output is basically the HTML page it produces or changes, or equivalently, the DOM representation of it. With the mechanisms of the illustrative embodiments, the web application can access the “outside” environment and simulate real behavior (including human input) that would not be possible by simple code instrumentation. The ability to inspect internal structures are missed by tools that can only inspect the program “appearance.”
| Number | Date | Country | |
|---|---|---|---|
| Parent | 13395302 | Mar 2012 | US |
| Child | 13771271 | US |
