Patent Application
20160103761
Embodiments described herein relate to application testing. More particularly, embodiments described herein relate to preparation of an application testing environment and to execution of an automated test script in an application testing environment.
Software testing is commonly utilized for providing information about the quality of a software product, service, or application under test. Test techniques include the execution of a program or application with the intent of finding software errors or other defects (often referred to as “software bugs”). Software testing typically involves the execution of one or more software components in a testing environment to evaluate one or more properties of interest. In general, these properties may indicate the extent to which the component or system under test meets requirements that guided its design and development, responds correctly to various inputs, performs its functions within an acceptable time, is sufficiently usable, can be installed and run in its intended environment, and achieves a desired result.
A particular type of software testing is GUI testing. Such testing involves testing a software component or system in a testing environment to ensure or determine whether it meets design and development requirements. This may typically be done through the use of various test cases. GUI testing and other types of testing can be a difficult task due, because software components and systems can include a large number of functions and code that need to be tested in a variety of different ways. For at least these reasons, it is desired to provide improved systems and techniques for software testing, particularly GUI testing.
The foregoing summary, as well as the following detailed description of various embodiments, is better understood when read in conjunction with the drawings provided herein. For the purposes of illustration, there is shown in the drawings exemplary embodiments; however, the presently disclosed subject matter is not limited to the specific methods and instrumentalities disclosed.
The presently disclosed subject matter is described with specificity to meet statutory requirements. However, the description itself is not intended to limit the scope of this patent. Rather, the inventors have contemplated that the claimed subject matter might also be embodied in other ways, to include different steps or elements similar to the ones described in this document, in conjunction with other present or future technologies. Moreover, although the term “step” may be used herein to connote different aspects of methods employed, the term should not be interpreted as implying any particular order among or between various steps herein disclosed unless and except when the order of individual steps is explicitly described.
As referred to herein, the term “computing device” should be broadly construed. It can include any type of device including hardware, software, firmware, the like, and combinations thereof. A computing device may include one or more processors and memory or other suitable non-transitory, computer readable storage medium having computer readable program code for implementing methods in accordance with embodiments of the present invention. A computing device may be, for example, retail equipment such as POS equipment. In another example, a computing device may be a server or other computer located within a retail environment and communicatively connected to other computing devices (e.g., POS equipment or computers) for managing accounting, purchase transactions, and other processes within the retail environment. In another example, a computing device may be a mobile computing device such as, for example, but not limited to, a smart phone, a cell phone, a pager, a personal digital assistant (PDA), a mobile computer with a smart phone client, or the like. In another example, a computing device may be any type of wearable computer, such as a computer with a head-mounted display (HMD). A computing device can also include any type of conventional computer, for example, a laptop computer or a tablet computer. A typical mobile computing device is a wireless data access-enabled device (e.g., an iPHONE® smart phone, a BLACKBERRY® smart phone, a NEXUS ONE™ smart phone, an iPAD® device, or the like) that is capable of sending and receiving data in a wireless manner using protocols like the Internet Protocol, or IP, and the wireless application protocol, or WAP. This allows users to access information via wireless devices, such as smart phones, mobile phones, pagers, two-way radios, communicators, and the like. Wireless data access is supported by many wireless networks, including, but not limited to, CDPD, CDMA, GSM, PDC, PHS, TDMA, FLEX, ReFLEX, iDEN, TETRA, DECT, DataTAC, Mobitex, EDGE and other 2G, 3G, 4G and LTE technologies, and it operates with many handheld device operating systems, such as PalmOS, EPOC, Windows CE, FLEXOS, OS/9, JavaOS, iOS and Android. Typically, these devices use graphical displays and can access the Internet (or other communications network) on so-called mini- or micro-browsers, which are web browsers with small file sizes that can accommodate the reduced memory constraints of wireless networks. In a representative embodiment, the mobile device is a cellular telephone or smart phone that operates over GPRS (General Packet Radio Services), which is a data technology for GSM networks. In addition to a conventional voice communication, a given mobile device can communicate with another such device via many different types of message transfer techniques, including SMS (short message service), enhanced SMS (EMS), multi-media message (MMS), email WAP, paging, or other known or later-developed wireless data formats. Although many of the examples provided herein are implemented on smart phone, the examples may similarly be implemented on any suitable computing device, such as a computer.
As referred to herein, the term “user interface” is generally a system by which users interact with a computing device. A user interface can include an input for allowing users to manipulate a computing device, and can include an output for allowing the computing device to present information and/or data, indicate the effects of the user's manipulation, etc. An example of a user interface on a computing device includes a graphical user interface (GUI) that allows users to interact with programs or applications in more ways than typing. A GUI typically can offer display objects, and visual indicators, as opposed to text-based interfaces, typed command labels or text navigation to represent information and actions available to a user. For example, a user interface can be a display window or display object, which is selectable by a user of a computing device for interaction. The display object can be displayed on a display screen of a computing device and can be selected by and interacted with by a user using the user interface. In an example, the display of the computing device can be a touch screen, which can display the display icon. The user can depress the area of the display screen where the display icon is displayed for selecting the display icon. In another example, the user can use any other suitable user interface of a computing device, such as a keypad, to select the display icon or display object. For example, the user can use a track ball or arrow keys for moving a cursor to highlight and select the display object.
As referred to herein, the term “serialization” refers to a process of translating data structures or objects into a format that can be stored and reconstructed subsequently in the same or another computing environment. For example, the serialized structure or object may be stored in memory, a file, a memory buffer, or transmitted across a network connection. When a serialized structure or object is reread according to the serialization format (or de-serialized), the de-serialized structures or objects may be used to create a semantically identical clone of the original structure or object.
As referred to herein, the term “test script” may refer to a set of instructions that are to be performed by an application or system under test to test that the application or system functions as expected. An example, manual testing is a process of manually testing software for defects. It requires a tester or user to play the role of an end user, and to use most of all features of the application to ensure correct behavior. In another example, test automation refers to the user of specialized software (apart from the application being tested) to control the execution of tests and the comparison of actual outcomes with predicted outcomes. Test automation can automate some repetitive but necessary tasks in a formalized testing process already in place, or add additional testing that may be difficult to perform manually.
Referring to
Each test script 100 may include one or more GUI test automation objects 106. Each of the GUI test automation objects 106 may be classified as either a test navigation object or a test essential object in accordance with embodiments described herein. The classification can be performed by the use of a tagging mechanism via annotations specially designated for this purpose, by the scripter, based on a functional understanding of the Application Under Test. The annotation can in turn facilitate the serialization of the automation objects by the coding engine, that are tagged by the scripter to be de-serialized in the subsequent runs to not just test the automation object, but also its state, as recorded during the script creation, providing more accuracy to the test and also improving the speed of execution.
As referred to herein, the term “test navigation objects” may refer to objects which are not critical to testing result and used for the purpose of navigation for testing. The test navigational objects are not critical to testing result, but they are still required to execute the test. The term critical or required is used in the context of their contribution to the outcome of the test. As referred to herein, the term “test essential objects” may refer to objects which are used to conclude whether a test case has been executed or not. The objects that decide the failure or passing of a test can be tagged as essential. The same automation object can become essential for one test or navigational for another test based on the objective of the test.
In a particular example of testing retail sales application, the testing is performed using Credit card/Debit card/Gift card payment. If the objective of the test is to verify the order number generated, a credit card button may become navigational. Because it is just a step to generate the order, which is replaceable by taking an alternate option other than a credit card to place an order and complete the transaction. But if the objective of the test is to check the order number when credit card is used, the credit card button is an essential object to generate such order number.
In another example of testing retail sales application, a testing is performed to check the presence of 2 items added in the cart, and to ensure that each item has a delete link to remove from the cart. In such case, for example, searching items with typing into text box is navigational because the items can be added to the cart even without it. In the meanwhile, it is essential that verifying the delete links are working because the objective of the test is to find these delete links and remove items.
With continuing reference to
The coding engine 104 may be configured to serialize the GUI test automation objects classified as a test navigation object for subsequent testing in the testing environment. Further, the coding engine 104 may store the serialized GUI test automation objects 106 at a predetermined location in a memory 106. The coding engine 104 may also identify the GUI test automation objects 106 classified as a test essential object as being classified as a test essential object. The coding engine 104 may also manage execution of the automated test scripts 100 for testing the application 102. Additional details of these functions and others are described in further detail herein. The coding engine 104 may implement a decision comparator that may initiate in a fail safe to use the object that is available at run time when the de-serialized object is not matching the navigation object. The objective may be to update the serialized object scheme to be used in subsequent testing runs.
Tags may be used in the subsequent tests to identify objects as either a test essential type or a test navigation type. These tags can be used by a suggested wrapper to determine whether a GUI test automation object has been serialized and saved at a predetermined location in memory, which can be a file or a database. Once the serialized objects have been updated or saved to the desired location, the subsequent testing runs can use the serialized objects at runtime by de-serialization without finding them each time at the application under test. As a result, resources at runtime can be saved in subsequent test runs.
In accordance with embodiments of the present subject matter, a wrapper can have a failsafe mechanism in place to re-discover and serialize any changes that may have taken place in navigation objects in case a failure occurs, such as an instance of an object not being found.
Objects marked as essential can be marked as “transient” so that they are not serialized and are always discovered at run time. This is such that a proper validation can be performed on these objects at each run to ensure an accurate result for each run.
The coding engine 104 may be include hardware, software, firmware, or combinations thereof configured to implement the functions described herein. For example, the coding engine 104 may include one or more processors and memory. The memory may include instructions for implementation by the process(s).
The system shown in
Referring to
The method of
In a particular example of classifying GUI test automation objects, the application 102 may be a retail sales application including functionality for user login, for placing an order, and for validating a sales order number for the order. These functions may include the following sequence of actions and events: logging into the application by entering a user name and password, and “clicking” or otherwise selecting a displayed “go” or login button; selecting an “add item” button for an item to purchase; selecting a “complete order” button for initiating a purchase and complete order action; and validating an order number on a window or “pop up” graphic on the display 110. In this example, GUI test automation objects may be associated with these actions and events. The user name, password, login button (or “go” button), “add item” button, and “complete order” button may be classified as test navigation objects and can be tagged as “@navigation.” The window (or “pop up” graphic) for the new order can be classified as a test essential object and can be tagged as “@essential.”
The method of
The method of
Once the test navigation objects that are serialized are de-serialized by the automated test script 100 at run time, they can be used during the entire duration of the automated test script 100 execution with an aim to navigate to the test essential object, which can be marked or otherwise identified as a non-serialized object and can be created at the execution time before the validation is performed.
In accordance with embodiments of the present subject matter,
Referring to
The method of
The method of
The method of
The present subject matter may be a system, a method, and/or a computer program product. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present subject matter.
The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.
Computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing/processing device.
Computer readable program instructions for carrying out operations of the present subject matter may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present subject matter.
Aspects of the present subject matter are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the subject matter. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.
These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present subject matter. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.
While certain embodiments have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the inventions. Indeed, the novel methods, devices, and systems described herein may be embodied in a variety of other forms. Furthermore, various omissions, substitutions, and changes in the form of the methods, devices, and systems described herein may be made without departing from the spirit of the inventions. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the inventions.
