Patent Application
20140026123
This application claims the benefit of Indian Patent Application Filing No. 3011/CHE/2012, filed Jun. 20, 2012, which is hereby incorporated by reference in its entirety.
The invention relates generally to the replication of a test case in a network environment. In particular, the invention relates to the reproduction of test results in the testing of mobile application behavior across a variety of network scenarios.
Recent years have seen a surge in the number and diversity of wireless and mobile network applications. A key differentiator among such applications may rest in their ability to deliver a consistent and reliable service to the user. Such delivery through mobile applications is a significant challenge, however, as it requires the coordination of a range of key components in the end-to-end mobile ecosystem. More particularly, the components of the ecosystem, including, for example, the mobile device, a mobile network, and an application server, may exhibit significant variations in their behavior.
In order to ensure reliability, a mobile application may therefore need to be tested against each or all of the variations exhibited by these components. In general, device variation may be addressed through device diversity testing, while application servers are subjected to load and performance testing. However, the network component is harder to address, as it may include many factors outside the control of the application, such as network connectivity, bandwidth, and network congestion.
A key requirement of application performance is to be resilient to such external and environmental disturbances, which, consequently, imposes a need for testing in all possible network conditions. Such testing can be done by means of network simulation, network emulation, live network testing, or a combination thereof. While simulation offers significant benefits in controllability of the network environment, network simulation is ultimately a best-guess or approximation of live network conditions, and may fail to account for the full extent of non-deterministic network behavior. In addition, live testing, which is testing carried out in an actual live network environment, as its name indicates, does present test-cases that closely approximate real-world use cases, but the reproducibility of test-cases is problematic, given the lack of controllability over the network environment.
In contrast, the emulation approach charts a middle path between these two scenarios. In the context of testing wireless applications, the real application runs on the actual device. The network stack, on the actual emulator device, is also real. Given this setup, network behavior may be controlled by the emulator, which decides what packets the wireless mobile device sends and receives.
Mere repetition of a test case supported by a network emulator, however, may not serve to ensure test result repeatability due to the fact that test cases of a statistical nature may create a variety of different and unpredictable network events, such as a random packet drop, when executed at different points of time, thus leading to different test results.
Accordingly, there is a need for a network emulator mechanism whereby an ability to emulate network conditions and reproduce recorded test results at any point of time is preserved while accounting for a variation in incoming traffic and random network events.
Embodiments of the present invention include a computer implemented method for replication of a test case in a network environment. The method describes comprising monitoring network traffic in a first instance of the test case by a network emulator node emplaced in the network environment, wherein the network emulator node comprises a processor and a processor readable memory, and network traffic comprises one or more data packets transmitted to the network emulator node by one or more remote nodes. Monitoring further comprises recording a time at which a connection open at the network emulator node completes a communication task, and deriving an order of task completion for each extant open connection thereby and detecting the occurrence of one or more events, wherein the one or more events are selected from a group consisting of a packet drop, a packet retransmission, a packet delay, a packet duplication, packet reordering, a change in the connection's IP address, and a disruption in a radio link associated with the network emulator node. The method additionally comprises recording the nature of the event and identification information associated with the packet to which the event is detected to have occurred, and associating the one or more events with each of the packets monitored, on the basis of the identification information and event recording.
In an additional embodiment, a system for replication of a test case in a network environment is described, the system comprising a network emulator node operable to connect to a network, the network emulator node comprising a processor and a processor readable memory and wherein the network emulator node is adapted to monitor network traffic in a first instance of the test case, wherein network traffic comprises one or more data packets transmitted to the network emulator node by one or more remote nodes, and the monitoring by the network emulator node further comprises recording a time at which a connection open at the network emulator node completes a communication task, deriving an order of task completion for each extant open connection thereby, and detecting the occurrence of one or more events, wherein the one or more events are selected from a group consisting of a packet drop, a packet retransmission, a packet delay, a packet duplication, packet reordering, a change in the connection's IP address, and a disruption in a radio link associated with the network emulator node. The system additionally records the nature of the event and identification information associated with the packet to which the event is detected to have occurred; and associates the one or more events with each of the packets monitored, on the basis of the identification information and event recording.
These and other features, aspects, and advantages of the present invention will be better understood when the following detailed description is read with reference to the accompanying drawings in which like characters represent like parts throughout the drawings, wherein:
While systems and methods are described herein by way of example and embodiments, those skilled in the art recognize that systems and methods for the replication of test results in a network environment are not limited to the embodiments or drawings described. It should be understood that the drawings and description are not intended to be limiting to the particular form disclosed. Rather, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope of the appended claims. Any headings used herein are for organizational purposes only and are not meant to limit the scope of the description or the claims. As used herein, the word “may” is used in a permissive sense (i.e., meaning having the potential to) rather than the mandatory sense (i.e., meaning must). Similarly, the words “include”, “including”, and “includes” mean including, but not limited to.
The following description is the full and informative description of the best method and system presently contemplated for carrying out the present invention which is known to the inventors at the time of filing the patent application.
Exemplary Computing Environment
One or more of the above-described techniques may be implemented in or involve one or more computer systems.
With reference to
A computing environment may have additional features. For example, the computing environment 100 includes storage 140, one or more input devices 150, one or more output devices 160, and one or more communication connections 170. An interconnection mechanism (not shown) such as a bus, controller, or network interconnects the components of the computing environment 100. Typically, operating system software (not shown) provides an operating environment for other software executing in the computing environment 100, and coordinates activities of the components of the computing environment 100.
The storage 140 may be removable or non-removable, and includes magnetic disks, magnetic tapes or cassettes, CD-ROMs, CD-RWs, DVDs, or any other medium which may be used to store information and which may be accessed within the computing environment 100. In some embodiments, the storage 140 stores instructions for the software 180.
The input device(s) 150 may be a touch input device such as a keyboard, mouse, pen, trackball, touch screen, or game controller, a voice input device, a scanning device, a digital camera, or another device that provides input to the computing environment 100. The output device(s) 160 may be a display, printer, speaker, or another device that provides output from the computing environment 100.
The communication connection(s) 170 enable communication over a communication medium to another computing entity. The communication medium conveys information such as computer-executable instructions, audio or video information, or other data in a modulated data signal. A modulated data signal is a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media include wired or wireless techniques implemented with an electrical, optical, RF, infrared, acoustic, or other carrier.
Implementations may be described in the general context of computer-readable media. Computer-readable media are any available media that may be accessed within a computing environment. By way of example, and not limitation, within the computing environment 100, computer-readable media include memory 120, storage 140, communication media, and combinations of any of the above.
Exemplary Architecture of a Network Emulator Node
As in
Exemplary Network Environment
An exemplary network environment in accordance with some embodiments of the invention is depicted in
In accordance with an embodiment, the client device 310 may be authenticated by the device driver WLAN layer 250 in the network emulator node. When the client 310 sends a request to the network emulator node 306, the network traffic controller 240 and the radio controller 230 induces network variations, i.e. in order to emulate a target network, in the traffic or signal flowing from the client to the server 308.
Replication of Test Results
The replication of test results requires an exact reproduction of network events that occur in application traffic. To that end, steps in the replication of test results, by means of a network emulator node, in a network environment are described by means of
Then, as in a block 404, a time at which a connection open at the network emulator node completes a communication task is recorded and an order of task completion is consequently derived for open connections thereby. More specifically, the timing relationship between connections or the time at which connections are initialized and terminated is recorded. Learning connection behavior or, more specifically, learning how existing connections are used by the application for sending and receiving data is useful in replicating the merging and splitting of connections.
Then, as in a block 406, the occurrence of one or more network events is detected. Such detection may be predicated upon maintenance of the monitoring step 402. The detection may be implemented by means of a session based recording of one or more of packet drops in the network, an on/off state of the wireless radio, variation in the network bandwidth, and the recording of packet duplication. For each session, a packet sequence number, a packet length, a packet flag and a packet event are recorded. Recorded events may be stored in file in a processor readable memory at the network emulator node for a user to replay at a later point of time.
Then, as in a step 408, the nature of the network event and identification information associated with the packet to which the event is detected to have occurred is recorded. Input identification may be performed by mapping incoming traffic sessions to corresponding sessions in recorded data. If the incoming traffic is encrypted, such mapping may be based on determining a timing relationship between socket connections, from the step 404, and additionally take into consideration a port number of the connections, the packet size information of the traffic flow recorded in the step 402, and user input. If incoming traffic is unencrypted, input identification may similarly be performed by mapping incoming traffic sessions to corresponding sessions in recorded data, where mapping, in this case, includes determining a timing relationship between socket connections, from the step 404, and additionally takes into consideration some or all of a port number of the connections, the packet size information of the traffic flow recorded in the step 402, user input, and the invariant content information as detailed in the input monitoring and recording step, 402.
Then, as in a step 410, one or more events are associated with the monitored packets, the event association on the basis of the identification information and the recorded events. Once sessions are identified and associated with corresponding sessions in recorded data, network events may be applied on incoming traffic based on the events recorded.
If, in the second test case where the test results are to be replicated, packets do not arrive in the same order as in the first recorded test case, a sequence number and a packet count number may be used to uniquely identify and map packets in incoming traffic to the recorded data. The packet count number is determined relative to the temporal position of the packet counting from the start of the application traffic, and the sequence number so used is a measure of bytes passed before the arrival of the packet at the network emulator node. In some embodiments, if incoming traffic utilizes the transmission control protocol (TCP), the sequence number in the TCP header may be used instead.
In some embodiments, there may be a significant deviation in the input traffic in the second test case, while still necessitating reproduction of test results from the first test case. Such deviation between incoming traffic and recorded data from the first test case may require additional steps that include detection of the exact nature and extent of the deviation, and correction thereto by the network emulator node.
Detection scenarios of deviation between incoming traffic and recorded data may include instances where the incoming traffic's packet size is not equal to the corresponding recorded packet size, and, as well, instances where the sum of consecutive packet sizes are not equal to the recorded packet size. Additional aberrations may include a variation in the number of time a packet has been retransmitted in the repeat test case from the recorded test case. In another example, an application may stop sending data prematurely, in comparison with the recorded test case. Event detection and mapping mechanisms explained in steps 402-410 above may then be applied in order to account for such deviation in the test results.
In some embodiments, once an input deviation is detected, the network emulator node may alert a user and ask to rerun the test case. In some embodiments, once a deviation is detected, the detection of the deviation may be conveyed to an end user by means of a display.
In some embodiments, packets traffic utilizing certain specific protocols may lack a sequence number. For example, UDP based traffic may not carry sequence numbers, rendering event recording based on a sequence number inapplicable. In such instances, the sequence number may be constructed based on the following information:
Firstly, detecting whether any protocol above the UDP layer is utilized in the network for handling packet retransmission. To do this, a packet may be dropped by the network emulator node and the traffic monitored for a reoccurrence of the same packet in future communication. If such retransmission capability is not supported in the network, then the network emulator node may simply tag a sequence number to a packet.
If a higher layer protocol is detected that supports retransmission, then the retransmission window size, i.e. the number of packets that may be transmitted before an acknowledgement is received, is determined. The retransmission window size can be estimated by the network emulator node by dropping or holding packets and measuring how many packets are sent without retransmission or complete stoppage of data transfer. Dropping or holding packets at different points of time during a network session may thereby allow the network emulator node to arrive at an estimate of window size.
Secondly, identification of the retransmitted packet. Packet identification is a key part of test case replication and for protocols such as the UDP, a checksum field and a packet size is used to detect and map packets across test cases. Based on the retransmission window size, the network emulator node may determine the depth of backward surveillance of packet traffic to determine whether one or more received packets have been retransmitted, or not.
The present description includes the best presently-contemplated method for carrying out the present invention. Various modifications to the embodiment will be readily apparent to those skilled in the art and some features of the present invention may be used without the corresponding use of other features. Accordingly, the present invention is not intended to be limited to the embodiments shown but is to be accorded the widest scope consistent with the principles and features described herein.
As will be appreciated by those ordinary skilled in the art, the aforementioned example, demonstrations, and method steps may be implemented by suitable code on a processor base system, such as general purpose or special purpose computer. It should also be noted that different implementations of the present technique may perform some or all the steps described herein in different orders or substantially concurrently, that is, in parallel. Furthermore, the functions may be implemented in a variety of programming languages. Such code, as will be appreciated by those of ordinary skilled in the art, may be stored or adapted for storage in one or more tangible machine readable media, such as on memory chips, local or remote hard disks, optical disks or other media, which may be accessed by a processor based system to execute the stored code.
| Number | Date | Country | Kind |
|---|---|---|---|
| 3011/CHE/2012 | Jul 2012 | IN | national |
