Patent Application
20130326202
Engineers use load testing to evaluate the performance of a computer program while being exposed to a heavy workload. Load testing is one of the tests carried out before a software application is shipped to customers. A test engineer may attempt to understand how a human user would interact with the software application and devise a test plan to automate the human interaction therewith. Such automation may be conducted with a software testing tool, such as LoadRunner distributed by Hewlett-Packard.
A test engineer may use a software testing tool to interact with a computer program and record those interactions in a script. Such a script may be replayed as many times as needed to evaluate the performance of the program. During load testing, test engineers may execute multiple concurrent instances of these scripts to determine how the program reacts under stress.
Introduction:
As noted above, a load test may include concurrent execution of multiple scripts to evaluate the performance of a computer program. Load tests may be implemented in a lab containing a plurality of load generators, which are computer apparatus used for executing the scripts. The cost to purchase these computers depend on the load testing plan or the scenarios that will be evaluated. However, when formulating a particular test plan, estimating the number of computers to purchase may be difficult. Furthermore, there are many types of computer apparatus in the market with different specifications. An inaccurate estimate may result in the purchase of too many or too few computers for a given load test or may result in the purchase of unsuitable computer hardware. If unsuitable hardware is purchased, the processor thereof may be overly consumed during the test, which may result in inaccurate measurements, such as invalid response times of the user simulations. Manually forecasting the amount and type of resources to purchase for a particular load test may be burdensome and time consuming.
In view of the foregoing, various examples disclosed herein provide a system, non-transitory computer-readable medium, and method that may provide test engineers with forecast information which allows them to appropriately budget for a particular load test. For example, the techniques herein may advise a test engineer how many scripts associated with a load test can execute concurrently in a particular computer apparatus. Such a forecast may be based at least partially from a metric associated with a different computer apparatus. The aspects, features and advantages of the present disclosure will be appreciated when considered with reference to the following description of examples and accompanying figures. The following description does not limit the application; rather, the scope of the disclosure is defined by the appended claims and equivalents. The present disclosure is divided into sections. The first, labeled “Components,” describes examples of various physical and logical components for implementing aspects of the disclosure. The second section, labeled “Operation,” provides a working example of the computer apparatus, non-transitory computer-readable medium, and method. Finally, the section labeled “Conclusion” summarizes the disclosure.
Components:
The computer apparatus 100 may also contain a processor 110 and memory 112. Memory 112 may store instructions that may be retrieved and executed by processor 110. In one example, memory 112 may be a random access memory (“RAM”) device. In a further example, memory 112 may be divided into multiple memory segments organized as dual in-line memory modules (DIMMs). Alternatively, memory 112 may comprise other types of devices, such as memory provided on floppy disk drives, tapes, and hard disk drives, or other storage devices that may be coupled to computer apparatus 100 directly or indirectly. The memory may also include any combination of one or more of the foregoing and/or other devices as well. The processor 110 may be any number of well known processors, such as processors from Intel® Corporation. In another example, the processor may be a dedicated controller for executing operations, such as an application specific integrated circuit (“ASIC”). Although all the components of computer apparatus 100 are functionally illustrated in
The instructions residing in memory 112 may comprise any set of instructions to be executed directly (such as machine code) or indirectly (such as scripts) by processor 110. In that regard, the terms “instructions,” “scripts,” “applications,” and “programs” may be used interchangeably herein. The computer executable instructions may be stored in any computer language or format, such as in object code or modules of source code. Furthermore, it is understood that the instructions may be implemented in the form of hardware, software, or a combination of hardware and software and that the examples herein are merely illustrative.
Testing application 115 may contain a capacity planner module 116 that may implement the techniques described in the present disclosure. In that regard, testing application 115 may be realized in any non-transitory computer-readable media for use by or in connection with an instruction execution system such as computer apparatus 100, an ASIC or other system that can fetch or obtain the logic from non-transitory computer-readable media and execute the instructions contained therein. “Non-transitory computer-readable media” may be any media that can contain, store, or maintain programs and data for use by or in connection with the instruction execution system. Non-transitory computer readable media may comprise any one of many physical media such as, for example, electronic, magnetic, optical, electromagnetic, or semiconductor media. More specific examples of suitable non-transitory computer-readable media include, but are not limited to, a portable magnetic computer diskette such as floppy diskettes or hard drives, a read-only memory (“ROM”), an erasable programmable read-only memory, or a portable compact disc.
Testing application 115 may configure processor 110 to record human interactions with a program being subjected to load testing, such as computer program 120. These interactions may be recorded as a series of functions in script 118, which may be executable by testing application 115. The recorded functions may trigger the same objects in the program being tested that were triggered during the recording. Testing application 115 may be any performance and load testing application for examining system behavior and performance while generating actual workload. One example of such application is the LoadRunner application distributed by Hewlett-Packard. However, it is understood that any appropriate network or load monitoring tool may be used and is considered to be within the scope of the present disclosure. In some implementations, LoadRunner or other load test application may execute concurrent instances of a script, such as script 118, to emulate hundreds or thousands of concurrent users and/or transactions. During such a load test, test engineers may be able to collect information from infrastructure components.
Operation:
One working example of the system, method, and non-transitory computer-readable medium is shown in
In
The resources of the first computer apparatus consumed by the script may be further determined by executing the script in the first computer apparatus. Referring now to
The script processor number and the aforementioned metric associated with the first computer apparatus may be used to determine the resources of the first computer apparatus consumed by the script. In one example, the resource consumption may be represented by a script score. The script score may be calculated by multiplying the metric associated with the first computer apparatus by the script processor number. Once again, the metric associated with the first computer apparatus may be 45 points/second. Thus, in the example discussed above, the script score would be 45 points/second×10 seconds=450 points/per second.
Referring back to
Furthermore, drop down box 202 may also allow the user to select his or her local computer for capacity planning. In this example, the first computer apparatus and the second computer apparatus would be the same, and the metric associated with the first computer apparatus would be the same as the metric associated with the second computer apparatus. As such, there would be no need to download metric information from a benchmark standard-setting entity, when a user selects his or her local computer.
Referring back to
script score=metric associated with the first computer×script processor number script score=45 points/second×10 seconds=450 points/second
As noted above, the script processor number may represent the time spent by the script using the processor of the first computer apparatus (e.g., computer apparatus 100). Assuming the metric associated with the first computer apparatus is 45 points/second and the script processor number is 10 seconds, the script score is 450 points/second. The number of instances of the computer executable instructions capable of executing concurrently in the second computer apparatus may be further determined by multiplying the metric of the second computer apparatus and the script runtime number associated with the first computer apparatus. Using the illustrative values discussed above, the product of the metric associated with the second computer apparatus and the script runtime number, may be the following:
Metric associated with second computer apparatus×script runtime number 15 points/second×200 seconds=3000 points/second
Multiplying the example metric obtained from the benchmark standard-setting entity for model “HP DL 120 G5” and the script runtime number associated with computer apparatus 100 results in 3000 points/second. This product may be divided by the script score calculated earlier:
3000 points/second÷450 points/second˜6.666
This may be rounded down to 6. Thus, the second computer apparatus (e.g., the computer model “HP DL 120 G5” chosen by the user from the drop down box) can execute approximately 6 instances of script 118 concurrently.
Referring now to
Conclusion:
Advantageously, the above-described computer apparatus, non-transitory computer readable medium, and method allow test engineers to better prepare for a particular load test. In this regard, the user may be able to predict how a particular computer would behave in a load test environment before purchasing the said computer. In turn, load test execution can be carried out in a more organized fashion.
Although the disclosure herein has been described with reference to particular examples, it is to be understood that these examples are merely illustrative of the principles of the disclosure. It is therefore to be understood that numerous modifications may be made to the examples and that other arrangements may be devised without departing from the spirit and scope of the disclosure as defined by the appended claims. Furthermore, while particular processes are shown in a specific order in the appended drawings, such processes are not limited to any particular order unless such order is expressly set forth herein. Rather, processes may be performed in a different order or concurrently and steps may be added or omitted.
