Various testing applications have been developed to assess system behaviour and performance under load. These testing applications may generate concurrent simulated users to simulate the load on a system under test. To do so, the testing applications may generate a load test schedule in which a number of simulated users may be scheduled over time. The simulated users may perform various actions such as submitting requests to applications running on the system for which the behaviour and performance under load is being tested or otherwise placing a load on the system.
Features of the present disclosure may be illustrated by way of example and not limited in the following figure(s), in which like numerals indicate like elements, in which:
For simplicity and illustrative purposes, the present disclosure may be described by referring mainly to examples. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be readily apparent however, that the present disclosure may be practiced without limitation to these specific details. In other instances, some methods and structures have not been described in detail so as not to unnecessarily obscure the present disclosure. Throughout the present disclosure, the terms “a” and “an” may be intended to denote at least one of a particular element. As used herein, the term “includes” means includes but not limited to, the term “including” means including but not limited to. The term “based on” means based at least in part on.
A problem that may arise out of automated and simulated load testing is that it may be difficult to generate the load test schedule. For instance, a (human) load tester may be required to configure the load test schedule by inputting each number of simulated users (also referred to herein as “virtual users” or “vusers”) to be scheduled at a given time or may schedule the same number of virtual users throughout the duration of the load test, either of which may not be complex enough to simulate real-world conditions. A virtual user may include an application or process that executes various activities on the system under test to simulate computational load imposed by real-world activity on the system under test.
Disclosed herein are apparatuses and methods for scheduling virtual users in a load test of a system under test. For example, an apparatus may access a plurality of points in which each point of the plurality of points represents a number of virtual users that is to place a load on the system under test at a given time. The apparatus may apply curve fitting to generate a curve, such as a best-fit curve, based on the plurality of points. One example of a curve fitting technique may include Bézier curve fitting, although the apparatus may use other curve fitting techniques as well. In some examples, the apparatus may use a smoothing factor to adjust the contours of the generated curve. The smoothing factor may be received from a load test designer.
The apparatus may use the curve to schedule virtual users. For example, the apparatus may use the curve to determine curve-fitted points (points along the curve other than the plurality of points from which the curve was generated) to schedule virtual users at times corresponding to the curve-fitted points. Because the plurality of points may be selected (by a human load test designer) to simulate real-world conditions at specified points in time and because the apparatus may automatically determine curve-fitted points to generate the load test schedule, the apparatus may simulate complex real-world conditions for load testing.
In some examples, the apparatus may access the plurality of points via a graphical user interface (GUI)-based graph that represents a load test schedule. A first axis (such as the x-axis) of the graph may represent time and a second axis (such as the y-axis) may represent a number of virtual users to be scheduled at a given point in time. A load test designer may plot the plurality of points on the graph via a GUI. The apparatus may then apply curve fitting to generate the curve and determine curve-fitted points along the curve. In some examples, the apparatus may access the plurality of points from a load test datastore. The load test datastore may store a historical set of points used for curve fitting in previous load tests.
In some examples, the apparatus may receive a modification to the plurality of points received via input at the graph or imported from the load test datastore. The modification may include a deletion, addition, or change to an existing point of the plurality of points. For example, a user may delete a point, add a point, or move a point plotted on the graph. After the modification to the plurality of points, the apparatus may generate (or re-fit) a curve based on the modified plurality of points.
In some examples, the number of virtual users instantiated to simulate load on the system under test may impose a computational burden (such as processing resources, memory resources, and the like) on the apparatus or other virtual user scheduler that instantiates the virtual users. To mitigate the load, the apparatus may use coroutines. Coroutines may be scheduled in user space and have flexible stack space. Coroutines may therefore enable a larger scale of virtual users since the coroutines may use less system resources.
The apparatus 100 may include a memory 110 that may have stored thereon machine-readable instructions (which may also be termed computer readable instructions) 112-118 that the processor 102 may execute. The memory 110 may be an electronic, magnetic, optical, or other physical storage device that includes or stores executable instructions. The memory 110 may be, for example, Random Access memory (RAM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a storage device, an optical disc, and the like. The memory 110 may be a non-transitory machine-readable storage medium, where the term “non-transitory” does not encompass transitory propagating signals.
Referring to
The processor 102 may fetch, decode, and execute the instructions 114 to apply curve fitting to the plurality of points. One example of a curve fitting technique may include Bézier curve fitting, although the apparatus may use other curve fitting techniques as well. In some examples, the process of curve fitting may include determining points other than the plurality of points along a curve that fits the plurality of points. Such points other than the plurality of points will be referred to herein as “curve-fitted points.”
The processor 102 may fetch, decode, and execute the instructions 116 to generate a curve (such as curve 310 illustrated in
In some examples, the curve may include a Quadratic Bezier curve in which the processor 102 may use the plurality of points (such as points 302A-N illustrated in
In some examples, the processor 102 may graphically represent the curve in a GUI by plotting some or all of the curve-fitted points and some or all of the plurality of points. In some examples, the processor 102 may access and use a smoothing factor to generate the curve or otherwise adjust the contours of the generated curve. The smoothing factor may be received from a load test designer.
The processor 102 may fetch, decode, and execute the instructions 118 to generate a load test schedule based on the curve. For example, to generate the load test schedule, the processor 102 may determine a plurality of curve-fitted points based on the curve. The load test schedule may be generated based on the plurality of curve-fitted points from the curve.
In some examples, the processor 102 may conduct a load test based on the load test schedule. In some examples, the processor 102 may determine a current number of virtual users during the load test and generate, in the graph, an indication of the number of virtual users currently in the load test. In some examples, the processor 102 may access a modification to the plurality of points and refit the curve based on the modification to the plurality of points. In these examples, the processor 102 may modify the load test schedule based on the refit curve.
In some examples, the processor 102 may conduct the load test by executing a virtual user according to the load test schedule. For example, the processor 102 may schedule a number of virtual users at a given time based on a point along the curve in the graph. In some examples, to mitigate computational load imposed by the virtual user, the processor 102 may execute the virtual user using a coroutine. Coroutines may be scheduled in user space and have flexible stack space. Coroutines may therefore enable a larger scale of virtual users since the coroutines may use less system resources.
In some examples, the processor 102 may access an indication of a number of instances that virtual users are to be scheduled during the load test. The number of the plurality of curve-fitted points may be determined based on the number of instances. For example, a load test designer may specify that fifty points be used during a load test. The processor 102 may accordingly select fifty points along the curve to instantiate virtual users. The processor 102 may space the fifty points substantially evenly across the load test such that an interval between any two adjacent points in the curve is substantially equal to another interval between two other adjacent points.
In some examples, the processor 102 may determine a first number of virtual users to be scheduled associated with a first curve-fitted point of the plurality of curve-fitted points. The processor 102 may determine a second number of virtual users to be scheduled associated with a second curve-fitted point of the plurality of curve-fitted points. The second curve-fitted point may be adjacent to the first curve-fitted point in the plurality of curve-fitted points on the curve. The processor 102 may determine whether to add or remove a virtual user in an interval between the first curve-fitted point and the second curve-fitted point based on the first number and the second number.
In some examples, the apparatus 100 may be part of a system 200 for scheduling virtual users for load testing. For example,
In some examples, the graph 300 (or other portion of the GUI 210) may receive user input that specifies locations in the graph 300 corresponding to a plurality of points that the apparatus 100 accesses. The user input may include GUI interactions such as a point, a click, and/or other inputs to the graph 300 that may specify a location on the graph at which a point is to be created. The user input may be provided from the client device 230, which may receive and display the GUI 210 (and graph 300) to a user, such as a load test designer. In these examples, the apparatus 100 may access the plurality of points based on the user input.
In some examples, the apparatus 100 may access the plurality of points from the load test store 220. The load test store 220 may store previously specified (and/or used) points for curve-fitting in previous load tests. In this manner, a load test designer may select previously-used points for curve-fitting. In some of these examples, the load test designer may modify the previously-used points by removing, adding, or moving a point in the previously-used points.
Regardless of the manner in which the apparatus 100 may access the plurality of points 302, the apparatus 100 may apply curve-fitting on the plurality of points 302 to generate a curve 310 (examples of which are illustrated in
The load test schedule 201 may include a schedule of the number of virtual users that are to be instantiated over time. For example, the load test schedule 201 may include a plurality of entries corresponding to points along the curve 310. In other words, each entry in the load test schedule 201 may correspond to a point on the curve 310. The point on the curve may include a point among the plurality of points 302 or a point among the curve-fitted points 312. Each point on the curve may correspond to a number of virtual users to be scheduled (according to the y-axis of the graph 300) at a given time (according to the x-axis of the graph 300). As such, the apparatus 100 may generate each entry in the load test schedule 201 to specify a number of virtual users that is to run at a given point in time during a load test.
For example, referring to
In some examples, the apparatus 100 may provide the generated load test schedule 201 to a vuser scheduler 203 to conduct the load test based on the load test schedule. The vuser scheduler 203 may conduct the load test by analyzing the load test schedule 201 and increasing, decreasing, or keeping the same number of virtual users from one time point to another during the load test.
It should be noted that the vuser scheduler 203 is illustrated as being separate from the apparatus 100 in
Some of the curve-fitted points 312 may be interpolated based on points 302. For example, curve-fitted point 312B may be interpolated based on points 302A and 302B in which the position of the curve-fitted point 312B may be based on the positions of points 302A and 302B. Some of the curve-fitted points 312 may be extrapolated. For example, a curve-fitted point 312A may be extrapolated based on only a single point 302A. In this example, the curve-fitting function may determine the position of the curve-fitted point 312A by using the position of the single point 302A.
Having described curve-fitting to generate a load test schedule, an example of graphically representing the curve will now be described with reference to
As shown in
At block 504, the processor 102 may access a plurality of points, such as points 302 illustrated in
For example, the plurality of points 302 may be stored at a datastore, such as the load test store 220. In some examples, the processor 102 may access a user input that modifies the predefined set of points (or user input points from a graph) and modify the plurality of points based on the user input. The modification to the plurality of points may impact generation of the curve and/or cause the curve to be refit to the modified plurality of points, such as illustrated in
At block 506, the processor 102 may generate a curve (such as curve 310 illustrated in
At block 508, the processor 102 may access telemetry information indicating a number of active virtual users that exist in the load test at a given time. For example, the telemetry information may be provided by the vuser scheduler 203, which may conduct the load test and instantiate the virtual users during the load test.
At block 510, the processor 102 may display the telemetry information in association with the curve to show the number of active virtual users at the given time in relation to the curve.
In some examples, the processor 102 may forward the plurality of points to a virtual user scheduler (such as vuser scheduler 203 illustrated in
The machine-readable instructions 602 may cause the processor to import a plurality of predefined points. Each point of the plurality of predefined points may identify a number of virtual users to be scheduled during a load test and a time at which the number of virtual users is to be scheduled during the load test.
The machine-readable instructions 604 may cause the processor to modify the plurality of predefined points based on a user input. The machine-readable instructions 606 may cause the processor to apply curve fitting to the modified plurality of predefined points. The result of such curve-fitting, in some examples, may be to generate curve-fitted points based on a curve-fitting technique. The machine-readable instructions 608 may cause the processor to generate a curve based on the curve fitting. The machine-readable instructions 610 may cause the processor to generate a load test schedule based on the curve.
Although described specifically throughout the entirety of the instant disclosure, representative examples of the present disclosure have utility over a wide range of applications, and the above discussion is not intended and should not be construed to be limiting, but is offered as an illustrative discussion of aspects of the disclosure.
What has been described and illustrated herein is an example of the disclosure along with some of its variations. The terms, descriptions and figures used herein are set forth by way of illustration only and are not meant as limitations. Many variations are possible within the spirit and scope of the disclosure, which is intended to be defined by the following claims—and their equivalents—in which all terms are meant in their broadest reasonable sense unless otherwise indicated.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/CN2019/097184 | 7/23/2019 | WO |