The subject matter described herein relates to methods and devices used for testing virtual deployments. More particularly, the subject matter described herein relates to methods, systems, and computer readable media for emulating computer processing usage patterns on a virtual machine.
At present, numerous network elements and service nodes are being deployed in continuously expanding telecommunications networks by system administrators. In order to properly address the processing needs of a network by provisioning the optimal number or amount of resources, a system administrator typically requires knowledge of the processing capabilities of a network element in light of the likely demands it will be subjected to by network users. To obtain this pertinent information, a system administrator would ideally be able to simulate specific CPU and IO usage patterns on a virtual machine hosted by a network element in a testing environment. Notably, such specific simulations enable test managers to stress test a virtual machine setup in a device under test (DUT) and subsequently monitor and measure the resulting performance behavior.
Accordingly, there exists a need for methods, systems, and computer readable media for emulating computer processing usage patterns on a virtual machine.
Methods, systems, and computer readable media for emulating computer processing usage patterns on a virtual machine are disclosed. According to one aspect, the subject matter described herein includes a method that generates a usage pattern specification based on a plurality of computer processing usage patterns provided by a user and receiving, by a virtual machine hosted by a device under test (DUT), the usage pattern specification containing the plurality of computer processing usage patterns, wherein each of the plurality of computer processing usage patterns includes execution parameters associated with the execution of simulated actions of an application. The method also includes generating a plurality of pattern instruction sequences using the execution parameters included in each of the plurality of computer processing usage patterns and distributing each of the plurality of pattern instruction sequences among a plurality of virtual processing cores of the virtual machine. The method further includes emulating the operation of the application on the virtual machine by executing the pattern instruction sequences on the virtual processing cores in a manner specified by the execution parameters.
The subject matter described herein may be implemented in hardware, software, firmware, or any combination thereof. As such, the terms “function” or “module” as used herein refer to hardware, which may also include software and/or firmware components, for implementing the feature being described (e.g., conducting simulations and emulations via executed software). In one exemplary implementation, the subject matter described herein may be implemented using a non-transitory computer readable medium having stored thereon computer executable instructions that when executed by the processor of a computer cause the computer to perform steps. Exemplary computer readable media suitable for implementing the subject matter described herein include non-transitory computer-readable media, such as disk memory devices, chip memory devices, programmable logic devices, and application specific integrated circuits. In addition, a computer readable medium that implements the subject matter described herein may be located on a single device or computing platform or may be distributed across multiple devices or computing platforms.
The subject matter described herein will now be explained with reference to the accompanying drawings of which:
The subject matter described herein relates to methods, systems, and computer readable media for emulating computer processing usage patterns on a virtual machine. The present subject matter facilitates the simulation of computer processing usage patterns to follow user-defined application actions or activities. As used herein, a usage pattern may include a processing activity or load associated with the operation of an application or a combination of applications. Further, the usage pattern can represent the activity generated by a single user or multiple users (e.g., multiple and contemporaneous instances of bank account access). In some embodiments, the subject matter simulates the CPU usage by performing the actions specified in user-defined usage patterns, such as pre-canned simulation usage patterns or by customized usage patterns. In some embodiments, the present subject matter provisions pre-canned CPU usage patterns for specific actions. Notably, the disclosed subject matter affords a user the ability to define a specific pattern of computer processing usage that can be used in a test tool environment to generate CPU consumption, file input/output (I/O) access (e.g., non-cache storage access, disk access, etc.), and/or memory usage (e.g., RAM consumption, cache utilization, processor cache consumption, and the like). The predefined levels of generated computer processing usage (e.g., CPU consumption, file I/O access, and/or memory usage) may therefore be used to validate the CPU management solution being employed by a physical device and/or virtual machine being tested.
Reference will now be made in detail to exemplary embodiments of the presently disclosed subject matter, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts. Various embodiments of the present subject matter are disclosed and described herein.
In some examples, design module 106 can be utilized by a user (e.g., a system administrator, network manager, test administrator, etc.) to create and/or construct computer processing usage patterns that emulates a user-defined pattern of user interaction associated with the execution of a software application (e.g., user activity corresponding to the operation of word processing application). In some embodiments, design module 106 enables a user to specify certain parameters and/or values that are to be included in the customized specification. For example, design module 106 may comprise a module configured to receive input and/or specifications from a user via a user interface. Specifically, design module 106 may be used to customize a computer processing usage pattern specification that specifies the individual activities, inputs, and/or commands a software application may experience under the operation of a user. In some embodiments, a user may initially select the executable actions of the application operation that need to be emulated on a device (e.g., a DUT) and/or associated virtual machines to be tested. Exemplary user actions may include the launching/opening the application, the closing of the application, the operation of the application, and the like). Examples of applications in this context may include a word processing application, a media player application, a spreadsheet application, web browsing application, and the like. In some embodiments, user-defined activities and actions may be represented in a graphical manner (as opposed to a task-based manner). For instance, the user-defined activities may include a timeline that defines and/or specifies different usage patterns of activity, such as a ramp-up of processing activity, a ramp-down of processing activity, a Poisson distribution pattern of processing activity, and the like.
In addition to specifying the actions to be emulated, the user may subsequently select associated execution parameters (e.g., number of cores to run the application, etc.) that will be applied to the specified actions of the user-defined computer processing usage pattern. In some instances, the execution parameters operate to establish and/or provide the rules for the execution of the selected actions on the DUT. For example, a user may utilize design module 106 (e.g., via a GUI or command line interface) to specify a number of execution parameters, such as i) a computer processing usage pattern count, ii) an amount of total time for executing the patterns, iii) a designation as to whether the patterns are to be executed serially or in parallel, iv) a designation as to whether the patterns should load processing cores in an evenly distributed manner, and the like. In addition to these ‘global’ input execution parameters, each of the patterns may further include pattern-specific execution parameters, including i) a specified thread count for executing the computer processing usage pattern, ii) the number of cores to use to execute the computer processing usage pattern, iii) an indication (e.g., a flag, a bit indicator, etc.) whether to automatically scale the computer processing usage pattern to run on all of the virtual processing cores, iv) a number of instances (e.g., “P” times) to repeat the execution of the computer processing usage pattern, and/or v) the total time (e.g., a timeline and/or a time limit duration) to execute the computer processing usage pattern. In some embodiments, the execution parameters may further include a central processing unit (CPU) usage threshold limit, a file I/O access threshold limit, and/or a memory usage threshold limit to be adhered to in the simulation testing. Notably, each of these threshold limits may include a minimum threshold limit and/or a maximum threshold limit.
After the actions/activities and associated execution parameters are designated by the user as described above, design module 106 may generate a computer processing usage pattern specification, such as a CPU usage pattern specification, a file I/O access pattern specification, and/or a memory usage pattern specification. Notably, design module 106 may combine or aggregate the designated actions and execution parameters to form a CPU usage pattern specification, which may be embodied as a software based package or object comprising XML data, binary data, or file data.
After being created by design module 106, the usage pattern specification may be forwarded to provisioning module 112 by design module 106 for delivery to DUT 104. Alternatively, the usage pattern specification may be stored in usage pattern database 110 by design module 106 via provisioning module 112 for future test use and/or DUT provisioning.
As indicated above, processing activity emulator 102 further includes capture module 108. In some examples, capture module 108 can include any software entity that is configured to monitor and record actual CPU usage patterns being executed by a deployed computing device and/or network element. For example, capture module 108 may comprise a utility software element that is executed by at least one processor of emulator 102 and that provides diagnostic tools for monitoring interactions (such as system calls, signal deliveries, changes in process state, etc.) between the simulated application processes and the processor. In other embodiments, capture module 108 includes a diagnostic software tool that monitors and records system calls. After capture module 108 records a plurality of the observed system calls and/or signal deliveries, capture module 108 may forward the recorded information to design module 106 (e.g., via an application programming interface (API)). Design module 106 can then process the information received from capture module 108 to generate one or more usage pattern specifications.
In some embodiments, usage pattern database 110 may comprise any data storage device configured to store a plurality of defined and/or customized-usage pattern specifications. The usage pattern specifications stored in database 110 may be a customized usage pattern specification generated by design module 106 or a usage pattern specification based on data captured by capture module 108. Moreover, usage pattern database 110 may also be configured to receive and store pre-canned CPU usage patterns generated and/or designed by a separate system. In some embodiments, provisioning module 112 may be configured to access database 110 per a user's direction or via a system call in order to obtain a stored usage pattern specification for delivery to and provisioning on DUT 104.
As shown in
In some embodiments, DUT 104 can comprise a processor 114 and memory 115. Memory 115 may include a virtual machine monitoring (VMM) module 116 that, when executed by processor 114, establishes a virtual environment 118 on DUT 104. In some embodiments, VMM module 116 may comprise a hypervisor or other software application that is configured to manage and support a virtual environment 118 that includes a plurality of virtual machines 1201 . . . N (or virtual ports). Within each virtual machine or port, there is an ability to specify the pattern as an input. As described below, each virtual machine has a pattern generation module 122 that understands the input and generates the pattern. For example, each of virtual machines 1201 . . . N depicted in
In some embodiments, usage pattern generation module 122 includes a software module that can be configured to receive a usage pattern specification as input from emulator 102. The functionality of usage pattern generation module 122 is discussed in greater detail below with regard to
In some embodiments, publishing module 124 may include a software module that is configured to monitor CPU usage across multiple processing cores (e.g., cores 212 in
In some embodiments, parser module 204 in UPGM 222 receives a CPU usage pattern specification 201 for processing. For example, parser module 204 may be configured to extract one or more patterns from the received usage pattern specification 201. Moreover, parser module 204 may further extract pattern parameters from each of the patterns and convert each of the execution patterns into a machine compatible format. For example, parser module 204 may convert the execution parameters into an XML file. Likewise,
Upon receiving the execution parameters included in usage patterns 202, pattern instruction sequence generation module 206 may generate a unique instruction sequence for each of the plurality of usage patterns. As used herein, an exemplary instruction sequence may include a software program and/or object. For example, the instruction sequence may be embodied in an XML format, a binary format, JavaScript Object Notation (JSON) format, or as file data. Once generated by pattern instruction sequence generation module 206, the instruction sequences are forwarded to pattern execution module 208 as input. For example,
In some embodiments, pattern execution module 208 initiates the distribution and/or loading of the instruction sequences among virtual processing cores 2121 . . . Q of virtual machine 222 for execution (of a CPU usage simulation). For example, pattern execution module 208 may implement a round-robin distribution of instruction sequences 210 across virtual processing cores 2121 . . . Q. Each of virtual processing cores 2121 . . . Q subsequently executes instruction sequences 210, which in turn simulates CPU usage as originally defined by the user. For example, the CPU usage may be defined via the pre-canned simulation patterns and/or the custom defined patterns managed by a processing activity emulator (e.g., processing activity emulator 102).
By provisioning the instruction sequences across virtual processing cores 2121 . . . Q, the selected CPU usage patterns are being emulated on a virtual machine 220 of the DUT. Such a simulation test affords a user the ability to control CPU usage and correlate it to underlying system comprising hardware and/or hypervisor to determine if the system is performing correctly. Notably, the simulation of multiple users utilizing multiple applications based on the number of instruction sequences are being generated and applied to processing cores 2121 . . . Q. In some embodiments, the disclosed subject matter is configured to allow a user to simulate multiple CPU usage patterns via profiles (e.g., pre-canned CPU usage patterns) across multiple servers and to monitor the performance.
After the pattern instruction sequences are executed by the virtual processing cores 212, pattern execution module 208 can generate an electronic status report that indicates the state and/or status of execution for all of the executed pattern instruction sequences 208. Notably, pattern execution module 208 may be configured to assess the status of the executed pattern instruction sequence and send the electronic status report reflecting such stats to statistics and reporting module 113 (shown in FIG. 1). Publisher module 124 may then provide and/or present the collected status information in a visual manner, such as via a graphical user interface (GUI) or a printed report.
After one or more CPU usage patterns is defined by the user as described above, a design module in the processing activity emulator may generate one or more usage pattern specifications by aggregating the CPU usage patterns into one or more usage pattern specifications. The design module may also be configured to forward the generated usage pattern specifications to a provisioning module in the processing activity emulator for distribution.
In step 304, a computer processing usage pattern specification containing a plurality of usage patterns is received. In some embodiments, a virtual machine hosted by a DUT receives a computer processing usage pattern specification, such as a CPU usage pattern specification, from the processing activity emulator (via the provisioning module). The usage pattern specification may comprise a CPU usage pattern specification including one or more CPU usage patterns that i) were customized by a user and/or ii) were predefined CPU usage patterns (e.g., as described in step 302). In some embodiments, a parser module in the virtual machine receives the CPU usage pattern specification and extracts the CPU usage patterns to conduct a translation process (e.g., convert the patterns included CPU usage pattern specification to a format that is comprehended by the pattern instruction sequence generator).
In step 306, a plurality of pattern instruction sequences using execution parameters included in each of the plurality of CPU usage patterns is generated. In some embodiments, the pattern instruction sequence generation module in the virtual machine processes the extracted execution parameters, and subsequently generates a pattern instruction sequence for each respective usage pattern (containing said execution parameters). After being generated, the pattern instruction sequences are sent to a pattern execution module residing in the virtual machine.
In step 308, the pattern instruction sequences are distributed. In some embodiments, the pattern execution module is configured to distribute the pattern instruction sequences among a plurality of virtual processing cores (e.g., virtual CPU cores) utilized by the virtual machine. For example, the pattern execution may distribute the pattern instruction sequences among the virtual processing cores in a round-robin fashion, or in accordance with some performance based or load based parameters.
In step 310, the operation of the application is emulated on the virtual machine. In some embodiments, the operation of the application is emulated by executing the pattern instruction sequences on the virtual processing cores in a manner specified by the execution parameters. Further, the performance of each of the virtual processing cores conducting the simulation may be monitored or measured by the statistics and reporting module 113.
It should be noted that processing activity emulator 102, DUT 104, and/or functionality described herein (e.g., see
It will be understood that various details of the subject matter described herein may be changed without departing from the scope of the subject matter described herein. Furthermore, the foregoing description is for the purpose of illustration only, and not for the purpose of limitation, as the subject matter described herein is defined by the claims as set forth hereinafter.
Number | Name | Date | Kind |
---|---|---|---|
6542854 | Yang et al. | Apr 2003 | B2 |
6769054 | Sahin et al. | Jul 2004 | B1 |
6792393 | Farel et al. | Sep 2004 | B1 |
7328134 | Burbidge, III et al. | Feb 2008 | B1 |
7890951 | Vinberg et al. | Feb 2011 | B2 |
8984341 | Chandrasekharapuram et al. | Mar 2015 | B1 |
9317252 | Roy et al. | Apr 2016 | B2 |
20020087282 | Millard | Jul 2002 | A1 |
20020184614 | Davia et al. | Dec 2002 | A1 |
20030036897 | Flores et al. | Feb 2003 | A1 |
20030154432 | Scott et al. | Aug 2003 | A1 |
20030182408 | Hu | Sep 2003 | A1 |
20040015600 | Tiwary et al. | Jan 2004 | A1 |
20040139437 | Arndt | Jul 2004 | A1 |
20050216234 | Glas et al. | Sep 2005 | A1 |
20060025985 | Vinberg et al. | Feb 2006 | A1 |
20060037002 | Vinberg et al. | Feb 2006 | A1 |
20070067374 | Iketani et al. | Mar 2007 | A1 |
20070069005 | Dickerson et al. | Mar 2007 | A1 |
20070112549 | Lau et al. | May 2007 | A1 |
20080208554 | Igarashi | Aug 2008 | A1 |
20080221857 | Casotto | Sep 2008 | A1 |
20090089038 | Nadgir et al. | Apr 2009 | A1 |
20090259704 | Aharoni et al. | Oct 2009 | A1 |
20100111494 | Mazzaferri | May 2010 | A1 |
20100153529 | Moser | Jun 2010 | A1 |
20100161864 | Barde et al. | Jun 2010 | A1 |
20110066786 | Colbert | Mar 2011 | A1 |
20110246171 | Cleeton et al. | Oct 2011 | A1 |
20110307739 | El Mahdy | Dec 2011 | A1 |
20120054740 | Chakraborty et al. | Mar 2012 | A1 |
20120060167 | Salsburg et al. | Mar 2012 | A1 |
20120192182 | Hayward et al. | Jul 2012 | A1 |
20120311387 | Santhosh et al. | Dec 2012 | A1 |
20130013657 | Emelko et al. | Jan 2013 | A1 |
20130055026 | Hatano et al. | Feb 2013 | A1 |
20130282354 | Sayers et al. | Oct 2013 | A1 |
20130297769 | Chang et al. | Nov 2013 | A1 |
20140006358 | Wang et al. | Jan 2014 | A1 |
20140013306 | Gounares et al. | Jan 2014 | A1 |
20140047272 | Breternitz et al. | Feb 2014 | A1 |
20140067940 | Li et al. | Mar 2014 | A1 |
20140068335 | Bromley et al. | Mar 2014 | A1 |
20140108001 | Brown et al. | Apr 2014 | A1 |
20140298335 | Regev et al. | Oct 2014 | A1 |
20140378057 | Ramon et al. | Dec 2014 | A1 |
20150046141 | Lahiri et al. | Feb 2015 | A1 |
20150120797 | Roy et al. | Apr 2015 | A1 |
20150135178 | Fischer et al. | May 2015 | A1 |
20150140956 | Prewitt, II et al. | May 2015 | A1 |
20150293826 | Sincan et al. | Oct 2015 | A1 |
20160034372 | Panda et al. | Feb 2016 | A1 |
Number | Date | Country |
---|---|---|
WO 2014160660 | Oct 2014 | WO |
WO 2015023369 | Feb 2015 | WO |
Entry |
---|
Final Office Action for U.S. Appl. No. 14/224,024 (Jun. 15, 2016). |
Applicant-Initiated Interview Summary for U.S. Appl. No. 13/969,085 (Jun. 7, 2016). |
Communication of European publication number and information on the application of Article 67(3) EPC for European Application No. 14836839.2 (May 25, 2016). |
Applicant-Initialed Interview Summary for U.S. Appl. No. 14/224,024 (May 13, 2016). |
Notice of Allowance and Fee(s) Due for U.S. Appl. No. 14/445,921 (May 12, 2016). |
Final Office Action for U.S. Appl. No. 13/969,085 (Apr. 19, 2016). |
Non-Final Office Action for U.S. Appl. No. 14/251,547 (Apr. 26, 2016). |
Applicant-Initiated Interview Summary for U.S. Appl. No. 14/445,921 (Apr. 14, 2016). |
Non-Final Office Action for U.S. Appl. No. 14/224,024 (Feb. 3, 2016). |
Corrected Notice of Allowability for U.S. Appl. No. 14/158,659 (Jan. 11, 2016). |
Notice of Allowance and Fee(s) Due for U.S. Appl. No. 14/158,659 (Dec. 11, 2015). |
Non-Final Office Action for U.S. Appl. No. 14/445,921 (Jan. 14, 2016). |
Non-Final Office Action for U.S. Appl. No. 13/969,085 (Sep. 24, 2015). |
“Fisher-Yates Shuffle,” http://en.wikipedia.org/wiki/Fisher-Yates—shuffle, pp. 1-11, (May 8, 2014). |
“ImpairNetTM—EIM1G4S, EIM10G4S, and EIM40G2Q Ethernet Impairment Lead Module,” pp. 1-5, (Jul. 2013). |
“IxVM: Validating Virtualized Assets and Environments,” Ixia, Data Sheet, pp. 1-8 (Jun. 2015). |
Notification of Transmittal of the International Search Report and the Written Opnion of the International Searching Authority, or the Declaration for PCT International Application No. PCT/US2014/045658 (Oct. 30, 2014). |
Notification of Transmittal of the International Search Report and the Written Opnion of the International Searching Authority, or the Declaration for PCT International Application No. PCT/US2014/031637 (Jul. 31, 2014). |
“IxLoad: Specifications,” https://web.archive.org/web/20130901094417/http://www.ixiacom.com/products/networktest/applications/ixload/specifications/index.php, pp. 1-5 (Sep. 1, 2013). |
“IxLoad” Ixia, Solution Brief, pp. 1-4 (Feb. 2012). |