Patent Application
20070046282
1. Field of the Invention
The present invention relates generally to an improved data processing system and in particular to a method, system, and computer usable code for processing data. Still more particularly, the present invention relates to a method, system, and computer usable code for generating test grid environments in grid computing.
2. Description of the Related Art
To save money related to the costs of maintaining computer resources, companies may outsource certain data processing operations. For example, a company maintains a certain amount of data processing resources to handle day-to-day data processing workloads, though from time to time the company may require additional data processing resources to handle overflow data processing workloads. Instead of paying the cost to maintain data processing resources sufficient to handle relatively infrequent workloads, the customer company pays a smaller cost to a provider to provide the data processing resources when needed. A customer that desires such outsourcing works with the provider to define and manage an arrangement that describes the work to be outsourced and the resources that need to be maintained. The arrangement between the customer and the provider may be referred to as a service level agreement.
Currently, grid computing is used to implement a service level agreement. Grid computing environments are data processing environments that enable software applications to integrate instruments, displays, and computational and information resources even when the software applications are managed by diverse organizations in widespread locations. Grid computing environments are different from other distributed network environments in that data processing systems in a grid computing environment share resources, even if the data processing systems are located in different geographic locations, are based on different architectures, or belong to different management domains. Thus, a computing grid represents a powerful pool of computing resources.
To implement a service level agreement, the local grid maintained by the customer is connected to one or more remote grids maintained by the provider. Resources from the remote grid are allocated to the local grid as defined in the service level agreement. However, the service level agreement may also provide that a customer may use a variable amount of resources on the remote grid. In this case, a customer may request remote resources to be allocated dynamically. However, a resource request in such an environment is often a complex description of requirements for hardware, software, networks, applications, and other systems which must be parsed before decisions may be made about available resource pools, pricing, and time to allocate such resources. Significant time, such as days to weeks, may be required to implement the requested changes.
In addition, the provider may not be able to predict which resources will be needed to handle a dynamic request for resources or which resources will physically function with other types of resources. Thus, it is desirable for a provider to build test grid environments in order to test the operation of a particular system configuration. Each test grid environment is designed to handle a particular type of customer request, though several test grid environments may have to be created and subsequent tests performed before a provider will know which system configuration should be finally implemented. Currently, a human operator performs the task of creating and running testing environments. However, due to the complexity of creating and running test grid environments, performing this task is time consuming, error prone, and difficult. Thus, it would be advantageous to have an improved method, apparatus, and computer usable code for generating test grid environments in grid computing.
The present invention provides a method, system, and computer usable code for generating a description of a test environment for use in a grid computing environment. A database containing a number of test snapshots is generated. Each test snapshot reflects a previously used grid test environment, and each test snapshot includes a grid configuration used to implement a particular test scenario for a particular application. When a new test scenario is generated, a description of the new test scenario is entered as a query to the database. Based on the information in the database, a proposed test grid environment description is produced.
The novel features believed characteristic of the invention are set forth in the appended claims. The invention itself, however, as well as a preferred mode of use, further objectives and advantages thereof, will best be understood by reference to the following detailed description of an illustrative embodiment when read in conjunction with the accompanying drawings, wherein:
The present invention provides a method, system, and computer usable code for generating a description of a test environment for use in a grid computing environment. The data processing device may be a stand-alone computing device or may be a distributed data processing system in which multiple computing devices are utilized to perform various aspects of the present invention. Therefore,
With reference now to the figures,
In the depicted example, server 104 connects to network 102 along with storage unit 106. In addition, clients 108, 110, and 112 connect to network 102. These clients 108, 110, and 112 may be, for example, personal computers or network computers. In the depicted example, server 104 provides data, such as boot files, operating system images, and applications to clients 108-112. Clients 108, 110, and 112 are clients to server 104. Network data processing system 100 may include additional servers, clients, and other devices not shown.
In the depicted example, network data processing system 100 is the Internet with network 102 representing a worldwide collection of networks and gateways that use the Transmission Control Protocol/Internet Protocol (TCP/IP) suite of protocols to communicate with one another. At the heart of the Internet is a backbone of high-speed data communication lines between major nodes or host computers, consisting of thousands of commercial, government, educational and other computer systems that route data and messages. Of course, network data processing system 100 also may be implemented as a number of different types of networks, such as for example, an intranet, a local area network (LAN), or a wide area network (WAN).
Referring to
Peripheral component interconnect (PCI) bus bridge 214 connects to I/O bus 212 provides an interface to PCI local bus 216. A number of modems may be connected to PCI local bus 216. Typical PCI bus implementations will support four PCI expansion slots or add-in connectors. Communications links to clients 108-112 in
Additional PCI bus bridges 222 and 224 provide interfaces for additional PCI local buses 226 and 228, from which additional modems or network adapters may be supported. In this manner, data processing system 200 allows connections to multiple network computers. A memory-mapped graphics adapter 230 and hard disk 232 may also be connected to I/O bus 212 as depicted, either directly or indirectly.
Those of ordinary skill in the art will appreciate that the hardware depicted in
The data processing system depicted in
With reference now to
In the depicted example, local area network (LAN) adapter 312, audio adapter 316, keyboard and mouse adapter 320, modem 322, read only memory (ROM) 324, hard disk drive (HDD) 326, CD-ROM drive 330, universal serial bus (USB) ports and other communications ports 332, and PCI/PCIe devices 334 connect to ICH 310. PCI/PCIe devices may include, for example, Ethernet adapters, add-in cards, PC cards for notebook computers, etc. PCI uses a card bus controller, while PCIe does not. ROM 324 may be, for example, a flash binary input/output system (BIOS). Hard disk drive 326 and CD-ROM drive 330 may use, for example, an integrated drive electronics (IDE) or serial advanced technology attachment (SATA) interface. A super I/O (SIO) device 336 may be connected to ICH 310.
An operating system runs on processor 302 and coordinates and provides control of various components within data processing system 300 in
Instructions for the operating system, the object-oriented programming system, and applications or programs are located on storage devices, such as hard disk drive 326, and may be loaded into main memory 304 for execution by processor 302. The processes for embodiments of the present invention are performed by processor 302 using computer implemented instructions, which may be located in a memory such as, for example, main memory 304, memory 324, or in one or more peripheral devices 326 and 330. These processes may be executed by any processing unit, which may contain one or more processors.
Those of ordinary skill in the art will appreciate that the hardware in
As some illustrative examples, data processing system 300 may be a personal digital assistant (PDA), which is configured with flash memory to provide non-volatile memory for storing operating system files and/or user-generated data.
A bus system may be comprised of one or more buses, such as system bus 206, I/O bus 212 and PCI buses 216, 226 and 228 as shown in
The present invention provides a method, apparatus, and computer usable code for generating a description of a test environment for use in a grid computing environment. A database containing a number of test snapshots is generated. Each test snapshot reflects a previously used grid test environment, and each test snapshot includes a grid configuration used to implement a particular test scenario for a particular application. When a new test scenario is generated, a description of the new test scenario is entered as a query to the database. Based on the information in the database, a proposed test grid environment description is produced.
The proposed test grid environment description may be compared to a current grid environment description. If desired, the current grid environment may be changed to match the test grid environment description. However, several test grid environment descriptions may be generated and compared to the current grid environment description before actually changing the current grid environment. After changing the current grid environment to match the test grid environment, if necessary, a test scenario may be implemented on the new grid environment.
The user request is transmitted across path 406, which may be implemented as a network such as network 102 shown in
These plug-ins are shown in section 420 and are a part of a plug-in library. These plug-ins may provide functions such as, for example, Login, System Request, System Software Request, System Hardware Request, Application Middleware Request, Geography, Security and Capacity/Performance, as shown in section 420. The System Request may be for a server or storage in this example, the System Software Request may be for a operating system, microcode, or drivers in this example, and the Systems Hardware Request may be for a processor, network, or storage in this example. An error or unavailable signal may be generated at any point based upon the state of the user request.
Similarly, server 520 may be described using operating system 526, basic input/output 528, on board memory 530, and processor 532. A network may be described by switch 534 and the type of connectivity 536. In each of these instances, a hierarchy of atomistic resources, such as processor 532, storage 522, database 516, and system management 510, and compound resources, such as software 504 and hardware 506, are used to define the grid environment. A plurality of such grid environments may be further organized into a larger grid environment.
The illustrative grid environment shown in
In the illustrative example shown in
Each of the major factors described above may have sub factors. For example, performance 604 includes application inputs 612 and system configuration 614. Application inputs 612 describe how application inputs affect the performance of the application in the particular configuration and on the particular grid environment. Similarly, system configuration 614 may affect application performance. In addition, scalability 606 may also be affected by application inputs 618, which may be different from or similar to application inputs 612. In addition, fault tolerance 608 may depend on error injections 620, which may in turn depend on errors from user inputs 622, bugs in software 624, and problems in hardware 628.
The illustrative application test scenario shown in
The database associated with test snapshot 700 may include basic information such as which particular application test scenario is associated with which particular grid environment. In this case, it may be assumed that a particular application test scenario will function adequately in the corresponding grid environment. However, the database associated with test snapshot 700 may also include additional information. For example, the database associated with test snapshot 700 may include information regarding how well a particular application test scenario operates in an associated grid environment, manually entered notes, or other information relevant to the application test scenario and the corresponding grid environment. The database may also include constructs that will be useful for comparison, such as the actual test cases and constants surrounding the test case. Those constructs may be items such as, successful execution of the test case, outcome of the test case, errors associated with the test case, time required to execute test case, resources available to the test case, or need of assistance for the test case to execute successfully.
Without library 900, if a direct comparison against all known test cases were attempted, the search would become very expensive. The idea is to gather a vast array of test templates, test environment templates, known use case scenarios, and already exercised components for utilization as compatible test grid environments. For example, as shown in block 902, library 900 may be used to compare an existing grid environment with a desired grid environment. The existing grid environment and the desire grid environment are each defined using workflow language.
Workflow language describes the ontology and taxonomy of test grid environments and testing scenarios. Specifically, ontology and taxonomy specifies the test environment components and dependencies in a hierarchical fashion. Current art specifies the hierarchy to build a cluster in terms of components and component based dependencies. For a test environment, the test cases themselves may be predicated on certain tests executing before others are performed. The union of the components and the dependencies is a novel concept for generating dynamic test grid environments. For example, a specific requirement may be a language for describing how a cluster may be constructed (servers, network, I/O, OS) as well as what tests may be run on that cluster simultaneously or independently, e.g. file system performance and gather/scatter computing operations. So each cluster may have a list of components and component dependencies, and relevant tests and test dependencies.
A data processing system may then compare the descriptions of each grid environment and produce a dependency graph to generate missing components in the grid environment, additional test scenarios to be run, and other useful information. This function is useful in any application test environment where repeatability is important.
In addition, as shown in block 904, library 900 may be used to generate a list of potential test grid environments based on a description of an application test scenario. In this case, a user describes the application test scenario using workflow language as previously described. The description of the application test scenario is entered as input for a query to library 900. In response, a search engine searches library 900 for similar application test scenarios. Because each application test scenario in library 900 has a corresponding test grid environment, the search engine returns a list of potential test grid environments that may be used with the desired application test scenario. Furthermore, the search engine may return a dependency graph including the test grid environment, application test scenarios, success criteria, and other factors. A user may use the dependency graph to decide how to modify an existing grid environment to a desired grid environment. An example of a dependency graph is shown in
In addition, as shown in block 906, library 900 may be used to adapt or modify a desired test scenario based on an existing grid environment. In this case, a description of the current grid environment is entered as input for a query to library 900. In response, the search engine returns a list of application test scenarios that are appropriate for use in the current grid environment. Based on information contained in the list, a user may adjust a desired application test scenario, if necessary, such that the application test scenario will function in the existing grid environment. Similarly, if time and resources exist to modify the existing grid environment in some manner, then a range of potential grid environments may be provided as input to a query. In response, the search engine will return a list of application test scenarios that would be appropriate for the list of potential grid environments. The user may then adjust the desired application test scenario accordingly.
Initially, a user or the workflow language program generates a description of a desired test scenario (step 1100). The user or the workflow language program uses workflow language to develop a provisioning workflow (step 1102). The provisioning workflow language, as previously described, is adapted to modify a grid environment to accommodate a particular application test environment.
Next, a determination is made whether the test scenario will be submitted without a particular grid configuration (step 1104). If the test scenario is submitted without a particular grid configuration, then the workflow program automatically generates a test grid environment (step 1106). The workflow program uses an agent installed on the current grid environment to describe the current grid environment. The description of the current grid environment is provided as input to the workflow program. The workflow program then uses the description of the current grid environment, a library, and the desired application test scenario to generate a description of the test grid environment. Alternatively, the agent itself automatically generates a description of the test grid environment by associating key application and test criteria using a library, such as library 900 shown in
Returning to step 1104, if the test scenario is to be submitted with a grid configuration, the workflow program queries a library, such as library 900 in
The workflow program then builds any bindings necessary to effectuate changes in the current grid environment (step 1116). For example, configuration scripts to configure the servers, these bindings may be as simple as changing operating system parameters (number of concurrent jobs, to start specific daemons on a UNIX® system) or even more complex bindings such as loading a specific dependant application for the test. Thereafter, the workflow program actually generates the test grid environment (step 1118). Optionally, the workflow program may cause the test scenario to actually be conducted on the test grid environment (step 1120).
Thereafter, the workflow program determines whether additional test scenarios are to be processed (step 1122). If additional test scenarios are to be processed, then the process returns to step 1100 and the process repeats. Otherwise, the process terminates.
The process begins as a user or the workflow program generates a description of the current grid configuration (step 1200). Then, the user or the workflow program generates a description of the desired grid configuration (step 1202). The workflow program then compares the current grid configuration to the desired-grid configuration (step 1204). Using the library and the results of the comparison in step 1204, the workflow program generates a dependency graph (step 1206). The dependency graph shows components missing from the current grid configuration that would be used to conduct one or more associated test scenarios (step 1208). An example of a dependency graph is shown in
Thus, the present invention provides a method, system, and computer usable code for generating a description of a test environment for use in a grid computing environment. A database containing a number of test snapshots is generated. Each test snapshot reflects a previously used grid test environment, and each test snapshot includes a grid configuration used to implement a particular test scenario for a particular application. When a new test scenario is generated, a description of the new test scenario is entered as a query to the database. Based on the information in the database, a proposed test grid environment description is produced.
The mechanism of the present invention has several advantages over currently available methods for conducting application test scenarios in a grid computing environment. Because the process is semi-automated, a user may design and experiment on test grid environments much more quickly and easily than by using the known manual system of developing and implementing test scenarios. Thus, a provider of grid resources may more quickly adapt to rapidly changing demands of a customer.
The present invention can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment containing both hardware and software elements. In an illustrative embodiment, the invention is implemented in software, which includes but is not limited to firmware, resident software, microcode, etc.
Furthermore, the invention can take the form of a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system. For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk. Current examples of optical disks include compact disk-read only memory (CD-ROM), compact disk-read/write (CD-R/W) and DVD.
A data processing system suitable for storing and/or executing program code will include at least one processor coupled directly or indirectly to memory elements through a system bus. The memory elements can include local memory employed during actual execution of the program code, bulk storage, and cache memories which provide temporary storage of at least some program code in order to reduce the number of times code must be retrieved from bulk storage during execution.
Input/output or I/O devices (including but not limited to keyboards, displays, pointing devices, etc.) can be coupled to the system either directly or through intervening I/O controllers.
Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modem and Ethernet cards are just a few of the currently available types of network adapters.
The description of the present invention has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiment was chosen and described in order to best explain the principles of the invention, the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.
