Patent Application
20130031416
Typical test frameworks, particularly unit-testing methods, revolve around source code, usually of a single application. The drawback of this approach is that it is difficult to test systems where multiple applications or hardware platforms are required to interact with each other. It is left to the test developer to write the code to start up and connect to each component, establish communication between them, and handle any errors. As a result, test development effort tends to be platform specific—a barrier to code re-use—and the test developer often ends up writing more auxiliary code than actual tests.
It is clearly desirable for tests to be retargettable, to be able to test different hardware platforms or components without changing the test itself. However, these components are often vastly disparate, perhaps requiring different connection methods or implementing different APIs.
Entity-oriented testing as taught here facilitates a shift in the way tests are conceived and developed, moving away from single-platform, single-application tests and frameworks. Test components—entities—are handled by the test framework and wrapped in a common API (application programming interface) which provides command execution, file handling and inter-communication. They become interchangeable parameters to the test, hiding platform-specific code from the test developer and promoting code re-use. Retargettability is enabled by allowing specific entity instances—physical machines, for example—to be specified on a per test run basis, without changing the generic test code.
Some of the purposes of the invention having been stated, others will appear as the description proceeds, when taken in connection with the accompanying drawings, in which:
While the present invention will be described more fully hereinafter with reference to the accompanying drawings, in which a preferred embodiment of the present invention is shown, it is to be understood at the outset of the description which follows that persons of skill in the appropriate arts may modify the invention here described while still achieving the favorable results of the invention. Accordingly, the description which follows is to be understood as being a broad, teaching disclosure directed to persons of skill in the appropriate arts, and not as limiting upon the present invention.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present invention has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the invention. The embodiment was chosen and described in order to best explain the principles of the invention and the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.
Processor 100 comprises an on-chip level one (L1) cache 190, an instruction buffer 130, control circuitry 160, and execution units 150. Level 1 cache 190 receives and stores instructions that are near to time of execution. Instruction buffer 130 forms an instruction queue and enables control over the order of instructions issued to the execution units. Execution units 150 perform the operations called for by the instructions. Execution units 150 may comprise load/store units, integer Arithmetic/Logic Units, floating point Arithmetic/Logic Units, and Graphical Logic Units. Each execution unit comprises stages to perform steps in the execution of the instructions received from instruction buffer 130. Control circuitry 160 controls instruction buffer 130 and execution units 150. Control circuitry 160 also receives information relevant to control decisions from execution units 150. For example, control circuitry 160 is notified in the event of a data cache miss in the execution pipeline.
Digital system 116 also may include other components and subsystems not shown, such as: a SP, a Trusted Platform Module, memory controllers, random access memory (RAM), peripheral drivers, a system monitor, a keyboard, a color video monitor, one or more flexible diskette drives, one or more removable non-volatile media drives such as a fixed disk hard drive, CD and DVD drives, a pointing device such as a mouse, and a network interface adapter, etc.
Digital systems 116 may include personal computers, workstations, servers, mainframe computers, notebook or laptop computers, desktop computers, or the like. Processor 100 also communicates with a server 112 by way of Input/Output Device 110. For example, I/O device 110 may comprise a network adapter. Server 112 may connect system 116 with other computers and servers 114. Thus, digital system 116 may be in a network of computers such as the Internet and/or a local intranet. Further, server 112 may control access to another memory 118 comprising tape drive storage, hard disk arrays, RAM, ROM, etc.
In one mode of operation of digital system 116, the L2 cache receives from memory 108 data and instructions expected to be processed in a pipeline of processor 100. L2 cache 102 is fast memory located physically close to processor 100 to achieve greater speed. The L2 cache receives from memory 108 the instructions for a plurality of instruction threads. Such instructions may include branch instructions. The L1 cache 190 is located in the processor and contains data and instructions preferably received from L2 cache 102. Ideally, as the time approaches for a program instruction to be executed, the instruction is passed with its data, if any, first to the L2 cache, and then as execution time is near imminent, to the L1 cache.
Execution units 150 execute the instructions received from the L1 cache 190. Execution units 150 may comprise load/store units, integer Arithmetic/Logic Units, floating point Arithmetic/Logic Units, and Graphical Logic Units. Each of the units may be adapted to execute a specific set of instructions. Instructions can be submitted to different execution units for execution in parallel. In one embodiment, two execution units are employed simultaneously to execute certain instructions. Data processed by execution units 150 are storable in and accessible from integer register files and floating point register files (not shown). Data stored in these register files can also come from or be transferred to on-board L1 cache 190 or an external cache or memory. The processor can load data from memory, such as L1 cache, to a register of the processor by executing a load instruction. The processor can store data into memory from a register by executing a store instruction.
Thus, the system of
In an embodiment of the invention, a server 112 is in electrical communication with a plurality of computers to be tested. The server comprises a sequencer 113 that sends command messages to each computer under test to cause execution of certain steps and programs by a computer to verify correct operation. The sequencer 113 is implemented as a program in a directory that is executed by a processor of the server. Each command message from sequencer 113 specifies at least one environment and at least one command. Server 112 further comprises a listener 115. The system under test, for example, digital system 116, comprises a listener 111 that implements the environment specified in a received command message and executes a received command within the environment. A listener 111 is implemented as a program in a directory that is executed by processor 100.
In
One example of an implementation of the embodiment of
There are three parts to implementing entity-oriented testing into a test framework:
(1) The interface for allowing entities to be passed as arguments to the test framework and instantiated internally;
(2) The platform-specific code for connecting and executing commands on these entities; and
(3) The common API for exposing entities to the test developer.
It is to be understood that, as here used, “entities” and “test framework” refer to program code written to be executed on the processor of a digital data handling system and to be stored in a storage element associated with such a system. “Entities” are smaller portions of code written with the intention of being used in a range of test routines. “Test frameworks” may be larger portions of code written with the intention of being used in test routines for specific systems or system configurations.
A key to understanding entities as discussed here is the distinctions among entity labels, entity types and entity identifiers. This separates a ‘type’ of entity—e.g. virtual machine, database, operating system—from a specific instance of it—e.g. a Vmware instance running locally, a remote MySQL server. The test developer is only interested in the entity label and the type. For example, a test may be written declaring that a ‘Unix shell’ entity is needed, knowing what commands will work on that type of entity, but not having to worry about what physical hardware or connection is used to interact with the entity.
On the other hand, the test framework is mostly concerned with the entity identifier and class, which it uses to construct an entity object and expose it to the test developer through its label.
It is contemplated that the list of classes which the test framework supports is finite, and there is an initial overhead in implementing the platform-specific code in the test framework before tests can be written to use that class of entity. It is a trade-off, however; the up-front work to add the support pays off during test development, in vastly reduced lines of code and increased maintainability.
Every class of entity in the test framework implements a common interface, since no knowledge of the underlying API is necessary. In the simplest implementation, the test framework just needs to know how to set up or connect to the entity (if required), and how to execute commands on it. Therefore, each class of entity has a setup method and an execute method, which handles the detail of how to execute the command and returns the result in a standard format. Basic error handling can be performed in the test framework.
As an example, imagine a simple client-server test: a UNIX server must set some environment variables and issue a start command to a MySQL database. A Windows client then connects to it, runs a simple query, and checks that the result is correct.
There could be three entities here:
The UNIX server. For the purposes of this example, the class is UNIX and the label server.
The MySQL database. Class mysql, label database.
The Windows client. Class Windows, label client.
It is contemplated that the test framework knows how to connect to and issue commands to each of these types of entity. The first thing the test developer must do is declare the entity labels they to be used in the test, along with the class for each one. This could look like (in a test configuration file):
The test developer uses these labels to retrieve the corresponding entity objects, which will be instantiated by the test framework in each test run. The pseudocode for the test might look something like:
Note how the entities are retrieved in the test from the test framework using only their label. The details of how the framework connects to each entity is hidden from the test developer. Every entity implements the execute method to the same interface. For the example, it is assumed that the return value of a call to execute has a ‘status’ property in each case.
The only remaining step is to run the test, providing an instance of each entity to be used for testing. This can be as simple as providing a mapping of label=>identifier; as long as the identifier is in a format understood by the test framework, this is sufficient to construct entity objects. For example, the identifier for client and server could be the hostnames or IP addresses assigned to those machines. The identifier for database could be the network port that the database is listening on.
As an example command to run the test framework:
During initialization, the test framework will construct the entity objects internally, using the platform-specific code that has been implemented in the test framework, retrieving the test components from storage and transforming the retrieved test components into a runtime instance operable within the computer program test execution framework. The test developer can then retrieve these objects using only the label, as demonstrated by getEntity in the pseudocode above, and begin to execute commands on them in the test.
The flowchart of
One or more aspects of the present invention can be included in an article of manufacture (e.g., one or more computer program products) having, for instance, tangible computer usable media, indicated at 400 in
In the drawings and specifications there has been set forth a preferred embodiment of the invention and, although specific terms are used, the description thus given uses terminology in a generic and descriptive sense only and not for purposes of limitation.
This application is continuation of prior copending application Ser. No. 13/189,805 filed Jul. 25, 2011, the priority of which is claimed.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 13189805 | Jul 2011 | US |
| Child | 13486703 | US |
