Management and workflow collaboration software systems are used by organizations to maintain the efficiency of workforces. Among other things, these systems, herein referred to as management systems, allow employees to communicate, obtain information, requisition products and services, generate documents, and perform online training. Management systems may also connect portions of an organization that are separated geographically. As a result, management systems are often spread across multiple servers coupled to the network. The distributed nature of management systems along with the requirement that they perform numerous and often times complex operations simultaneously makes them very large and difficult to test and trouble shoot.
Adding to the complexity of management systems is the fact that many of these systems have evolved over time. This evolution has largely been driven by customer needs and has resulted in rather basic management systems evolving into the large, complex ones predominant today. In addition, users must be able to write their own software applications to extract maximum utility from management systems. The combination of old and new system software executable instructions and customer developed software instructions may further contribute to software environments that are difficult to troubleshoot. For example, an enterprise, or management system may be executing hundreds or thousands of software agents and instructions as a matter of normal operation. A software agent as used herein refers to any self-contained potentially adaptive software application or an executable sequence of instructions, typically written by an end user. These software agents can be short lived, i.e. having lifetimes of less than a second, or can be long-lived having lifetimes measured in hours or days. Some of these software agents may need to operate according to a schedule, while others may be event driven. If software agents are not properly written, they may fail to operate at scheduled times, may fail to start when a triggering event occurs, may operate in infinite loops, run too long, or consume too many system resources such as memory.
End users may also write software instructions for performing input/output interactions with the management system. And, users may write applications making use of templates for facilitating usage of forms and data entry associated with the management system. Template-driven applications refer to software enabled applications which are primarily document based when they are accessed, edited or created. Examples of template-driven applications are, but are not limited to, spread sheets, database user interfaces, on-line requisition forms, personnel data input screens, and the like.
In many situations, these user developed software applications must be tested for suitability in addressing their intended purposes and for troubleshooting applications that are not operating properly. In some environments, individuals performing software testing are not experienced software developers; and therefore, they may not be capable of operating sophisticated software debugging tools such as those used by experienced computer programmers.
Performing software testing associated with management systems may require that a user, such as a quality engineer, duplicate the environment in which a software application will be used in order to test the software application. In addition, the quality engineer may need to populate the duplicate environment with data, representative of actual data used in the application, to facilitate meaningful testing. By way of example, if a quality engineer is testing an email application for use in a management system, he/she may need to create thousands of user accounts spread across multiple hardware platforms. In addition, the quality engineer may need to populate the user accounts with sent and received messages of varying sizes and formats. When the quality engineer is finished with testing, he/she may need to restore the system to its state prior to testing. In the above example, the quality engineer may have populated actual user accounts with messages in order to test a new email server configuration, and after confirming that the server configuration operated correctly, the quality engineer may wish to remove only the messages employed during testing. Prior art systems and methods may not allow users, such as quality engineers, to easily and accurately scrub test data and restore system functionality to pre-testing configurations.
What is needed is a tool that lets non-programmers delete test logic and test environments after testing of complex software applications. In addition, such a tool should not adversely impair or impact the software applications that were tested once the application environment has been restored to its pre-tested state.
The foregoing and other features and advantages of the system and method for client and template validation will be apparent from the following more particular description of preferred embodiments of the system and method as illustrated in the accompanying drawings in which like reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention.
[TO BE COMPLETED AFTER CLAIMS ARE FINALIZED]
Embodiments of the invention let a user, such as a quality engineer, create test environments and populate these environments with data for testing software/hardware systems such as, for example, management systems. In addition, embodiments make it easy for the user to scrub, or remove, test data and test software so that the system can be restored to its pre-test state. Furthermore, the embodiments make it possible for the user to re-use test routines and test data for subsequent system testing. In addition, test routines and test data can be updated without requiring that the user employ complex and time consuming dedicated test configuration management applications. Test routines and test data employed by the invention can be input to automated test solutions such as, for example, SILK.
SILK is a general-purpose simulation and testing language based on the Java™ programming language, and it lets a user create reusable object-oriented simulation/test components and models. SILK is Java based and supports multi-threaded execution which makes it useful for testing and simulating large scale multi-user applications such as enterprise management systems.
Systems and methods consistent with preferred embodiments can operate in essentially any computer hardware and software environment. For example, embodiments can be employed for testing Windows™ based personal computers in standalone or networked environments, Solaris™ or AIX™ based work stations operating alone or in a networked environment, and mainframe computers. In addition, embodiments can be employed in distributed applications operating across a plurality of computer platforms and networking components. One such distributed application environment is that of an enterprise management system such as Lotus Notes™ marketed by International Business Machines Corporation (IBM). By way of illustrative example, a Lotus Notes environment will be used when discussing preferred embodiments of the invention herein; however, it should be understood that Lotus Notes is not a limiting example since other embodiments can be used in conjunction with other applications and/or environments without departing from the spirit and scope of the invention.
Preferred embodiments operate at the application programming interface (API). An API is the specific method prescribed by a computer operating system or by an application program by which a programmer writing a software application program can make requests of the operating system or another application. As such, an API is distinguishable from direct user interfaces such as graphical user interfaces (GUI) or a command interface. An API typically provides a set of commonly used functions which can be used by programmers to achieve a level of abstraction between higher level and lower level software agents or software objects.
Servers 102A, 102B are coupled to clients 106A, 106B, respectively. Clients 106A, 106B and servers 102A, 102B may also interact with a test environment creation and cleanup utility 108A, 108B, hereinafter referred to as “Utility 108”, which is used to create a test environment for testing applications running on system 100. Utility 108 uses an XML based syntax that facilitates creation of test environments and then facilitates restoration of system 100 to its pre-test configuration once testing is complete.
Network 110 may be used for communicatively coupling elements of system 100. Network 110 may be any type of communications network running any networking protocol. For example, network 110 may be an intranet such as a local area network (LAN) operating within a corporate location or university campus, a metropolitan area network (MAN) operating within a geographic region such as a city and its surrounding suburbs, or a wide area network (WAN) such as the world wide web. In addition, network 110 may run any type of networking protocol such as, for example, transmission control protocol and Internet protocol (TCP/IP), asynchronous transfer mode (ATM), synchronous optical network (Sonet), frame relay, integrated services digital network (ISDN), and open shortest path first (OSPF). Network 110 may employ a plurality of links 114 for coupling network elements and locations. Links 114 may include hardwired links and/or wireless links. Examples of hardwired links include, but are not limited to, coaxial cable, twisted pair cable, and optical fibers; and examples of wireless links include, but are not limited to, radio frequency (RF) such as IEEE 802.11 based links, or free space optical links.
System 100 may also include gateways and/or firewalls 112A, 112B for providing access to network 110 and for providing protection against undesirable network traffic such as denial-of-service attacks and network traffic containing malicious code such as computer worms and viruses. Links 116 are used to connect elements on the protected side of gateway/firewall 112A and 112B. The implementation of management system 100 shown in
Servers 102C, 102D are coupled to clients 106C, 106D, respectively, which may be Domino clients and are hereinafter generally referred to as Clients 106. Management system 100B may further include one or more test environment creation and cleanup utilities 108C and D, respectively. Utilities 108C and D facilitate application programming interface (API) layer based testing of management applications developed for use on Clients 106 using an XML based syntax.
Database object 120 may be populated with one or more documents 122. Since document 122 is represented one layer below database object 120, it is a child object. Document 122 may contain a plurality of fields 124 and 126, respectively, as grand children. Fields 124 and 126 may contain other lower level software objects such as type 128, 132 and value 130, 134. When a test environment is created for database 120, document 122, fields 124 and 126, types 128 and 132, and values 130 and 134 may be populated with test data. After testing is completed, test data can be deleted from the nested objects of
Since the Utility operates at the API, it is beneficial to have it load at application, or system, startup and to be available to all applications running on system 100A. The Utility is loaded as a dynamic linking library (DLL) in a preferred embodiment. A DLL is a binary file containing functions which are called by one or more processes operating on a computer. A DLL is loaded into system memory at system startup, or at the startup of a particular software application. Once a DLL is loaded, multiple applications can access it simultaneously. For example, if a database application, such as that of
In an alternative embodiment, the Utility can be implemented as an executable, or .exe, file. Once an XML file is created and associated with one or more objects, it may be launched using a GUI or by way of a command line interface, such as Microsoft DOS™. For example, a user can use a GUI based tool to generate an XML file. Then the XML file can be distributed across a plurality of platforms such as, for example, a personal computer operating system such as Windows™, Apple Computing's MAC OS™ operating system, Sun Microsystems Solaris™ operating system, and the like, for execution by way of a command line instruction. Alternatively, the system 100B may generate the XML or an application such as SILK can generate XML. In addition, XML may be used to perform functions such as serializing the display of pop up windows such as those shown in
Returning to
If the environment creation tool should be configured as a .exe file, the method goes to “A” in
Returning again to
A user may specify one or more include files for association with the menu command (per step 170). For example. Include files having # define in the header file may be associated with the menu command. Continuing with the example, a user can use MY_DATABASE_NAME value which may be defined in the header file for Database Name by way of the application. If the user needs to change this Database Name, the user can employ a different header file in which MY_DATABASE_NAME is defined using a different value.
After specifying the include files in step 170, the user may enter a start ID and password, if required (per step 172). Next, the user enters the name of the Cleaner File for cleaning the environment after testing is complete (per step 174).
Now referring to
If the user selects creating a database environment object type, the method goes to
Returning to 4D, the user fills in required fields using an input device such as a keyboard (per step 202). Then the user clicks on Add New Object which prompts the user to select the object type from a pop up window (per step 204). An exemplary pop up window is shown in
Next the user selects a database object (per step 206) and clicks OK (per step 208). Then a Database Object Dialog window is presented to the user as shown in
Returning again to
If the user creates a person environment object type in step 190, the method proceeds to
Returning now to
When OK is clicked, the system writes the person object's attributes to an XML file. By way of example, the data from a person object attribute window may be saved in the following format:
If the user desires to create a cleaner environment object type in step 192, the method illustrated in
Returning now to
After performing the methods associated with
The Utility determines if existing menus are being modified (per step 270). If an existing menu is being modified, Modify Menu Item is selected by checking on the File menu in the menu bar of the display (per step 272). The user then selects a menu item to modify using a pointing device and clicks OK to enter the selection.
In contrast, a determination is made as to whether an existing menu is being deleted (per step 276) if a pre-existing menu is not being modified in step 270. If an existing menu is being deleted, Delete Menu Item is selected by clicking on File menu in the menu bar (per step 278). Then a menu item for deletion is selected using a point device (per step 280).
Returning to
Returning to
Turning to
Many computing platforms are also provided with one or more storage drives 664, such as a hard-disk drives (HDD), floppy disk drives, compact disc drives (CD, CD-R, CD-RW, DVD, DVD-R, etc.), and proprietary disk and tape drives (e.g., Iomega Zip™ and Jaz™, Addonics SuperDisk™, etc.). Additionally, some storage drives may be accessible over a computer network. The RAM is capable of storing machine-readable instructions and information necessary to operate software applications under test as well as validation and test automation tools.
Many computing platforms are provided with one or more communication interfaces 666, according to the function intended of the computing platform. For example, a personal computer is often provided with a high speed serial port (RS-232, RS-422, etc.), an enhanced parallel port (EPP), and one or more universal serial bus (USB) ports. The computing platform may also be provided with a local area network (LAN) interface, such as an Ethernet card, and other high-speed interfaces such as the High Performance Serial Bus IEEE-1394.
Computing platforms such as wireless telephones and wireless networked PDA's may also be provided with a radio frequency (RF) interface with antenna, as well. And in some cases, the computing platform may be provided with an infrared data arrangement (IrDA) interface, too.
Computing platforms are often equipped with one or more internal expansion slots 668, such as Industry Standard Architecture (ISA), Enhanced Industry Standard Architecture (EISA), Peripheral Component Interconnect (PCI), or proprietary interface slots for the addition of other hardware, such as sound cards, memory boards, and graphics accelerators.
Additionally, many units, such as laptop computers and PDA's, are provided with one or more external expansion slots 670 allowing the user the ability to easily install and remove hardware expansion devices, such as PCMCIA cards, SmartMedia cards, and various proprietary modules such as removable hard drives, CD drives, and floppy drives.
Often, the storage drives 664, communication interfaces 10, internal expansion slots 668 and external expansion slots 670 are interconnected with the CPU 650 via a standard or industry open bus architecture 672, such as ISA, EISA, or PCI. In many cases, the bus 672 may be of a proprietary design.
A computing platform is usually provided with one or more user input devices, such as a keyboard or a keypad 674, and mouse or pointing device 676, and/or a touch-screen display 678. In the case of a personal computer, a full size keyboard is often provided along with a mouse or pointer device, such as a track ball or TrackPoint™. In the case of a web-enabled wireless telephone, a simple keypad may be provided with one or more function-specific keys. In the case of a PDA, a touch screen 678 is usually provided, often with handwriting recognition capabilities.
Additionally, a microphone 680, such as the microphone of a personal computer, is supplied with the computing platform. This microphone may be used for simply reporting audio and voice signals, and it may also be used for entering user choices, such as voice navigation of web sites or auto-dialing telephone numbers, using voice recognition capabilities.
Many computing platforms are also equipped with a camera device 682, such as a still digital camera or full motion video digital camera which can be used for employee collaboration such as web conferencing.
One or more user output devices, such as a display 686, are also provided with most computing platforms. The display 686 may take many forms, including a Cathode Ray Tube (CRT), a Thin Flat Transistor (TFT) array, or a simple set of light emitting diodes (LED) or liquid crystal display (LCD) indicators.
One or more speakers 688 and/or annunciators 690 are often associated with computing platforms, too. The speakers 688 may be used to reproduce audio and music, such as the speaker of a wireless telephone or the speakers of a personal computer. Annuciators 690 may take the form of simple beep emitters or buzzers, commonly found on certain devices such as PDAs and PIMs.
These user input and output devices may be directly interconnected (672′, 672″) to the CPU 650 via a proprietary bus structure and/or interfaces, or they may be interconnected through one or more industry open buses such as ISA, EISA, PCI, etc.
The computing platform is also provided with one or more software and firmware 684 programs to implement the desired functionality of the computing platforms.
Turning now to
Additionally, one or more portable or device-independent programs 686 may be provided, which must be interpreted by an OS-native platform-specific interpreter 688, such as Java™ scripts and programs.
Often, computing platforms are also provided with a form of web browser or micro-browser 690, which may also include one or more extensions to the browser such as browser plug-ins 692.
The computing device is often provided with an operating system 694, such as, for example, Microsoft Windows™, UNIX®, IBM OS/2™, or AIX®, LINUX, MAC OS™, Sun Solaris™, or other platform specific operating systems. Smaller devices such as PDA's and wireless telephones may be equipped with other forms of operating systems such as real-time operating systems (RTOS) or Palm Computing's PalmOS™.
A set of basic input and output functions (BIOS) and hardware device drivers 696 are often provided to allow the operating system 694 and programs to interface to and control the specific hardware functions provided with the computing platform.
Additionally, one or more embedded firmware programs 696 are commonly provided with many computing platforms, which are executed by onboard or “embedded” microprocessors as part of the peripheral device, such as a microcontroller or a hard drive, a communication processor, network interface card, or sound or graphics card.
As such,