Patent Application
20050289517
Management and workflow collaboration software systems are used by organizations to maintain the efficiency of workforces. Among other things, these systems, herein referred to as management systems, allow employees to communicate, obtain information, requisition products and services, generate documents, and perform online training. Management systems may also connect portions of an organization that are separated geographically. As a result, management systems are often spread across multiple servers coupled to the network. The distributed nature of management systems along with the requirement that they perform numerous operations simultaneously makes them very large and complex.
Adding to the complexity of management systems is the fact that many of these systems have evolved over time. This evolution has largely been driven by customer needs and has resulted in rather basic management systems evolving into the large, complex ones predominant today. In addition, users must be able to write their own software applications to extract maximum utility from management systems. The combination of old/new software executable instructions and customer developed software instructions may produce software environments that are difficult to troubleshoot. For example, an enterprise, or management system may be executing hundreds or thousands of software agents and instructions as a matter of normal operation. A software agent as used herein refers to any self-contained potentially adaptive software application or an executable sequence of instructions, typically written by an end user. These software agents can be short lived, i.e. having lifetimes of less than a second, or can be long-lived having lifetimes measured in hours or days. Some software agents may need to operate according to a schedule, while others are event driven. If software agents are not properly written, they may fail to operate at scheduled times, may fail to start when a triggering event occurs, may operate in infinite loops, run too long, or consume too many system resources such as memory.
Users may also write software instructions for performing input/output interactions with the management system. And, in addition, users may write applications making use of templates for facilitating usage of forms and data entry associated with the management system. Template-driven applications refer to software enabled applications which are primarily document based when they are accessed, edited or created. Examples of template-driven applications are, but are not limited to, spread sheets, database user interfaces, on-line requisition forms, personnel data input screens, and the like.
In many situations, user developed software must be tested for suitability in addressing its intended purpose and for troubleshooting applications that are not operating properly. In some environments, individuals performing software testing are not experienced software developers; and therefore, they may not be capable of operating sophisticated software debugging tools such as those used by programmers.
What is needed is a tool that allows non-programmers to create test logic and perform testing of complex software applications. In addition, such a tool should not adversely impair or impact the software applications under test.
A preferred embodiment of the present invention includes a method for validating a software application comprised of machine-readable instructions operating on a computer. The method includes the steps of making an in process memory available, having at least a first portion and a second portion, making a processor available for executing the machine-readable instructions, loading the machine-readable instructions into the memory to cause the instructions to occupy the first portion and receiving a validator having function-executable instructions, wherein the validator is used for verifying the operation of the software application.
In accordance with another aspect of the present invention, an apparatus for validating the operation of a software application includes machine-readable instructions. The apparatus has a memory having a first portion and a second portion. The first portion of the memory stores a first set of instructions associated with the software application and the second portion stores a second set of instructions associated with a validator that validates the software while operating. The memory is further capable of letting the validator monitor the first portion of memory while the software application is operating. A processor is communicatively associated with the memory and responsible for executing the first and second sets of instructions thereby facilitating validation of the software application by the validator. A user interface is also communicatively associated with the processor and is used for displaying a validation result to a user in human-readable form. An output interface is provided and communicatively associated with the processor and used for providing a validation result to an external application.
A preferred embodiment of the present invention includes a method operating on a Notes client for validating the operation of a software application communicatively associated with a server coupled to a network and includes the steps of making an in process memory available. This memory is capable of holding a first set of machine-readable instructions in a first portion of the memory and a second set of machine-readable instructions in a second portion of the memory. The memory is further communicatively associated with a processor. The method includes loading a first set of instructions associated with the software application into the first portion of memory. And, further includes loading a second set of instructions associated with a validator into a second portion of memory. The second set of instructions is further capable of monitoring the first set of instructions to produce a validation result. The validation result being associated with the operation of the software. The method then makes the validation result available to a user interface.
In accordance with another aspect of the present invention, a computer program product having machine-readable instructions for causing a processor to perform a method for validating the operation of a software application operating on a computer. The method includes the steps of making a memory available, the memory having at least a first portion and a second portion loading a validator for use in validating operation of the software application, making the processor available for executing the machine-readable instructions, the machine-readable instructions including a first set associated with the software application and a second set associated with the validator, loading the first set of instructions into the first portion of memory, loading the second set of instructions into the second portion of memory, making the second portion available to the first portion, determining a validation result using the first set and the second set of instructions; and making the validation result available to a user interface.
The foregoing and other features and advantages of the system and method for client and template validation will be apparent from the following more particular description of preferred embodiments of the system and method as illustrated in the accompanying drawings in which like reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention.
The embodiments of the invention allow a user such as a quality engineer, to validate the in process memory utilized by client centric applications. In addition, the embodiments of the present invention facilitate testing of the possible states associated with documents in a system, and further make it possible to test memory in an ad hoc fashion. Furthermore, outputs generated by the preferred embodiments can be input to automated test solutions such as, for example, SILK.
SILK is a general-purpose simulation and testing language based on the Java™ programming language, and it lets a user create reusable object-oriented simulation/test components and models. SILK is comprised of modeling components based on the Java Beans™ component architecture. SILK has several key strengths that make it desirable for simulation activities. First, SILK provides a rich set of Java class libraries which can be assembled into a plurality of modeling constructs. Second, SILK is a process-oriented simulation language that makes it possible to program within a standard programming language using industry standard development environments. Third, SILK provides a visual modeling and test environment where components can be graphically assembled for quickly creating simulation applications. Since SILK is Java based, it supports multi-threaded execution making it useful for simulating large scale multi-user applications such as enterprise management systems.
The servers 102A, 102B are coupled to clients 106A, 106B, respectively, which may be Domino clients and are hereinafter generally referred to as Clients. The Clients 106A, 106B may also interact with client and template validation tool (CTVT) 108A, 108B which is used to monitor the operation of user-developed applications and documents. In addition, CTVT 108A, 108B also provide users with the ability to monitor applications in real-time without adversely impairing the operation of the application under test.
Network 110 may be used for communicatively coupling other elements of management system 100. Network 110 may be any type of communications network running any networking protocol. For example, network 110 may be an intranet such as a local area network (LAN) operating within a corporate location or university campus, a metropolitan area network (MAN) operating within a geographic region such as a city and its surrounding suburbs, or a wide area network (WAN) such as the world wide web. In addition, network 110 may run any type of networking protocol such as, for example, transmission control protocol and Internet protocol (TCP/IP), asynchronous transfer mode (ATM), synchronous optical network (Sonet), frame relay, integrated services digital network (ISDN), and open shortest path first (OSPF). Network 110 may employ a plurality of links 114 for coupling network elements and locations. Links 114 may include hardwired links and/or wireless links. Examples of hardwired links include, but are not limited to, coaxial cable, twisted pair cable, and optical fibers; and examples of wireless links include, but are not limited to, radio frequency (RF) such as IEEE 802.11 based links, or free space optical links.
Management system 100 may also have gateways and/or firewalls 112A, 112B for providing access to network 110 and for providing protection against undesirable network traffic such as denial-of-service attacks as well as network traffic containing malicious code such as computer worms and viruses. Links 116 are used to connect elements on the protected side of gateway/firewall 112A and 112B. The implementation of management system 100 shown in
Many computing platforms are also provided with one or more storage drives 9, such as a hard-disk drives (HDD), floppy disk drives, compact disc drives (CD, CD-R, CD-RW, DVD, DVD-R, etc.), and proprietary disk and tape drives (e.g., Iomega Zip™ and Jaz™, Addonics SuperDisk™, etc.). Additionally, some storage drives may be accessible over a computer network. The RAM is capable of storing machine-readable instructions and information necessary to operate software applications under test as well as validation and test automation tools.
Many computing platforms are provided with one or more communication interfaces 10, according to the function intended of the computing platform. For example, a personal computer is often provided with a high speed serial port (RS-232, RS-422, etc.), an enhanced parallel port (EPP), and one or more universal serial bus (USB) ports. The computing platform may also be provided with a local area network (LAN) interface, such as an Ethernet card, and other high-speed interfaces such as the High Performance Serial Bus IEEE-1394. The computing platform may also be provided with a radio frequency (RF) interface with antenna, as well. And in some cases, the computing platform may be provided with an infrared data arrangement (IRDA) interface, too.
Computing platforms are often equipped with one or more internal expansion slots 11, such as Industry Standard Architecture (ISA), Enhanced Industry Standard Architecture (EISA), Peripheral Component Interconnect (PCI), or proprietary interface slots for the addition of other hardware, such as sound cards, memory boards, and graphics accelerators.
Additionally, many units, such as laptop computers and PDA's, are provided with one or more external expansion slots 12 allowing the user the ability to easily install and remove hardware expansion devices, such as PCMCIA cards, SmartMedia cards, and various proprietary modules such as removable hard drives, CD drives, and floppy drives.
Often, the storage drives 9, communication interfaces 10, internal expansion slots 11 and external expansion slots 12 are interconnected with the CPU 1 via a standard or industry open bus architecture 8, such as ISA, EISA, or PCI. In many cases, the bus 8 may be of a proprietary design.
A computing platform is usually provided with one or more user input devices, such as a keyboard or a keypad 16, and mouse or pointer device 17, and/or a touch-screen display 18. In the case of a personal computer, a full size keyboard is often provided along with a mouse or pointer device, such as a track ball or TrackPoint™. In the case of a portable device such as a PDA, a simple keypad may be provided with one or more function-specific keys as well as a touch-screen 18.
Additionally, a microphone 19, such as the microphone of a web-enabled wireless telephone or the microphone of a personal computer, is supplied with the computing platform. This microphone may be used for simply reporting audio and voice signals, and it may also be used for entering user choices, such as voice navigation of web sites or auto-dialing telephone numbers, using voice recognition capabilities.
Many computing platforms are also equipped with a camera device 29, such as a still digital camera or full motion video digital camera which can be used for employee collaboration such as web conferencing.
One or more user output devices, such as a display 13, are also provided with most computing platforms. The display 13 may take many forms, including a Cathode Ray Tube (CRT), a Thin Flat Transistor (TFT) array, or a simple set of light emitting diodes (LED) or liquid crystal display (LCD) indicators.
One or more speakers 14 and/or annunciators 15 are often associated with computing platforms, too. The speakers 14 may be used to reproduce audio and music, such as the speaker of a wireless telephone or the speakers of a personal computer. Annuciators 15 may take the form of simple beep emitters or buzzers, commonly found on certain devices such as PDAs and PIMs.
These user input and output devices may be directly interconnected (8′, 8″) to the CPU 1 via a proprietary bus structure and/or interfaces, or they may be interconnected through one or more industry open buses such as, for example, ISA, EISA, and PCI.
The computing platform is also provided with one or more software and firmware 30 programs to implement the desired functionality of the computing platforms.
Additionally, one or more portable or device-independent programs 24 may be provided, which must be interpreted by an OS-native platform-specific interpreter 25, such as Java™ scripts and programs.
Often, computing platforms are also provided with a form of web browser or micro-browser 26, which may also include one or more extensions to the browser such as browser plug-ins 27.
The computing device is often provided with an operating system 20, such as, for example, Microsoft Windows™, UNIX®, IBM OS/2™, or AIX®, LINUX, MAC OS™, Sun Solaris™, or other platform specific operating systems. Smaller devices such as PDA's and wireless telephones may be equipped with other forms of operating systems such as real-time operating systems (RTOS) or Palm Computing's PalmOS™.
A set of basic input and output functions (BIOS) and hardware device drivers 21 are often provided to allow the operating system 20 and programs to interface to and control the specific hardware functions provided with the computing platform.
Additionally, one or more embedded firmware programs 22 are commonly provided with many computing platforms, which are executed by onboard or “embedded” microprocessors as part of the peripheral device, such as a microcontroller or a hard drive, a communication processor, network interface card, or sound or graphics card.
As such,
A DLL is a binary file containing functions which can be called by one or more processes operating on a computer. A DLL is loaded into system memory at system startup, or at the startup of a particular software application. Once a DLL is loaded, multiple applications can access it simultaneously. For example, if a database application has three modules input, output and processing, respectively, that need to access a database, a programmer can write a DLL for handling communication to the database. The three modules can then call the DLL to communicate with the database.
A testing tool known in the art may be loaded and initialized (per step 308). By way of example, SILK may be loaded in step 308 to facilitate automation of testing and diagnostics of an application under test. The particular application under test is then executed (per step 310). If the application under test requires user inputs, they may be entered (per step 312). The application under test is monitored (per step 314). The application under test then interacts with the CTVT during operation (per step 316). The CTVT captures information about the application under test, such as, for example, memory usage, sequence of operational steps, input/output operations and data and formulates them into results which are reported to a user such as a quality control engineer (per step 318).
The CTVT provides a nonprogrammer the ability to create and deploy test logic for monitoring the in memory Notes of a Client. This allows the user to validate fields, hide-when formula validations, databases, directories and documents, as well as many other in memory or back end processing not otherwise available to a user of the application under test.
Validations for all field hide-when formulas are added (per step 410), and then the validation may be saved as, for example, “Cal Entry—State 1—New” (per step 412). Calendar entry fields may then be populated in a second state (per step 414).
A CVAddin add validation may be selected using the mouse 17 (per step 416). Then, validations for all field values, field types and field hide-when formulas currently in the open document may be added (per steps 418, 420 and 422, respectively). The validation may then be saved as, for example, “Cal Entry—State 2—Document populated” (per step 424).
A third state may commence when a keystroke combination such as CTRL+S is entered to save a document (per step 426). A CVAddin “ADD” validation is selected (per step 428). Validations for all field values, all field types and all field hide-when formulas may be added in the open document (per steps 430, 432 and 434, respectively). Then the validation may be saved as, for example, “Cal Entry—State 3—Document saved, Computed Occurred” (Per step 436).
The method illustrated with respect to
A user, such as a quality engineer, verifies that all validations pass either manually or using an automated tool such as SILK for accomplishing the verification (per step 446). Calendar entry fields are populated (per step 448). For example, a user may click validator “Cal Entry—State 2—Document populated” using a mouse 17. Then a verification is made that all validations pass (per step 450). The calendar entry is then saved (per step 452). By way of example, the calendar entry may be saved by clicking validator “Cal Entry—State 3—Document saved Computed Occurred” using a computer mouse. Then a verification is made that all validations pass the testing criteria (per step 454).
For the exemplary method of
FIGS. 5A-P illustrate exemplary user displays and graphical user interfaces that are useful when practicing a preferred embodiment of the invention. Embodiments of the invention may be used to evaluate both front end data and back end data created through the use of a client application such as a Notes client. Since the invention operates as a DLL, it runs within the memory space of the application under test. As a result, the invention may utilize data from a document currently open in a user interface (GUI) editor, thus giving the invention access to all items associated with the document under test.
Aspects of the embodiment discussed in conjunction with FIGS. 5A-P were implemented as a Notes client add-in using core Notes application program interfaces (APIs) and the Microsoft® Windows SDK. Controls utilized were standard Windows® controls and were implemented in an object oriented fashion using C++. The validation tool code was built into a DLL located in the notes binary directory. The illustrated embodiment executes tests from the run menu and displays results in the list view of a main screen. The embodiment of the validator illustrated in FIGS. 5A-P produces three types of results which are denoted as follows:
PASS: The result is displayed with a green check mark signaling that the test executed and that the validation was successful.
FAIL: The result is displayed with a red X signaling that the test executed and the validation was unsuccessful.
WARNING: The result is displayed with a yellow exclamation icon signaling the test executed and the validation was undetermined.
To run the validator, all Notes processes running on the client device may be closed down. Next, a file associated with the DLL is copied into the Notes client binary directory. For example, a file NCVAddin.DLL may be copied. The notes.ini file on the client is modified to include an entry for the validator such as, for example, “Addin Menus—NCVAddin.DLL”. To begin using the validator tool, a user may click on the Actions menu entry and select the validator as shown in
Validation of Item Type may occur by having a user select it from the validation type selection dialog. A user may then be asked if they want to create the command off of a user interface note, which may be a note not yet saved to disk. If the user selects yes, another dialog box may appear containing all of the item names in the document and further containing a check box next to each item as shown in
In contrast to
An item value may be validated by creating an Item Value validation command. A user may be asked if they would like to create the command off of the user interface note if there is an open note in the user interface workspace. If “Yes” is selected, another dialog box may appear containing all of the item names in the document along with a check box next to each one as shown in
The display of
If a user wants to create a Busytime Lookup validation command, Busytime Lookup may be selected from the validation type selection dialog as shown in
The popup windows utilized with embodiments of the invention disclosed herein may be XML driven.
While exemplary embodiments of the invention have been described and illustrated hereinabove, the invention is not limited thereto. Many alternative embodiments and implementations are possible in light of the disclosure without departing from the spirit of the invention. For example, the validation tool may be deployed in a distributed architecture where the user is located at a first geographic location, the application under test is located at a second geographic location and the validation tool components are located at a third geographic location. The three locations may be connected by way of a high speed data network such as the world wide web. When the user launches the validation tool, the necessary functional code may be sent to the hardware running the application at the second location by way of the network. The functional code may then be placed in system memory as a DLL where the application resides. Then the user may run the application and view validation results on a display device at the first location.
Many changes in the details, materials and arrangements of parts, herein described and illustrated, can be made by those skilled in the art in light of teachings contained hereinabove. Accordingly, it will be understood that the following claims are not to be limited to the embodiments disclosed herein and can include practices other than those specifically described, and are to be interpreted as broadly as allowed under the law.
