Internet server system test and measurement

Abstract

A software product for monitoring the performance of an e-commerce business by reading log files from a plurality of tiers, making queries to a database, and interacting with an e-commerce infrastructure. Resulting information is presented as measurements varying over time, where the measurements are business and financial measures.

Description

BACKGROUND OF THE INVENTION

[0002] Internet server systems are now a critical component to many successful businesses. Many Internet server systems are configured to function as e-commerce web sites where computer users can purchase goods and services. The efficient and reliable operation of the e-commerce web site is vital to many businesses.

[0003] In response to the need for efficient and reliable e-commerce web sites, test systems have been developed to ensure that the web site is operating within tolerable thresholds. These test systems perform automated tests using transactions that were previously recorded. To the web site, the test transaction appears like another customer. These test user transactions are able to determine how long a typical transaction takes and whether or not the e-commerce web site is responding at all.

[0004] Unfortunately, current test systems treat the Internet server system that provides the e-commerce web site as a black box—meaning that the test system sends in stimulus and measures response. Test systems do not effectively correlate user test results with internal performance measurements from the Internet server system. If there is a problem, the test system does not effectively isolate the responsible component within the Internet server system.

[0005] Current test systems also fail to correlate system testing and performance data with business performance data. Business performance data may only be produced in weekly or monthly reports. If the web site operator receives an alarm from a test system, another system must be used to assess the financial damage due to the system error. The use of multiple systems is complex and time consuming.

SUMMARY

[0006] The invention addresses the above problems with a software solution that delivers key business and financial measures in near real time, providing information on current (“live”) business operations.

[0007] An e-commerce installation is typically constructed from several tiers. These tiers may or may not be physically realized as separate systems. An Internet server system would typically comprises a web servers tier, a session servers tier, a transaction servers tier and a database servers tier. Each of these tiers potentially generates information detailing operations as they are performed.

[0008] The invention monitors in near real time the business and financial performance of the e-commerce business. The invention reads the log files from each of the tiers, makes queries to databases, and interacts with the e-commerce infrastructure. This information is presented as measurements varying overtime, where the measurements are comprised of key business and financial measures such as the number of new orders per minute, the dollar volume of completed orders per minute, the number of abandoned “shopping carts” per minute, lost sales due to system errors, etc.

[0009] The invention assesses and reports key business and financial measurements. The system of the invention delivers these measurements in near-real time allowing e-commerce businesses to quickly determine current financial and business performance. This in turn allows quick detection and response to variances.

DESCRIPTION OF THE DRAWINGS

[0010] FIG. 1 is a diagram illustrating a four tier construction of an Internet server system.

DETAILED DESCRIPTION OF THE INVENTION

[0011] An e-commerce installation is typically constructed from several tiers.

[0012] These tiers may or may not be physically realized as separate systems. A preferred embodiment of the invention is shown in FIG. 1. FIG. 1 shows a four tier construction of an Internet server system. This construction typically comprises a web servers tier 101, a session servers tier 102, a transaction servers tier 103 and a database servers tier 104. The web servers 101 are responsible for handling direct customer interaction with the e-commerce web pages and delivering content back to the customer. A session server 102 keeps track of individual customers and what actions those customers are performing. The e-commerce transactions are supported by the transaction servers 103 and the database servers 104 manage the e-commerce database. Each of these tiers potentially generates information detailing operations as they are performed. Also shown in FIG. 1 is a plurality of agents 105. The agents 105 are shown connected to each of the elements in the four tiers. For purposes of this example, an agent 105 is shown connected to each server in each tier, but it is understood that agents 105 are connected to those servers in the tiers 101,102,103,104 as desired by a user. The agents 105 are software programs that run on each desired server. Agent programs 105 are termed “tests”.

[0013] The invention monitors in near real time the business and financial performance of the e-commerce business. Applications and computer systems or subsystems generate log files in their normal course of operation. For example, a web server in the web server tier 101 generates a log file containing hit rates, client/server errors, error rates, etc. which can be used to indicate health and performance of the server. Generally, log files generated by any subsystem in the tiers 101,102,103,104 track what occurs in that portion of the system. Accordingly, log files will contain data regarding events, state, transactions and anything of interest. It is understood that a log file may contain any type or class of data dependent upon programming.

[0014] Agents 105 associated with the web servers tier 101 are web server log readers. Examples of web servers 106 are Netscape, Apache, MicroSoft/IIS, America On-Line, etc., and include generic web servers. Events in a web server 106 log file include but are not limited to hit rate, hosts served, data volume, error rates, log file size and system load. System load includes but is not limited to CPU utilization memory usage and I/O transfers.

[0015] Agents 105 associated with the session servers tier 102 are session server application tests and log readers. Examples of sessions servers 107 are session management process tests. Events in a session server 107 log file include but are not limited to users served, interaction manager (IM) restarts, IM status, IM load database connectivity, error rates and system load.

[0016] Agents 105 associated with the transaction servers tier 103 are transaction server application tests and log readers. Transaction servers 108 process information on a variety of levels, including system level performance to transaction process count and availability, sales and orders, shopping statistics, payment service and order fulfillment (shipping). Events in a transaction server 108 log file may include but are not limited to inventory status, system load, open and discarded cart rates, shoppers in store, authorized, cancelled and declined data and payment service errors. Transaction servers 108 also compile and maintain databases. The agents 105 associated with transaction servers 108 query the transaction server databases using standard SQL database queries. It is understood that any type of database query capable of obtaining data from a database can be used. Examples of data in a transaction server 108 data base may include but are not limited to sales per hour, items sold per hour, new order rates, processed order rates, fulfilled order rates orders awaiting authorization and orders awaiting shipping.

[0017] Agents 105 associated with the database servers tier 104 are database server tests. Database servers 109 process information and create and maintain databases. The agents 105 associated with database servers 109 query the database server databases using standard SQL database queries. It is understood that any type of database query capable of obtaining data from a database can be used. While any data contained in the database can be retrieved, for the examples used in accordance with the invention, the pertinent data to query is health information such as number of the transactions that have occurred, the number of users, sales volume and other application or business health indicators of the e-commerce website which the database may contain.

[0018] In the preferred embodiment of the invention shown in FIG. 1, each agent 105 is shown as being associated with a particular server. The agents are software programs, or tests, that are resident on the servers due to the need for direct access to the relevant log files or databases. It is understood, that a particular agent 105 will be programmed to read the format, or formats, of log files and/or databases resident on a particular server. For example, if the agent 105 is interested in events relating to sale amount, it knows that column four (4) of the log file contains sale amount data. The agent 105 retrieves the appropriate data from the log files or databases and sends that information to a user site. The user site is an Internet service management solution such as the Firehunter® products offered by Agilent Technologies that enables a user to measure and monitor an e-commerce solution's performance. It is understood that the invention is not limited to use with Firehuntere and can be used in any manner capable of displaying data.

[0019] The data from the agents 105 is sent to a user site via means capable of transferring data. Preferably, the data is transmitted via the Internet or an intranet. It is understood that any network means can be used to transmit data from the agents 105 to a user site. It is also considered that a direct connection be used to transmit data from the agents to the user site.

[0020] The invention reads the log files from each of the tiers 101,102,103,104 in which there is an agent, makes queries to the database, and interacts with the e-commerce infrastructure. This information is presented as measurements varying overtime, where the measurements are comprised of system infrastructure (such as load or error information) and key business and financial measures such as the number of new orders per minute, the dollar volume of completed orders per minute, the number of abandoned “shopping carts” per minute, etc. Gathering these infrastructural and business measurements together into a system like Firehunter® then enables correlated results, such as lost sales due to system errors and the impact of system load on the rate and number of business transactions.

[0021] The invention assesses and reports key business and financial measurements. The system of the invention delivers these measurements in real time allowing e-commerce businesses to quickly determine current financial and business performance. This in turn allows quick detection and response to variances.

[0022] It is understood that the invention is not limited to the specific examples and illustrations discussed above, but only by the following claims and their equivalents. Accordingly, the invention is applicable to a broad variety of log files and databases that contain information that has been logged by an application.

Claims

1. A method for monitoring an e-commerce installation, said comprising a plurality of tiers and said tiers generating log files and databases, wherein said method comprises: reading the log files and querying the databases to obtain information stored therein, said information comprising performance measurements of the installation; and graphically presenting said measurements.

2. The method of claim 1, wherein said plurality of tiers comprises a web serverstier, a session servers tier, a transaction servers tier and a database servers tier.

3. The method of claim 2, wherein; said web servers tier comprises a plurality of web servers with each of said plurality of web servers generating log files, said log files comprising hit rate data, hosts served data, data volume data, error rates data, log file size monitor data, system load data and generic data, said session servers tier comprises a plurality of session servers with each of said plurality of session servers generating a plurality of log files, said plurality of log files comprising users served data, restarts data, IM status data, IM load data, database connectivity data, error rates data and system load data, said transaction servers tier comprises a plurality of transaction servers with each of said plurality of transaction servers generating a plurality of log files and a plurality of databases, said log files comprising open and discarded cart rates data, shoppers in store data, authorized, cancelled, declined data, payment service errors data, inventory status data, system load data and said databases comprising sales per hour data, items sold per hour data, new order rates data, processed order rates data, fulfilled order rates data shipping network connectivity data, orders awaiting authorization data and orders awaiting shipment data; and said database servers tier comprises a plurality of database servers with each of said plurality of database servers generating a plurality of databases, said plurality of databases comprising database access performance data, database size monitor data, database connectivity error rate data and system load data.

4. The method of claim 3, wherein said user site is a fully functional web-based business application or e-commerce monitoring solution.

5. A software product for testing and monitoring a web-based business application or e-commerce installation comprising at least one web server, at least one session server, at least one transaction server, and at least one database server, the software product comprising: test instructions configured to direct a processor to configure and execute user transaction tests and report user transaction test results; system performance instructions configured to direct the processor to measure system performance data for each of the web server, the session server, the transaction server, and the database server; business performance instructions configured to direct the processor to measure business performance data comprising monetary volume transacted by the e-commerce installation during a time period; and a first storage media configured to store the user test instructions, the system performance instructions, and the business performance instructions.

6. The software product of claim 5, wherein the business performance data further comprises new orders transacted by the e-commerce installation during a time period.

7. The software product of claim 5, wherein the business performance data further comprises sales volume transacted by the e-commerce installation during a time period.

8. The software product of claim 5, wherein the business performance dta further comprises lost sales due to system errors during a time period.

9. The software product of claim 5, wherein the business performance data further comprises abandoned shopping carts during a time period.

10. A method of operating a computer system to test and monitor an Internet server system comprising a web server, session server, transaction server, and database server, the method comprising: configuring and executing user transaction tests and reporting user transaction test results; measuring system performance data for each of the web server, the session server, the transaction server, and the database server; and measuring business performance data comprising monetary volume transacted by the e-commerce installation during a time period.

11. The method of claim 10, wherein the business performance data further comprises financial data transacted by the e-commerce installation during a time period.

12. The method of claim 10, further comprising: processing the system performance data to generate system graphics illustrating system performance measured against system performance baselines and system performance thresholds; and processing the business performance data to generate business graphics illustrating billing performance measured against business performance baselines and business performance thresholds.

13. The method of claim 10, further comprising: collecting the system performance data and the business performance data from the e-commerce installation; and transferring the system performance data and the business performance data to the computer system.