Patent Application
20060184498
This invention is related to the field of electrical computers and digital processing systems in general, and specifically to processes used to efficiently obtain a summary from a log file and to locate data in a log file.
Currently, many computer systems track the performance of processes and log performance data so that it can be viewed and analyzed by users. The most common technique for retrieving data from a log file is to read the data from the beginning of the log file to its end sequentially. However, this approach has some drawbacks. First, the user cannot obtain a summary view of the entire log file without waiting for the entire file to be read and summarized. Second, it is difficult to view data in a selected time range when the time range is located far from the beginning of the file because all of the preceding records must be read before the desired records can be located.
While these drawbacks are not significant when the file size is small, they become considerable as the size of the log file grows. Many log files cover many activities over a substantial period of time, so large log files that require analysis are quite common.
The prior art discloses approaches for obtaining summary data and efficiently accessing records in log files and databases. For example, U.S. Pat. No. 5,819,066 discloses, inter alia, benchmarking a database server by generating analysis reports from log information stored in transition log files and process log files. U.S. Pat. No. 6,114,967 to to Nock (the '967 patent) discloses generation of a custom log analysis framework encapsulating the common attributes needed by log analysis tools. Similarly, U.S. Pat. No. 6,493,699 to Colby et al. (the '699 patent) discloses defining and characterizing an analysis space for analysis on a user defined subset of detail data to reduce analysis time. U.S. Patent Application Publication 2003/0055809 to Bhat (the '809 publication) discloses configuring log files with header information to allow a logging service to directly access various locations of the log file. Furthermore, U.S. Patent Application Publication 2003/0220940 to Futoransky et al. (the '940 publication) discloses secure auditing of information systems that analyze audit log data. U.S. Pat. No. 5,961,598 to Sime (the '598 patent) discloses a system and method for internet gateway performance charting that displays selected performance charts based upon gathered statistics. U.S. Patent Application Publication 2002/0111887 to McFarlane et al. (the '887 application) discloses an employee online activity monitoring system that monitors employee online activity. The '066 patent, '967 patent, the '699 patent, the '809 publication, '940 publication, the '598 patent, and the '887 application disclose methods for obtaining summary data, but these approaches do not include generation of summary data inside a log during the logging process.
In addition to the patents and publication discussed above, U.S. Patent Application Publication 2002/0174136 to Cameron et al. (the '136 publication) discloses high-performance transaction processing using a relational data base. However, the '136 publication neither maintains summary data within the log file, nor improves the efficiency of retrieving data records in a non-sequential way. U.S. Pat. No. 6,789,115 to Singer et al. (the '115 patent) discloses a system that captures, analyzes, stores, and reports system users' usage of multiple internet and/or intranet web servers. However, the system disclosed in the '115 patent does not aid in efficiently retrieving data records in a non-sequential way, and also does not reduce the number of input/output operations for retrieving summary data and individual data records. Furthermore, U.S. Patent Application Publication 2002/0138762 to Horne (the '762 publication) discloses management of log archival and reporting for data network security systems. However, the '762 publication does not generate summary data during the logging process, and does not integrate the archival and analysis processes.
What is needed beyond the prior art is a method to generate summary data from a a log file and to locate data in a log file during the logging process so that data records are retrieved efficiently in a non-sequential way, the number of input/output operations for retrieving the summary data is reduced, and the archival process is integrated with the analysis process.
The invention that meets the needs described above comprises of a log writing program to monitor a process and write data records, summary records, and summary information to a log file as needed, a log reading program to read the summary information, summary records, and user-specified data records and display them to a user, and a configuration program to permit a user to define the frequency of summary record creation and the duration of monitoring, which is recorded in a configuration data file. The log writing program monitors a process and writes data records to the log file to record activities. The log writing program writes a summary record whenever the criteria defined in the configuration data file is met. Once the monitoring terminates, the log writing program checks the last record in the log file to see if it is a summary record. If it is not, then the log writing program writes a summary record. Then, prior to terminating, the log writing program writes the summary information to the log file.
The log reading program begins by reading the summary information to locate the summary records and then reading the summary records and displaying them to the user. After the user requests a specific data record or range of data records, the log reading program identifies matching summary records and uses their location to access the requested data records quickly. The requested data records are then displayed to the user before the log reading program terminates.
The log writing program may be run automatically by a computer. The log reading program may be run in response to receiving a request from a user. The data records may include an event type, a start time, an end time, and an event detail. The summary records may include a total number of events for each event type covered by the summary record, a total number of events covered by the summary record, a start time of the first data record covered by the summary record, and an end time of the last data record covered by the summary record. The summary information may include a number of summary records, a summary record name, an offset in the log file, a start time, an end time, and a number of data records.
The novel features believed characteristic of the invention are set forth in the appended claims. The invention itself, however, as well as a preferred mode of use, further objectives and advantages thereof, will best be understood by reference to the following detailed description of an illustrative embodiment when read in conjunction with the accompanying drawings, wherein:
A person of ordinary skill in the art will appreciate that the present invention may be implemented in a variety of software and hardware configurations. It is believed, however, that the invention is described best as a computer program that configures and enables one or more general-purpose computers to implement the novel aspects of the invention.
As used herein, “offset” means a position in a file relative to the current view, expressed as a count.
As used herein, “computer” means a machine having a processor, a memory, and an operating system, capable of interaction with a user or other computer, and shall include without limitation desktop computers, notebook computers, tablet computers, personal digital assistants (PDAs), servers, handheld computers, and similar devices.
As used herein, “memory” means a device where information can be stored and retrieved including without limitation internal and external storage devices, magnetic and optical disks, magnetic tape, compact disc, random access memory (RAM), read only memory (ROM), and available addressable storage space including Directly Attached Storage (DAS) and Network Attached Storage (NAS).
As used herein, “process” means a series of actions or operations conducing to an end, typically carried out by a computer program.
As used herein, “event” means something that happens during a process.
As used herein, “repository” means a portion of a memory, or of a distributed memory, in which a configuration data file and a log file can be stored for later retrieval.
Local computer 110 has first memory 112 and first processor 114 that receive input from and display results to a user. Server 200 has second memory 212 and second processor 214 that run configuration program 300, log writing program 400, and log reading program 500. Server 200 is linked to database 210 and stores configuration data file 600 and log file 700.
First memory 112 and second memory 212 may reside in server 200, database 110, or may be distributed in network 116 among one or more of server 200 and local computer 110. Alternatively, first memory 112 may be combined with second memory 212. First memory 112 is connected to network 116 by first processor 114. Second memory 212 is connected to network 116 by second processor 214.
The internal configuration of a computer, including connection and orientation of the processor, memory, and input/output devices, is well known in the art. The present invention is a methodology that can be embodied in a computer program. Referring to
With respect to the above description, it is to be realized that the optimum dimensional relationships for the parts of the invention, to include variations in size, materials, shape, form, function, manner of operation, assembly, and use are deemed readily apparent and obvious to one of ordinary skill in the art. The present invention encompasses all equivalent relationships to those illustrated in the drawings and described in the specification. The novel spirit of the present invention is still embodied by reordering or deleting some of the steps contained in this disclosure. The spirit of the invention is not meant to be limited in any way except by proper construction of the following claims.
