Patent Application
20200250140
A log file is a file that stores information regarding events or messages that occur in a computer system or a computer network. Network administrators find it helpful to have access to such log files and to perform analysis on the contents of such log files. Traditionally, log files have been stored locally on a local machine or network where the contents of the log files may be easily accessed.
A number of software tools traditionally have been used to review and analyze the contents of the log files. For example, in Unix® environments, a number of tools have been available for use in reviewing and analyzing log files. Once such tool is the cat utility. This program name is short for concatenate. The cat utility reads files sequentially and writes them to a standard output in the same sequence. Standard output is the stream where a program writes its output data.
Another tool that has commonly been used to process log files is the less utility. The less utility is a terminal pager that enables the display of information one page at a time. This avoids all of data being viewed in one continuous scrolling sequence.
One of the tools is the grep utility, which performs a global search of a regular expression and prints the matching lines. The grep utility locates instances of a regular expression globally and then prints the lines with the regular expression that are located in the file. The grep utility is useful for locating a string of words, for example.
An additional tool that is used with log files is the tail utility. The tail utility displays the tail end of a text file or a pipe data. It ouputs the last ten lines of standard output by default.
In accordance with an exemplary embodiment a method is practiced in a computer system. The method includes requesting that a file system be mounted on the computer system, wherein the file system includes file system constructs. A request to access a selected at least one of the file system constructs by an application running on the computer system is received. In response to the request, a search request is sent to a remote data store. The remote data store stores files that are not part of the file system. Results of the search are received and access is provided to the received results of the search to the application as if they were stored in the selected one or more file system constructs.
The remote data store may store log files. The computer system may run an operating system and the request to access the selected at least one of the file system constructs may be a system call to the operating system. The request may be received by hooking the system call to the operating system from the application in some instances. The search request may comprise sending a request to the remote data store to provide contents for the selected at least one construct. The selected one or more file system constructs may comprise one or more files or one or more folders. File System in User Space (FUSE) may be used to receive the request and send the search request. The application may take many forms including one of the less utility, the tail utility, the grep utility or the cat utility.
In accordance with an exemplary embodiment, a method is practiced in a computing system having a file in a file system. A request to access contents of the file is received. Contents of the file are dynamically obtained from a remote data store in response to the request. In further response to the request, access is provided to the dynamically obtained contents as if the contents were stored in the file.
The method may also include receiving a second request to access the file and dynamically obtaining new contents for the file from the remote data store in response to the second request. In further response to the second request, access to the new contents may be provided as if the results were new contents stored in the file.
In some instances, the dynamically obtained contents may be cached for subsequent requests to access the files. The remote data store may store log files. The computer system may run an operating system, and the request to access may be a system call to the operating system. The method may include the additional steps of hooking a system call and generating a search request to the remote data store to obtain the contents.
These methods may be performed by instructions stored on a non-transitory computer readable storage medium that are executable by a processor of a computing device.
The review and analysis of log files has changed due to the volume of log file information and location of log files within organizations. More recently, log files have been accessed by way of web interfaces, where the log files are stored remotely relative to the client that is requesting access to the log files. Moreover, the log files may be stored on different devices and at disparate locations.
One difficulty encountered when the log files are accessed by way of web interface is that network administrators may no longer use the tools, such as the less, cat, grep and tail utilities, that traditionally have been used locally on the terminal to access and process log files. This poses a hardship to such network administrators. The exemplary embodiments described herein provide a mechanism for such tools to be used on log file data that is located remotely. Exemplary embodiments are able to mount a sub-tree containing desired log file information on a local file system. The illustrative embodiments may use the mount command available in the Unix operating system, for example, to achieve this objective. A mounted sub-tree may hold dynamic search results obtained from a remote database of log file information. Virtual files may be associated with the dynamic search results, and such virtual files may be accessed by applications, such as the tools mentioned above, to gain access to the desired log file information as if the search results were stored in the associated virtual files. The exemplary embodiments may exploit the FUSE technology to access the mounted log file information in the local file system.
The exemplary embodiments allow the network administrator to act as if the files are located locally and accessible on the local system. These files actually are remotely located and are virtual files representing dynamic search results. A FUSE handler is provided that is responsible for receiving requests to access the virtual files, sending search requests to the remote data store for the log files and returning the results via the operating system kernel to the requesting application. Thus, the exemplary embodiments map a file system call to a search query and return the results of the search query so that the results may be accessed as if they are part of a file.
As will be explained in more detail below, the mounted sub-tree is a representation of how a party wants to see the log file data. The directory may be set up in whatever fashion the party desires to hold the appropriate dynamic search results.
The client computing device 102 may have access to one or more storage devices 120. The storage devices may hold information regarding a file system 124 and may hold one or more configuration files 122 that may be used to configure and mount appropriate subdirectories into the file system provided by the operating system 116.
Those skilled in the art will appreciate that the client computing device 102 may take many different forms, including but not limited to that of a workstation, a desktop computer, a laptop computer or other computing device, such as a tablet computing system, a smartphone or the like. The client computing device 102 may execute computer readable instructions stored on a non-transitory storage medium. These instructions may perform the operation described herein.
Instructions may be stored on magnetic disc, optical disc, solid state storage, or other storage devices. The computing device 102 may include storage, such as random access memory (RAM), read only memory (ROM) and variants thereof.
The server computing device 104 may include one or more processes 130 for executing instructions. The server 104 may execute instructions for providing a web interface 32 to enable clients to access a database 136. A data store system 134 may be provided and may manage the database 136. The database 136 may include log files 138 for an enterprise, an organization, a computing network or the like. The log files 138 need not be stored in a single physical device but rather may be stored across multiple devices. The storage may be realized in a number of different forms, such as on magnetic state storage, magnetic tape storage, optical state storage, solid state storage or the like. For purposes for exemplary embodiments, it is assumed that the server 106 provides access to the log files in response to request from clients.
Those skilled in the art will appreciate the depiction of the distributed environment 100 in
As is mentioned above, the exemplary embodiments mount a sub-tree into a local file system. The sub-tree may hold directory entries and files for search results obtained from searches conducted by the data store system 134 of the server computing device 104. This enables a network administrator or user to locally access the log file search results locally as if the log file search results were stored locally in files of the local file system.
When a party wishes to mount the sub-tree, the configuration file is processed in step 204. The processing of the configuration file identifies what portion of the log files are of interest and specifies directory tree structure for the sub-tree that is to be mounted into local file system. A “mount” command or equivalent command may be executed so that the directory structure is mounted to the local file system in step 206.
Once the sub-tree is mounted, the dynamic search results may be associated with virtual files that are visible in the sub-tree of the local file system and thus may be accessed as if the data was locally held.
Each time a virtual file is read or opened the results associated with the file may be different. The results may be for a unique search each time. Alternatively, the content of the virtual file (i.e. the search results for a particular search) may be cached and preserved for multiple accesses. This may be a configurable option.
The configuration file may also account for server side filters. The syntax of the file may account for such filters.
These examples illustrate the configurability and filtering is available in the exemplary embodiments.
While the present invention has been described with reference to exemplary embodiments herein, those skilled in the art will appreciate the various changes in form and detail may be made without departing from the intended scope of the present invention as defined in the appended claims.
