Patent Application
20080033991
The above and other items, features and advantages of the invention will be better understood by reading the following more particular description of the invention in conjunction with the accompanying drawings, wherein:
The present invention would now be explained with reference to the accompanying figures. Unless the context clearly requires otherwise, throughout the description and the claims, the words “comprise,” “comprising,” and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is to say, in a sense of “including, but not limited to.” Words using the singular or plural number also include the plural or singular number respectively. Additionally, the words “herein,” “hereunder,” “above,” “below,” and words of similar import refer to this application as a whole and not to any particular portions of this application. When the word “or” is used in reference to a list of two or more items, that word covers all of the following interpretations of the word: any of the items in the list, all of the items in the list and any combination of the items in the list.
The present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment containing both hardware and software elements. In accordance with an embodiment of the present invention, the invention is implemented in software, which includes, but is not limited to firmware, resident software, microcode, etc.
Furthermore, the invention may take the form of a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system. For the purposes of this description, a computer-usable or computer readable medium may be any apparatus that may contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus or device.
The medium may be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk. Current examples of optical disks include compact disk-read only memory (CDROM), compact disk-read/write (CD-R/W) and DVD.
Processor 110 may be constructed from one or more microprocessors and/or integrated circuits. Processor 110 executes program instructions stored in main memory 120. Main memory 120 stores programs and data that computer system 100 may access. Main memory 120 includes one or more application programs 122, data 124, operating system 126, data extracting module 128 extracting the historical data from a database 129, trend building module 130, future predicting module 132 and warning module 134. When computer system 100 starts, processor 110 initially executes the program instructions that make up operating system 126. Operating system 126 manages the resources of computer system 100 for example, processor 110, main memory 120, mass storage interface 140, network interface 150 and system bus 160.
Application programs 122 are executed by processor 110 under the control of operating system 126. Application programs 122 may be run with program data 124 as input. Application programs 122 may also output their results as program data 124 in main memory. In one embodiment of the present invention, computer system 100 includes data extracting module 128 to extract the historical data of at least on operating parameter from database 129. Computer system 100 also includes trend building module 130 to build a trend of the historical data for each of the operating parameters and a future predicting module 132 to predict future performance of a Database management system (DBMS). Further, computer system 100 includes warning module 134 to warn an operator about the future performance of the DBMS based on the predicted future performance. The “modules” are software codes that may be a callable routine or embedded into another program, i.e., an operating system or application program. For example, although the modules are shown as a part of operating system 126 in accordance with one embodiment of the invention, it is equally within the scope of the present invention to provide a separate software application or utility that could also provide data extracting module 128, trend building module 130, future predicting module 132 and warning module 134. In accordance with an embodiment of the present invention, the modules may be provided as independent modules. In accordance with another embodiment of the present invention, the modules may be clubbed together.
Mass storage interface 140 allows computer system 100 to retrieve and store data from auxiliary storage devices such as magnetic disks (hard disks, diskettes) and optical disks (CD-ROM). These mass storage devices 180 are commonly known as Direct Access Storage Devices (DASD), and act as a permanent store of information. One suitable type of DASD 180 is a floppy disk drive 180 that reads data from and writes data to a floppy diskette 186. The information from the DASD may be in many forms. Common forms are application programs and program data. Data retrieved through mass storage interface 140 is usually placed in main memory 120 where processor 110 may process it.
While main memory 120 and DASD 180 are typically separate storage devices, computer system 100 may use well known virtual addressing mechanisms that allow the programs of computer system 100 to run smoothly as if having access to a large, single storage entity, instead of access to multiple, smaller storage entities (e.g., main memory 120 and DASD device 180). Therefore, while certain elements are shown to reside in main memory 120, those skilled in the art will recognize that these are not necessarily all completely contained in main memory 120 at the same time. It should be noted that the term “memory” is used herein to generically refer to the entire virtual memory of computer system 100. In addition, an apparatus in accordance with the present invention may include any possible configuration of hardware and software that contains the elements of the invention, whether the apparatus is a single computer system or is comprised of multiple computer systems operating in sync with each other.
Network interface 150 allows computer system 100 to send and receive data to and from any network connected to computer system 100. This network may be a local area network (LAN), a wide area network (WAN), or more specifically, the Internet 170. Suitable methods of connecting to a network include known analog and/or digital techniques, as well as networking mechanisms that are being developed or may be developed in the future. Various different network protocols may be used to implement a network. These protocols are specialized computer programs that allow computers to communicate across a network. TCP/IP (Transmission Control Protocol/Internet Protocol), used to communicate across the Internet, is an example of a suitable network protocol.
System bus 160 allows data to be transferred among the various components of computer system 100. Although computer system 100 is shown to contain only a single main processor and a single system bus, those skilled in the art will appreciate that the present invention may be practiced using a computer system that has multiple processors and/or multiple buses. In addition, the interfaces that are used in the preferred embodiment may include separate, fully programmed microprocessors that are used to off-load compute-intensive processing from processor 110, or may include I/O adapters to perform similar functions.
The present invention provides a method for predicting the future performance of a database management system (DBMS). On basis of early warnings generated as a result of prediction, the operator can take one or more corrective actions, to reduce the chances of a system failure. The performance of the DBMS depends on various operating parameters of the DBMS. The operating parameters may include memory usage, CPU time, transactions per time, file system fill grade, transaction log space used, CPU utilization, disk utilization, buffer pool hit ratio at database level, table space level, buffer pool level and caching. These operating parameters govern the effective usage of the DBMS during interaction of the operator. In accordance with an embodiment of the present invention, the prediction of the future performance of the DBMS is provided to the operator by predicting future values of the operating parameters.
Trend analysis or time series techniques are applied to predict the future values of the operating parameters. Trend analysis or time series analysis is known to predict and forecast future values of different variables in a huge number of domains. The domains include market research, stock market analysis and prediction, finance, politics (such as election campaigns), population study, economics, crime analysis and forensic applications, chemistry, geographical and geological analysis, medical data analysis, Electro-Encephalogram (EEG) and Magneto-Encephalogram (MEG) data analysis, web intelligence, intrusion detection, sequential data mining and various other fields. The method of an embodiment of the present invention predicts the future value of the operating parameters on the basis of the trend of data points of historical data of the operating parameters. The data points of historical data are past values of the operating parameters.
In accordance with the present invention, the future values of the operating parameters are predicted using the historical data stored in the database. Many of the operating parameters of the DBMS are non-stationary, i.e., they change during the course of operation of the DBMS, for example CPU time. Moreover, the historical data may consist of spikes and exhibit no patterns. The spikes and absence of pattern can cause an error in prediction of the future values. In an embodiment of the present invention, the extracted historical data is qualified before building the trend 204.
where xp+i is the value of operating parameter at a point p+i and μp is the computed mean of the historical data at the point p over a window size (2W+1) spanning from p−W to p+W. At step 306, all the values of the operating parameters away from the mean μp by a predetermined factor of the standard deviation are removed. In an embodiment of the present invention the predetermined factor is taken as 2. This may restore 95% of the points and 5% of points as outliers. The outliers in a set of numerical data are any value that is markedly smaller or larger than other values in the data set. In the instant invention the historical data is the numerical data set from which the outliers are removed. In an embodiment of the present invention, next step 308, smoothening of the historical data removes the spikes from the historical data for the restrictive time window (2W+1). It will be obvious to one skilled in the art that different smoothening algorithms such as Gaussian Convolution may be used for smoothening the historical data.
Where xn is most recent observed value of the operating parameter that is to be predicted, xn−T+1, xn−T+2, . . . , xn is the historical data for a time length T and a1s, a2s, a3s, . . . aws are auto correlation coefficients. It is known to those skilled in the art that, if number of the historical observation w is more than the required time length, then the extra auto correlation coefficients take very small value. In the matrix notation the auto regressive mathematical model can be expressed as
Minimization of Gaussian noise ε, considering a generating Gaussian process, the mathematical model fit with the historical data can be calculated as
a
s=(XsTXs)−1XsTys
Further a decay matrix may be used to minimize the Gaussian noise ε more for the recent values of operating parameters and less for the past observations. In an embodiment of the present invention an exponential decay is used for estimation of as. The exponential decay is defined as [1 1/k . . . 1/kw], such that the error for the most recent observation has the maximum weight of unity and the error in the next observation has a weight of 1/k, the weight of the next one is 1/k2 and so on. The mathematical model fit with the historical data can be calculated as
One of ordinary skill in the art will appreciate that various decay, including logarithmic decay, may be used in the embodiments without departing from the scope of the present invention.
At step 404, error checks for the mathematical model are provided. The error check may include, if any of the historical value of the operating parameter is not accessible, and if the number of the historical observations is not enough to enable a proper learning of the mathematical model.
In accordance with the method of the present invention, the future values fixed time-interval ahead is predicted using the current value and the previous w values of the operating parameter in the history.
x
n+s
=z
T
a
s
Where z=[xn xn−1 . . . xn−w+1]T, are the current values and the previous w values of the operating parameter. After observing the current value of the operating parameters, the matrix XS is recomputed incrementally in step 504. Using this approach the mathematical model need not to built at every instance in the time period. Using the present values of the operating parameters and the latest mathematical model, a new mathematical model can be arrived at. The first row of the matrix Xs is pushed to the second row and so on by obtaining only first element of the matrix. The current value of the operating parameter is used as the first element of the matrix. Every time one row at the top of Xs is included, and one row from the bottom of Xs is deleted. A similar operation is performed on matrix DXs. At step 506, the mathematical model is updated using the new matrices Xs and DXs as
a
s=(XsTDXs)−1XsTDys
An embodiment of the present invention includes a module for analyzing a reliability of the predicted future values.
E
s
=∥y
s
−v
s∥2
Where, ys is actual observation of the operating parameter. At step 604, the reliability of the predicted future values is calculated as
Where, C is a predetermined threshold and is provided by the operator. At step 606, an overall reliability of the future values over a given prediction length of s is calculated. The overall error is calculated as
The overall reliability over the prediction length s is calculated as
Integration with IBM DB2 Performance Expert®
For illustrative purposes, exemplary use cases of implementation of the present invention, in the IBM DB2 Performance expert current product would now be explained. The exact implementation may vary for different DBMS, but the inventive concept can be applied to other systems without departing from the spirit and scope of the invention. Application Programming Interfaces (APIs) are used to interact with the database.
Use Case 1: Time Series Learning
A time series learning module is provided for the building of the mathematical model from a single time series. The time series learning module is provided with a set of numeric time series segments as input. The time series segments are tagged with the operating parameter and consist of finite sets of data points. Each of the data points consists of a numeric value of the operating parameter and an associated timestamp. The time interval between the consecutive data points is “chunk-wise” constant. For each time series segment specified in the input, the time series learning module learns the mathematical model that describes its trend. The mathematical model can also be used to predict the future values of the operating parameter.
The time series learning module is provided with a set of option settings that can be used to control behaviors of the time series learning. The time series learning options include incremental, renewed learning, and additional suggestions about the time series. Further, the time series learning option permits to specify number of the data points that will be used to compute the smoothening factor of internal algorithm used to construct the mathematical model.
IDataRange is an Interface for a data container having a simple time-data range. The implementing class is used to provide a set of timestamp and double values. The amount of timestamps and values is the same and matching by index. So the timestamp at index [n] corresponds to the double value at index [n]. Timestamps are provided in workstation format (ticks since the 1 Jan. 1970). The implementing class is simply a data transporter/container for a range of values over time, such as values of the same counter through a specific time frame. The timestamps are ordered so that the data values with a lower index occur before the ones with a higher index.
Use Case 2: Trend Detection
A trend detection module is provided to detect the trend of the historical data of the operating parameters. The trend detection is explicitly targeted for one or more of the following operation parameters:
The trend detection module calculates a trend value for each corresponding timestamp in the time series using the mathematical model learned in USE CASE 1.
The start of the time period over which the trend detection is requested must not be earlier than the start of the time interval of corresponding time series specified in the learning phase. The end of the time period over which trend detection is requested must not be later than the end of the time interval of corresponding time series specified in the learning phase.
Use Case 3: Trend Prediction
A trend prediction module is provided to predict future data points using the mathematical model learned in the USE CASE 1. For each timestamp in the output time series, corresponding predicted values are calculated. Along with each predicted value, a confidence indicator is calculated.
Start time of the tine period over which the trend prediction is requested must be later than the end of the time period of the time series specified in the learning phase.
ITrendAnalyzer is an Interface for the trend analyzer backend engine. The class which implements this interface is supposed to compute trends, trend predictions and confidence levels on data ranges, which are passed in the form of a training set to the learnDataRange method. The IDataRange interface is used to pass in the training data. After the training has been performed, the implementing class returns trend values for specific time frames and granularity through the getTrend method. The time frame may be within the same time range as the training data, or can also specify a time window beyond the training data to predict a trend in the future. Incremental training should also be supported by the learnDataRange method. This means that the training data is not added in a single range but as an initial training set. Then a trend might be requested through the getTrend method on the base of the already learned data. Later on, another set of training data (which is the next valid time slice after the already learned training data) is added through the learnDataRange method. The class then needs to optimize the trend prediction, based on the previously added training data with the help of the new, incremental specified data range. Therefore, a subsequent call to the getTrend method should return a data range trend with improved quality. Therefore, the learning process can be called several times to incrementally increase the quality. Only by calling the reset method, all so far learned values are dropped and the next learning cycle starts from scratch again. The getTrend method can only be called after training data has been processed.
Use Case 4: Prediction Quality Statement
A module is provided for a quality statement to describe the prediction quality of the chosen mathematical model. The module calculates an expected error of the predicted future data. The module also calculates the reliability of the predicted future values. Thereafter, an overall reliability of the future values over a given prediction period is calculated.
IOutputDataRange is an extended data container, adding confidence information. This interface defines a container that has the same support for raw timestamp/double value sets as the inherited IDataRange interface but it extends this information by confidence levels. The scalar overall confidence level returns the average confidence, detected for a computed trend and the confidence level for each of the data points, which are returned. Therefore the index based confidence returns the value of the confidence function which is computed in parallel to the trend values. This extended interface can be the output data container of a trend computation. Therefore, it carries out the resulting trend together with the detected confidence values.
Use Case 5: Iterative Time Series Learning
An iterative time series learning module is provided for incremental learning of the data points in the future. Instead of rebuilding the mathematical model at every instance of time when parameters are to be predicted, the existing mathematical model is incrementally updated to reflect the new subsequent time series data input.
In the aforesaid description, specific embodiments of the present invention have been described by way of examples with reference to the accompanying figures and drawings. One of ordinary skill in the art will appreciate that various modifications and changes can be made to the embodiments without departing from the scope of the present invention as set forth in the claims below. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of present invention.
