Patent Application
20080235182
1. Field of the Invention
The field of the invention is data processing, or, more specifically, methods, apparatus, and products for isolating database queries for performance processing.
2. Description of Related Art
The development of the EDVAC computer system of 1948 is often cited as the beginning of the computer era. Since that time, computer systems have evolved into extremely complicated devices. Today's computers are much more sophisticated than early systems such as the EDVAC. Computer systems typically include a combination of hardware and software components, application programs, operating systems, processors, buses, memory, input/output devices, and so on. As advances in semiconductor processing and computer architecture push the performance of the computer higher and higher, more sophisticated computer software has evolved to take advantage of the higher performance of the hardware, resulting in computer systems today that are much more powerful than just a few years ago.
Information stored on a computer system is often organized in a structure called a database. A database is a grouping of related structures called ‘tables,’ which in turn are organized in rows of individual data elements. The rows are often referred to as ‘records,’ and the individual data elements are referred to as ‘fields.’ In this specification generally, therefore, an aggregation of fields is referred to as a ‘data structure’ or a ‘record,’ and an aggregation of records is referred to as a ‘table.’ An aggregation of related tables is called a ‘database.’
A computer system typically operates according to computer program instructions in computer programs. A computer program that supports access to information in a database is typically called a database management system or a ‘DBMS.’ A DBMS is responsible for helping other computer programs access, manipulate, and save information in a database.
A DBMS typically supports access and management tools to aid users, developers, and other programs in accessing information in a database. One such tool is the structured query language, ‘SQL.’ SQL is query language for requesting information from a database. Although there is a standard of the American National Standards Institute (‘ANSI’) for SQL, as a practical matter, most versions of SQL tend to include many extensions. Here is an example of a database query expressed in SQL:
This SQL query accesses information in a database by selecting records from two tables of the database, one table named ‘stores’ and another table named ‘transactions.’ The records selected are those having value “Minnesota” in their store location fields and transactions for the stores in Minnesota. In retrieving the data for this SQL query, an SQL engine will first retrieve records from the stores table and then retrieve records from the transaction table. Records that satisfy the query requirements then are merged in a ‘join.’
Application developers frequently have to debug various performance problems with Structured Query Language (SQL)—from application usage problems, poor index configurations, poor system/environment configurations, to DBMS or application defects. In a live environment, the rate of SQL queries per second may be very high (many thousands), and thus it is very hard for a user to stop the right job with the right query they intend to investigate. In dynamic SQL environments, the user may not even know when and from where the exact statements are issued, making the investigation very difficult.
The problem also exists for developers of the DBMS in the various components such as the parser, optimizer, engine, and contributes to high amount of resources spent fixing customer problems when there are thousands of jobs and thousands of queries per job that must be scrutinized to find the problem query in question.
Conventional analysis tools such as IBM's Visual Explain can capture information about queries and query implementations. Such conventional analysis tools are useful to pinpoint the location of a performance defect. It is often the case that users will run multiple queries in a job where one or more of the queries run poorly. In order to identify performance defects, the user must first start a performance monitor and then run the entire job to completion. The monitor generates a monitor file which is then imported into a tool that lists the explainable queries that can be retrieved from the file.
Because conventional analysis tools are run as a separate application and gather data about many queries at one time, such analysis tools do not provide the option of focusing on a particular query at run time in order to try various performance adjustments, or to debug the query at run time. For example, a typical Visual Explain diagram of an application may have hundreds and thousands of query statements. A Visual Explain filter can be applied to narrow these down so that one can find a query of interest, such as to display only queries that run more than 10 seconds, but this is after the query is run and therefore too late to focus on that query during its actual run, such as to apply a performance option, database monitor, or to debug that query during the next time it is run.
Methods, systems, and computer program products are provided for isolating database queries for performance processing. Embodiments typically include presenting to a user a prioritized list of potentially poorly performing queries; receiving from the user a selection of one or more potentially poorly performing queries; and executing performance processing of the selected queries.
The foregoing and other objects, features and advantages of the invention will be apparent from the following more particular descriptions of exemplary embodiments of the invention as illustrated in the accompanying drawings wherein like reference numbers generally represent like parts of exemplary embodiments of the invention.
Exemplary methods, apparatus, and products for isolating database queries for performance processing in accordance with the present invention are described with reference to the accompanying drawings, beginning with
Presenting to a user a prioritized list of potentially poorly performing queries may be carried out, for example, in a SQL module of a database management system capable of receiving a plurality of queries; identifying one or more metrics describing the queries; ordering the queries in dependence upon the metrics; and displaying the ordered queries. Many different metrics may be used to order the queries. Examples of such metrics include absolute performance, performance relative to other runs of the same query, number of access plan rebuilds, number of Full Opens as opposed to pseudo opens, age of the query, number and type of negative SQL codes returned from the query, and others as will occur to those of skill in the art. Such a prioritized list of potentially poorly performing queries may also be updated as new queries are received. In such cases, presenting to a user a prioritized list of potentially poorly performing queries may also include receiving a new query; identifying one or more metrics describing the new query; and inserting the query in a prioritized list of queries in dependence upon the metrics. A current prioritized list of potentially poorly performing queries may therefore be maintained.
A prioritized list of poorly performing queries allows a user to select from the prioritized list a particular query and execute performance processing on only the selected queries. The performance processing of the selected queries may occur upon receiving the selected query, such as the next time the query is run without further user interaction. Alternatively, executing performance processing of the selected queries may include prompting the user for an instruction to execute a performance processing tool for the selected query upon receiving the selected query. Prompting the user provides notice to the user that the selected query has been received and performance processing tools are being executed on the isolated query.
In the example of
In the system of
The arrangement of servers and other devices making up the exemplary system illustrated in
For further explanation,
The exemplary SQL module (260) of
This access plan represents database functions to scan through the stores table and, for each stores record, join all transactions records for the store. The transactions for a store are identified through the storeID field acting as a foreign key. The fact that a selection of transactions records is carried out for each store record in the stores table identifies the join function as iterative.
The exemplary access plan generator (256) of
The exemplary access plan generator (256) also includes an optimizer (254) implemented as computer program instructions that optimize the access plan in dependence upon database management statistics (264). Optimizer (254) optimizes the execution of SQL queries against DBMS (250). Optimizer (254) is implemented as computer program instructions that optimize execution of a SQL query in dependence upon database management statistics (264). Database statistics are typically implemented as metadata of a table, such as, for example, metadata of tables of database (262) or metadata of database indexes. Database statistics may include, for example:
These three database statistics are presented for explanation only, not for limitation.
The exemplary SQL module (260) of
The SQL module (260) of
A prioritized list of poorly performing queries allows a user to select from the prioritized list one or more particular queries and execute performance processing on only the selected queries. The query isolation and performance processing module (150) therefore provides a user with the ability to monitor the performance and tune the performance in real time of select queries.
As mentioned above, isolating database queries for performance processing in accordance with the present invention is generally implemented with computers, that is, with automated computing machinery. For further explanation,
The SQL module (260) of
A prioritized list of poorly performing queries allows a user to select from the prioritized list a particular query and execute performance processing on only the selected queries. The query isolation and performance processing module (150) therefore provides a user with the ability to monitor the performance and tune to performance in real time of select queries.
Also stored in RAM (168) is an application (232), a computer program that uses the DBMS (250) to access data stored in a database. Also stored in RAM (168) is an operating system (154). Operating systems useful in computers according to embodiments of the present invention include Unix, Linux, Microsoft NT™, i50S, and many others as will occur to those of skill in the art. Operating system (154), DBMS (250), and application (154) in the example of
The computer (152) of
The exemplary computer (152) of
The example computer of
For further explanation,
Once a prioritized list of potentially poorly performing queries is created, that list may be updated query by query. Presenting to a user a prioritized list of potentially poorly performing queries that updates an existing list may therefore may also be carried out by receiving a new query; identifying one or more metrics describing the new query; and inserting the query in a prioritized list of queries in dependence upon the metrics as discussed below with reference to
The method of
The method of
Executing (410) performance processing of the selected queries (408) may be carried out in real time when the selected query is run. In such cases, identifications of the selected queries are maintained and each query received is compared to the selected queries. If the received query matches a user selected query then performance processing on the received query is executed. The comparison between each received query and identifications of those selected by a user may be carried out at SQL Open for each query and performance processing of the selected query may be ended at SQL close.
In some cases, once a user has selected one or more queries no further user intervention is required. Executing performance processing of the selected queries may be carried out in such case by receiving the selected query and executing a performance processing tool for the selected query without further user intervention.
Alternatively, a user may wish be notified when a selected query is run. Executing performance processing of the selected queries may be carried out in these situations by receiving the selected query and prompting the user for an instruction to execute a performance processing tool for the selected query. Prompting the user for an instruction to execute a performance processing tool for the selected query provides to a user real time notification of the receipt of the query and provides a user the option of a user to determine whether to execute the performance processing.
Executing (410) performance processing of the selected queries may be carried out by executing performance analysis of the selected queries. Executing performance analysis of the selected queries (408) may be carried out by running a performance analysis tool on the selected queries. Examples of performance analysis tools useful in accordance with the present invention include IBM's Visual Explain, Microsoft's SQL Profiler, and others as will occur to those of skill in the art. Visual Explain, for example, is a database tool that graphically represents the implementation of a query request. Visual Explain provides a method of identifying and analyzing database performance problems. The implementation of a query is broken down into the individual element of an access path and organized in a tree structure. The resulting tree (made up of these base components) provides a visual explanation of the implementation of a query. Other examples of performance analysis tools useful in accordance with the present invention include Microsoft's SQL Profiler, and others as will occur to those of skill in the art.
Executing performance processing of the selected queries further may also be carried out by executing performance tuning of the selected queries. Executing performance tuning of the selected queries may be carried out by providing hints through a DBMS function, such as for example, a Change Query Attributes or CHGQRYA options for the selected query. Such hints may include instructions to use different indexes. Performance tuning may also include changing environmental conditions, modifying an access plan, creating statistics, creating indexes, and others as will occur to those of skill in the art.
Executing performance processing of the selected queries therefore often leaves other unselected queries to operate without interruption or change. That is, the queries not selected may continue to operate as usual and are unaffected by the method of
For further explanation,
The method of
The method of
A single metric may not be sufficient to accurately order the queries from greatest to least potentially poorly performing queries. In some cases, therefore, ordering (510) the queries (504) in dependence upon the metrics (508) may include a weighted ordering scheme that prioritizes some metric types over others.
A raw metrics value may also be more useful when combined with the metric type. That is, metrics are more useful when taken in context of the actual query. Ordering (510) the queries (504) in dependence upon the metrics (508) therefore also may be carried out by ordering the queries in further dependence upon one or more attributes of the query. Such query types may include certain SQL operations such as SQL Open, SQL Pseudo Open, SQL Fetch, or others.
The method of
As mentioned above, once a prioritized list of potentially poorly performing queries is created, that list may be updated query by query. For further explanation,
Exemplary embodiments of the present invention are described largely in the context of SQL. This is for ease of explanation and not for limitation. Isolating database queries for performance processing is not limited to SQL. In fact, other query languages exist such as XML, QRY/400, Open Query File (‘OPNQUERYF’), DLL and isolating database queries for performance processing may include queries of all such query languages and many others as will occur to those of skill in the art.
Exemplary embodiments of the present invention are described largely in the context of a fully functional computer system for isolating database queries for performance processing. Readers of skill in the art will recognize, however, that the present invention also may be embodied in a computer program product disposed on signal bearing media for use with any suitable data processing system. Such signal bearing media may be transmission media or recordable media for machine-readable information, including magnetic media, optical media, or other suitable media. Examples of recordable media include magnetic disks in hard drives or diskettes, compact disks for optical drives, magnetic tape, and others as will occur to those of skill in the art. Examples of transmission media include telephone networks for voice communications and digital data communications networks such as, for example, Ethernets™ and networks that communicate with the Internet Protocol and the World Wide Web as well as wireless transmission media such as, for example, networks implemented according to the IEEE 802.11 family of specifications. Persons skilled in the art will immediately recognize that any computer system having suitable programming means will be capable of executing the steps of the method of the invention as embodied in a program product. Persons skilled in the art will recognize immediately that, although some of the exemplary embodiments described in this specification are oriented to software installed and executing on computer hardware, nevertheless, alternative embodiments implemented as firmware or as hardware are well within the scope of the present invention.
It will be understood from the foregoing description that modifications and changes may be made in various embodiments of the present invention without departing from its true spirit. The descriptions in this specification are for purposes of illustration only and are not to be construed in a limiting sense. The scope of the present invention is limited only by the language of the following claims.
