1. Field of the Invention
The field of the invention is data processing, or, more specifically, methods, apparatus, and products for query expression evaluation for query optimization.
2. Description of Related Art
The development of the EDVAC computer system of 1948 is often cited as the beginning of the computer era. Since that time, computer systems have evolved into extremely complicated devices. Today's computers are much more sophisticated than early systems such as the EDVAC. Computer systems typically include a combination of hardware and software components, application programs, operating systems, processors, buses, memory, input/output devices, and so on. As advances in semiconductor processing and computer architecture push the performance of the computer higher and higher, more sophisticated computer software has evolved to take advantage of the higher performance of the hardware, resulting in computer systems today that are much more powerful than just a few years ago.
Information stored on a computer system is often organized in a structure called a database. A relational database is a grouping of related structures called ‘tables,’ which in turn are organized in rows of individual data elements. The rows are often referred to as ‘records,’ and the individual data elements are referred to as ‘fields.’ In this specification generally, therefore, an aggregation of fields is referred to as a ‘data structure’ or a ‘record,’ and an aggregation of records is referred to as a ‘table.’ An aggregation of related tables is called a ‘database.’
A computer system typically operates according to computer program instructions in computer programs. A computer program that supports access to information in a relational database is typically called a database management system or a ‘DBMS.’ A DBMS is responsible for helping other computer programs access, manipulate, and save information in a database.
A DBMS typically supports access and management tools to aid users, developers, and other programs in accessing information in a database. One such tool is the structured query language, ‘SQL.’ SQL is query language for requesting information from a database. Although there is a standard of the American National Standards Institute (‘ANSI’) for SQL, as a practical matter, most versions of SQL tend to include many extensions. Here is an example of a database query expressed in SQL:
This SQL query accesses information in a database by selecting records from two tables of the database, one table named ‘stores’ and another table named ‘transactions.’ The records selected are those having value “Minnesota” in their store location fields and transactions for the stores in Minnesota. In retrieving the data for this SQL query, an SQL engine will first retrieve records from the stores table and then retrieve records from the transaction table. Records that satisfy the query requirements then are merged in a ‘join.’
Each SQL query is carried out by a sequence of database operations specified as an access plan. An access plan generator creates an access plan for a query. An access plan generator uses an optimizer to optimize the execution of SQL queries against the DBMS based upon database statistics. Another database management statistic useful in creating an access plan includes the percentage of rows of a database table returned by a query. Such information may be used for example in determining whether an access plan includes retrieving a record using a tablescan or an index.
Methods, systems, and computer program products are provided for query expression evaluation for query optimization. Embodiments include receiving a query including a plurality of predicates; creating a truth table representing the possible solutions to the query; identifying a plurality of intersections of the truth table, the plurality of intersections including each predicate of the query; assigning a selectivity factor to each predicate such as by probing database statistics; determining in dependence upon the selectivity factors for each predicate a selectivity rating for each of the intersections of the truth table; selecting one or more of the intersections in dependence upon the selectivity rating; and determining in dependence upon the selected intersections a percentage of rows in a database table returned by the query; and providing the percentage of rows in the database table to an access plan generator for query optimization.
The foregoing and other objects, features and advantages of the invention will be apparent from the following more particular descriptions of exemplary embodiments of the invention as illustrated in the accompanying drawings wherein like reference numbers generally represent like parts of exemplary embodiments of the invention.
Exemplary methods, systems, and products for query expression evaluation for query optimization in accordance with the present invention are described with reference to the accompanying drawings, beginning with
In the example of
In the system of
The arrangement of servers and other devices making up the exemplary system illustrated in
For further explanation,
The exemplary SQL module (260) of
access plan generator (256) may generate the following exemplary access plan for this SQL query:
This access plan represents database functions to scan through the stores table and, for each stores record, join all transactions records for the store. The transactions for a store are identified through the storeID field acting as a foreign key. The fact that a selection of transactions records is carried out for each store record in the stores table identifies the join function as iterative.
The exemplary access plan generator (256) of
The exemplary access plan generator (256) also includes an optimizer (254) implemented as computer program instructions that optimize the access plan in dependence upon database management statistics (264). Optimizer (254) optimizes the execution of SQL queries against DBMS (250). Optimizer (254) is implemented as computer program instructions that optimize execution of a SQL query in dependence upon database management statistics (264). Database statistics are typically implemented as metadata of a table, such as, for example, metadata of tables of database (262) or metadata of database indexes. Database statistics may include, for example:
These three database statistics are presented for explanation only, not for limitation.
Another statistic useful in creating an access plan is the percentage of rows of a database table returned by a query. The access plan generator (256) of
The exemplary SQL module (260) of
As mentioned above, query expression evaluation for query optimization in accordance with the present invention is generally implemented with computers, that is, with automated computing machinery. For further explanation,
The access plan generator (256) of
Also stored in RAM (168) is an application (232), a computer program that uses the DBMS (250) to access data stored in a database. Also stored in RAM (168) is an operating system (154). Operating systems useful in computers according to embodiments of the present invention include Unix, Linux, Microsoft N™, i50S, and many others as will occur to those of skill in the art. Operating system (154), DBMS (250), and application (154) in the example of
The computer (152) of
The exemplary computer (134) of
The example computer of
For further explanation,
The method of
(C1=2 OR C2>4) AND (C3=6 OR C4<=8)
and truth table representing the possible solutions to the query expression:
In the example above, the query expression contains four predicates C1=2, C2>4, C3=6, and C4<=8 labeled A, B, C, and D respectively. The truth table representing this four predicate query includes sixteen possible solutions to the query in rows 0-15.
Truth tables representing the possible solutions to query expressions may be further simplified or condensed as a plurality of intersections of the query expression. The method of
The semantics of the intersections of the truth table differ from those of the truth table itself in that a value of 1 indicates that a predicate is true while a value of X indicates a disregard for the whether the predicate is true or false. That is, X is a ‘do not care’ with respect to whether the predicate is true or false.
The plurality of intersections (412) above includes each predicate (405) of the query (404) and also implicitly includes the relationships among the predicates (405). The plurality of intersections maintain the relationship among the predicates because each intersection explicitly includes a true solution and implicitly includes other true solutions. For example, intersection row 1 above includes the true solution B (C2>4) AND D (C4<=8) each represented as true by a value of ‘1’ thereby making the query relationship true and implicitly maintains other true solutions because predicates A (C1=2) and C (C3=6) may be either true or false respectively because the value of ‘X’ in the intersections representing either true or false.
The method of
Consider for example the following exemplary intersections having predicates assigned a selectivity factor.
In the example intersections above, each predicate A, B, C, and D have been assigned a selectivity factor equal to the percentage of rows in the database table containing the predicate. Predicate A (C1=2) is assigned a selectivity factor of 0.2 because C1=2 is true for 20 percent of the rows of the target database table. Predicate B (C2>4) is assigned a selectivity factor of 0.7 because C2>4 is true for 70 percent of the rows of the target database table. Predicate C (C3=6) is assigned a selectivity factor of 0.6 because C3=6 is true for 60 percent of the rows of the target database table. And similarly, predicate D (C4<=8) is assigned a selectivity factor of 0.3 because C4<=8 is true for 30 percent of the rows of the target database table.
The method of
For further explanation, consider the following exemplary intersections whose selectivity ratings have been determined.
In the example above, the selectivity rating of intersection 1 is determined to be 0.21 or 21 percent by multiplying the selectivity factor of predicate B (selectivity factor=0.7) by the selectivity factor of predicate D (selectivity factor=0.3), the predicates (B and D) having been identified as true in intersection 1. The selectivity rating of intersection 2 in the example above is determined to be 0.42 or 42 percent by multiplying the selectivity factor of predicate B (selectivity factor=0.7) by the selectivity factor of predicate C (selectivity factor=0.6), the predicates (B and C) having been identified as true in intersection 2. The selectivity rating of intersection 3 in the example above is determined to be 0.06 or 6 percent by multiplying the selectivity factor of predicate A (selectivity factor=0.2) by the selectivity factor of predicate D (selectivity factor=0.3), the predicates (A and D) having been identified as true in intersection 3. Similarly, the selectivity rating of intersection 4 in the example above is determined to be 0.12 or 12 percent by multiplying the selectivity factor of predicate A (selectivity factor=0.2) by the selectivity factor of predicate C (selectivity factor=0.6), the predicates (A and C) having been identified as true in intersection 4.
The method of
A first-tiered method of selecting (418) one or more of the intersections (412) in dependence upon the selectivity rating may be carried out by determining whether any of the intersections has a selectivity rating of zero and selecting the intersections having a selection rating greater than zero if any of the intersections has a selectivity rating of zero as described below with reference to
In some cases, there may be few or no intersections having a selectivity rating of zero but there may be enough intersections having relatively low selectivity ratings that a threshold may be used to select intersections according to the present invention. In such cases, a second-tier methodology uses a threshold greater than zero and in such a second-tier methodology selecting one or more of the intersections in dependence upon the selectivity rating may be carried out by determining whether any of the intersections has a selectivity rating greater than a threshold and selecting the intersections having a selectivity rating greater than the threshold if any of the intersections has a selectivity rating greater than the threshold as described below with reference to
In still other cases, there may be a number of intersections having relatively similar selectivity ratings. In such cases, a third-tiered methodology may be used to select intersections for determining a projected number of rows returned by the query. In such cases, selecting one or more of the intersections in dependence upon the selectivity rating is carried out by selecting a subset of intersections as described below with reference to
After selecting (418) one or more of the intersections (412) in dependence upon the selectivity rating, the method of
Alternatively, and often when there are more selected intersection such as by using the third-tier methodology for intersection selection described below, selecting one or more of the intersections in dependence upon the selectivity rating is carried out by selecting a subset of intersections and determining (422) in dependence upon the intersections (412) a percentage of rows in a database table returned by the query is carried out by determining a projected percentage of rows in the database table determined in dependence upon the subset of intersections as described below with reference to
As mentioned above, selecting one or more of the intersections in dependence upon the selectivity rating may be carried out using a tiered methodology. For further explanation,
For further explanation, consider the following example:
In the example above, the selectivity rating of intersection 1 is determined to be zero ‘0’ by multiplying the selectivity factor of predicate B (selectivity factor=0) by the selectivity factor of predicate D (selectivity factor=0.3), the predicates (B and D) having been identified as true in intersection 1. The selectivity rating of intersection 2 in the example above is also determined to be zero ('0) by multiplying the selectivity factor of predicate B (selectivity factor=0) by the selectivity factor of predicate C (selectivity factor=0.6), the predicates (B and C) having been identified as true in intersection 2. The selectivity rating of intersection 3 in the example above is determined to be 0.06 or 6 percent by multiplying the selectivity factor of predicate A (selectivity factor=0.2) by the selectivity factor of predicate D (selectivity factor=0.3), the predicates (A and D) having been identified as true in intersection 3. Similarly, the selectivity rating of intersection 4 in the example above is determined to be 0.12 or 12 percent by multiplying the selectivity factor of predicate A (selectivity factor=0.2) by the selectivity factor of predicate C (selectivity factor=0.6), the predicates (A and C) having been identified as true in intersection 4.
In this first-tier example, two intersections, intersection 1 and intersection 2, are determined to have a selectivity rating of zero. Selecting one or more of the intersections in dependence upon the selectivity rating according to the method of
In many cases, after selecting the intersections having a selection rating greater than zero, or some other small threshold, there are relatively few intersections selected. For further explanation,
The method of
In some relatively rare cases, only one intersection has a selection rating greater than zero. In such cases, summing (606) the percentage (604) of rows in the database table returned by each of the intersections (506) having a selection rating greater than zero includes simply summing the percentage of rows returned by that single non-zero intersection with zero.
As mentioned above, in some embodiments of the present invention, there may be few or no intersections having a selectivity rating of zero but there may be enough intersections having relatively low selectivity ratings such that a threshold may be used to select intersections according to the present invention. In such cases, a second-tier methodology may be used that includes threshold greater than zero for selecting intersections for use in determining a percentage of rows in a database table returned by the query for query optimization. For further explanation, therefore,
If any of the intersections has a selectivity rating greater than the threshold, the method of
For further explanation, consider the following example:
In the example above, the selectivity rating of intersection 1 is determined to be 0.007 or 0.7 percent by multiplying the selectivity factor of predicate B (selectivity factor=0.01) by the selectivity factor of predicate D (selectivity factor=0.7), the predicates (B and D) having been identified as true in intersection 1. The selectivity rating of intersection 2 in the example above is determined to be 0.006 or 0.6 percent by multiplying the selectivity factor of predicate B (selectivity factor=0.01) by the selectivity factor of predicate C (selectivity factor=0.6), the predicates (B and C) having been identified as true in intersection 2. The selectivity rating of intersection 3 in the example above is determined to be 0.14 or 14 percent by multiplying the selectivity factor of predicate A (selectivity factor=0.2) by the selectivity factor of predicate D (selectivity factor=0.7), the predicates (A and D) having been identified as true in intersection 3. Similarly, the selectivity rating of intersection 4 in the example above is determined to be 0.12 or 12 percent by multiplying the selectivity factor of predicate A (selectivity factor=0.2) by the selectivity factor of predicate C (selectivity factor=0.6), the predicates (A and C) having been identified as true in intersection 4.
In this first second-tier example, a threshold is established at, for example, one percent. Intersection 1 and intersection 2 are determined to have a selectivity rating below the threshold of one percent. Selecting one or more of the intersections in dependence upon the selectivity rating according to the method of
In many cases, after selecting the intersections having a selection rating greater than the threshold there are relatively few intersections selected. In such cases, determining a percentage of rows in the database table returned by each of the intersections may be carried out as described above with reference to
As mentioned above, in some embodiments, there may be a number of intersections having relatively similar selectivity ratings. In such cases, a third-tiered methodology may be used to select intersections for determining a projected number of rows returned by the query. For further explanation,
For further explanation, consider the following exemplary intersections whose selectivity ratings have been determined.
In the example above, selecting (802) a subset (804) of intersections (412) according to the method of
In the method of
For further explanation,
The method of
As mentioned above, total percentage of rows returned by the subset of the intersections is calculated only from the subset. The total percentage of rows returned by the subset is therefore scaled to create a projected percentage. The method of
Σ total intersections/[Σ subset
Where:
As an aid to understanding, the following example case with data is presented for better understanding of determining a projected percentage of rows returned by a query from a subset of intersections as will occur to those of skill in the art. In this example, a ten row database table includes the following data.
In this example the following query expression is (C1=2 OR C2>4) AND (C3=6 OR C4<=8) is applied to the database table having the data above. The following table illustrates that the 8 of the 10 rows (80 percent) are in the result set of the query expression.
The intersections of a truth table representing the possible solutions to the query expression are illustrated in the chart below. The selectivity factors of each predicate in this example are determined by probing the database table above and a selectivity rating for each intersection is included in the table.
Using the method of
A projected percentage of rows returned by the query determined in dependence upon a subset of intersection 2 and intersection 3 by determining a percentage of rows in the database table returned by each of the intersections of the subset. In this example, the database table is probed to determine that intersection 2 returns 0.5 or 50 percent of the rows and intersection 3 returns zero (‘0’) rows.
Determining a projected percentage of rows in this example includes summing the percentage of rows in the database table returned intersection 2 and intersection 3 which in this example is 0.5 or 50 percent.
Determining a projected percentage of rows in this example includes determining a scalar in dependence upon the selectivity ratings of the intersections of the subset (in this case, 0.42+0.06=0.48 percent) and scaling the summed percentage in dependence upon the scalar. Determining a scalar in dependence upon the selectivity ratings of the intersections of the subset is typically carried out by summing the selectivity ratings of all the intersections (in this case 0.21+0.42+0.06+0.12=0.81) and summing the selectivity ratings of the intersections in the subset (in this case 0.42+0.06=0.48), and determining the ratio of the sum of the selectivity ratings of all the intersections and the sum of the selectivity ratings of the intersections of the subset (in this case 0.81/0.48). The scalar is therefore calculated according the following formula:
Σ total intersections/[Σ subset
Where:
In this case the calculated ratio is therefore 0.81/0.48
Scaling the scaling the summed percentage in dependence upon the scalar in this example therefore yields:
(0.81/0.48)*(0.5+0.0)=0.84
The example above yields a projected percentage of rows of 84 percent that is determined from probing only the subset of intersections not all the intersections. The projected percentage of rows which is 84 percent differs slightly from the actual percentage of rows in the database table above returned by the query which is 80 percent.
Exemplary embodiments of the present invention are described largely in the context of a fully functional computer system for query expression evaluation for query optimization. Readers of skill in the art will recognize, however, that the present invention also may be embodied in a computer program product disposed on signal bearing media for use with any suitable data processing system. Such signal bearing media may be transmission media or recordable media for machine-readable information, including magnetic media, optical media, or other suitable media. Examples of recordable media include magnetic disks in hard drives or diskettes, compact disks for optical drives, magnetic tape, and others as will occur to those of skill in the art. Examples of transmission media include telephone networks for voice communications and digital data communications networks such as, for example, Ethernets™ and networks that communicate with the Internet Protocol and the World Wide Web as well as wireless transmission media such as, for example, networks implemented according to the IEEE 802.11 family of specifications. Persons skilled in the art will immediately recognize that any computer system having suitable programming means will be capable of executing the steps of the method of the invention as embodied in a program product. Persons skilled in the art will recognize immediately that, although some of the exemplary embodiments described in this specification are oriented to software installed and executing on computer hardware, nevertheless, alternative embodiments implemented as firmware or as hardware are well within the scope of the present invention.
It will be understood from the foregoing description that modifications and changes may be made in various embodiments of the present invention without departing from its true spirit. The descriptions in this specification are for purposes of illustration only and are not to be construed in a limiting sense. The scope of the present invention is limited only by the language of the following claims.