Patent Grant
7162485
B.1 Field of Invention
This disclosure teaches techniques, methods and systems related in general to the field of information processing. More particularly, the teachings relate to methods, systems, and computer-program products for querying tree-structured databases, Web pages, or XML documents using formalisms or query languages that share certain characteristics and navigational constructs such as path expressions with the standardized XPath 1 formalism [W4] of the World Wide Web Consortium.
B.2 Basic Concepts, Terminology, and Introduction
The reader is assumed to be familiar with XML (cf. [W2]), XPath and, in particular, with standard notions such as axes and location steps in XPath (cf. [W4]). XPath has been proposed by the W3C [W4] primarily as a practical language for selecting nodes from XML document trees. But it is also designed to be used for formulating expressions that evaluate to a string, a number or a boolean value. The importance of XPath stems from
B.3 Desirable Properties of Methods and Systems Evaluating Xpath Queries and Xpath Expressions over XML Documents
Since XPath and related technologies will be tested in ever-growing deployment scenarios, its implementations need to scale well both with respect to the size of the XML data and the growing size and intricacy of the queries (usually referred to as combined complexity). In particular, this evaluation should profitably be feasible in polynomial time for any given XML document, any given XPath query or XPath expression and, optionally, any given context.
B.4 References
The following documents provide background information helpful in understanding this disclosure, and to that extent, they are incorporated herein by reference. They are referred to, using the abbreviated notations shown below, in subsequent discussions to indicate specific relevance wherever necessary.
In this section, we evaluate the efficiency of three XPath engines, namely Apache XALAN (the Lotus/IBM XPath implementation which has been donated to the Apache foundation) and James Clark's XT, which are, as we believe, the two most popular freely available XPath engines, and Microsoft Internet Explorer 6 (IE6), a commercial product. We have not made explicit experiments with Saxon, which is another popular XSLT processor (thus including an XPath engine), see [H2]. At any rate, also Saxon, in general, exhibits the exponential behavior that we observed for the other three systems mentioned above.
The version of XALAN used for the experiments was Xalan-j—2—2_D11 (i.e., a Java release). We used the current version of XT (another Java implementation) with release tag 19991105, as available on James Clark's home page, in combination with his XP parser through the SAX driver. We ran both XALAN and XT on a 360 MHz (dual processor) Ultra Sparc 60 with 512 MB of RAM running Solaris. IE6 was evaluated on a Windows 2000 machine with a 1.2 GHz AMD K7 processor and 1.5 GB of RAM.
XT and IE6 are not literally XPath engines, but are able to process XPath embedded in XSLT transformations. We used the xsl:for-each performative to obtain the set of all nodes an XPath query would evaluate to.
We show by experiments that all three implementations require time exponential in the size of the queries in the worst case. Furthermore, we show that even the simplest queries, with which IE6 can deal efficiently in the size of the queries, take quadratic time in the size of the data. Since we used two different platforms for running the benchmarks, our goal of course was not to compare the systems against each other, but to test the scalabilities of their XPath processing methods. The reason we used two different platforms was that Solaris allows for accurate timing, while IE6 is only available on Windows. (The IE6 timings reported on here have the precision of ±1 second).
For our experiments, we generated simple, flat XML documents. Each document DOC(i) was of the form
and its tree thus contained i+1 element nodes.
Experiment 1: Exponential-time Query Complexity of XALAN and XT
In this experiment, we used the fixed document DOC(2) (i.e., a
b/
b/
/a
). Queries were constructed using a simple pattern. The first query was ‘//a/b’ and the i+1-th query was obtained by taking the i-th query and appending ‘/parent::a/b’. For instance, the third query was ‘//a/b/parent::a/b/parent::a/b’.
It is easy to see that the time measurements reported in
Discussion
This behavior can be explained with the following pseudocode fragment, which seems to appropriately describe the basic query evaluation strategy of XALAN and XT.
It is clear that each application of a location step to a context node may result in a set of nodes of size linear in the size of the document (e.g., each node may have a linear number of descendants or nodes appearing after it in the document). If we now proceed by recursively applying the location steps of an XPath query to individual nodes as shown in the pseudocode procedure above, we end up consuming time exponential in the size of the query in the worst case, even for very simple path queries. As a (simplified) recurrence, we basically have
where |Q| is the length of the query and |D| is the size of the document, or equivalently
Time(|Q|)≈|D||Q|.
The class of queries used puts an emphasis on simplicity and reproducibility (using the very simple document a
b/
b/
/a
). Interestingly, each ‘parent::a/b’ sequence quite exactly doubles the times both systems take to evaluate a query, as we first jump (back) to the tree root labeled “a” and then experience the “branching factor” of two due the two child nodes labeled “b”.
Our class of queries may seem contrived; however, it is clear that we make a practical point. First, more realistic document sizes allow for very short queries only. We will show this in the second experiment for IE6, and have verified it for XALAN and XT as well. At the same time, XPath query engines need to be able to deal with increasingly sophisticated queries, along the current trend to delegate larger and larger parts of data management problems to query engines, where they can profit from their efficiency and can be made subject to optimization. The intuition that XPath can be used to match a large class of tree patterns (cf. [S4]) in XML documents also implies to a certain degree that queries may be extensive.
Moreover, similar queries using antagonist axes such as “following” and “preceding” instead of “child” and “parent” do have practical applications, such as when we want to put restrictions on the relative positions of nodes in a document. Finally, if we make the realistic assumption that the documents are always much larger than the queries (|Q|<<|D|), it is not even necessary to jump back and forth with antagonist axes. We can use queries such as //following::*/following::*/. . . /following::* to observe exponential behavior.
Experiment 2: Exponential-time Query Complexity of Internet Explorer 6
In our second experiment, we executed queries that nest two important features of XPath, namely paths and arithmetics, using IE6. The first three queries were
The experiment was carried out for four document sizes (2, 3, 10, and 200).
Experiment 3: Quadratic-time Data Complexity for Simple Path Queries (IE6)
For our third experiment, we took a fixed query and benchmarked the time taken by IE6 for various document sizes. The query was ‘//a’+q(20)+‘//b’ with
(Note: The size of queries q(i) is of course O(i).)
For instance, the query of size two according to this scheme, i.e. ‘//a’+q(2)+‘//b’, is //a//b [ancestor::a//b [ancestor::a//b]/ancestor::a//b]/ancestor::a//b
The granularity of measurements (in terms of document size) was 5000 nodes.
The disclosed teachings provide methods, systems and computer-program products for the efficient evaluation of XPath expressions over XML documents. Features of this evaluation are:
Note that these methods are not confined to a single embodiment but can be embodied in various ways. It should be clear that the detailed description given in this document is only an example implementation and should not be construed to restrict the scope of the claims in any way.
Further, computer-program products including computer-readable media with instructions that implement the systems and methods disclosed, completely or partially, are also contemplated as being within the overall scope of the disclosed teachings. It should be noted that the media could be anything including but not limited to RAMs, ROMs, hard disks, CDs, tapes, floppy disks, Internet downloads, etc. In short, any medium that can fix all or a subset of the instructions, even for a transient period of time, is considered as a computer-readable media for the purposes of this disclosed teaching. Further the type of computer that can implement is also not restricted to any particular type, but includes personal computers, workstations, mainframes and the like. Also, it can be implemented on a stand-alone computer or in a distributed fashion across a network, including but not limited to, the Internet.
1.
2.
3.
4.
5.
The following description of a preferred embodiment of our invention and of some variations and ramifications thereof is made for illustrating the general principles of the invention and is not to be taken in any limiting sense.
A Basic Notions
The following assumptions and simplifications do not restrict the applicability of the invention. In fact, anybody skilled in the art can adapt our methods to all other versions of XPath (notably XPath 2), languages similar in nature to XPath, and to full XML data models.
In this document, we use an XML document model simplified as follows. An XML document is viewed as an unranked (i.e., nodes may have a variable number of children), ordered, and labeled tree. All of the artifacts of this and the next section are defined in the context of a given XML document. Given a document tree, let dom denote the set of its nodes, and let us use the two functions
The axes self, child, parent, descendant, ancestor, descendant-or-self, ancestor-or-self, following, preceding, following-sibling, and preceding-sibling are binary relations χ⊂dom×dom. Let self:={x,x
|xεdom}. The other axes are defined in terms of our “primitive” relations “firstchild” and “nextsibling” as shown in Table 1 (cf. [W4]). R1.R2, R1∪ R2, and R1*denote t union, and reflexive and transitive closure, respectively, of binary relations R1 and R2. Let E(χ) denote the regular expression defining χ in Table 1. It is important to observe that some axes are defined in terms of other axes, but that these definitions are acyclic.
where S ⊂dom is a set of nodes of an XML document, e1 and e2 are regular expressions, R, R1, . . . , Rn are primitive relations, χ1 and χ2 are axes, and χ is an axis other than “self”.
Clearly, some axes could have been defined in a simpler way in Table 1 (e.g., ancestor equals parent.parent*). However, the definitions, which use a limited form of regular expressions only, allow to compute χ(S) in a very simple way, as evidenced by Procedure 3.
Consider the directed graph G=(V, E) with V=dom and E=R1 ∪ . . . ∪ Rn. The function eval(R
Then Procedure 3 has the following properties:
Let S ⊂ dom be a set of nodes of an XML document and χ be an axis. Then,
The O(|dom|) upper bound on the running time is due to the fact that each of the eval functions can be implemented so as to visit each node at most once and the number of calls to eval functions and relations joined by union is constant (see Table 1).
C Data Model
Let dom be the set of nodes in the document tree as introduced in Section A. Each node is of one of seven types, namely root, element, text, comment, attribute, namespace, and processing instruction. As in DOM [W1], the root node of the document is the only one of type “root”, and is the parent of the document element node of the XML document. The main type of non-terminal node is “element”, the other node types are self-explaining (cf. [W4]). Nodes of all types besides “text” and “comment” have a name associated with it.
A node test is an expression of the form τ( ) (where τ is a node type or the wildcard “node”, matching any type) or τ(n) (where n is a node name and τ is a type whose nodes have a name). τ(*) is equivalent to τ( ). We define a function T which maps each node test to the subset of dom that satisfies it. For instance, T(node( ))=dom and T(attribute(href)) returns all attribute nodes labeled “href”.
Consider DOC(4) of Part I, Section B.5. It consists of six nodes—the document element node a labeled “a”, its four children b1, . . . , b4 (labeled “b”), and a root node r which is the parent of a. We have T(root( ))={r}, T(element( ))={a, b1, . . . , b4}, T(element(a))={a}, and T(element(b))={b1, . . . , b4}.
Now, XPath axes differ from the abstract, untyped axes of Section B in that there are special child axes “attribute” and “namespace” which filter out all resulting nodes that are not of type attribute or namespace, respectively. In turn, all other XPath axis functions remove nodes of these two types from their results. We can express this formally as
attribute(S):=child(S)∩T(attribute( ))
namespace(S):=child(S)∩T(namespace( ))
and for all other XPath axes χ (let χ0 be the abstract axis of the same name),
χ(S):=χ0(S)−(T(attribute( ))∪T(namespace( )).
Node tests that occur explicitly in XPath queries must not use the types “root”, “attribute”, or “namespace”. Actually, these node tests are also redundant with ‘/’ and the “attribute” and “namespace” axes. In XPath, axis applications χ and node tests t always come in location step expressions of the form χ::t. The node test n (where n is a node name or the wildcard *) is a shortcut for τ(n), where τ is the principal node type of χ. For the axis attribute, the principal node type is attribute, for namespace it is namespace, and for all other axes, it is element. For example, child::a is short for child::element(a) and child::* is short for child::element(*).
Note that for a set of nodes S and a typed axis χ, χ(S) can be computed in linear time—just as for the untyped axes of Section B.
Let <doc be the binary document order relation, such that x<doc y (for two nodes x, y εdom) iff the opening tag of x precedes the opening tag of y in the (well-formed) document. The function first<
Given an XML Document Type Definition (DTD) [W2] that uses the ID/IDREF feature, some element nodes of the document may be identified by a unique id. The function deref_ids:string→2dom interprets its input string as a whitespace-separated list of keys and returns the set of nodes whose ids are contained in that list.
The function strval:dom→string returns the string value of a node, for the precise definition of which we refer to [W4]. Notably, the string value of an element or root node x is the concatenation of the string values of descendant text nodes descendant ({x})∩T(text( )) visited in document order. The functions to_string and to_number convert a number to a string resp. a string to a number according to the rules specified in [W4].
This concludes our discussion of the XPath data model, which is complete except for some details related to namespaces. This topic is mostly orthogonal to our discussion, and extending our framework to also handle namespaces (without a penalty with respect to efficiency bounds) is an easy exercise. To be consistent, we will not discuss the “local-name”, “namespace-uri”, and “name” core library functions [W4] either. Note that names used in node tests may be of the form NCName:*, which matches all names from a given namespace named NCNAME.
D Standard Semantics of Xpath
In this section, we present a concise definition of the semantics of XPath 1 [W4]. We assume the syntax of this language known, and cohere with its unabbreviated form [W4]. This means that
These assumptions do not cause any loss of generality, but reduce the number of cases we have to distinguish in the semantics definition below. In particular, the assumptions and restrictions imposed here are purely syntactical and our invention can be easily applied to the general XPath language by anybody skilled in the art.
The main syntactic construct of XPath are expressions, which are of one of four types, namely node set, number, string, or boolean. Each expression evaluates relative to a context {right arrow over (c)}=x, k, n
consisting of a context node x, a context position k, and a context size n [W4]. By the domain of contexts, we mean the set
C=dom×{(k, n
|1≦k≦n≦|dom|}.
Let
/π
(x) := P
π
(root)
π1/π2
(x) := ∪yεP
π1
(x) P
π2
(y)
π1|π2
(x) := P
π1
(x) ∪ P
π2
(x)
χ::t[e1] . . . [em]
(x) :=
ei
(y, idxχ(y, S), |S|) = true};
Definition 5 (Semantics of XPath) Each XPath expression returns a value of one of the following four types: number, node set, string, and boolean (abbreviated num, nset, str, and bool, respectively). Let T be an expression type and the semantics e
:C→T of XPath expression e be defined as follows.
π
(
x, k, n
):=P
π
(x)
position( )
(
x, k, n
):=k
last( )
(
x, k, n
):=n
text( )
(
x, k, n
):=strval(n)
For all other kinds of expressions e=Op(e1, . . . , em) mapping a context {right arrow over (c)} to a value of type T,Op(e1, . . . , em)
({right arrow over (c)}):=F
Op
(
e1
({right arrow over (c)}), . . . ,
em
({right arrow over (c)})),
where FOp
:T1× . . . ×Tm→T is called the effective semantics function of Op. The function P is defined in Table 2 and the effective semantics function F is defined in Table 3.
To save space, we at times reuse function definitions in Table 3 to define others. However, our definitions are not circular and the indirections can be eliminated by a constant number of unfolding steps. Moreover, we define neither the number operations floor, ceiling, and round nor the string operations concat, starts-with, contains, substring-before, substring-after, substring (two versions), string-length, normalize-space, translate, and lang in Table 3, but it is very easy to obtain these definitions from the XPath 1 Recommendation [W4]. Table 3 can be considered as a parameterization of our invention, and changes of or additions to this table allow anybody skilled in the art to adapt the query language discussed here to past or future versions of XPath or a similar language.
The compatibility of our semantics definition (modulo the assumptions made in this paper to simplify the data model) with [W4] can easily be verified by inspection of the latter document.
It is instructive to compare the definition of Pπ1/π2
in Table 2 with the procedure process-location-step of Part I, Section B.5 and the claim regarding exponential-time query evaluation made there. In fact, if the semantics definition of [W4] (or of this section, for that matter) is followed rigorously to obtain an analogous functional implementation, query evaluation using this implementation requires time exponential in the size of the queries.
E The Context-Value Table Principle
The main notion that we propose at this point to obtain an XPath evaluation method or system with polynomial-time complexity is that of a context-value table. Given an expression e that occurs in the input query, the context-value table of e specifies all valid combinations of contexts {right arrow over (c)} and values v,
E
constant number v : → num
( ) := v
ArithOp : num × num → num
(v1, v2) := v1ArithOp v2
count : nset → num
(S) := |S|
sum : nset → num
(S) := ΣnεS to_number(strval(n))
id : nset → nset
(S) := ∪nεSF
id
(strval(n))
id : str → nset
(s) := deref_ids(s)
constant string s : → str
( ) := s
and : bool × bool → bool
(b1,b2) := b1
b2
or : bool × bool → bool
(b1, b2) := b1
b2
not : bool → bool
(b) :=
b
true( ) : → bool
( ) := true
false( ) : → bool
( ) := false
RelOp : nset × nset → bool
(S1, S2) :=
RelOp : nset × num → bool
(S,v) :=
RelOp : nset × str → bool
(S,s) := ∃n ε S : strval(n) RelOp s
RelOp : nset × bool → bool
(S,b) := F
boolean
(S) RelOp b
EqOp : bool × (str ∪ num ∪ bool) → bool
(b,x) :=
boolean
(x)
EqOp : num × (str ∪ num) → bool
(v,x) := v EqOp F
number
(x)
EqOp : str × str → bool
(s1,s2) := s1 EqOp s2
GtOp : (str ∪ num ∪ bool) × (str ∪ num ∪ bool) → bool
(x1,x2) :=
number
(x1) GtOp F
number
(x2)
string : num → str
(v) := to_string(v)
string : nset → str
(S) := if S = ∅ then “” else strval(first<doc(S))
string : bool → str
(b) := if b = true then “true” else “false”
boolean : str → bool
(s) := if s ≠ “” then true else false
boolean : num → bool
(v) := if v ≠ ±0 and v ≠ NaN
boolean : nset → bool
(S) := if S ≠ ∅ then true else false
number : str → num
(s) := to_number(s)
number : bool → num
(b) := if b=true then 1 else 0
number : nset → num
(S) := F
number
(F
string
(S))
such that e evaluates to v in context {right arrow over (c)}. Alternatively, where appropriate, a context-value table may only hold a subset of these context-value tuples, obtained by ignoring tuples that are of no interest in a given query processing setting.
In the following, by atomic query operations, we mainly refer to ones at the (small) granularity of those of Table 3. However, any kind of operation that can be executed in polynomial time in the size of its input is applicable.
The general context-value table principle for obtaining a polynomial-time technique for processing XPath queries is, simply speaking, as follows.
In the following subsections, we discuss two main embodiments of this principle:
Clearly, these are just two natural interpretations of the context-value table principle. Notably, we envision the possibility of processing XPath expressions not just via their strict syntactic subexpressions, but also through equivalent but syntactically different subexpressions, for optimization reasons.
Moreover, for similar reasons, it is also possible to trade strict bottom-up or top-down navigation by adaptive methods and systems which combine these two strategies or use different tree navigation strategies altogether.
Finally, as a third embodiment, we show how existing XPath processors can be improved to polynomial time methods by incorporating the context-value table principle into them.
E.1 Embodiment 1
Bottom-up Evaluation of Xpath
In this section, we present a semantics and a method for evaluating XPath queries in polynomial time which both use a “bottom-up” intuition. We discuss the intuitions which lead to polynomial time evaluation (which we call the “context-value table principle”), and establish the correctness and complexity results.
Definition 6 (Semantics) We represent the four XPath expression types nset, num, str, and bool using relations as shown in Table 4. The bottom-up semantics of expressions is defined via a semantics function
ε↑:Expression→nset∪num∪str∪bool,
given in Table 5 and as
E↑Op(e1, . . . , em)
:={
{right arrow over (c)}, F
Op
(v1, . . . , vm)
|{right arrow over (c)}εC,
{right arrow over (c)}, v1
εE ↑
e1
, . . . ,
{right arrow over (c)}, vm
εE↑
em
}
for the remaining kinds of XPath expressions.
Now, for each expression e and each (x, k, n)εC, there is exactly one v s.t. (x, k, n, v)εE↑e
, and which happens to be the value
e
(
x, k, n
) of e on (x, k, n) (see Definition 5). We thus immediately get the following correctness result:
Query Evaluation. The idea of Procedure 7 below is so closely based on our semantics definition that its correctness follows directly from the above correctness result.
E
e
}> | <x0, k0, n0, S> ∈ ε↑
E
}
π
}
π2
, y ∈ Y}> | 1 ≦ K ≦ n ≦ |dom|, <x, k1, n1, Y> ∈ ε↑
π1
}
π1
, <x, k, n, S2> ∈ ε↑
π2
}
Q
.
l
root(Tree(Q))
∉ R do
l1
, . . . ,ε↑
ln
∈ R;
Op(l1, . . . ,ln)
using ε↑
l1
, . . . ,ε↑
ln
;
Op(l1, . . . ,ln)
to R;
root(Tree(Q))
.
Now let Q denote an arbitrary XPath expression and let D be an XML document. Moreover, suppose that we want to evaluate Q for the context x, k, n
εC. Then this can be achieved as follows: First the context-value tables of all subexpressions of Q are computed via Procedure 7. Then we select the quadruple
x′, k′, n′, v′
from the context-value table of the root of the query-tree, such that the input context
x, k, n
coincides with the first three components of
x′, k′, n′, v′
, i.e., x=x′, k=k′, n=n′. Thus, by the definition of context-value tables, the final result of the evaluation of Q for the context
x, k, n
is precisely the fourth component v′ of the selected quadruple
x′, k′, n′, v′
. Example 8 below will help to illustrate this method.
E3
E2
E1
Q
Consider document DOC(4) of Part I, Section B.5. Let dom={r, a, b1, . . . , b4}, where r denotes the root node, a the document element node (the child of r, labeled a) and b1, . . . , b4 denote the children of a in document order (labeled b). We want to evaluate the XPath query Q, which reads as
Moreover, we have
The most interesting step is the computation of E↑E2
from the tables for E3 and E4. For instance, consider
b1, k, n, {b2, b3, b4}
εE↑
E3
. b2 is the first, b3 the second, and b4 the third of the three siblings following b1. Thus, only for b2 and b3 is the condition E2 (requiring that the position in set {b2, b3, b4} is different from the size of the set, three) satisfied. Thus, we obtain the tuple
b1, k, n, {b2, b3}
which we add to E↑
E2
. In order to read out the final result from the context-value table of Q, we have to select the second row of this table since this entry corresponds to the input context-node a. Hence, we get the final result {b2, b3}.
As far as the complexity of Procedure 7 is concerned, the following property can be shown:
Numbers and strings computable in XPath are of size O(|D|·|Q|): “concat” on strings and arithmetic multiplication on numbers are the most costly operations (w.r.t. size increase of values) on strings and numbers. Recall that for the conversion from a node set to a string or number, only the first node in the set is chosen. Of the string functions, only “concat” may produce a string longer than the input strings. The “translate” function of [W4], for instance, does not allow for arbitrary but just single-character replacement, e.g. for case-conversion purposes.
Hence, the lengths of the argument values add up such that we get to sizes O(|D|·|Q|) at worst, even in the relation representing the “top” expression Q itself. The overall space bound of o(|D|4·|Q|2) follows. Note that no significant additional amount of space is required for intermediate computations.
Let each context-value table be stored as a three-dimensional array, such that we can find the value for a given context (x, k, n) in constant time. Given m context-value tables representing expressions e1, . . . , em and a context x, k, n
, any m-ary XPath operation Op(e1, . . . , em) on context
x, k, n
can be evaluated in time O(|D|·I); again, I is the size of the input values and thus O(|D|·|Q|). This is not difficult to verify; it only takes very standard techniques to implement the XPath operations according to the definitions of Table 3 (sometimes using auxiliary data structures created in a preprocessing step). The most costly operator is RelOp : nset×nset→bool, and this one also takes the most ingenuity. We assume a pre-computed table
It becomes clear that each of the expression relations can be computed in time O(|D|3·|D|2·|Q|) at worst when the expression semantics tables of the direct subexpressions are given. (The |Q| factor is due the size bound on strings and numbers generated during the computation.) Moreover, O(|Q|) such computations are needed in total to evaluate Q. The O(|D|5·|Q|2) time bound follows.
Note that contexts can also be represented in terms of pairs of a current and a “previous” context node (rather than triples of a node, a position, and a size), which are defined relative to an axis and a node test (which, however, are fixed with the query). For instance, the corresponding ternary context for {right arrow over (c)}=(x0, x) w.r.t. axis χ and node test t is (x, idxχ(x, Y), |Y|), where Y={y|x0χy, y εT(t)}. Thus, position and size values can be recovered on demand.
Moreover, it is possible to represent context-value tables of node set-typed expressions just as binary unnested relations (indeed, a construction that contains a previous node, a current node, and a node set as value would represent two location steps rather than one). Note that to move to this alternative form of representation, a number of changes in various aspects of our construction are necessary, which we do not describe here in detail. Through these two changes, it is possible to obtain an improved worst-case time bound of O(|D|4·|Q|2) for the bottom-up XPath query evaluation.
E.2 Embodiment 2: Top-down Evaluation of XPath
In the previous section, we obtained a bottom-up semantics definition which led to a polynomial-time query evaluation method for XPath. Despite this favorable complexity bound, the bottom-up technique of Embodiment 1 can be further improved, since usually many irrelevant intermediate results are computed to fill the context-value tables which are not used later on. Next, building on the context-value table principle of Section E.1, we develop a top-down evaluation method based on vector computation for which the favorable (worst-case) complexity bound carries over but in which the computation of a large number of irrelevant results is avoided.
Given an m-ary operation Op: Dm→D, its vectorized version Op: (Dk)m→Dk is defined as
Op(
x1,1, . . . , x1,k
, . . . ,
xm,1, . . . , xm,k
) :=
Op(x1,1, . . . , xm,1), . . . , Op(x1,k, . . . , xm,k)
For instance, X1, . . . , Xk)∪
Y1, . . . , Yk
:=
X1 ∪Y1, . . . , Xk ∪Yk
. Let
S↓: LocationPath→List(2dom)→List(2dom)
be the auxiliary semantics function for location paths defined in Table 7. We basically distinguish the same cases (related to location paths) as for the bottom-up semantics E↑π
. Given a location path π and a list
X1, . . . , Xk
of node sets, S↓ determines a list
Y1, . . . , Yk
of node sets, s.t. for every
iε{1, . . . k}, the nodes reachable from the context nodes in Xi via the location path π are precisely the nodes in Yi. S↓π
can be obtained from the relations E↑
π
as follows. A node y is in Yi iff there is an xεXi and such that
x, p, s, y
εE↑
π
for some numbers p and s.
Definition 9 The semantics function E↓ for arbitrary XPath expressions is of the following type:
E↓: XPathExpression→List(C)→List(XPathType)
Given an XPath expression e and a list ({right arrow over (c)}1, . . . , {right arrow over (c)}l) of contexts, E↓ determines a list r1, . . . , rl
of results of one of the XPath types number, string, boolean, or node set. E↓is defined as
Consider the XPath query
Given the query Q, data, and context a, 1, 1
of Example 8, we evaluate Q as E↓
Q
(
a, 1, 1
)=S↓
E2
(S↓
descendant::b
({a})). Again, E2 is the subexpression
following-sibling::*[position( ) !=last( )].
First, we obtain S↓descendant::b
({a})=
{b1, b2, b3, b4}
. To compute the location step S↓
E2
(
{b1, b2, b3, b4}
), we proceed as described in the method of Table 7. We initially obtain the set
S={b1, b2
,
b1, b3
,
b1, b4
,
b2, b3
,
b2, b4
,
b3, b4
}
and the list of contexts {right arrow over (t)}=b2, 1, 3
,
b3, 2, 3
,
b4, 3, 3
,
b3, 1, 2
,
b4, 2, 2
,
b4, 1, 1
.
The check of condition E4 returns the filter
{right arrow over (r)}=true, true, false, true, false, false
.
which is applied to S to obtain
S={b1, b2
,
b1, b3
,
b2, b3
}
Thus, the query returns {b2, b3}
.
The correctness of the top-down semantics follows immediately from the corresponding result in the bottom-up case and from the definition of S↓ and E↓. We thus have:
S↓ and E↓ can be immediately transformed into function definitions in a top-down evaluation method. We thus have to define one evaluation function for each case of the definition of S↓ and E↓, respectively. The functions corresponding to the various cases of S↓ have a location path and a list of node sets of variable length (X1, . . . , Xk) as input parameter and return a list (R1, . . . , Rk) of node sets of the same length as result. Likewise, the functions corresponding to E↓ take an arbitrary XPath expression and a list of contexts as input and return a list of XPath values (which can be of type num, str, bool or nset). Moreover, the recursions in the definition of S↓ and E↓ correspond to recursive function calls of the respective evaluation functions.
Now suppose that we want to evaluate an arbitrary XPath expression Q over some XML document D for some context x, k, n
εC. Then we would simply have to call the functional implementation of E↓
Q
({right arrow over (c)}) with {right arrow over (c)}=
x, k, n
. The return value of this function call is the desired evaluation.
Analogously to the complexity of the bottom-up procedure from Section E.1, the following property of the top-down evaluation method can be shown:
Finally, note that using arguments relating the top-down method of this section with (join) optimization techniques in relational databases, one may argue that the context-value table principle is also the basis of the above mentioned polynomial-time bound of the top-down evaluation method.
E.3 Embodiment 3
Improving Existing Xpath-Processors
Those skilled in the art will recognize that an embodiment of the disclosed invention can also be obtained from an existing method or system for evaluating XPath queries such as, e.g., IE6, Saxon, Xalan-C++, Xalan-Java, XT (cf. [H1, H2, H3, H4]), etc. by improving said systems as follows:
During the evaluation process of some input XPath query Q, all of the existing methods or systems repeatedly evaluate subexpressions e of Q for contexts {right arrow over (c)}εC, where {right arrow over (c)} is of the form {right arrow over (c)}=x, k, n
for some context-node x, context-position I, and context-size n. In contrast to the method described in this document, the existing methods and systems, in general, do the evaluation of the same subexpression e of Q for the same context {right arrow over (c)}εC more than once. This is the very reason why their time complexity is, in general, exponential. By incorporating data structures analogous to the context-value tables described in Section E.1, multiple evaluations of the same subexpression e of Q for the same context {right arrow over (c)}εC can be avoided, thus reducing the time complexity to polynomial time. The data structures used for this purpose will be referred to as “data pool”. It contains triples of the form
e, {right arrow over (c)}, v
, where e is a subexpression of the input XPath query Q, {right arrow over (c)}εC is a context, and v is the result value obtained when evaluating e for the context {right arrow over (c)}. In other terms, ({right arrow over (c)}, v) can be considered as a row in the context-value table corresponding to e. Initially, the data pool is empty, i.e., it contains no such triples.
In order to guarantee that no evaluation of the same subexpression e for the same context {right arrow over (c)} is done more than once, we have to add two further components to the existing methods and systems, namely a “storage procedure” and a “retrieval procedure”. Prior to the evaluation of any subexpression e for any context {right arrow over (c)}, the retrieval procedure is called, which checks whether a triple e′, {right arrow over (c)}′, v
with e=e′ and {right arrow over (c)}={right arrow over (c)}′ already exists in the data pool. If this is the case, then the result value v of e for the context {right arrow over (c)} is returned without further computation. On the other hand, after the evaluation of any subexpression e for any context {right arrow over (c)} yielding the result value v, the storage procedure stores the triple
e, {right arrow over (c)}, v
in the data pool.
Let the basic evaluation step of an existing method or system be referred to as “atomic-evaluation”, which takes an XPath expression e and a context {right arrow over (c)} as an input and returns the corresponding result value v. Then this “atomic-evaluation” simply has to be replaced by the following procedure based on the context-value table principle:
Analogously to the Sections E.1 and E.2, we get the following complexity result:
Linear Time Evaluation of Core Xpath
In this section, we define a fragment of XPath (called Core XPath) which constitutes a clean logical core of XPath. The only objects that are manipulated in this language are sets of nodes (i.e., there are no arithmetical or string operations). Besides from these restrictions, the full power of location paths is supported, and so is the matching of such paths in condition predicates (with an “exists” semantics), and the closure of such condition expressions with respect to boolean operations “and”, “or”, and “not”.
We define a mapping of each query in this language to a simple algebra over the set operations ∩, ∪, ‘−’, χ (the axis functions from Definition 2), and an operation
is dom if rootεS and θ otherwise.
Note that each XPath axis has a natural inverse: self−1=self, child−1=parent, descendant−1=ancestor, descendant-or-self−1=ancestor-or-self, following−1=preceding, and following-sibling−1=preceding-sibling. Then the following relation between axes and inverse axes can be proved by a very easy induction.
where N0 is a set of context nodes or dom and a query π evaluates as S→π
(N0).
The Core XPath Query
(Note that there are alternative but equivalent query-trees due to the associativity and commutativity of some of our operators.)
The semantics of XPath and Core XPath (defined using S←, S→, and E1) coincide in the following way:
using S→, S←, and ε1 in time O(|Q|). Each of the operations in our algebra can be carried out in time O(|D|). Since at most O(|Q|) such operations need to be carried out to process E, the complexity bound follows.
F.2 Embodiment 5
Linear Time Evaluation of XPatterns
We extend our linear-time fragment Core XPath by the operation id: nset→nset of Table 3 by defining “id” as an axis relation
id: id:={x0,x
|x0εdom, xεderef_ids(strval(x0))}
Queries of the form π1/id(π2)/π3 are now treated as π1/π2/id/π3.
Then, analogously to Section F.1, the following relationship between the semantics functions S↓ and S→ holds:
Example. Let id(i)=ni. For the XML document t id=1
3
t id=2
1
/t
t id=3
1 2
/t
/t
, we have ref:={
n1, n3
,
n2, n1
,
n3, n1
,
n3, n2
}.
“ref” can be efficiently computed in a preprocessing step. It does not satisfy any functional dependencies, but it is guaranteed to be of linear size w.r.t. the input data (however, not in the tree nodes). Now we can encode id(S) as those nodes reachable from S and its descendants using “ref”.
We may define XPatterns as the smallest language that subsumes Core XPath and the XSLT Pattern language of [W3] (see also [S5] for a good and formal overview of this language) and is (syntactically)
x : nextsibling(x, y)}
y : nextsibling(x, y)}
contained in XPath. Stated differently, it is obtained by extending the language of [W3] without the first-of-type and last-of-type predicates (which do not exist in XPath) to support all of the XPath axes. As pointed out in the introduction, XPatterns is an interesting and practically useful query language. It can be shown that XPatterns queries have the following computational property:
Let Σ be a finite set of all possible node names that a document may use (e.g., given through a DTD). Note that the unary first-of-type and last-of-type predicates can be computed in time O(|D|*|Σ|) when parsing the document, but are of size O(|D|):
where R+=R.R*.
G Empiric Results
In the previous sections, we presented the first XPath query evaluation methods and systems that run in polynominal time with respect to the size of both the data and of the query. Our results will empower XPath engines to be able to deal efficiently with very sophisticated queries.
We have made a main-memory implementation of the top-down evaluation method of Section E.2. Table 9 compares it to IE6 along the assumptions made in Experiment 2 (i.e., the queries of which were strictly the most demanding of all three experiments). It shows that our method and system scales linearly in the size of the queries and quadratically (for this class of queries) in the size of the data.
This application claims benefit of Provisional Application No. 60/389,513 filed Jun. 19, 2002; the disclosure of which is incorporated herein by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 20030074352 | Raboczi et al. | Apr 2003 | A1 |
| 20040073541 | Lindblad et al. | Apr 2004 | A1 |
| 20040167864 | Wang et al. | Aug 2004 | A1 |
| Number | Date | Country | |
|---|---|---|---|
| 20040060007 A1 | Mar 2004 | US |
| Number | Date | Country | |
|---|---|---|---|
| 60389513 | Jun 2003 | US |
