Patent Application
20070016851
This invention relates to the processing of XML data and, more particularly, to the integration of XML data from one or more sources into a single XML document.
eXtensible Markup Language (XML) is rapidly emerging as the standard for data representation and exchange on the World Wide Web. The diversity of next-generation Web applications that rely on XML as a data-exchange format dictates the need for tools that can efficiently collect data from multiple distributed XML sources and incorporate same in a target XML document. In practice, such XML integration is typically DTD-directed. That is, the integration task is constrained by a predefined Document Type Definition (DTD) that the target XML document is required to conform. DTD-conformance is evident when, for example, enterprises agree on a common DTD and then exchange and interpret their XML data based on this predefined DTD. Another important application of DTD-conformance concerns security. That is, the integrated XML document, as a view of the original data, is required to conform to a pre-specified “view DTD” in order to both hide confidential information and facilitate effective formulation of user queries over the secure integrated view.
Ensuring the conformance of an integrated XML document (created through multiple XML data sources) to a predefined target DTD is a non-trivial problem. First, the target DTD itself may specify a fairly complex schema structure, e.g., recursive and/or non-deterministic with disjunctions. Second, the integration task may be large-scale and naturally “hierarchical”. In other words, the integration may involve a large number of distributed data sources, where some of the sources are virtual, in the sense that they are views that need to be created via XML integration. This latter requirement indicates that effective XML-integration specifications should be composable, such that large, complex integration tasks can be built via composition of simpler sub-tasks.
A straightforward solution to DTD-directed XML data integration employs a well-known XML query language (e.g., XQuery, XSLT) to define an integrated XML view, and then check whether the resulting view conforms to the prescribed DTD. Unfortunately, such an approach fails for a number of reasons. First, using full XML query languages to define an integrated view cannot guarantee DTD-conformance. Specifically, type inference for such derived XML views is too expensive to be used in practice. That is, it is intractable for extremely restricted view definitions, and undecidable for realistic views. Similarly, accurate XML type checking is difficult. Accordingly, languages such as XQuery typically implement only approximate type checking. Such an approach provides no guidance on how to specify a DTD-conforming XML view. This means that DTD-directed integration becomes a trial-and-error process where, if a resulting view fails to type-check, the view definition needs to be modified and the type-checking process must be repeated. For complex integration mappings, reaching a DTD-conforming integrated view through repeated trial-and-error is a time consuming and arduous process. Second, while Turing-Complete XML query languages (such as XQuery) can express very complex integration mappings, optimization for such languages still remains to be explored, and their complexity makes it desirable to work within a more limited formalism. That is, when it comes to large scale XML data integration, it is often desirable to trade expressive power for efficiency and ease-of-use.
Attribute Integration Grammars (AIGs), a grammar-based formalism for schema-directed integration of relational data in XML, are another possible solution. AIGs extend a target DTD with tuple-valued attributes and SQL queries over the relations, whereas a mild variation of AIGs can support DTD-directed XML-to-XML transformations. However, these proposals are inadequate for XML integration because they are restricted to flat, relational sources and they are not composable. Developing an effective, modular solution for large-scale, DTD-directed XML data integration poses a new set of challenges including the need for a significantly more powerful, composable formalism technique. Accordingly, there is a need for language construction rules or grammars that collect XML data from various sources and cost effectively and correctly integrate such data into a single XML document that conforms to a desired target DTD.
The disadvantages associated with the prior art are overcome by the present invention of a grammar for mapping a first grouping of XML data having at least one parent element and a set of corresponding child elements into a second grouping of XML data and a method for accomplishing same to incorporate the first grouping of XML data into the second grouping of XML data. In an embodiment of the invention, the grammar includes a first rule for computing a first child element attribute by extracting data from one or more sources via a query which takes a first parent element attribute as a constant; and a second rule for computing a second parent element attribute by grouping one or more second child element attributes. The first rule and second rule vary according to a production of an element type of the first grouping of XML data that is being evaluated. The element types capable of being evaluated include PCDATA, disjunctive, conjunctive and Kleene star, each having a unique rule set for defining inherited and synthesized attributes of the parent and child elements.
Additionally, the method for performing the integration (thereby specifying Document Type Definition (DTD) integration of XML data) includes the step of executing a mapping of a first grouping of XML data having at least one parent element and a set of corresponding child elements into a second grouping of XML data. The mapping further includes following at least one rule for computing a first child element attribute and a second parent element according to a production type of the XML data. The production types being selected from those identified above with respect to the grammar of the subject invention.
The teachings of the present invention can be readily understood by considering the following detailed description in conjunction with the accompanying drawings, in which:
The inventors provide a novel formalism, denoted as XML Integration Grammars (XIGs), for a modular specification of complex, DTD-directed XML integration tasks. The invention is described Within the context of specific functions and processes. However, it will be appreciated that the teachings of the invention are adaptable to many other functions and processes. An XIG is a (partial) function defined with a target DTD D, a fragment of a query language (such as XQuery), and tree-valued attributes that, given a collection of XML data sources, constructs an integrated XML document that conforms to D. The definition of an XIG is built on top of the individual element productions in the target DTD D using a collection of localized semantic rules. These rules for a DTD element production are used to compute intermediate results in tree-valued attributes that correspond to the appropriate data elements in the integrated XML document, and include (1) queries over the XML sources expressed in the XQuery fragment used, and (2) embedded XIG calls which can be either local (i.e., executed at the same site) or remote (i.e., executed remotely).
An XIG call is essentially treated as a “black box” that returns an XML tree conforming to its target DTD, thus allowing for modular XIG specifications of complex integration tasks. Based on this XIG formalism, a middleware-system architecture for DTD-directed XML integration and novel techniques for optimizing the evaluation of XIGs is proposed. Taking a middleware-based approach to XIG evaluation, results in several effective XIG-specific optimization techniques that can be applied outside the generic XQuery engine. More specifically, it is possible to capture recursive DTDs and recursive XIGs in a uniform framework, and propose a cost-based algorithm for scheduling local XML queries/XlGs and remote XIGs to maximize parallelism. An algorithm for merging multiple XQuery expressions into a single query without using “outer-union/outerjoin” is also proposed. Combined with possible optimization techniques for the XQuery fragment used in XIG definitions, such optimizations can yield efficient evaluation strategies for DTD-directed XML integration.
Note that XIG formalism is not an XML transformation language. XIGs serve as a user/application level interface for specifying DTD-directed integration in query languages. Being built around the productions in the target DTD, XIGs provide guidance on how to specify XML integration in a manner that automatically guarantees DTD conformance. Furthermore, XIGs rely on semantic rules that are local to each DTD production, thereby allowing integration sub-tasks to be declaratively specified for each production in isolation. Such a feature allows XIGs to simplify a complex integration task by breaking it into small, production/element-specific pieces that can be specified independently. XIG definitions rely solely on DTDs and a query language. As such, there is no need to study any new, specialized integration language. XIGs can be compiled into, for example, XQuery functions that impose control over computation and data flow of queries to automatically ensure DTD-conformance. Moreover, XIGs can be defined using some specific XQuery fragment that allows for more optimizations than full-fledged XQuery, thus providing better performance. Other types of query languages known to those skilled in the art can also be used to achieve the desired results and are within the scope of this invention.
A brief review of DTDs and XQuery expressions is useful in describing the subject invention. Without loss of generality, a DTD is defined as (Ele, P, r), where Ele is a finite set of element types; r is a distinguished type in Ele, called the root type; P defines the element types: for each A in Ele, P(A) is a regular expression of the following form:
α::=PCDATA|ε|B1, . . . , Bn|B1+ . . . +Bn|B*
where ε is the empty word, B is a type in Ele (referred to as a child type of A), and ‘+’, ‘,’ and ‘*’ denote disjunction, concatenation and the Kleene star, respectively. A→P(A) is referred to as the production of A. It has been shown that all DTDs can be converted to this form in linear time by introducing new element types. Examples of DTDs can be found in
As another example, consider a view for car dealers. Each dealer maintains a local XML document specified by a source DTD Ddealer 104 which describes the dealer, cars carried by the dealer, and invoice, as depicted in
Also consider integration of XML data for car dealers in a region together with sale promotion data. The regional integration is to extract data from XML sources and construct a single target document that consists of sale data, information of all the dealers in the region, and cars carried by these dealers and promoted by sale. As shown in
This integration task is rather complex. First, the target DTD 204 is recursive and non-deterministic; its DTD graph 206 given in
An XML document (tree) T conforms to a DTD D if (1) there is a unique node, the root, in T labeled with r; (2) each node in T is labeled either with an Ele type A, called an A element, or with PCDATA, called a text node; (3) each A element has a list of children of elements and text nodes such that their labels are in the regular language defined by P(A); and, (4) each text node carries a string value (PCDATA) and is a leaf of the tree. T is referred to as a document (instance) of D if T conforms to D.
XIGs can be defined with any fragment of XQuery that supports FLWR constructs as discussed in XQuery 1.0: An XML Query Language, by D. Chamberlin et al., W3C Working Draft, June 2001, herein incorporated by reference, and permits effective optimization. Given a fragment of XQuery, the syntax is extended by incorporating XIG calls in the top level let clauses. Specifically, consider the class of queries defined as follows:
Q::=q|let$ χ:=XIG_callQ XIG_call::=UV:V(U)|V(U)
where q is a query in the fragment, V is an XIG, Uv is the URI of V (for remote XIG), and U is the URI of a source XML document. Here UV:V(U) denotes a remote XIG call, and V(U) is a local XIG call. The semantics of a query “let$ χ:=XIG_call Q” is to first evaluate the XIG, assign the result of the evaluation to $x as a constant, and then evaluate the XQuery expression q. This extension is referred to as XQe. An XIG is defined with a target DTD D and is evaluated to an XML document of D; thus, the XIG can be viewed as an XML expression of “type” D.
An XIG V is a partial function from a collection X of XML sources to documents of a target DTD D, referred to as an XIG from X to D and denoted by V:X→D. Specifically, let D=(Ele, P, r); then V is defined on top of D as follows.
For a production p=A→α, the semantic rules rule(p) enforce that Syn(A) is indeed an A element as follows.
P=A→PCDATA. Then rule(p) is defined as
Syn(A)={Q(Inh(A))/value( )},
Where Q is an XQe query that returns PCDATA and treats Inh(A) as a constant parameter. See, e.g., the rule for production name→PCDATA in the XIG Vdealer of
P=A→B1, . . . , Bn. Then rule(p) consists of
Inh(Bi)=Qi(Inh(A))/* for iε[1,n]*/, Syn(A)=<A>{Syn(Bi) . . . Syn(Bn)}</A>,
Where for each i ε[1,n], Qi is an XQe query that returns a single element (subtree). As an example, see the rules for car→make, model, price, inStock in Vdealer.
P=A→B1+ . . . +Bn. Then rule(p) is defined as:
P=A→B*. Then rule(p) is defined as:
Inh(B)→for $b in Q(Inh(A)) where C($b) return $b,Syn(A)=<A>i Syn(B)</A>,
where Q is an XQe query that may return a (probably empty) set of elements, C is an XQe Boolean expression, and “i” is a list constructor. For each $b generated by Q, the rules for processing B are evaluated, treating $b as a value of Inh(B). Then the rule for Syn(A) groups together the corresponding Syn(B)'s into a list using
i in the same order as the one when the $b's are generated. For example, see the rules for cars→car* in Vdealer.
P=A→ε. Then rule(p) is defined by Syn(A)=Q(Inh(A)) where Q is an XQe query such that Q(Inh(A)) returns either <A/>, or <empty/> if the value of Syn(A) is not to be included in the target document. For example, see the rule for the production yes→ε in Vdealer.
Several subtleties are worth mentioning. First, recall that Syn(A) is defined in terms of Syn(Bi). In the rule for computing Syn(A) one may replace Syn(Bi) with the XQe query for computing Syn(Bi)(defined in the rules for Bi). For example, in the XIG Vdealer, the rules for dealer and car can be rewritten as:
The operational semantics of an XIG (i.e., V: X→D as presented above) are now considered. Given an instance of X, V evaluates its attributes via its rules and returns Syn(r) of the root r of D as its output. The evaluation is carried out in a top-down manner, using a stack. The root r is first pushed onto the stack. For each node A at the top of the stack, a subtree Syn(A) is computed. This is accomplished by first identifying the production p=A→α in D, and for each B in α, Inh(B) is evaluated with rule(p) as follows:
p=A→PCDATA—First, Q(Inh(A) is evaluated, treating Inh(A) as a constant. Note that if Q has an embedded XIG, the XIG is invoked and evaluated. Then, the PCDATA of Q(Inh(A) is assigned to Syn(A).
p=A→B1, . . . , Bn—For each Bi, Inh(Bi) is computed by evaluating Qi(Inh(A)). Then, Bi is pushed onto the stack and each is processed in the same way using the value of Inh(Bi). After all of the Bi's are evaluated and popped off of the stack, i.e., when all the Syn(Bi)'s are available, Syn(A) is computed by collecting all of the Syn(Bi)'s such that A has a unique Bi child for each iε[1,n].
p=A→B1 + . . . +Bn—First, the condition query Qc(Inh(A)) is evaluated and the first Bi is selected such that its corresponding Ci is true with respect to the result of Qc. Next, the rule for computing Inh(Bi) is evaluated. This Bi is pushed onto the stack and is processed inductively using the value of Inh(Bi). After this Bi is popped off of the stack and Syn(Bi) is available, Syn(A) is computed by means of Syn(Bi). Note that Inh(Bj) and Syn(Bj) are not computed if j≠i, i.e., Bi is the only child of A.
P=A→B*—First, Q(Inh(A)) is evaluated in the rule for computing Inh(B) For each $c returned by Q, a copy of B is pushed onto the stack, taking $c as its inherited value. The B nodes are then processed in the same way. After these nodes have been evaluated and popped off of the stack, Syn(A) is assigned the list consisting of all the Syn(B)'s. Note that is Q is empty, then Syn(A) is <empty/>, i.e., A has no children.
p=A→ε—Q(Inh(A)) is evaluated and its result is treated as Syn(A). After Syn(A) is computed, A is popped off of the stack and Syn(A) is used to evaluate other nodes until no more nodes are in the stack. At this stage, Syn(r) is computed and returned as the output of the XIG evaluation. Note that for each A, its inherited attribute is evaluated first, then its synthesized attribute, which is an A-subtree. The evaluation takes one-sweep. That is, each A element is visited twice, first pushed onto the stack and then popped off after its subtree is constructed. Accordingly, such an XIG is capable of taking XML from various sources and integrating same into a single target DTD.
Although various embodiments which incorporate the teachings of the present invention have been shown and described in detail herein, those skilled in the art can readily devise many other varied embodiments that still incorporate these teachings.
