Patent Application
20140143280
The present invention relates to data storage and data queries.
Resource Description Framework (RDF) is the de-facto standard for data representation on the World Wide Web. The amount of RDF data from disparate domains grows rapidly. For instance, the Linked Open Data (LOD) initiative integrates billions of entities from hundreds of sources. Just one of these sources, the DBpedia dataset, describes more than 3.64 million things using more than 1 billion RDF triples, of which 385 million are extracted from the English edition of Wikipedia.
Keyword searching is used to explore and search large data corpuses whose structure is either unknown or constantly changing and has already been studied in the context of World Wide Web data, graphs, relational databases and XML documents. More recent efforts considered applying keyword searching over RDF data; however, the solutions considered by these efforts have serious limitations. Most notably, these previous attempts suffer from either false positives, i.e., the keyword search returns answers that do not correspond to real subgraphs from the underlying RDF data or false negatives, i.e., the search misses valid matches from the RDF data. A severe limitation of existing techniques is the inability to scale to handle typical RDF datasets with tens of millions of triples. When presented with such workloads, existing techniques often return empty results for meaningful keyword queries that do have matches from the underlying RDF data.
Exemplary embodiments of systems and methods in accordance with the present invention provide improved keyword searching over large volumes of resource description framework (RDF) data. A scalable and exact solution handles realistic RDF datasets with tens of millions of distinct triples and achieves accurate search results. A succinct and effective summarization structure is built from the underlying RDF graph based on the type system in the RDF graph. Given a keyword search query, the summarization structure prunes out the keyword search space, which leads to increased efficiency compared to approaches that process queries directly on the RDF graph. This summarization is accomplished without any loss of information or data contained in the underlying RDF graph dataset.
A critical error in the termination condition of existing keyword search techniques misses correct results even if this error is fixed due to the limitations in its summarization. The present invention provides a correct termination. Efficient algorithms summarize the structure of RDF data into a summarization that is indexable, lightweight, and easy to update. In addition, the summarization is scalable and gives exact results. Keyword search queries are answered based on the summarization. In addition, the summarization can be updated incrementally and efficiently, with insertions or deletions to the underlying RDF data.
A resource description framework (RDF) dataset is a graph, i.e., an RDF graph, containing a plurality of triples. Each triple is formed by a subject, a predicate and an object such that the predicate connects the subject to the object. Therefore, a triple is regarded as a directed edge (the predicate) connecting two vertices, subject and object. Referring initially to
The use of the prescribed unified vocabularies on an RDF graph facilitates a classification of vertices and edges into three distinct groups, string vertices 106, type vertices 108 and entity vertices 110. VE is the set of entity vertices 110, i.e., uniform resource identifiers (URIs), and VT is the set of type vertices 108. V, is the set of keyword vertices 106. The division on vertices results in a corresponding division on the RDF predicates, i.e., the edges in a directed graph. The RDF graph includes entity-entity predicates 112, entity-keyword predicates 114 and entity-type predicates 116. ER is the set of edges for entity-entity predicates, i.e., connecting two vertices in VE. EA is the set of edges for entity-keyword predicates, i.e., connecting an entity to a keyword, and ET is the set of edges for entity-type predicates, i.e., connecting an entity to a type. The main structure of an RDF graph is captured by the entity-entity predicates or edges 112 represented by the set ER. Using these set representations, a given RDF dataset is viewed as an RDF graph G =(V, E), where V is the union of disjoint sets, VE,VE and VW, and E is the union of disjoint sets ER, EA and EE.
An alternate view of the RDF dataset is generated by treating a given entity vertex along with its associated type and keyword vertices as a single combined vertex. For example, the entity vertices URI5, URI1 and URI3 from
For an RDF graph G={V,E}, the alternate view of the RDF dataset is incorporated to generate the condensed view of the RDF graph G, denoted as Gc={V′E,ER}. While every vertex v′ε VE contains not only the entity value of a corresponding vertex vεVE, but also the associated keywords and types of v. For ease of presentation, a single keyword and a single type are associated to each entity, which works for the general case without additional effort or cost. In addition, hereinafter, G={V,E} is used to represent the condensed view of an RDF graph.
SPARQL is a pattern-matching query language. For example, to extract the vertices that are connected by predicates LaunchPad and Booster in
SELECT*WHERE{?x launchPAD?y. ?xbooster?z. OPTIONAL{?x previousMission ?w}}
The actual bindings for the variables (whose names begin with “?”) are retrieved by evaluating the query on the dataset. An optional triple pattern is provided in the query, where its matching is not enforced in the evaluation of the query. Evaluating the query on the data in
Given a condensed RDF graph G={V,E}, for any vertex vεV, let w(v) be the keyword stored in v. Formally, a keyword search query q in an RDF data G={V,E} is defined by m unique keywords {w1 , w2, . . . , wm}. A set of vertices {r,v1, . . . , vm} from V is a qualified candidate when:
rεV is a root answer node and vi εV for i ε[1, m], and
w(vi)=wi.
If we define the answer for q as A(q) and the set of all qualified candidates in G with respect to q as C(q), then
where d(r,vi) is the graph distance between vertices r and vi when treating G as an undirected graph. Intuitively, this definition looks for a subgraph in an RDF graph that has minimum length to cover all query keywords from a root node r. In prior works concerning keyword search in RDF data, the graph distance of d(v1, v2) is simply the shortest path between v1 and v2 in G, when each edge is assigned a weight, i.e., distance. By default, every edge in E has a weight of 1. In this case, d(v1, v2) simply gives the minimum number of hops required to move from v1 to v2 in G. When v1 and v2 belong to disconnected parts of G, i.e., v1 cannot reach v2 at all, d(v1, v2)=+∞.
In addition, equation (1) defines the answer of a keyword search query in G as the subgraph g in G that connects all query keywords with the minimum sum of weights from a root node rεg to every other query keyword node in g. This problem is extended to the top-k version, when a user supplies an integer parameter k. The answer is the top k qualified candidates from C (q) that minimize the value of equation (1). By defining the score of a qualified candidate gεC(q) as s(g)=Σrεg,v
Many techniques for keyword searching on generic graphs assume that graphs will fit within memory. This assumption, however, is unrealistic for common, large RDF graphs. In addition, certain approaches maintain a distance matrix for all vertex pairs. These approaches do not scale for graphs containing millions of vertices. In addition, previous approaches do not consider how to handle updates.
One approach is the baseline method. A baseline solution is based on the “backward search” heuristic on generic graphs. Intuitively, the “backward search” for the root node r starts simultaneously from each vertex in the graph G that corresponds to a query keyword and expands to its neighboring nodes recursively until a candidate answer is generated. A termination condition is used to determine whether the search is complete. Keyword searching on RDF graphs that applies backward searching utilizes a termination condition to stop the search whenever the expansions originating from m vertices and corresponding to m distinct query keywords meet at a node r for the first time. Therefore,
the set is returned as the answer. Unfortunately, this termination condition is incorrect. Using the above termination condition, the three expansions for the three vertices
covering the query keywords
meet for the first time in the second iteration, so the candidate answer
is returned and
. Continuing to the next iteration, the three expansions meet again at
, with
and
, which is the correct answer. Even if this error in the terminating condition is corrected, incorrect results are still returned due to the limitations in the summary that is built.
In addition to these limitations, the backward method is not scalable for large disk-resident RDF graphs as it initiates many random accesses to the data on disk and constructs numerous search paths in order to complete the search. However, the majority of the random accesses and search paths will not produce any answers. In order to perform the backward search only on the most promising sections of the RDF dataset, exemplary embodiments in accordance with the present invention create and utilize a type-based summarization of the RDF dataset. By operating the keyword search initially on the type-based summary, which is much smaller than the actual underlying RDF dataset, large portions of the RDF dataset graph that are irrelevant to a given query are eliminated. This also reduces the computation cost and facilitates the handling of larger RDF graphs. In accordance with exemplary embodiments of the present invention, partitions are induced over the RDF graph G based on the types in G. To build these partitions, the inherent structures that are unique to the RDF dataset are leveraged. In general, RDF graphs contain types for their vertices. The distinct number of types in a RDF graph G is usually much smaller than the number of vertices in G. The interconnections or relationships among different types in a given RDF graph are inferred from the relationships between entity vertices in the graph and are used to summarize the common structures of the condensed RDF graph G.
Neighborhoods in close proximity around vertices of the same type often share similar structures in how they connect to vertices of other types. Considering the RDF graph illustrated in
In order to generate the common type based structures, the RDF graph is initially split into a plurality of smaller partitions. Then, a minimal set of common type based structures is defined that summarizes each partition. The summarization maintains distinct structures from all the partitions. In general, keyword searching benefits from the summarization, because the summarization obtains the upper and lower bounds for the distance traversed in keyword exploration without frequently turning to the RDF dataset stored, for example, on disk and provides for the efficient retrieval of every partition from the RDF dataset by interacting with RDF query engine. The summarization is kept as small as possible without compromising these searching benefits so that it can be cached in memory for query optimization.
The present invention utilizes two notions from graph theory, graph homomorphisms and the core of a graph. As illustrated in to a graph
, written as
, is a mapping function
such that
implies
. When such an f exists, G is homomorphic to
.
Referring to and
imply
.
A core is a graph that is only homomorphic to itself, but not to any one of its proper subgraphs. Formally, a core c of a graph G is a graph with the following properties: there exists a homomorphism from c to G; there exists a homomorphism from G to c; and c is graph having these properties that has a minimal number of vertices. Therefore, cores reduce the size of a given graph and replace it with a graph having a minimum number of vertices. Therefore, homomorphisms are used to reduce the number of partitions, and cores are used to reduce the size of any given partition or summary or partitions. In one exemplary embodiment in accordance with the present invention, a core is identified for each one of the plurality of partitions before homomorphisms are used to generate the summaries. Alternatively, homomorphisms are used to generate the summaries and a core is identified for each generated summary.
Referring again to
Partitioning splits the input RDF dataset G into a plurality of smaller but semantically similar and edge disjoint subgraphs. Given that nodes with the same type often share similar type-neighborhoods, a distinct set of partitions for G are induced based on the types in G, using small subgraphs surrounding vertices of the same type. The partitioning algorithm treats an input RDF dataset as a directed graph G concerning only the type information, i.e., the condensed view of an RDF graph with each vertex only keeping its type information. For any vertex that does not have a type specified by the underlying dataset, a universal type, NA, is assigned. Systems and methods in accordance with the present invention build semantically similar partitions.
If the RDF dataset graph has n distinct number of types
, and Vi represents the set of vertices from V that have a type Ti, the z-neighborhood surrounding a vertex is defined, where α is a parameter used to produce a set of edge disjoint partitions P over G. For any vertex
and a constant α, the α-neighborhood of is the subgraph from G obtained by expanding v with α hops in a breadth-first manner, denoted
, subject to the constraint that the expansion only uses edges which have not been included yet in any partition in P. The i-hop neighboring nodes of v are defined as the set of vertices in G that can be connected to v through a directed path with exactly i directed edges. Since directed edges are used, the i-hop neighboring nodes of v can be an empty set. The nodes in
are a subset of the α-hop neighboring nodes of v since some may have already been included in another partition. To produce P, P is initialized to be an empty set and iterated through a plurality of different types. For type Ti and each vertex
, the α-neighborhood
is found and
is added as a new partition into P.
To summarize the properties of the partitions constructed in accordance with the present invention, the partitions in P are edge disjoint, and the union of all partitions in P covers the entire graph G. The order of iteration through different types may affect the final partitions P. However, no matter which order is chosen, vertices in the same type always induce a set of partitions based on their α-neighborhoods, which is what matters for building good summarization structures. Therefore, in general any traversal order over different types will yield a good partition of G to be used in the summarization procedure, as long as partitions are produced using small subgraphs surrounding entity vertices of the same type. Different traversal orders over the types might still lead to the same partitions P. Since G is treated as a directed graph, the inherent type structures in G already pose a limitation on what their 's will be regardless of the current state in P, i.e., the traversal order. For example, the partitions P for the RDF dataset of
Having generated the plurality of partitions, a summary S(G) for the RDF dataset graph G is generated. The summary is initialized to an empty set, and each partition in P is considered by visiting the α-neighborhood of v in a breadth-first manner. During this traversal, a covering tree is constructed for the edges in
, denoted
. For each visited vertex in
, its type is extracted and a new node is created in
even if a node for this type already exists. A tree
is built that represents all the distinct type-paths in
. Referring to
702 is built for the first partition
704. The vertex
705 in the first partition is visited three times in the traversal across three different paths through the first partition using single hops from vertex to vertex. This yields three distinct nodes in the covering tree with the same type, designated
703,
707 and
708 in
. In addition, an isomorphic structure to tree
702 is
, which resulted from the partition
706. This illustrates two advantages of building type-based covering trees. First, the size of the resulting summary S(G) can be reduced. As illustrated in
and
, share an identical structure at the type level. Taking advantage of such overlaps is the easiest way to reduce the number of distinct structures in S(G). The second advantage is efficiency. Building S(G) necessitates testing for subgraph isomorphism. Whereas this is computationally hard for generic graphs (NP-complete), there are polynomial time solutions for the problem since the testing on trees can be restricted, leading to improved efficiency in constructing S(G).
The size of the summary tree structure for a partition is further reduced by extracting its core and using it to represent the structure of the partition. This is achieved with a bottom up and recursive procedure in which homomorphic branches are merged under the same parent node in a given summary tree. An example of such a merging is shown in the reduced type based structure 500 illustrated in
to its core. Again, the merging steps used to generate the core for any given summary tree are logged and stored in association with the summary tree structure core.
Once a core c is constructed for a given partition, the existing summary structures in S(G) are scanned to check if any existing structure in S(G) is homomorphic to c or if c is homomorphic to any existing structure
in S(G). In the former case,
is removed from S(G) and the scan is continued. In the latter case, the scan is terminated, and S(G) is returned without c. When S(G) is empty or c is not homomorphic to any of the structures in S(G) after a complete scan on S(G), c is added into S(G), and S(G) is returned.
To facilitate keyword searching, a plurality of auxiliary, i.e., inverted, indexes are maintained in combination with the summary of the RDF graph. A portal node is a data node that is included in more than one partition. This is possible because the partitions are created to be edge-disjoint and not node disjoint. A portal node joins different partitions. A partition may have multiple portals but typically has fewer portals than the total number of nodes in the partition. Portal nodes allow different partitions to be pieced together. In a first index, for each partition h, a unique id,
is assigned and associated with the list of portals in that partition. Since
is used to represent
, where a vertex in
could correspond to more than one vertex in
,
can represent the mappings in
for a vertex
in
. All vertices in
are of the same type. Let
. Considering the first partition
and the covering tree for the first partition
as illustrated in
. The second index is to map the
of h to
.
A third index maps data nodes in partitions to summary nodes in S(G). In particular, a unique id, is assigned to each summary in S(G) and each node in S(G) is denoted with a unique id nid. For any data node u in a partition
with partition id
, d is defined as the distance of u from v. This third index maps the data node u in
to an entry that stores the partition root v, the partition id, the distance d, the id
of the summary and the id
of the summary node that u corresponds to.
In order to obtain the homomorphic mappings from each to a summary in S(G), a log is maintained for all the homomorphism found during the construction of S(G). Once S(G) is finalized, the mappings in this log are traced to find all the mappings from data to summaries. As each partition, i.e., represented by its core, is either in the final S(G) or is homomorphic to one other partition, the size of the log is linear to G. Referring to
is the covering tree for the ith partition. The homomorphism log includes sets of trees 802 in combination with their homomorphic mappings 804. Each set is associated with one of the summaries in S(G) that all trees in that set are homomorphic to. To find the final mappings, each set of trees in the log is scanned, and the homomorphisms of each entry are mapped in a set to the corresponding entry from S(G).
Exemplary embodiments of systems and methods in accordance with the present invention utilize a scalable and exact search algorithm by leveraging graph partitions and the summarization of the present invention. In accordance with one embodiment, a two-level backward search is conducted. One backward search is conducted at the summary level, and one backward search is conducted at the data level. For identified connected partitions that are found to contain all the distinct keywords at the summary level, a backward search at the data level is initiated. Path length computation is at the heart of backward search. While working at the summary level, exact path lengths are not available. Therefore, the path length of the actual data represented by the summary is estimated.
At the summary-level, any shortest path in the underlying RDF graph passes through a plurality of partitions. For each partition the path includes two of its portals, i.e., an entrance and an exit node. By construction, the distance from the root node v of a partition to any vertex u in the same partition is known and has been indexed. By triangle inequality, the distance for any two vertices v1 and v2 in a partition with a root node v can be upper bounded by
, and lower bounded by
. A possibly tighter lower bound can be found by using the summary and the recognition that given two graphs
and h, if
, then
,
and their homomorphic mappings
,
.
Referring to , and its summary s in S(G) are illustrated 902. There is no direct homomorphic relation between h and s. In order to obtain a lower bound for the distance of any two vertices in h, a homomorphic structure is constructed for h. First, a mapping function ω is defined that outputs a new graph
from a given graph
. The mapping function ω takes as input g and a set of vertices of the same type
, denoted as
. Next, via is added to V′ to represent all vertices in
. Then,
, v is added to V′. Finally,
, add
if
, or add
or to E′ if
respectively. In addition, labels are copied accordingly. Therefore, a homomorphism exists from g to
.
A homomorphism for h is derived by using ω and its summary s. Initially, h is constructed from by recursively applying ω on
, with each
as input, denoted as
. The set
can be retrieved by the stored indexes. For example, in
. Applying
results in a structure that is isomorphic to the partition
. Again in
can be rewritten by the associativity of ω as nested ω s, i.e.,
. Applying the inner mapping generates an isomorphic structure to partition
, while applying the outer mapping returns a homomorphism of
. Using these properties and the transitivity of homomorphism, a homomorphism for any partition h can be constructed. However, there is a practical issue as ht is not explicitly stored. Only a summary s is kept to represent the structure of ht, where ht is homomorphic to s. Thus, a homomorphism for h is constructed through s.
Letting h1 and h2 represent two partitions, and supposing and
, where
, there exists
. This implies that by merging
on ht, i.e.
, and merging
on its summary s, i.e.,
, there is a homomorphism from
to
. It follows that
is homomorphic to
. As discussed above, h is homomorphic to
; therefore, h is homomorphic to
. Here,
is a part of the third auxiliary inverted index, which maps a vertex in data to a vertex in summary. Referring again to
. In one embodiment, the larger lower bound from either the summary or the triangle inequality is used.
The set is defined, where
is the set of vertices in G that contains query keyword
and m priority queues
are initialized. A set M of entries is maintained, one for each considered partition. Each entry stores a partition id followed by m lists. The ith list records all the reachable vertices found so far that contain keyword wi and how they connect to the partition
, in the form of quadruples—
. M tracks what keywords have reached its associated partition in their backward partition-based expansion. In the quadruple, the vertex stands for the first vertex in the backward expansion; the expansion reaches the current partition by routing through a sequence of the portals from some partitions, stored in S as a sequence of (portal,partition) pairs.
A sequence S defines a path that begins at vertex. For instance, a sub-sequence means the indicated path enters the partitions
at the portal l (exiting from partition
) and uses l′ as its next portal for exit. The interest is for the shortest path that connects a portal to its next portal in a sequence. The lower and upper bounds for the length of this path defined by S are denoted as
and
. For example, where m=2 and an entry for a partition
in M, an entry for keyword
) which indicates that there is a path (of partitions) from w1 that reaches
. This path starts at v1, enters
at portal l2 and has a length of at least 5 hops and at most 7 hops. To reach
, it passes through
. For a second keyword w2,
is reachable from v2 and
as indicated by
and
.
Like the BACKWARD methods, the algorithm proceeds in iterations. In the first iteration, for each vertex v from , the
of the partition is retrieved that contains v, from the third inverted index. Next, if there is an entry for
in M, a quadruple
is appended to the ith list of the entry. Otherwise a new entry for
in M is initialized with m empty lists, and the ith list is updated with t. In addition, an entry (
) is added to the priority queue αi. Entries in the priority queue are sorted in ascending order by their lower bound distances. This process is repeated for all
's for
, which completes the first iteration.
In the iteration, the top entry is popped from ai, for example
. The last pair in S is said to be
, and for the partition
, its portals
are found from the first inverted index. Then, for each
in
, the lower and upper bounds
and
in the partition are computed from
(or v if
) to
using the approach discussed above. From the inverted index, a portal
connects
to a set of neighboring partitions, for example a set
of partitions. For each partition
, a quadruple
is constructed.
In addition, the entry for in M is searched, and its ith list is updated in the same way as for the first iteration. But t′ is only appended to the ith list if the following two conditions are satisfied. First, for every
in the ith list, if l is the last portal in S″, then
. Second, l′ is not in S, i.e., no cycle. Finally, if t′ has indeed been appended,
is inserted to
, which completes the
iteration.
At any iteration, if a new quadruple t has been appended to the ith list of an entry in M, and all of its m lists become non-empty, then partition
contains potential roots for a candidate answer.
First, all the possible combinations of the quadruples from the other lists are found, one from each list, and combined with t, denoted as
. This is referred to as a candidate subgraph, where
is the
quadruple in the lth list. For example, letting t1 be the new quadruple just inserted for the 1st list in the entry for pid in M, since both of its lists are now non-empty, two candidate subgraphs can be found, i.e.,
and
. Using the information in a quadruple t, two subgraphs are found (linked by the partitions and their portals) that have candidate answers rooted at a node in partition
.
To track the top-k answers, a priority queue is maintained for the top-k answers found so far. Once the partition, that contains the possible root of the candidate answers, is retrieved from the data, BACKWARD searching is used to find the answer roots at that partition. Instead of taking all the vertices in as input for the backward search, only vertices in
that are part of the tailing portals are considered at each of the m sequences. Furthermore, the shortest path algorithm is used to find the distance from a portal to its succeeding portal in the respective partitions. For instance, for the candidate subgraph
,
is found on the partition
using the shortest path algorithm (similarly,
on
) and backward search only is used on the partition pid with
(and k) as the input.
Finally, in any iteration, whenever the same answer root with a different score is found, only the one with the smaller score is kept in the priority queue. To complete the algorithm, the correct termination condition can be found by letting be an entry in the priority queue. Then
partition
and for any path from v that is defined by S, it has
. In addition, let
be the top entry in the priority queue
. Then for any explored path p from
in
, it has a
. The set of all unexplored partition ID's in G is denoted as
. For any
that has not been included in M, clearly,
. The best possible candidate answer rooted at a node in partition
is to use the current top entries from the m expansion queues, i.e.,
. Let these m top entries be
respectively. This yields the following results.
Let be the possible unexplored candidate answer rooted at a vertex in partition
, with
,
Next, consider the set of partitions whose ID's have been included in M. Let the first entry from each of the m lists for a in M be:
Each list is sorted by the lower bound distance, and
if the jth list of
is empty or the path of its first entry is nil. Based on this, let the best possible unexplored candidate answer rooted at a vertex in partitions
, where
, be
, then
where if
otherwise
.
Finally, the termination condition is derived for the search. The score of the best possible answer in an unexplored partition is denoted as , as defined by the RHS of equation (3). In addition, the score of the best possible answer in all explored partitions as
, as defined by the RHS of equation (4). Let g be the candidate answer in the priority queue with the kth smallest score. The search can safely terminate when
. This algorithm is denoted the SUMM method. The SUMM method finds the top-k answers
for any top-k: keyword search query q on a RDF graph.
The SUMM algorithm uses the summaries to reduce the amount of data accessed during the backward search. For the algorithm to be effective, the subgraphs of the data graph that correspond to the different partitions should be efficiently identified. One option is to store the triples by partitions and index on their partition ID's, e.g., using the namegraph feature in any standard engine where each namegraph defines a partition. But then whenever an update on the partition happens, the index is updated. Furthermore, the approach enforces a storage organization that is particular to the present methods (i.e., not general). In one embodiment, an alternative efficient approach is used that has no update overhead and requires no special storage organization. Approaches in accordance with the present invention work by dynamically identifying the data of a partition using appropriately constructed SPARQL queries that retrieve only the data for that partition.
Since graph homomorphism is a special case of homomorphism on relational structure, the Homomorphism Theorem is used to characterize the results of two homomorphic graph query patterns. According to the Homomorphism Theorem, q and q′ are relational queries over the same data D. Then there exists a homomorphism mapping
. Recall that
and for each ht, a core c is extracted from ht. By definition, c is homomorphic to
. Therefore, c is homomorphic to h (transitivity). Using c as a SPARQL query pattern can extract h due to the Homomorphism Theorem.
Addressing, two practical issues, there is usually a many-to-one mapping from a set of ht's to the same core c—leading to a low selectivity by using c as the query pattern. To address this issue, constants are bound from the targeted partition to the respective variables in query pattern. These constants could include the root and the portals of the targeted partition which are retrievable from the inverted indexes. The second issue is that in the construction of S(G), every c is not explicitly kept. Instead, c could be embedded (by finding homomorphism) to a summary S(G), where c is a subtree of s. To construct a SPARQL query from s, a mapping is found for the root node of the targeted partition in s. The triple patterns corresponding to the subtree in s are expressed in (nested) OPTIONALs from the root to the leaves. For example, the SPARQL query for the partition rooted at URI5 using the summary in
One important limitation of previous work on summarizing RDF data is the inability to handle updates in the data in an incremental way. Summaries in accordance with exemplary embodiments of the present invention can be incrementally updated. This includes providing for insertion and deletion in the RDF graph dataset. Insertions are handled efficiently. A new subgraph, i.e., a set of triples, is simply treated as a data partition that has not been traversed. Indexing structures and the summarization are updated accordingly. Regarding deletions, let t be the triple deleted. Then all the partitions that visit the subject/object of t are updated. As a deletion only affects the nodes in the α-neighborhood of t's subject and object, this can be done efficiently. To update S(G), there are two cases to consider. In the first case, if the core of an updated partition is not in S(G), i.e., it is homomorphic to a core in S(G). Its core is rebuilt, and the correspondent inverted indexes are updated. In the second case, if the core of an updated partition is in S(G), this will lead to a removal for the core in S(G). In addition, all the partitions homomorphic to the deleted core are retrieved and summarized together with the updated partition as if they are new data. To access these partitions efficiently, techniques discussed herein are utilized with the deleted core as the query pattern.
In accordance with one exemplary embodiment, the present invention is directed to systems for summarizing resource description framework datasets. These systems include one or more computers or computing systems that are in communication across one or more local or wide area networks. These include computing systems the have access to locations and other computing resources across the world wide web or Internet. The system also includes one or more databases that are in communication with the computer and that store the datasets and summaries of the present invention. The computer is capable of generating the datasets and summaries of the present invention and of processing user defined queries such as keyword searches over the datasets and summaries. Suitable computer, computing systems and databases are known and available in the art.
The database includes an identification of one or more resource description framework dataset graphs. Each resource description framework dataset graph includes a plurality of entity vertices associated with data accessible across the network. These entity vertices include, for example, an identification of the location of information or data accessible across the network, for example, URIs. In addition, the resource description framework dataset graph includes a plurality of type vertices associated with the entity vertices and indicating the type of a given entity vertex and a plurality of keyword vertices associated with the entity vertices. A plurality of predicate edges are provided in the resource description framework dataset graph to connect pairs of vertices selected from the entity vertices, type vertices and keyword vertices.
The database includes a plurality of partitions. Each partition represents a portion of the vertices and predicate edges from the resource description framework dataset graph. The plurality of partitions are preferably a plurality of predicate edge disjoint partitions, and the union of all predicate edge disjoint partitions represents the entire resource description framework dataset graph. In one embodiment, a condensed view of the resource description framework dataset graph is used. Therefore, the database includes this condensed view of the resource description framework dataset graph. The condensed view includes a plurality of condensed vertices linked only by inter entity vertex predicate edges from the resource description framework dataset. These are predicate edges between pairs of entity vertices in the original dataset graph. Each condensed vertex is associated with an entity vertex in the resource description framework dataset graph; however, the vertices in the condensed view only contain type information from a given type vertex associated with that entity vertex. Therefore, the condensed view of the dataset graph is a type based view, and the types are linked by the structure of the original dataset graph as defined by the relationships among the entity vertices.
In one embodiment, the condensed view is used to generate the partitions, and each partition in the plurality of partitions includes a portion of the condensed vertices and the inter entity vertex predicate edges from the condensed view of the resource description framework data graph. Each partition can be formed from one or more predicate edge disjoint subgraphs, where each subgraph is formed starting at a given condensed vertex and adding condensed vertices extending out a predetermined number of hops through the condensed view of the resource description framework from the given condensed vertex. In one embodiment, the given condensed vertices from which the predicate edge disjoint subgraphs in a given partition are initiated comprise common type information. Therefore, each partition represents subgraphs around a common type of vertex from the condensed view.
A minimum set of common type based structures summarizing the plurality of partitions is stored in the database. This summary is used for purposes of keyword searching. The minimum set of common type based structures summarizes the plurality of partitions. In addition to using the condensed view of the dataset graph and the partitions, the size of the summary can be further reduced using cores and homomorphisms. Therefore, the summary stored in the database includes a plurality of covering trees representing the plurality of partitions. Each covering tree represents all distinct paths through the vertices of the partitions. Since a given covering tree may contain, for example, duplicate nodes or branches, the covering tree has a core that contains a minimum number of vertices for the covering tree. Each core in the plurality of covering trees cores in the minimum set of common type based structures represents a superset of other covering tree cores having common type based information that are not include in the minimum set of common type based structures. This is accomplished by adding covering tree cores to the summary using a process that employs homomorphisms.
The database also includes a plurality of auxiliary indexes in combination with the minimum set of common type based structures. The plurality of auxiliary indexes takes into account the condensing of the dataset graph, the creation of partitions and the generation of the summary using cores and homomorphisms such that it is sufficient to recreate the resource description framework dataset graph from the minimum set of common type based structures and the plurality of partitions. Therefore, in generating the ultimate summary used for keyword searching, the underlying original dataset graph information is not lost through summarization. In one embodiment, the plurality of auxiliary indexes includes a first index comprising an identification of portals in each partition, a second index mapping each partition to a covering tree associated with that partition and a third index mapping data nodes in each partition to summary nodes in the minimum set of common type based structures.
Exemplary embodiments in accordance with the present invention are also directed to methods for summarizing resource description framework datasets. These summaries are then used to respond to user defined keyword searches over the resource description framework datasets. In this method, the resource description framework dataset graph containing a plurality of entity vertices, type vertices and keyword vertices connected by a plurality of predicate edges is split into a plurality of partitions. Each partition contains a plurality of vertices and predicate edges connecting the vertices. In one embodiment, the resource description framework dataset graph is split into a plurality of predicate edge disjoint partitions. A union of all of these predicate edge disjoint partitions contains all vertices and predicate edges in the resource description framework dataset graph.
To reduce the number and size of the partitions, the partitions are generated from a condensed view of the dataset graph. Therefore, a condensed view of the resource description framework dataset graph is created by combining entity, keyword and type vertices into a plurality of condensed vertices linked only by inter entity vertex predicate edges from the resource description framework dataset. Entity information and keyword information are removed from each condensed vertex, and only type information is maintained in each condensed vertex, yielding a type based condensed graph and type based partitions. The condensed view of the resource description framework data graph is split into the plurality of partitions.
In one embodiment, splitting of the condensed view into the partitions includes creating a plurality of predicate edge disjoint subgraphs from the condensed view. Each subgraph begins at a given condensed vertex and extends out a predetermined number of hops through the condensed view of the resource description framework. Each partition represents a grouping of all subgraphs beginning at condensed vertices comprising common type information. Therefore, partitions are type specific. The creation of type specific partitions is further aided by grouping the plurality of condensed vertices by common type information. The condensed vertices on which to begin predicate edge disjoint graphs are selected by group, exhausting all condensed vertices in a given group before advancing to a subsequent group.
The method also includes defining a minimum set of common type based structures summarizing the plurality of partitions. This results in the summary that is used for keyword searching. In order to create this summary, a plurality of covering trees is created to represent the plurality of partitions by traversing each partition to create an associated covering tree comprising all distinct paths through the vertices of that partition. A core is extracted for each covering tree. This core represents a minimum number of vertices for the covering tree and is used to represent the structure of that covering tree. Homomorphisms among the plurality of covering trees are used to create the minimum set of common type based structures. This use of homomorphisms among the plurality of covering trees includes sequentially comparing each extracted core to existing structures in the minimum set of common type based structures, removing existing structures from the minimum set of common type based structures that represent a subset of a given extracted core being compared, terminating comparison of a given extracted core upon determination that the given extracted core represents a subset of existing structures in the minimum set of common type based structures and adding a given extracted core to the minimum set of common type based structures upon completing a comparison of that given extracted core to all existing structures in the minimum set of common type based structures and determining that the given extract core is not a subset of any existing structure.
A plurality of auxiliary indexes are maintained in the database in combination with the minimum set of common type based structures. These auxiliary indexes are used to recreate the resource description framework dataset graph from the minimum set of common type based structures and the plurality of partitions, for example, in response to a keyword search in order to obtain the underlying data from the original dataset graph. The plurality of auxiliary indexes includes a first index containing an identification of portals in each partition, a second index mapping each partition to a covering tree associated with that partition and a third index mapping data nodes in each partition to summary nodes in the minimum set of common type based structures.
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.
Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).
Aspects of the present invention are described above with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Methods and systems in accordance with exemplary embodiments of the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment containing both hardware and software elements. In a preferred embodiment, the invention is implemented in software, which includes but is not limited to firmware, resident software and microcode. In addition, exemplary methods and systems can take the form of a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer, logical processing unit or any instruction execution system. For the purposes of this description, a computer-usable or computer-readable medium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. Suitable computer-usable or computer readable mediums include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems (or apparatuses or devices) or propagation mediums. Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk. Current examples of optical disks include compact disk—read only memory (CD-ROM), compact disk—read/write (CD-R/W) and DVD.
Suitable data processing systems for storing and/or executing program code include, but are not limited to, at least one processor coupled directly or indirectly to memory elements through a system bus. The memory elements include local memory employed during actual execution of the program code, bulk storage, and cache memories, which provide temporary storage of at least some program code in order to reduce the number of times code must be retrieved from bulk storage during execution. Input/output or I/O devices, including but not limited to keyboards, displays and pointing devices, can be coupled to the system either directly or through intervening I/O controllers. Exemplary embodiments of the methods and systems in accordance with the present invention also include network adapters coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Suitable currently available types of network adapters include, but are not limited to, modems, cable modems, DSL modems, Ethernet cards and combinations thereof.
In one embodiment, the present invention is directed to a machine-readable or computer-readable medium containing a machine-executable or computer-executable code that when read by a machine or computer causes the machine or computer to perform a method for summarizing resource description framework datasets in accordance with exemplary embodiments of the present invention and to the computer-executable code itself. The machine-readable or computer-readable code can be any type of code or language capable of being read and executed by the machine or computer and can be expressed in any suitable language or syntax known and available in the art including machine languages, assembler languages, higher level languages, object oriented languages and scripting languages. The computer-executable code can be stored on any suitable storage medium or database, including databases disposed within, in communication with and accessible by computer networks utilized by systems in accordance with the present invention and can be executed on any suitable hardware platform as are known and available in the art including the control systems used to control the presentations of the present invention.
While it is apparent that the illustrative embodiments of the invention disclosed herein fulfill the objectives of the present invention, it is appreciated that numerous modifications and other embodiments may be devised by those skilled in the art. Additionally, feature(s) and/or element(s) from any embodiment may be used singly or in combination with other embodiment(s) and steps or elements from methods in accordance with the present invention can be executed or performed in any suitable order. Therefore, it will be understood that the appended claims are intended to cover all such modifications and embodiments, which would come within the spirit and scope of the present invention.
