A functional graph traversal language is a functional language for traversing a graph database and comprises one or more functions or steps, each of which may input a object flow and output a new object flow. Graph database is a graph-oriented or graph-based database for storing data and querying data in the data structure of the graph. The basic storage unit of the graph database includes vertex, edge, property and so on. The process of graph traversal is to start from a vertex in the graph and access other nodes based on the property of the vertex or edge, thereby accomplishing query for the graph database.
Structured query language (SQL) is a declarative language designed for managing data in relational databases. It may use SQL query statements to retrieve, insert, update, and delete data from the database. The most common query in SQL is the SELECT statement which generally consists of SELECT-FROM-WHERE as well as some peripheral clauses, such as GROUP-BY, ORDER-BY, and HAVING. As a declarative language, SQL generally only needs to specify the tasks to be performed without specifying the implementations details of the tasks. Therefore, SQL statements usually do not have complex statement structure and syntax requirements.
In embodiments of the present disclosure, there is provided a scheme for translating a functional graph traversal language to extended Structured Query Language. After obtaining a first query compiled with THE functional graph traversal language, the first query is translated to a second query in the form of extended Structured Query Language based on translating rules so as to optimize the first query. By extending the standard
Structured Query Language, embodiments of the present disclosure enable retaining directed edges between vertices in the graph database in the second query and retaining flow control, iteration, temporary variable definition, advanced data structure (array, dictionary, tree structure), side effect and dependency between steps included in the first query. Therefore, according to embodiments of the present disclosure, by translating the functional graph traversal language, it is possible to identify queries with the same semantics in the functional graph traversal language so as to optimize queries in the functional graph traversal language.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
With reference to the drawings and the detailed description below, the above and other features, aspects, and advantages of the embodiments of the present disclosure will become more apparent. In the drawings, the same or similar reference signs usually represent the same or similar components, in which:
The embodiments of the present disclosure will be described in details with reference to the drawings. Although the drawings demonstrate some embodiments of the present disclosure, it is to be understood that the present disclosure may be implemented in various manners other than the ones described herein. To the contrary, these implementations are provided for a more thorough and comprehensive understanding of the present disclosure. It is to be understood that the drawings and embodiments of the present disclosure are only for the purpose of illustration, rather than suggesting any limitations on the scope of the subject matter.
As used herein, the term “comprises/include” and its variants are to be read as open terms that mean “comprise/include, but is not limited to”. The term “based on” is to be read as “based at least in part on”. The term “one embodiment” and “an embodiment” are to be read as “at least one embodiment”. The term “another embodiment” is to be read as “at least one other embodiment”; the term “some embodiments” is to be read as “at least some embodiments.” Definitions related to other terms will be provided in the following description.
Traditionally, queries in the functional graph traversal language are generally consisted of one or more steps and the semantics of the queries is defined by the order of implementing the steps. However, implementing the steps in order limits the space of query optimization. Thus, it is generally impossible to identify queries with the same semantics in the functional graph traversal language. To identify queries with the same semantics in the functional graph traversal language, one improvement is to translate the functional graph traversal language to another standard declarative language, such as standard SQL. However, in the standard SQL, the directed edges between vertices in the graph database cannot be reflected, and moreover, standard SQL cannot support translation of some steps (such as the side effect step) in the functional graph traversal language. Therefore, the above improvement will ignore the translation of some query steps during the language translation process, and thus it is error-prone and fails to truly identify queries with the same semantics.
For this purpose, embodiments of the present disclosure provide a scheme for translating a functional graph traversal language into an extended Structured Query Language. In embodiments of the present disclosure, after a first query compiled with the functional graph traversal language is obtained, the first query is translated to a second query in the form of extended Structured Query Language based on a translating rule so as to optimize the first query. Therefore, in accordance with embodiments of the present disclosure, by translating the functional graph traversal language, it is possible to identify queries with the same semantics in the functional graph traversal language so as to optimize queries in the functional graph traversal language.
In addition, in accordance with embodiments of the present disclosure, after quires with the same semantics in the functional graph traversal language are identified, it is possible to select a query with the highest implementation efficiency for implementation, thereby improving the querying efficiency. Moreover, embodiments of the present disclosure support user-defined table-valued functions and enable reservation of flow control, iteration, temporary variable definition, advanced data structure (array, dictionary, tree structure), side effect and dependency between steps in the functional graph traversal language, so as to achieve fully-fidelity semantic translation for the functional graph traversal language. Basic principles and several embodiments of the present disclosure will be described below with reference to
As shown in
The computing system/server 100 usually includes a plurality of computer storage media, which can be any available media accessible by the computing system/server 100, including but not limited to volatile and non-volatile media, and removable and non-removable media. The memory 120 can be a volatile memory (for example, a register, cache, Random Access Memory (RAM)), non-volatile memory (for example, a Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), flash memory), or any combination thereof. The storage device 130, which may be removable or non-removable media, includes machine-readable media, such as flash memory drive, disk or any other media and may be used to store information and/or data, and can be accessed in the computing system/server 100.
The computing system/server 100 may further include additional removable/non-removable and volatile/non-volatile storage media. Although not illustrated in
The communication unit 140 communicates with other computing devices through communication media. Additionally, the functions of the components in the computing system/server 100 may be realized by a single computing cluster or a plurality of computing machines, and computing machines communicate through communication connections.
Therefore, the computing system/server 100 may be operated in the networked environment through a logic link with one or more other servers, a networked personal computer (PC) or a further general network node.
The input device 150 may be one or more various input devices, such as a mouse, a keyboard and a trackball. The output device 160 may be one or more various output devices, such as a display, a loudspeaker and a printer. The computing device/server 100 may further communicate through the communication unit 140 as required with one or more external devices (not shown), such as a storage device, a display device and the like, one or more devices that enable the interaction between a user and the computing system/server 100 or any devices (such as a network card, a modem and the like) that enable the communication between the computing system/server 100 and one or more other computing devices. Such communication can be executed through an input/output (I/O) interface (not shown).
As shown in
Those skilled in the art shall appreciate that although
At 202, a first query (also referred to as “a source language query”) in the functional graph traversal language is obtained, and the first query relates to a directed edge depicting neighboring relation between vertices in a graph. The basic storage unit in the graph database includes vertices, edges and properties, and the vertices and the edges may have their respective properties. The traversal process of the graph generally starts from one vertex to another vertex through a directed edge. For example, referring to first query 180 in the example depicted in
At 204, the first query is translated to a second query (also referred to as “a target language query”) in extended structured query language (SQL) so as to optimize the first query, and the second query includes clauses describing the directed edge, such as MATCH clause. For example, the translating module 125 translates the source language query 180 in the input to the target language query 190 in the output through the translating rule. As shown in
It is to be understood that as SQL is a declarative language for a relational database, FROM clause generally defines k reference tables, and WHERE clause specifies a Boolean expression. The result of the SELECT statement is the Cartesian product of tuples from k tables, while satisfying the Boolean expression in the WHERE clause. This Cartesian product is commutative and associative, and as a result, it is possible to rearrange the k tables when performing the Cartesian product using queries with the same semantics. This capability of rearranging is the important source of query optimization. In other words, standard SQL does not specify the specific steps of executing query which are instead constructed by the optimizer of an SQL engine. Thus, the optimization of the query may be achieved.
As the functional graph traversal language needs to formulate the specific steps specified by the query, it is impossible to identify queries with the same semantics therein.
Therefore, translation of the functional graph traversal language can identify queries with the same semantics, thereby achieving optimization of the query. However, the standard SQL cannot cover all the steps in the functional graph traversal language, for example, the standard SQL neither includes corresponding clauses of directed edge(s) between vertices nor corresponding clauses of dependency relation between steps. In view of this, embodiments of the present disclosure extend standard SQL and can retain the directed edge between the vertices in the graph database in the second query so as to achieve full-fidelity semantic translation for the functional graph traversal language. Therefore, embodiments of the present disclosure can identify queries with the same semantics existing in the functional graph traversal language by completely translating the source language query to the target language query, thereby optimizing the first query.
It is to be understood by those skilled in the art that the functional graph traversal language is a functional language for traversing a graph database, and one typical example of functional graph traversal language is Gremlin language. As a functional and data-flow language, Gremlin consists of a set of combinable steps so that the user can express traversal or query of a complex property graph in a concise manner. For example, some graph databases (such as distributed graph database as a service) support Gremlin as query language and translate Gremlin to internal language (IL), compile internal language into computation operators, execute the operators and return the results. Each step in Gremlin injects a stream of objects as input and projects a new stream of objects as output. A Gremlin query is a sequence of steps, and the semantics of the query is defined by executing the steps one after another in the strict order as they are composed in the sequence. Objects produced by Gremlin have various types, including vertex, edge, scalar (such as integer, string and so on), and composite types (such as array and map).
In some embodiments, Gremlin objects may be implemented as Java objects and Gremlin steps are implemented as virtual functions of Java objects. In this implementation, a Gremlin query is directly mapped to a pipeline that carries a stream of Java objects. Every Gremlin step is a junction in the pipeline that pulls objects from the upstream, and for each of incoming objects, it invokes the virtual function of the corresponding step, produces zero or more objects and pushes them to the downstream. The objects produced by the last junction of the pipeline are the results of the Gremlin query.
It is to be understood that although embodiments of the present disclosure depict an example of translating the functional graph traversal language with reference to Gremlin as below, other functional graph traversal languages are also possible. The scope of embodiments of the present disclosure is not limited to Gremlin.
Continuing to refer to
As shown in
In some embodiments, it is possible to determine the number of columns output by the second query based on the SELECT statement in the second query. Though node conceptually has infinite columns, during query execution, only columns referenced in WHERE and SELECT clauses have practical meaning and should be populated in records. For example, for the query 180, even when the vertex shown in
In embodiments of the present disclosure, a first extension of the standard SQL is MATCH clause which consists of one or more path expressions, each of which specifies a list of directed edges connecting two vertex variables defined in the FROM clause. A common pattern of an edge is in the following format:
[sourceAlias]−[Edge As edgeAlias]−>[sinkAlias],
Where sourceAlias and sinkAlias are optional but at least one of them must exist. From the SQL perspective, the edge variable also defines a table reference. Similar to node, this table has an infinite number of columns due to flexible schemas of edges and includes a special column * (star) for the JSON representation of edges. In addition, the edge table has several predefined columns: sourceId, sourceLabel, sinkId and sinkLabel.
In the extended SQL according to embodiments of the present disclosure, FROM and MATCH clauses altogether define vertex and edge variables and their topological relationships. The WHERE clause further makes additional constraints on the variables through Boolean expressions and the execution order of variables is determined by the query optimizer. In essence, the topological relationships describe a special type of join conditions. By adding and extending MATCH clause in standard SQL, it is possible to allow the optimizer to exploit graph semantics so as to find Gremlin queries having the same semantics. As shown in
Continuing to refer to
Since data flow is directional and Gremlin itself does not specify equivalent compositions, it is impossible to identify equivalent execution plans of the same data flow and find an optimal one to execute at runtime. Therefore, the source language queries 180 and 185 in Gremlin are translated to the same target language query 190 of the extended SQL 420 by means of the translating module 125 according to embodiments of the present disclosure, so as to exploit SQL syntax/semantics to explicitly represent semantic equivalence in graphs, thereby optimizing source language query. In this manner, extended SQL forms the basis for query optimization and will be the input of the compiler that optimizes the query and produces execution operators to generate results. For example, as queries 180 and 185 have the same semantics, as long as one of them is executed, the output result of the two queries may be obtained, thereby improving the execution efficiency of Gremlin query.
Besides, query 185 is more efficient than query 180, because there are only 100 Senators in US and yet millions of CA residents. Starting from a much smaller vertex set reduces execution complexity and leads to higher query speed. Therefore, the execution efficiency of Gremlin query can be improved by using Gremlin to execute query 185 instead of query 180.
At 502, a third query in the functional graph traversal language is obtained. For example, a third query 185 compiled with Gremlin is obtained. At 504, the third query is translated to a fourth query in the form of extended SQL. For example, the translating module 125 translates the third query 185. At 506, it is determined whether the second query and the fourth query have the same execution plan. As extended SQL may find queries with the same semantics, it is possible to compare translated target language queries to identify the same source language queries.
If it is determined that the second query and the fourth query have the same execution plan, then at 508, it is determined that the first query and the third query have the same semantics. For example, in the example shown in
At 510, it is determined whether the execution efficiency of the third query is higher than that of the first query, and the execution efficiency may represent time costs and/or computation costs to execute the query. If it is determined that the execution efficiency of the third query is higher than that of the first query, at 512, the third query is used to replace the first query. For instance, in the example shown in
Besides, if it is determined at 506 that the second query and the fourth query have different execution plans, then it is determined at 516 that the first query and the third query have different semantics, which indicates that the first query and the third query cannot be optimized therebetween. However, the first query or third query may be compared with other queries to determine whether to perform optimization.
Therefore, according to embodiments of the present disclosure, it is possible to identify queries with the same semantics in the functional graph traversal language by translating the functional graph traversal language so as to optimize queries in the functional graph traversal language. Moreover, after the queries with the same semantics in the functional graph traversal language are identified, it is possible to select a query with the highest execution efficiency for execution, thereby improving the execution efficiency.
It is to be understood that as Gremlin is a data-flow command language, basic statements therein may involve operation statement, cyclic statement, condition branch statement, and unconditional branch statement. These statements may involve flow control, iteration, temporary variable definition, advanced data structure (array, dictionary, tree structure), side effect and dependency between steps.
Taking dependency relation as an example, dependency relation determines that some steps must be completed before other steps so as to prevent “rearrangement” during query optimization. For example, optional( ) and coalesce( ) steps in Gremlin represent if—else logic, and all the steps in the if-else system can only be implemented after steps prior to optional( ) and coalesce( ) are completed. Moreover, side effect steps aggregate( ) and store( ) in Gremlin will also generate dependency relation. The two steps both transfer data in a temporary variable so as to be retrieved through other steps (such as cap( ). To ensure data stored in the variable are correct, all the steps prior to the side effect step must be completed firstly, and all the steps after the side effect step must be performed later.
In the traditional standard SQL, it is generally impossible to express content involved in basic statements of the command language. To overcome this defect, embodiments of the present disclosure provide a self-defined table-value function (TVF) to preserve flow control, iteration, temporary variable definition, advanced data structure (array, dictionary, tree structure), side effect and dependency between steps in Gremlin accurately. TVF semantics in embodiments of the present disclosure is mostly unchanged and compatible with other SQL constructs.
TVF is a user-defined function that takes input as values from an incoming row and outputs zero or more rows each with a predefined number of columns. Taking a table as an input, the TVF is invoked repeatedly for each row of the input table and the results of all invocations are concatenated vertically. The concatenated rows form a new table which may be assigned a new alias and referenced in other parts of extended SQL, for example, a Boolean condition in the WHERE clause references a column of the table produced by the TVF, or some columns of this table are the input of another TVF.
In some embodiments, for steps in Gremlin, the corresponding TVF may be provided. TVF according to embodiments of the present disclosure can express complex Gremlin steps and has three important properties. First, TVF may change cardinality and may be downsized and upsized. This is an unavoidable requirement, given that many Gremlin steps can go either direction. By comparison, Boolean expressions only downsize the results, and scalar expressions do not change cardinality. Second, TVF is user-defined and what rows an invocation produces is completely defined by the user. As long as enough information is passed in as input parameters, the computation logic of the TVF can be made exactly the same as its Gremlin counterpart. By comparison, table joins and their variants (natural joins, outer joins) have very rigid semantics and it is generally impossible to twist them to fit Gremlin steps. Third, the input of a TVF may come from other vertices/edge variables and/or TVFs, thereby creating an implicit dependency graph such that a TVF can only be evaluated after the parent variables/TVFs have been executed. In this way, steps that cannot be rearranged or optimized can be identified in extended SQL and meanwhile, steps that can be rearranged or optimized are identified.
When the TVF corresponding to the Gremlin step is defined, it is semantically necessary to define the input of the function and the columns generated by TVF. For example, the function's input is a list of scalar values from prior table references that are necessary to specify the computation logic of the function. The output columns should contain all columns requested by the Gremlin steps following the TVF's corresponding steps. The columns are either fixed or dynamic. The dynamic set of columns is specified by the TVF's parameters too, either explicitly through a list of column names or implicitly through scalar sub-queries. Therefore, different from the conventional standard SQL, for the extended SQL according to embodiments of the present disclosure, a TVF's output columns are always fixed and predefined when the function is declared and registered in the database
That is, during the process of translating Gremlin query to extended SQL query, it is possible to identify the dependency between a plurality of steps included in Gremlin query and retain the dependency in the extended SQL query using a table-valued function. Optionally, the side effect step in Gremlin query may also be retained in extended SQL query, although the output of the executed Gremlin query itself is generally not influenced by whether the side effect step is executed.
Moreover, filter steps in Gremlin are not always mapped to Boolean predicates in the WHERE clause. In some cases, they should be mapped to Boolean predicates to execute TVF of filter logic. A Boolean predicate in WHERE clause would lose dependency to other table reference. When one filter step follows a Gremlin side effect step, it is necessary to perform further processing according to embodiments of the present disclosure.
As shown in
If the dependency of the side effect step Store( ) is neglected when Gremlin query 615 is translated to standard SQL, then as shown in
Continuing to refer to
In some embodiments, based on whether Gremlin query has sub-traversals, TVFs may be divided into two types: Gremlin queries having no sub-traversal and Gremlin queries having sub-traversals. For Gremlin steps having no sub-traversals, a TVF in this category takes input as a scalar value list, and the output is either a fixed number of columns (without any specification at all), or a dynamic set of columns whose names are provided in the scalar value list. In other words, the input value list consists of two parts, the first of which defines the computation logic and the second defines a list of column names to output.
For a Gremlin query with one or more sub-traversals, a TVF in this category takes input as one or more SELECT sub-queries, which reference vertex/edge variables and TVFs in its parent context, when the sub-traversal references data from prior Gremlin steps. The TVF's output columns are specified by the projected columns in the SELECT clause of the sub-queries. When there are a plurality of sub-traversals, the SELECT elements in all SELECT sub-queries must be aligned, that is, the number of columns output by a plurality of sub-queries respectively is the same, so that the output table presents a unified view of all output records, which may be produced by any sub-traversals.
In accordance with embodiments of the present disclosure, Gremlin query 715 may be translated into query 725 in the extended SQL 420. In the translated SQL query 725, each sub-traversal of coalesce( ) is translated to a SELECT sub-query. The two sub-queries are two input parameters of the TVF Coalesce( ). The computation logic of this TVF is that each record from N_18 is evaluated against the two SELECT sub-queries and the result returned by the first sub-query is taken as the result of the step. For each invocation, the input record from N_18 is digested through VertexToForwardEdge( ) (which is another TVF in the SELECT sub-query). This TVF takes input as a vertex in the JSON format, that is, N_18.*, and outputs a two-field record, one for the edge label and the other for the JSON representation of the edge. Both fields are populated because they are needed: the “label” field is referenced in the WHERE clause for an equality check; the “*” field is projected in the SELECT clause, given that both sub-traversals return edges. The two SELECT clauses of the two sub-queries also define that the TVF Coalesce( ) produces a one-column table in this query and the column name is “*”. As coalesce( ) in the original Gremlin query is the last step, this column is eventually projected in the top-level SELECT clause, and it outputs edges.
In accordance with embodiments of the present disclosure, Gremlin 815 may be translated into a query 825 in extended SQL 420. Though the optional( ) step has only one sub-traversal, its TVF's input contains an additional SELECT sub-query, which has only the SELECT clause. The selected element (namely, N 18.name) comes from the table reference of the prior Gremlin step. Therefore, it only projects values of input records. For each input record, the TVF's results come from the second sub-query, if its result is non-empty, or the result is the projection of the input record. No matter what branch an input record goes through, the sub-queries project the same set of columns, so the TVF presents a unified view: values( ). The projected column (“name”) is what the following step asks for.
Besides, count( ) sum( ) and sample( ) steps in Gremlin queries always output a fixed number of columns, where count( ) and sum( ) output a numeric column, whereas sample( ) outputs no columns. The numeric column's name is predefined, because it is fixed for any query. In general, most TVFs have a default projection column, and a column is returned when no additional column is requested by the following steps. The default column's name is “*” (star) when the TVF returns to vertex/edge objects or predefined otherwise. Data modification steps in Gremlin query (such as addV( ) and addE( ) are example steps outputting a dynamic set of columns, which are vertex/edge properties needed by the following Gremlin steps. This is because by Gremlin semantics, data modification steps return target objects (vertices/edges/properties) in addition to persisting changes and pass them to the following steps.
As shown in
In accordance with embodiments of the present disclosure, some transition functions have strict requirements on the input state, while others are applicable to any states. Applying a function to an unacceptable state will yield an exception, which means that the Gremlin query is invalid. All vertex steps, including out( ) in( ) both( ) outEO, inEO, bothE, outV( ) inV( ) bothV( ) and otherV( ) belong to the former category. For example, the out( ) step moves from a vertex to its outgoing neighbors by Gremlin semantics. Hence, it is invalid when the prior step leads to a non-vertex state. From the compiler perspective, checking the input state requirements is similar to static type checking, which is done at compilation time or translation time.
It is to be understood that each transition function will lead to an output state. Some transition functions do not yield a new state and always stay in its input state. These functions mostly correspond to Gremlin steps that filter/re-arrange input in a certain way but do not produce any new data, such as and( ) or( ) has( ) where( ) range( ) order( ). Other functions always move to a new state. Some functions' outputs are deterministic. For example, transition functions of vertex steps and data modification steps (such as addV( ) and addE( )) lead to either a vertex or an edge state. Value aggregation steps (such as sum( ) and count( ) always lead to a numeric-value state. Fold( ) and Project( ) are a special type of aggregation steps that generates an array and a map respectively. But not all functions' output can be determined at translation time. These functions correspond to branch steps (such as optional( ) union( ) and coalesce( )) that execute if-else logic, where the “if” branch and the “else” branch may produce different states and steps that extract data from a prior step whose state is itself obscure (such as select( ) and unfold( )). For these cases, instead of maintaining all possible output states in the FSM, which may eventually lead to a combinatorial explosion, a “composite” state is created for each such function, in which the following transition function is applied. A direct consequence of this design is that those transition functions that restrict the input to a certain type must be extended to support these composite states. The validity of query in return can only be checked at runtime.
For example, consider the following Gremlin query:
Where the expected input of the last out( ) step is vertex, and the query will only be valid if the coalesce( ) step outputs results from the second sub-traversal. Since which sub-traversal to execute is unknown at translation time, the translation algorithm assumes either sub-traversal of coalesce( ) is possible and adds a composite state to the FSM on which the out( ) step will create a new state.
As the FSM moves from one state to another, the translation algorithm builds extended SQL queries through transition. In general, vertex steps' output states are mapped to node table references in the FROM clause or edge names in the MATCH clause. Filter steps are mapped to Boolean predicates in the WHERE clause. Other steps are mapped to their corresponding TVFs in the FROM clause.
However, exceptions may exist in the translation process. For example, when the translation state following the vertex step is composite Gremlin step, there is an additional Boolean predicate to connect the TVF of the former Gremlin step to the node table reference of the latter vertex step. For example, for the example coalesce( ) out( ) in additional to a node table reference of out( ) and a TVF of coalesce( ) there will be an equality-comparison predicate, binding the vertex ID column of the node table to one of the TVF's columns. Such a connection is conventionally expressed as an edge names in the MATCH clause, which is a free variable that is bound to all edges. But since in this case edges are implicitly derived from the composite state, they cannot be defined in the MATCH clause. Besides using a Boolean predicate, an alternative approach for this case is to map out( ) to a TVF too, whose computation logic is to load a set of vertices which are bound by the output of the prior TVF.
Moreover, the key of translating Gremlin query to extended SQL query is to obtain columns projected by each TVF. In SQL, the table (including the dynamic table generated by TVF) generally has a fixed number of columns. SQL is considered valid and thus can be compiled only if all scalar expressions reference available columns. So a basic requirement of the translated expanded SQL query is that every TVF outputs all columns needed/referenced by other SQL constructs. Gremlin, however, adopts the object-oriented model when defining semantics: an object contains the full state of the current traversal and fields/properties are pulled from the object upon requests of Gremlin steps. Missing fields/properties will only be detected at runtime.
Populate( ) is a safeguard mechanism in translation that populates output columns for each table reference and TVF and ensures that the output columns contain all referenced by SQL counterparts of later Gremlin steps. It is implemented as a virtual function of every FSM state. The function follows a contract that each time Populate(string colName) is invoked, the state's corresponding TVF must output a column “colName”. This includes the situation when the state itself cannot produce “colName” but still needs to populate the column in the output table, whose values are therefore all null. This also includes the situation when producing such a column needs to further invoke Populate( ) of other FSM states, creating a cascade effect.
Populate( ) is a virtual function and its implementation varies by semantics of different FSM states (and their corresponding Gremlin steps). For example, in the following example:
Where has( ) step adds a WHERE-clause predicate to the FSM. But before that it needs to make sure the column “name” is available in the target table. The has( ) step points to a vertex variable and its state's Populate( ) is invoked. Since the vertex variable is mapped to a Node table and we assume a Node table contains all properties due to flexible schemas, a vertex variable's Populate( ) is a void function.
Besides, the following provides another Gremlin query example that triggers cascade effects: g.V(1). coal esce(_.in(‘x’), _.in(‘y’)). optional(_.out( )).has(‘name’, ‘John’), where the last step has( ) will add a Boolean predicate and need to invoke Populate( ) to make sure the predicate's target table has the column. In this case, it is TVF of optional( ). Since the output of optional( ) may come from its sub-traversal or its input, the implementation of optional( )'s Populate( ) must make sure both the sub-traversal and its input have the “name” column, thereby invoking Populate( ) for the sub-traversal (_.out( )) and the input coalesce state respectively.
Many Gremlin steps involve nested sub-traversals, for instance, coalesce( ), optional( ), flatMap( ). At the high level, a sub-traversal is a new query and is translated separately by executing a new FSM and translating it to an extended SQL query. The only difference is that the sub-traversal is linked to the outer traversal through “_” (double underscore), which passes the states of the outer traversal to the inner sub-traversal as its input. Hence, the Gremlin steps in the sub-traversal can see and reference data from the outer traversal. In return, the translated SQL of the sub-traversal is a sub-query and may reference a table and its columns in the parent query.
It is to be understood that the double underscore “_” is mapped to a surrogate state in the FSM that points to the sub-traversal's input state in the parent traversal. This state is not mapped to any SQL construct when building the SQL IL, but does implement Populate( ) such that any outside data referenced by the inner Gremlin steps are populated appropriately in the input state. As such, the translated SQL sub-query is sound in that all the column references of the parent query are valid.
In addition, the scheme of translating Gremlin to extended SQL query in accordance with embodiments of the present disclosure can translate query processing to record-oriented. In a standard implementation of Gremlin, Gremlin query is comprised by Gremlin steps which are directly mapped to a pipe of objects, and the computation logic of Gremlin steps is implemented as the object's virtual functions. While the object-oriented semantics is convenient for an object-oriented runtime, the object-oriented model mixes data and computation and is difficult to support many runtime features, such as concurrency control, batch processing and query pagination.
According to embodiments of the present disclosure, a record-oriented model is used to decouple data and computation: execution runtime consists of one or more execution operators chained together, each injects and projects a stream of records with a fixed set of columns/fields. This requirement is well aligned with SQL, because SQL tables have a fixed number of columns and several of them are concatenated. To enable record-oriented execution, the translation component is responsible for reasoning the input and output objects of each Gremlin step and converting them into collections of multi-field records, which are naturally represented as tables in the extended SQL. Therefore, embodiments of the present disclosure achieve decoupling of data and operation and can reduce invoking and realize batch processing, thereby improving query efficiency.
The functionally described herein can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-Programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Application-specific Standard Products (AS SPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.
Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowcharts and/or block diagrams to be implemented. The program code may execute entirely on a machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine readable medium may be any tangible medium that may contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine readable medium may be a machine readable signal medium or a machine readable storage medium. A machine readable medium may include but not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine readable storage medium would include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are contained in the above discussions, these should not be construed as limitations on the scope of the present disclosure, but rather as descriptions of features that may be specific to particular implementations. Certain features that are described in the context of separate implementations may also be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation may also be implemented in multiple implementations separately or in any suitable sub-combination.
Some exemplary embodiments of the present disclosure are listed below.
In one aspect, a computer-implemented method is provided. The method includes: obtaining a first query in a functional graph traversal language, where the first query involves a directed edge describing neighboring relation between vertices in a graph; and translating the first query to a second query in an extended structured query language (SQL), where the second query includes clauses describing the directed edge.
In some embodiments, the method further includes: obtaining a third query in the functional graph traversal language; translating the third query to a fourth query in the extended SQL; and in response to determining that the second query and the fourth query have a same execution plan, determining that the first query and the third query have same semantics.
In some embodiments, the method further includes: comparing execution efficiency of the third query with execution efficiency of the first query; and in response to determining that the execution efficiency of the third query is higher than the execution efficiency of the first query, replacing the first query with the third query.
In some embodiments, the translating the first query to the second query comprises: identifying a basic statement in command language included in the first query; and retaining content of the basic statement in the second query using a table-valued function, the table-valued function is a user-defined function that is able to define input and a number of output columns.
In some embodiments, the identifying a basic statement in the command language includes: identifying a side effect step included in the first query, it does not influence output of the first query whether the side effect step is executed, and the retaining content of the basic statements includes: retaining the side effect step in the second query.
In some embodiments, the translating the first query to the second query comprises: determining whether the first query involves a plurality of sub-traversals; and in response to determining that the first query involves a plurality of sub-traversals, translating the plurality of sub-traversals in the first query to a plurality of sub-queries in the second query, the plurality of sub-queries each output a same number of columns.
In some embodiments, the translating the first query to the second query comprises: determining the number of columns output by the second query based on SELECT statement in the second query.
In some embodiments, the translating the first query to the second query comprises: generating a state in a finite state machine (FSM) based on steps in the first query; and generating the second query based on the state in the FSM.
In some embodiments, the functional graph traversal language is Gremlin language, and the extended SQL is an extended version of standard SQL.
In another aspect, there is provided an electronic device. The electronic device comprises a processing unit and a memory coupled to the processing unit and storing instructions. The instructions, when executed by the processing unit, execute the following acts: obtaining a first query in a functional graph traversal language, wherein the first query involves a directed edge describing neighboring relation between vertices in a graph; and translating the first query to a second query in an extended structured query language (SQL), wherein the second query includes a clause describing the directed edge.
In some embodiments, the acts further comprise: obtaining a third query in the functional graph traversal language; translating the third query to a fourth query in the extended SQL; and in response to determining that the second query and the fourth query have a same execution plan, determining that the first query and the third query have same semantics.
In some embodiments, the acts further comprise: comparing execution efficiency of the third query with execution efficiency of the first query; and in response to determining that the execution efficiency of the third query is higher than the execution efficiency of the first query, replacing the first query with the third query.
In some embodiments, the translating the first query to the second query comprises: identifying a basic statement in command language included in the first query; and retaining content of the basic statement in the second query using a table-valued function, the table-valued function is a user-defined function that is able to define input and the number of output columns.
In some embodiments, the identifying a basic statement in the command language includes: identifying a side effect step included in the first query, it does not influence output of the first query whether the side effect step is executed, and retaining content of the basic statements includes: retaining the side effect step in the second query.
In some embodiments, the translating the first query to the second query comprises: determining whether the first query involves a plurality of sub-traversals; and in response to determining that the first query involves a plurality of sub-traversals, translating the plurality of sub-traversals in the first query to a plurality of sub-queries in the second query, the plurality of sub-queries each output a same number of columns.
In some embodiments, the translating the first query to the second query comprises: determining the number of columns output by the second query based on SELECT statement in the second query.
In some embodiments, the translating the first query to the second query comprises: generating a state in a finite state machine (FSM) based on steps in the first query; and generating the second query based on the state in the FSM.
In some embodiments, the functional graph traversal language is Gremlin language, and the extended SQL is an extended version of standard SQL.
In still another aspect, there is provided a computer program product being tangibly stored on a non-transient computer readable medium and including computer executable instructions. The computer executable instructions, when executed, cause the computer to implement acts. The acts include: obtaining a first query in a functional graph traversal language, wherein the first query involves a directed edge describing neighboring relation between vertices in a graph; and translating the first query to a second query in extended structured query language (SQL), wherein the second query includes a clause describing the directed edge.
In some embodiments, the computer executable instructions, when executed, cause the computer to: obtain a third query in the functional graph traversal language; translate the third query to a fourth query in the extended SQL; and in response to determining that the second query and the fourth query have a same execution plan, determine that the first query and the third query have same semantics.
In some embodiments, the computer executable instructions, when executed, cause a computer to: compare execution efficiency of the third query with execution efficiency of the first query; and in response to determining that the execution efficiency of the third query is higher than execution efficiency of the first query, replace the first query with the third query.
In some embodiments, the translating the first query to the second query comprises: identifying basic statement in command language included in the first query; and retaining content of the basic statement in the second query using a table-valued function, the table-valued function is a user-defined function that is able to define input and a number of output columns.
In some embodiments, the identifying basic statements in the command language includes: identifying a side effect step included in the first query, it does not influence output of the first query whether the side effect step is executed, and the retaining content of the basic statements includes: retaining the side effect step in the second query.
In some embodiments, the translating the first query to the second query comprises: determining whether the first query involves a plurality of sub-traversals; and in response to determining that the first query involves a plurality of sub-traversals, translating the plurality of sub-traversals in the first query to a plurality of sub-queries in the second query, the plurality of sub-queries each outputs a same number of columns.
In some embodiments, the translating the first query to the second query comprises: determining the number of columns output by the second query based on SELECT statement in the second query.
In some embodiments, the translating the first query to the second query comprises: generating a state in a finite state machine (FSM) based on steps in the first query; and generating the second query based on the state in the FSM.
In some embodiments, the functional graph traversal language is Gremlin language, and the extended SQL is an extended version of standard SQL.
Although the present disclosure has been described with languages specific to structural characteristics and/or method logic actions, it should be appreciated that the subject matter defined by the attached claims is not limited to the above described particular characteristics and actions. Conversely, the above described particular characteristics and actions are only example forms for realizing the claims.
Number | Date | Country | Kind |
---|---|---|---|
201810101533.7 | Feb 2018 | CN | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/US2019/015042 | 1/25/2019 | WO | 00 |