Patent Application
20180158034
This application relates to ordering upcoming transactions on a blockchain and more specifically to identifying upcoming transactions and dynamically reordering execution of any transactions that require ordering for optimizing performance of a blockchain system.
Blockchains can be broadly divided into two classes, permissioned chains and permission-less chains, which are sometimes referred to as ‘trustless’. The leading example of a permission-less chain is the blockchain used by BITCOIN. In a permission-less chain, the committers (i.e., miners) compete to extend the chain and therefore do not agree in advance on the order of transactions within a block. Should two miners extend the chain concurrently, which is call a ‘fork’, there are methods to identify which fork to keep and discard all other forks which may have occurred. Transactions in the discarded forks, not already in the retained fork, must be reprocessed. For permissioned chains there are varying degrees of trust placed in the committers, which are the entities or nodes that update the blockchain.
Systems participating in a blockchain are typically divided into clients and committers. The clients are entities that submit transactions to committers. Committers process and “verify” these transactions, updating states or variables that are written to the blockchain. The function of the committer can be split into separate nodes. Blockchains are also distributed systems, all committers are expected to have and maintain the entire blockchain. For a permissioned blockchain, to accomplish this and assure that correctly functioning committers have identical blockchains, consensus algorithms are structured to enable the committers to independently process the transactions in the same order. A typical approach is to have one of the committing nodes in the blockchain network be designated as the “leader”. The leader organizes multiple transactions into an ordered set and communicates this ordered set to the other committers in the blockchain network. All committers then execute the transactions in the order specified by the leader. An alternative approach is to utilize a communications infrastructure that guarantees that all committers see all transactions in the same order. With this approach the committers use any method known to the art to agree on block size. For either approach, the committers communicate amongst themselves to see if they can reach consensus on the new state of the system. Once consensus has been reached, the state of the blockchain is updated with the agreed upon new state. The problem with such an approach is that existing systems require a set of transactions to be processed/verified serially and in succession one after another. The reason for this is that transaction T1 (first transaction) modifies variable V2, and transaction T2 (second transaction) reads V2 and updates variable V3, and transaction T3 reads both V2 and V3, producing V4, according to one example. If the transactions are not processed in the correct order, the committers may end up with inconsistent state data and may be unable to reach consensus on the new system state. A pure serialization scheme used to process all transactions slows down the process/verification, which increases the time required to reach consensus on the new state of the blockchain.
One example embodiment may include a method that includes at least one of receiving an ordered set of proposed transactions intended for inclusion in a blockchain block, creating a lattice structure containing the proposed transactions for the blockchain block, the lattice structure comprising a top and a bottom and a plurality of nodes representing the proposed transactions, determining an order of execution of the proposed transactions for the blockchain block via the lattice structure, and processing the proposed transactions in the lattice structure in parallel based on a configuration of the lattice structure.
Another example embodiment may include an apparatus that includes a receiver configured to receive an ordered set of proposed transactions intended for inclusion in a blockchain block, and a processor configured to create a lattice structure containing the proposed transactions for the blockchain block, the lattice structure comprising a top and a bottom and a plurality of nodes representing the proposed transactions, determine an order of execution of the proposed transactions for the blockchain block via the lattice structure, and process the proposed transactions in the lattice structure in parallel based on a configuration of the lattice structure.
Another example embodiment may include a non-transitory computer readable storage medium configured to store instructions that when executed causes a processor to perform receiving an ordered set of proposed transactions intended for inclusion in a blockchain block, creating a lattice structure containing the proposed transactions for the blockchain block, the lattice structure comprising a top and a bottom and a plurality of nodes representing the proposed transactions, determining an order of execution of the proposed transactions for the blockchain block via the lattice structure, and processing the proposed transactions in the lattice structure in parallel based on a configuration of the lattice structure.
It will be readily understood that the components, as generally described and illustrated in the figures herein, may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of at least one of a method, apparatus, non-transitory computer readable medium and system, as represented in the attached figures, is not intended to limit the scope of the application as claimed, but is merely representative of selected embodiments.
The instant features, structures, or characteristics as described throughout this specification may be combined in any suitable manner in one or more embodiments. For example, the usage of the phrases “example embodiments”, “some embodiments”, or other similar language, throughout this specification refers to the fact that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment. Thus, appearances of the phrases “example embodiments”, “in some embodiments”, “in other embodiments”, or other similar language, throughout this specification do not necessarily all refer to the same group of embodiments, and the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
In addition, while the term “message” may have been used in the description of embodiments, the application may be applied to many types of network data, such as, packet, frame, datagram, etc. The term “message” also includes packet, frame, datagram, and any equivalents thereof. Furthermore, while certain types of messages and signaling may be depicted in exemplary embodiments they are not limited to a certain type of message, and the application is not limited to a certain type of signaling.
In operation, when a block of transactions must be “verified”, prior to execution, the committer(s) may decide on the order of execution of the transactions in the ordered transaction set for the block. Each transaction in the set may have a “read set” and a “write set”. These two sets represent database variables in the blockchain that are to be read, updated and/or written to the blockchain and/or any associated state of the system. Logically, any variable read or write operation is dependent on any variable write operation from a prior write transaction to the same variable in the transaction set. This dependence relationship can be represented as a graph, lattice or logic diagram that can be used to generate an ideal order of parallel execution of the transactions, assuming available resources that will result in a consistent execution of the transactions across all of committer nodes for a blockchain. On average, transaction executed base on an order generated from the lattice that properly handles joins and forks can proceed in parallel rather than being serialized. In the ideal case, execution would be faster by a factor of N, where N is the number of CPUs/cores/threads available in a validating peer node. The validating peer may be a logical construct and may include more than one computer, each with one or more CPUs/cores/threads that are scheduled as a single entity.
A blockchain may store ‘state’ information. In the current distributed ledger model, there is a state in the blockchain and in the “world state”. The world state is a database of key/value pairs. CHAINCODE is an example software that may read and/or write to both the blockchain and the global state. For some implementations of blockchains, all the state is stored in the blockchain. In these environments, there is no difference between a blockchain state and world state. Each function in a CHAINCODE application can be labelled as reading and/or writing to one or more of the variables (i.e., blockchain state or world state key/value pairs). (For the purposes of this description, we treat chaincode functions and transactions as used interchangeably.) It is possible, through the use of one or more different approaches, such as annotation, static analysis, dynamic analysis and metadata analysis, to label each CHAINCODE function with read set and write set annotations. For example, CHAINCODE ‘C’ has a function ‘F’ that reads world state variable ‘wsV’ and writes blockchain variable ‘bcV’. C would be labelled with read set {wsV} and write set {bcV}. The read set and write set annotations are also updated when chaincode calls other chaincode. The dependence sets of ‘{wsV}’ and {bcV}' of the callee must be propagated to the caller's dependency sets. An alternative to propagating the read and write sets is to maintain the list of functions called by each chaincode. If this alternative is used, the function dependencies must also be propagated and taken into account during construction of the lattice. For simplicity, the propagation of the read and write sets of the callee are performed by propagating the variables into the caller's read and write sets respectively, and the propagated variables are appropriately annotated with the name of the function making the change. Each annotated variable is also annotated with the name of the chaincode that actually changes the variable. Dynamic analysis determines and records the actual read and write sets for each of the chaincode functions after the chaincode functions execute. Dynamic analysis may also include modeling how input values to chaincode functions have an effect on a set of ledger (blockchain and/or world state) variables read and/or written by the chaincode. This approach to dynamic analysis can yield a more precise description of the read and write sets for chain code functions. The set of ordered transactions to be verified (chaincode function executions) may be used to construct a lattice/graph, where ‘Top’ is the start of execution of the transactions and ‘Bottom’ is the end of all transactions. There are other embodiments using less precise constructions of the lattice than the one described herein that still properly maintain serial order of execution for parallel scheduling.
Constructing the lattice for a proposed block of transactions includes using a dependency graph that is based on the read and write sets for each of the blockchain transactions to be invoked. One such representation of a dependency graph is a lattice, although other graph representations are possible to represent the dependency relationships between the read sets and write sets. The lattice is initialized with a ‘Top’ and a ‘Bottom’ nodes. Next, the read and write sets of each transaction must be determined by any number of ways (i.e., annotations, static analysis, or dynamic analysis). Another such method is to create a data structure that points to each transaction that also contains a pointer to the read set of the transaction and a pointer to the write set of the transaction. Next, an iteration over all transactions in the transaction set is performed, including retrieving the next transaction, ‘NT’, in the transaction set. Each time a transaction is considered, it is place in a node connected to the ‘Bottom’ and the lattice is searched breadth first starting at Bottom. By iterating over the lattice in a breadth first manner starting at the ‘Bottom’, the node containing NT is placed into the lattice below all existing nodes in the lattice that have a write set containing a variable in NT write set, or write set containing a variable in NT's read set, or read set that contains a variable in NT's write set. When a node is found that NT is dependent upon, a link to NT is added in the located node and the located node is added as a predecessor to NT. If NT has no read/write set dependency on any prior transactions in the transaction set, the node containing NT can be inserted into the lattice just below the ‘Top’, and parallel to any other nodes (transactions) just below the ‘Top’. Next, if the located node, Top or an internal node, has Bottom as a successor, that link is removed (in this case NT is inserted between the located node and Bottom). Repeated iterations must be performed over the set of transactions until all transactions have been inserted into the lattice. The lattice constructed represents the maximum parallel function execution opportunity for the transaction set. Assuming an unlimited number of CPUs/cores/threads for parallel execution, the system processes the lattice, starting from the Top, and executes in parallel all transactions at each level of the lattice. Those skilled in the art will recognize that there are further performance optimizations possible when the transaction execution times or system resource consumptions are non-uniform. Once such optimization is to dynamically construct the lattice as the transactions arrive, not waiting until a full block of transactions has been received.
In operation, when there is a merge of two paths in the lattice at a node, synchronization is required, waiting for all paths prior to the merge node in the lattice path to complete execution before executing the node at the join in the lattice. Synchronization is needed because this next node in the lattice has a dependency on variables being read or written/updated farther up in the lattice closer to the ‘Top’ node. Whenever there is a fork below a node, there will be two or more links below the fork node. The transactions represented by nodes along the paths below a fork node can be executed in parallel, assuming all of the dependencies (variable read/write operations), or joins, of each node have been satisfied.
Another level of optimization may include ordering the execution of the transactions subject to the constraint(s) on the number of available processors (e.g., CPUs, cores, threads). As an alternative, to maximize throughput, execution time of each of the transaction can be considered. An estimate of execution time can be derived from execution traces and/or metadata, such as annotations, that specifies the maximum amount of time that the function or transaction is expected, or allowed, to execute. The available resources can be determined using techniques from a compiler technology related to multi-core and/or MIMD systems. Validation of transactions can be parallelized on the same, or collaborating application systems, and transactions can be reordered without affecting consensus, for properly behaving (i.e., non-faulty) validating peers, by observing dependency relationships between transactions (e.g., the read and write sets). For example, independent transactions can be executed in parallel. The computing resources (e.g., processing cores) and dependent transactions are serialized to maintain an effective order prior to execution. The committer(s) can decide on the order of commitment of the transactions in the ordered transaction set for the block. Other operations include creating a dependence graph to show the relationship between read and write sets for transactions, where a variable read is dependent on any variable write operations from a prior transaction's write to the same variable in the transaction set representing this dependence relationship. A lattice represents an ideal order of parallel execution of the transactions.
Referring to
Example embodiments provide validating peers in a shared ledger permissioned blockchain configuration. By shared ledger we mean the complete ledger is located and maintained by each validating peer. Other approaches to maintain the ledger are possible, such as splitting the ledger state between validating peers. These approaches are documented in the distributed systems research publications. It does not matter whether a validating peer and a consenter are a single node or separate nodes. In the case were they are separate nodes, the node containing the validating peer will be used. In order to function, it is required that there is agreement between the validating peers or components performing validation on the order of the transactions to be executed. PBFT is an example of a consensus algorithm where the transactions are ordered before being processed. Ordering of transactions is a common feature of distributed consensus algorithms to help minimize thrashing. Chaincode is another type of program, Chaincode can contain a “function” or subroutines which are executable by other Chaincode. When having an ordered list of transactions, the maximum degree of parallelism that can be achieved may be desired for optimal performance of transaction processing.
When constructing a lattice of nodes having one or more transactions, given an ordered set of transactions to be verified, an empty lattice is initialized. This includes designating the ‘Top’ node, representing the start of execution of the transactions, and a ‘Bottom’ node representing the end of all transaction executions. Next, the read and write sets of each transaction must be determined by any number of means (including through annotations, static analysis or dynamic analysis). An iteration over all transactions in the transaction set is performed, including retrieving the next transaction, ‘NT’, in the transaction set. Each time a transaction is considered, it is place in a (new) node connected to Bottom. For each transaction considered, by iterating over the lattice in a breadth first manner starting at bottom, the node containing NT is placed into the lattice below all existing nodes in the lattice that have a write set containing a variable in NT's read set or write set or that read a variable in NT write set. If NT has no read/write set dependency on any prior transactions in the transaction set, the node containing NT can be inserted into the lattice just below ‘Top’, and parallel to any other nodes (transactions) just below Top. Repeated iterations must be performed over the set of transactions until all transactions have been inserted into the lattice. The lattice constructed represents the maximum parallel function execution opportunity for the transaction set. Assuming an unlimited number of CPUs/cores/threads for parallel execution, the system application processed the lattice, starting from the Top, and execute in parallel all transactions at each level of the lattice. Those versed in the art will recognize that there are further performance optimizations possible when the transaction execution times are non-uniform.
For clarity, in this description, each node in the lattice contains one transaction. Those skilled in the art understand that each node in the lattice could contain an ordered list of one or more proposed transactions. Such a lattice could be derived from the lattice constructed herein or constructed directly without first creating a lattice were each node contains one transaction. The dependence graph is based on the relationship between the read and write sets for each of the blockchain functions to be invoked by the transactions and the given order of the transactions iterate over all of the transactions in the transaction set in the given order. Next, the next transaction (NT) is received from the ordered list of transactions.
In the final lattice, “forks” represent opportunities for parallel execution. For example, in
Referring to
One way to accomplish the processing is to schedule a block of proposed transactions on a thread and have them return to the scheduler when they complete. At this point the scheduler would identify what else could be scheduled at any given time. When all threads have completed and there is nothing left to schedule, the validating node has finished validating the proposed transactions and can continue with the consensus algorithm. This technique works whether the validator has multiple cores with multiple threads or consists of multiple machines. The synchronization and signaling techniques differ depending on the architecture of the validator and are well understood in the art.
Before discussing the flow chart illustrated in
‘E’ is a list of nodes (proposed transactions) that are ready for execution; this list is constructed by the function find_ready. ‘F’ is a reference to a node in the lattice. ‘get_lock’ gets the lock for that node. The lock is a semaphore that only permits one process to decrement the predecessor count for the node. Manipulating semaphores for all system architectures is well understood in the art. ‘find_ready’ is a function which goes through a list of nodes in the lattice that still need to be scheduled and returns the list of nodes where the predecessor node count is zero and has not already executed the proposed transaction(s) represented by the node.
A zero count indicates that all predecessor nodes (proposed transactions) have completed execution so that the node can be scheduled for execution. ‘L’ is a set that represents all of the nodes (proposed transactions) in the lattice, exclusive of the ‘Top’ and ‘Bottom’. ‘Lattice’ is a directed graph without cycles with a single start node, ‘Top’, and a single end node ‘Bottom’. ‘NextNode(X)’ is a function over the ready transaction in ‘R’ that removes a node from the list of nodes, X, and returns the node. A ‘Node’ represents each node of the lattice which contains, or references, a transaction request, a lock, a count of the number of predecessor nodes (exclusive of the Top), and a list (or set) of its predecessor and successor nodes in the lattice. The ‘release_lock’ releases the lock associated with a particular node. ‘R’ is a set that represents all of the nodes (proposed transactions) that are ready for execution. These nodes do not have any predecessor nodes in the lattice that still need to execute. Specifically, each node in R has a count of zero. ‘Schedule’ is a function, which takes a node and executes the transaction that is associated with that node. The call to schedule is asynchronous, returning to the caller as soon as the system has created a thread or process to execute the scheduling routine. The maximum number of concurrent transactions that are executing is dependent on the architecture of the system and available resources where the transactions are being executed. A flowchart of the scheduling routine is included. This concept is well known in the art. SL is a copy of the list of successors of a node. If the only successor is the Bottom, then it is empty. The Top is the first node in the lattice (a directed graph) which has no predecessors. All paths through the lattice start with the Top and terminate with the Bottom. The Top is represented by the symbol T (capital t). The ‘transaction’ is a routine that returns the transaction associated with a node. ‘Φ’ represents the empty set. ‘∪’ is used to represent set union. Each node in the lattice represents or contains a transaction to be executed. In simpler processing models, this may be transaction execution. In other execution models, this may be transaction speculative execution. Each node also contains a list of predecessor nodes in the lattice and a list of successor nodes in the lattice.
Schedule(node) function 532 proceeds by obtaining the transaction from the node 534 and executing the transaction 536. Once the transaction has completed execution, the predecessor count on all successor nodes must be decremented. Next, a list of successor nodes ‘SL’ is obtained 538. The next successor, F, is removed from the list 542. If the successor is empty 544 there are no remaining successors that need their predecessor count decremented so the scheduler exits 554. Otherwise, it obtains the lock for the successor node 546, decrements the counter 548, and then releases the lock 552. Once the lock has been released it returns to obtain the next successor from the list of successors 542.
Before discussing the build lattice algorithm as represented in
The above embodiments may be implemented in hardware, in a computer program executed by a processor, in firmware, or in a combination of the above. A computer program may be embodied on a computer readable medium, such as a storage medium. For example, a computer program may reside in random access memory (“RAM”), flash memory, read-only memory (“ROM”), erasable programmable read-only memory (“EPROM”), electrically erasable programmable read-only memory (“EEPROM”), registers, hard disk, a removable disk, a compact disk read-only memory (“CD-ROM”), or any other form of storage medium known in the art.
An exemplary storage medium may be coupled to the processor such that the processor may read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an application specific integrated circuit (“ASIC”). In the alternative, the processor and the storage medium may reside as discrete components. For example,
As illustrated in
Although an exemplary embodiment of at least one of a system, method, and non-transitory computer readable medium has been illustrated in the accompanied drawings and described in the foregoing detailed description, it will be understood that the application is not limited to the embodiments disclosed, but is capable of numerous rearrangements, modifications, and substitutions as set forth and defined by the following claims. For example, the capabilities of the system of the various figures can be performed by one or more of the modules or components described herein or in a distributed architecture and may include a transmitter, receiver or pair of both. For example, all or part of the functionality performed by the individual modules, may be performed by one or more of these modules. Further, the functionality described herein may be performed at various times and in relation to various events, internal or external to the modules or components. Also, the information sent between various modules can be sent between the modules via at least one of: a data network, the Internet, a voice network, an Internet Protocol network, a wireless device, a wired device and/or via plurality of protocols. Also, the messages sent or received by any of the modules may be sent or received directly and/or via one or more of the other modules.
One skilled in the art will appreciate that a “system” could be embodied as a personal computer, a server, a console, a personal digital assistant (PDA), a cell phone, a tablet computing device, a smartphone or any other suitable computing device, or combination of devices. Presenting the above-described functions as being performed by a “system” is not intended to limit the scope of the present application in any way, but is intended to provide one example of many embodiments. Indeed, methods, systems and apparatuses disclosed herein may be implemented in localized and distributed forms consistent with computing technology.
It should be noted that some of the system features described in this specification have been presented as modules, in order to more particularly emphasize their implementation independence. For example, a module may be implemented as a hardware circuit comprising custom very large scale integration (VLSI) circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A module may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, graphics processing units, or the like.
A module may also be at least partially implemented in software for execution by various types of processors. An identified unit of executable code may, for instance, comprise one or more physical or logical blocks of computer instructions that may, for instance, be organized as an object, procedure, or function. Nevertheless, the executables of an identified module need not be physically located together, but may comprise disparate instructions stored in different locations which, when joined logically together, comprise the module and achieve the stated purpose for the module. Further, modules may be stored on a computer-readable medium, which may be, for instance, a hard disk drive, flash device, random access memory (RAM), tape, or any other such medium used to store data.
Indeed, a module of executable code could be a single instruction, or many instructions, and may even be distributed over several different code segments, among different programs, and across several memory devices. Similarly, operational data may be identified and illustrated herein within modules, and may be embodied in any suitable form and organized within any suitable type of data structure. The operational data may be collected as a single data set, or may be distributed over different locations including over different storage devices, and may exist, at least partially, merely as electronic signals on a system or network.
It will be readily understood that the components of the application, as generally described and illustrated in the figures herein, may be arranged and designed in a wide variety of different configurations. Thus, the detailed description of the embodiments is not intended to limit the scope of the application as claimed, but is merely representative of selected embodiments of the application.
One having ordinary skill in the art will readily understand that the above may be practiced with steps in a different order, and/or with hardware elements in configurations that are different than those which are disclosed. Therefore, although the application has been described based upon these preferred embodiments, it would be apparent to those of skill in the art that certain modifications, variations, and alternative constructions would be apparent.
While preferred embodiments of the present application have been described, it is to be understood that the embodiments described are illustrative only and the scope of the application is to be defined solely by the appended claims when considered with a full range of equivalents and modifications (e.g., protocols, hardware devices, software platforms etc.) thereto.
