Patent Application
20220283878
The subject matter described relates generally to distributed computing and, in particular, to using a data router to stream data to nodes in a distributed computing system.
A workflow can be modeled as a directed acyclic graph between various computational nodes where each directed edge indicates a data source and destination for the data. Workflows distribute a set of tasks across a collection of nodes to parallelize and accelerate execution of the workflow as a whole. These workflows are often carried out in a cloud computing architecture where the computational power can be scaled appropriately to the task at hand. In such architectures, nodes do not have a local shared memory and thus data are typically shared via communication channels. For some workflows (e.g., those involving quantum computers) the amount of data can be large and latencies in the communications channels can become a significant limiting factor on run times.
In typical distributed computing workflows, at least some of the tasks assigned to certain nodes are dependent upon computations performed by other nodes. Thus, the ability to perform a task depends on the exchange of information between nodes. Efficient workflows distribute tasks in a manner that reduces delays due to this communication. To communicate data from one node to another, most computer architectures wait for a task to complete, take the output of the task from memory and write it to disk, and then transmit the output over a communication channel. For tasks involving long processes or large files this can be a significant efficiency hurdle. Nodes that are dependent upon the data generated in the task are held up until this entire process finishes. This problem is particularly notable for workflows using computational nodes that are physically separated (e.g. quantum computers or quantum simulators) or cannot have a shared memory (e.g. where data is maintained locally for confidentiality or security reasons).
The figures and the following description describe certain embodiments by way of illustration only. One skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods may be employed without departing from the principles described. Wherever practicable, similar or like reference numbers are used in the figures to indicate similar or like functionality. Where elements share a common numeral followed by a different letter, this indicates the elements are similar or identical. A reference to the numeral alone generally refers to any one or any combination of such elements, unless the context indicates otherwise.
As described previously, in distributed computing environments, certain workflows can suffer from significant latencies, particularly where large amounts of data are to be transferred between different parts to he distributed computing environment. Furthermore, unlike traditional supercomputers that take advantage of high-speed shared memory, distributed computing environments typically lack consistent access to such resources as the computing components performing the tasks in workflows may be separated by significant distances, both physically and in terms of network path lengths/latencies. These and other problems may be addressed by a process and device for dependency-based data routing.
In various embodiments, a task executing on a node produces data that is streamed from memory of the node to a data router as it is produced. The data router may route the streamed data to another node which has a dependency for that data, where the receiving node may then carry out its task as it receives the data. The data routing method may be executed as a monolithic application or set of microservices.
In one embodiment, a computer-implemented method of data routing includes a data router receiving data from a data source and storing the data in a buffer of the data router. The data router analyzes the data in the buffer to identify the data source. The method further includes using a routing map to identify a destination for the data based on the data source and streaming the data from the buffer to the destination.
In some embodiments, the data router is used to add quantum enhanced optimization (“QEO”) or other additional capacity to an existing optimization system. The data router may transparently extract data (e.g., unevaluated and evaluated solutions) from the existing optimization system, provide it to the QEO (or other additional systems) for processing, and inject the results back into the existing optimization system.
The client devices 110 are computing devices with which users may define and modify workflows as well as review the results generated by workflows. A client device may be any suitable computing device, such as a desktop PC, a laptop, a tablet, a smartphone, or the like. In one embodiment, a client device 110 provides a first user interface for defining a workflow. Once defined, the user may submit the workflow (e.g., via an API) to the task manager 120 for execution. The client device 110 may also provide a second user interface for viewing information about and results generated by the workflow before execution, during execution, on completion, or any combination thereof. For example, the second user interface may provide predictions of when the workflow will be started and completed, preliminary or predicted results while execution is on-going, and final results once the workflow is completed. Various embodiments of client device 110 are described in greater detail below, with reference to
A node 130 is a physical or virtual machine (or a partition/other division of such a machine) configured to perform one or more tasks within the networked computing environment 100. The nodes 130 may be arranged into pods or clusters spread across multiple physical locations. Different nodes may be optimized or otherwise configured to perform different tasks based on factors including processing power, memory, speed, type (e.g., quantum versus classical), operating system, available software, physical location, and network location, etc. Various tasks may be assigned to nodes 130 by a workflow with at least some of the tasks being dependent of data generated by other tasks in the workflow. For example, the output from a first node 130A may be communicated via a data connection 170 to a second node 130B, which uses the received output in completing one or more tasks. Thus, efficient workflows distribute tasks in a manner that reduces delays due to this communication. In one embodiment, communication delays are reduced by at least some of the nodes 130 streaming data from memory as it is generated rather than waiting to complete a task and then transmitting the generated data (e.g., from disk storage).
The task manager 120 receives workflows from client devices 110 and selects nodes 130 to complete the tasks in those workloads. The task manager 120 may select one or more nodes 130 for each task based on one or more of: the requirements of the task, any dependencies on or by other tasks in the same workload, the properties of the nodes (e.g., type of node, processing power, memory, etc.), the locations of the nodes (e.g., preferentially selecting nodes that are closer together), and the availability of the nodes (e.g., how many other tasks are queued for each node). In one embodiment, the task manager 120 receives workloads via an API and thus the workloads may be independent of the programming language used to define it at a client device 110.
In
The persistent storage 140 includes one or more computer readable media configured to store some or all of the data received by the task manager from nodes 130. In one embodiment, the data router 125 analyzes received data to determine whether it should be stored and, if so, forwards it to the persistent storage 140. For example, the definitions of tasks in a workflow may include a flag or other indicator of whether persistent storage is required. For each data stream, the data router 125 may identify the task that generated the data, check the flag or other indicator for that task in the workflow, and forward the received data to the persistent storage 140 if the flag or other indicator indicates the data should be stored.
The data connections 170 are communication channels via which the other elements of the networked computing environment 100 communicate. The data connections 170 may be provided by a network that can include any combination of local area and wide area networks, using wired or wireless communication systems. In one embodiment, the data connections 170 are part of a network (e.g., the internet) that uses standard communications technologies and protocols. For example, the network can include communication links using technologies such as Ethernet, 802.11, worldwide interoperability for microwave access (WiMAX), 3G, 4G, 5G, code division multiple access (CDMA), digital subscriber line (DSL), etc. Examples of networking protocols used for communicating via the network include multiprotocol label switching (MPLS), transmission control protocol/Internet protocol (TCP/IP), hypertext transport protocol (HTTP), simple mail transfer protocol (SMTP), and file transfer protocol (FTP). Data exchanged over the network may be represented using any suitable format, such as hypertext markup language (HTML) or extensible markup language (XML). In some embodiments, some or all of the communication links of the network may be encrypted using any suitable technique or techniques.
The workflow definition module 210 provides a user interface with which a user can define a workflow. The user interface may be part of an integrated development environment (IDE). In one embodiment, the IDE provides a user interface with which the user can define a directed acyclic graph indicating the relationships and dependencies between tasks in the workflow and provide code modules for performing the tasks in the workflow. The code modules may use any suitable programming language.
The packaging module 220 packages workflows defined using the workflow definition module 210 for submission to the task manager 120. In one embodiment, the packaging module 220 creates a container object including the code and dependencies for the tasks in the workflow. The container object can use a standardized format. The container object can also be configured to provide validation checks and enable workflow scheduling using a standardized approach. The packaging module 220 provides the packaged workflow to the task manager for implementation (e.g., using an API).
The results module 230 provides a user interface with which a user can view information about the workflow after submission to the task manager 120. In one embodiment, the results module 230 receives the results of execution of the workflow after it is complete. The results may be displayed to the user in any suitable format for the particular workflow. Additionally or alternatively, the results module 230 may provide information about the workflow before or during execution. For example, the results module 230 may provide a user interface identifying the nodes 130 that are scheduled to execute or that are currently executing tasks in the workflow along with information such as predicted start and end times for each task. In some embodiments, the results module 230 may enable the user to request that a different node is used for one or more tasks in the workflow. The user interface may also provide preliminary or predicted results of the workflow during execution.
The local datastore 240 is one or more computer-readable medium configured to store the software and data used by the client device 110. In one embodiment, the local data stores includes copies of the workloads created by the user. This may be useful to enable the user to repeat execution of workloads or to restart workloads in the event of a crash or other data loss event at the task manager 120. The local datastore 240 may additionally or alternatively include cached copies of information generated or retrieved by the results module 230 to reduce loading times and network bandwidth requirements.
The data consumers 310 receive arbitrary streams of incoming data and write them to the buffer 350. Although
The stream manager 320 identifies the data in the buffers 350 and uses the routing map 360 to identify one or more destinations for the data. In one embodiment, the stream manager 320 analyzes data received from by a data consumer 310 to identify the source of the data. For example, the stream manager 320 may parse the incoming data stream to identify explicit identification information included in the stream (e.g., in a header portion of data packets in the stream). Explicit identification information may include an identifier of the node 130 from which the stream originates (e.g., a node ID, IP address, or MAC address, etc.) or an identifier of the task that generated the data. Additionally or alternatively, the stream manager 32 may parse the incoming data stream for implicit identifiers of the source. For example, the specific data types or formats included in the stream may originate uniquely from a single node or task. In some instances, the identity of the origin node or task may be irrelevant if the processing instructions can be derived from the format or nature of the data independently from its origin.
Having identified the data, the stream manager 320 uses the routing map 360 identify one or more destinations for the data stream. Data that is needed locally at a future time may be identified to be written to disk, while data which is needed immediately may be identified to be passed to memory. If the data stream is needed at one or more nodes 130, the stream manager uses the routing map 360 to identify one or more channels to which the data will be streamed and notifies one or more corresponding stream producers 330. Note that multiple destinations may be identified for a single data stream and the data may be streamed to each of the destinations simultaneously. In one embodiment, the routing map 360 includes both dependency information for the workflow and infrastructure information indicating properties of the nodes 130 and how to route data to them.
The stream producers 330 stream received data to nodes 130 that are identified as destinations by the stream manager 320. The stream producers 330 may stream data directly from the buffer 350 once the destinations have been determined without waiting for the task generating the data to complete. This can be particularly advantageous with tasks that generate a large amount of data as waiting for all of the data to be received can result in a significant delay in tasks that depend on that data being able to begin. In one embodiment, a stream producer 330 determines the appropriate network protocols and serialization formats for the channel, encodes the data accordingly, and sends the encoded data to the identified destination or destinations. Alternatively, data may be streamed to the destinations in the same format it was received.
The persistence manager 340 manages the storage of data in persistent storage 140. In one embodiment, the destinations identified by the stream manager 320 using the routing map 360 can include persistent storage 340. If persistent storage in an identified destination, the persistence manager 340 saves a copy of the received data from the buffer 30 into persistent storage 140. The data may be stored in conjunction with a timestamp, an identifier of the task that generated it, an identifier of the node 130 that generated it, an identifier of the workflow that generated it, an identifier of the user that created the workflow, or any other desired identifying information. Generally, saving data to persistent storage is lower priority than streaming the data to other destinations, so, unlike the stream producers 330, the persistence manager 340 may wait until all of the data has been received before starting to save it to persistent storage 140. Additionally or alternatively, the persistence manager 340 may compress, encrypt, or otherwise process the data as appropriate for any give use case.
In the embodiment shown in
The data router 125 identifies 430 one or more destinations for the received data using the routing map 360. In one embodiment, the routing map 360 includes both dependency information from the corresponding workflow and infrastructure information regarding the network topology and properties of the nodes 130. Thus, having determined the source of received data, the data router 125 can identify one or more nodes 130 that are executing or will execute tasks that depend on the received data and determine how to route the data to the identified nodes. The data router 125 distributes 440 the data to the identified destinations (e.g., by saving the data to local storage, streaming the data to other nodes 130, sending the data to persistent storage 140, or any combination thereof).
Thus, an optimizer in this setting is a source of new unevaluated solutions which are generated based on previous observations of evaluated solutions. Typically, the optimization happens in a sequential fashion, alternating between the plant 520 and the source 540 where the plant generates an evaluated solution, followed by a source proposing a new unevaluated solution, which is then fed to the plant for evaluation. The plant 520 then generates the evaluated solution, and the cycle continues. However, in one embodiment, a streaming approach is used, where the plant 520 keeps taking unevaluated solutions and producing evaluated ones, while the source 540 keeps producing new unevaluated solutions while receiving evaluated ones.
In some embodiments, at least one of the plant 520 and the source 540 uses quantum computation. A plant may use quantum computation by executing a set of quantum circuits as part of its process for evaluating the quality metric of the input (unevaluated solution). For example, and plant 520 may use a variational quantum eigensolver (VQE). In this example, the input is a set of (classical) parameters for the quantum circuit while the output is the estimated expectation of the observables, which involves running quantum circuits on a quantum computer backend. A source may use quantum computation by executing a set of quantum circuits as part of a process for generating new unevaluated solutions based on the input evaluated solutions. An example is a quantum generative model such as a quantum circuit Born machine (QCBM) or a hybrid quantum-classical generative adversarial network, where the quantum computer supplies the source of randomness that enables the generation of new unevaluated solutions.
The first source 660 and the second source 662 generate new unevaluated solutions using the evaluated solutions which are both passed to the first and second plants 620, 622 via corresponding fan-out operations 670, 672 and proxies 680, 682. The new unevaluated solutions are evaluated by the plants 620, 622 and passed to the optimality check 640 via the proxy 630 and the process repeats until a solution is found that meets the criteria of the optimality check. Similar to the optimization system shown in
The plants 620, 622 may be running different or identical evaluation routines, Similarly, the sources 660, 662 may use different or identical optimizers. In the case of VQE, a classical black-box optimization program may be used as a single source with multiple plants each running quantum circuits and measuring different Pauli operators in the Hamiltonian whose ground state is sought. In the case of QEO, where the quantum generative model serves as a “booster” to a classical optimizer, there may be a single plant evaluating the quality of a solution and multiple sources, some of which are running the quantum generative model while others are running a classical optimizer.
The default routing of data may be altered through a new routing map 360 provided by the data router 125 or an external data mapping service. Because the data router 125 is only extracting information that the existing optimization system 710 is already generating and routing between components and injecting back in evaluated and/or unevaluated solutions in the same format used by the existing optimization system, the QEO 720 can essentially be added transparently. The existing optimization system 710 continues to operate as it did previously and simply receives additional solutions to process. One of skill in the art will recognize that similar techniques may be used to transparently add additional source and plants of different types to an existing optimization system 710 with minimal or no alteration to the existing system.
In the embodiment shown in
The types of computers used by the entities of
Some portions of above description describe the embodiments in terms of algorithmic processes or operations. These algorithmic descriptions and representations are commonly used by those skilled in the computing arts to convey the substance of their work effectively to others skilled in the art. These operations, while described functionally, computationally, or logically, are understood to be implemented by computer programs comprising instructions for execution by a processor or equivalent electrical circuits, microcode, or the like. Furthermore, it has also proven convenient at times, to refer to these arrangements of functional operations as modules, without loss of generality.
As used herein, any reference to “one embodiment” or “an embodiment” means that a particular element, feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment. Similarly, use of “a” or “an” preceding an element or component is done merely for convenience. This description should be understood to mean that one or more of the elements or components are present unless it is obvious that it is meant otherwise.
Where values are described as “approximate” or “substantially” (or their derivatives), such values should be construed as accurate +/−10% unless another meaning is apparent from the context. From example, “approximately ten” should be understood to mean “in a range from nine to eleven.”
As used herein, the terms “comprises,” “comprising,” “includes,” “including,” “has,” “having” or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Further, unless expressly stated to the contrary, “or” refers to an inclusive or and not to an exclusive or. For example, a condition A or B is satisfied by any one of the following: A is true (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present).
Upon reading this disclosure, those of skill in the art will appreciate still additional alternative structural and functional designs for a system and a process for efficient dependency-based routing that may reduce downtime of nodes 130 or communications lag when implementing distributed workloads. Thus, while particular embodiments and applications have been illustrated and described, it is to be understood that the described subject matter is not limited to the precise construction and components disclosed. The scope of protection should be limited only by the following claims.
This application claims the benefit of U.S. Provisional Application No. 63/155,521, filed Mar. 2, 2021, which is incorporated by reference.
| Number | Date | Country | |
|---|---|---|---|
| 63155521 | Mar 2021 | US |
