Patent Application
20240394271
Cloud-based computation typically is accessed using a user facing control system. This control system coordinates submitted job requests and acts as a channel for the output of the job request. However, this can lead to the control system becoming a bottleneck especially in the event that the output of the job request is large and multiple users and jobs are receiving service from the control system.
Various embodiments of the invention are disclosed in the following detailed description and the accompanying drawings.
The invention can be implemented in numerous ways, including as a process; an apparatus; a system; a composition of matter; a computer program product embodied on a computer readable storage medium; and/or a processor, such as a processor configured to execute instructions stored on and/or provided by a memory coupled to the processor. In this specification, these implementations, or any other form that the invention may take, may be referred to as techniques. In general, the order of the steps of disclosed processes may be altered within the scope of the invention. Unless stated otherwise, a component such as a processor or a memory described as being configured to perform a task may be implemented as a general component that is temporarily configured to perform the task at a given time or a specific component that is manufactured to perform the task. As used herein, the term ‘processor’ refers to one or more devices, circuits, and/or processing cores configured to process data, such as computer program instructions.
A detailed description of one or more embodiments of the invention is provided below along with accompanying figures that illustrate the principles of the invention. The invention is described in connection with such embodiments, but the invention is not limited to any embodiment. The scope of the invention is limited only by the claims and the invention encompasses numerous alternatives, modifications and equivalents. Numerous specific details are set forth in the following description in order to provide a thorough understanding of the invention. These details are provided for the purpose of example and the invention may be practiced according to the claims without some or all of these specific details. For the purpose of clarity, technical material that is known in the technical fields related to the invention has not been described in detail so that the invention is not unnecessarily obscured.
A system for fetching query results through cloud object stores is disclosed. The system comprises a processor and a memory. The processor is configured to: receive a client request; determine one or more executors to generate a response to the user request; provide each of the one or more executors with an indication; receive for each indication a response including an output of either a cloud output or an in-line output to generate a group of in-line outputs and a group of cloud outputs; determine whether the group of in-line outputs comprises all outputs for the client request; in response to determining that the group of in-line outputs not comprising all outputs for the client request: a) convert the group of in-line outputs to a converted group of cloud outputs; b) generate metadata for the converted group of cloud outputs and the group of cloud outputs (e.g., generate metadata for the entire set of cloud outputs); and c) provide response to the client request including the metadata for the converted group of cloud outputs and the group of cloud outputs. The memory is coupled to the processor and configured to provide the processor with instructions. In some embodiments, the reference/pointers to the entire set of cloud outputs (e.g., part of the metadata) are stored in the memory.
In some embodiments, the system for fetching query results through cloud object stores enables direct and parallel access to output data for the client requesting results as stored in a cloud object store. The direct access for a client to outputs responding to the client request on a cloud storage system avoids a data access path through a control system (e.g., a control plane system). A client makes a request, and the system receives the request at a control system. The control system provides the request to a coordinating node to coordinate execution of jobs to generate a response for the client request. The coordinating node determines one or more executors for executing the jobs, and for each job, the coordinating node determines whether the output for the job will be all in-line outputs, a mix of in-line outputs and cloud outputs, or all cloud outputs. In-line outputs are selected for a job in the event that the outputs are relatively small, in which case supporting the overhead of storing the outputs (e.g., indexing, establishing and storing metadata, storing on the cloud storage system in a location indicated using the metadata, supporting client direct access using signed URLs, and removing output files after client access of the output files as indicated by metadata) is not efficient. The small amount of data associated with an in-line output can be efficiently passed through the control system without posing a problem of the control system becoming a bottleneck. So, for the situation in which all of the job outputs are in-line outputs, the job outputs will be passed back through the control system. However, for the situation in which not all of the job outputs are in-line outputs, the system enables a client to directly fetch the output data through a cloud storage system (e.g., a cloud object storage system). For the larger outputs, this enables avoiding the bottleneck of the control system. For the case in which all of the outputs for the job are designated as cloud outputs (e.g., that all the output files are large), then the efficiency of the direct fetching for the client is achieved. For the case in which some of the outputs are in-line outputs and some are cloud outputs, efficiency can be improved by gathering the in-line outputs into a cloud output file and adding that to the cloud storage system to be accessed and managed by the metadata as a larger unit. In some embodiments, the in-line outputs are gathered into a plurality of cloud output file. In some embodiments, the in-line outputs are compacted into one or more cloud output files.
In some embodiments, metadata regarding the output data for job output data is generated. For example, for a cloud output file returned from a job executed by an executor, a metadata entry is stored. In various embodiments, the metadata entry includes a client request identifier, a job identifier, a mapping information regarding how the cloud output is part of the total output responding to the client request (e.g., which columns of an output table the file stores, which rows of an output table the file stores, which table or area in a table the file stores, etc.), a creation date (e.g., to be used for determining a garbage collection date), a storage location on a cloud storage system, encryption key for decrypting the file, or any other appropriate information. In some embodiments, the metadata includes an indication of whether the file is compressed or not compressed. In some embodiments, the metadata includes a file size in bytes so that the client knows how to allocate buffers to load the data. In some embodiments, the metadata, a portion of the metadata, or data derived from the metadata is returned to the client that submitted the client request. The metadata, the portion of the metadata, or the data derived from the metadata is then used by client to request a transfer of the cloud output file(s) from the cloud storage system (e.g., by providing a request to transfer output data associated with a client request identifier). The cloud storage system responds to the request by providing pointers that can be used to transfer one or more of the cloud output files. For example, the cloud storage system provides a pointer (e.g., a URL with security token) to transfer a batch of cloud output files. In some embodiments, the cloud storage system sets a timer to monitor the time it takes to transfer the batch of cloud output files. In response to the batch transfer taking too long of a time (e.g., more than a threshold amount of time), the cloud storage system makes a next batch of cloud output files smaller (e.g., a percentage times the prior batch—for example, 50%, 30%, 25%, etc.—of the prior batch). In response to the batch transfer taking too short of a time (e.g., less than a threshold amount of time), the cloud storage system makes a next batch of cloud output files larger (e.g., a percentage—for example, 200%, 300%, 400%, etc.—of the prior batch). In some embodiments, the transfer of batches proceeds until all the cloud output files associated with the client request are transferred. In some embodiments, the cloud output files transferred include a cloud output file that is a group of in-line outputs that have been aggregated to form a cloud output file.
In some embodiments, the client system receives the metadata, the portion of the metadata, or the data derived from the metadata related to the client request, requests the transfer, and then assembles the transferred files into appropriate data structures. In various embodiments, the data structures comprise tables, summary results, data displayable as graphics or dashboards, models, or any other appropriate request result data structures.
In some embodiments, the system improves the computer by avoiding bottlenecking of cloud computation systems when transferring output data back to a requester. The system enables direct access to larger files from a cloud storage system. The system avoids inefficiencies associated with small output files by transferring a group of aggregated small output files or by directly transferring these smaller files back to a requestor.
In some embodiments, coordinating node 108 provides metadata, portion of metadata, or information derived from metadata and in-line outputs (if appropriate—for example, when only in-line outputs are generated in response to the request) to client system 102 via control plane system 106. Client system 102, if the outputs are cloud based, uses metadata, portion of metadata, or information derived from metadata to request the output data from cloud storage system 118. Cloud storage system 118 receives the request and provides a link for accessing a batch of the files. Client system 102 uses the link to transfer the batch of files. In some embodiments, cloud storage system 118 tracks the time it takes to transfer the batch of files. In the case that the time is too long (e.g., greater than a threshold time), cloud storage system 118 will create a next batch of files that is smaller than the prior batch of files. In the case that the time is too short (e.g., less than a threshold time), cloud storage system 118 will create a next batch of files that is larger than the prior batch of files.
In some embodiments, client system 102 receives the in-line files or the batches of files and then reconstructs the output response using the metadata. Client system 102 is then able to provide data to client requestor.
In some embodiments, processor 208 includes submission engine 210 and results engine 212. Submission engine 210 receives a client request and processes the client request to prepare for submission to a cloud computing system. In some embodiments, the submission steps to a cloud computing system are the steps needed for executing the query on a runtime system. The steps are as follows: 1) submission engine 210 determines whether the request can be handled using cloud storage and if the results can be compressed; submission engine 210 also applies a heuristic to blacklist queries that are likely to return with small results so it can bail out early from uploading; 2) submission engine 210 generates a query execution plan that includes the result serialization method (e.g., Arrow), compression library (e.g., LZ4), and a utility class for generating presigned URLs; the result serialization method, compression library, and utility class are used by tasks to transform the raw rows generated from computation to the appropriate cloud format and to upload them; 3) submission engine 210 creates a callback function that is used to collect results from the tasks; once results are collected, submission engine 210 resolves the result format to cloud or in-line. In some embodiments, the submission is provided to a control plane system. Results engine 212 receives information responsive to a request. Results engine 212 receives in-line output(s) and metadata information or receives metadata information on how to retrieve cloud output(s). To retrieve cloud output(s), results engine 212 makes a request to cloud storage system based on metadata information and receives batches of files. Results engine 212 constructs output responsive to request based on received in-line output(s) or received batches of files that comprise the cloud output(s). In some embodiments, some of the batches of files include in-line output that has been aggregated to create a cloud output. In some embodiments, results engine 212 generated output is provided to a client via user interface 206.
In some embodiments, storage 214 includes request storage 216 and results storage 218. Request storage 216 stores data and instructions related to one or more client requests for processing. In various embodiments, one or more of input data, submission data, instructions, or any other appropriate request related data are stored related to a request. Results storage 218 stores data and metadata related to one or more requests results. In various embodiments, one or more of in-line outputs, cloud outputs, metadata information are stored related responsive to a request, or any other appropriate result data.
In some embodiments, processor 228 includes coordinator job handler 230, metadata handler 232, and output handler 231. Coordinator job handler 230 processes of a client request and creates a submission for the request to a coordinating node of a cloud computation system (e.g., a cluster system). Metadata handler 232 processes a received metadata and provides the appropriate metadata to the client in order for output data to be understood. In various embodiments, metadata includes information related to the structure of the output files (e.g., in-line outputs, cloud outputs, relation of the outputs to the request, data location, job information, etc.), the location of the output files (e.g., cloud storage system location), or any other appropriate information. Output handler 231 passes on in-line outputs (e.g., if all in-line outputs for a response to the request) to a client system.
In some embodiments, storage 234 includes job storage 236 and metadata storage 238. Job storage 236 stores data and instructions related to a client request. Metadata storage 238 stores metadata information related to responses to the request. For example, metadata storage 238 includes storage of job identifier(s), request identifier, storage location, mapping of outputs to response data structures, etc.
In some embodiments, processor 248 includes executor job determiner 250, metadata generator 252, and output consolidator 251. Executor job determiner 250 determines smaller jobs for an executor to process the entire job responsive to the client request. Executor job determiner 250 also determines output type for each job and therefore for the client request. For example, the jobs for the client request are 1) all in-line output type, 2) a mix of in-line output type and cloud type, or 3) all cloud output type. In response to the jobs being a mix of in-line output type, the in-line outputs will be aggregated, using output consolidator 251 to create a cloud output for the in-line parts and stored along with the cloud outputs for direct fetching of the outputs to a client system. Metadata generator 252 generates metadata information related to the outputs from executors. For example, metadata information includes information related to the structure of the outputs and their relations (e.g., the type of data, the relationship of outputs to each other—for example, portions of a table, columns of a table, rows of a table, portions of a model, etc.). In various embodiments, metadata information includes identifiers (e.g., request identifier, job identifiers, client identifier, cloud storage identifier, executor identifier, etc.), data locations (e.g., cloud storage system locations, file names, directories, etc.), labels (e.g., table names, row names, column names, model names, etc.), encryption key information (e.g., key value, key location, etc.), or any other appropriate information. In some embodiments, the metadata that is associated with a task result includes a sequence of result information. It is a sequence because a single task may generate multiple Arrow batches (e.g., the in-lined result format that serializes multiple result rows) or multiple Cloud Files. In some embodiments, the Arrow batch result includes one or more of the following metadata: number of rows, size in bytes (compressed and/or uncompressed), array [byte]—the actual result. In some embodiments, the cloud file results include one or more of the following metadata: number of rows, size in bytes (compressed and/or uncompressed), upload count (e.g., an internally used count to detect whether some of the upload attempts have failed), a URL link. In some embodiments, the cloud file includes metadata (e.g., a result schema that can be used to deserialize the result by the client).
In some embodiments, storage 254 includes job storage 256 and metadata storage 258. Job storage 256 stores information related to jobs for processing a client request (e.g., input data, instructions, code, job data, job breakdowns, executor assignments, output types, etc.). Metadata storage 258 stores information related to outputs received in response to the executor jobs (e.g., in-line outputs, cloud outputs prior to transfer to cloud storage system, etc.).
In some embodiments, processor 268 includes storage engine 270 and retrieval engine 272. Storage engine 270 stores cloud outputs or aggregated in-line outputs and provides storage locations and pointer/indexing information back to coordinating node so that this information can be included in metadata associated with the cloud outputs or aggregated in-line outputs. Retrieval engine 272 provides batches of files in response to a request to fetch data from a client system. Retrieval engine 272 manages the creation of batches to not overwhelm the transfer of the files to the client system (e.g., large batches for fast transfers, smaller batches for slow transfers, etc.).
In some embodiments, storage 274 includes in-line results storage 276 and results storage 278. In-line results storage 276 stores aggregated in-line outputs. Results storage 278 stores cloud outputs.
In some embodiments, a query comprises a client request that needs to be executed. In some embodiments, a job comprises the execution plan of the query that comprises multiple stages each of which computes an intermediate result. In some embodiments, a task comprises an atomic unit of execution. In some embodiments, multiple identical tasks (e.g., a plurality of tasks where each task performs the same computation) form a stage.
In some embodiments, client system-cloud storage system interaction for transferring output files comprises:
The results may be returned in compressed format with LZA. The results are homogeneous, either all compressed or uncompressed, as it will be indicated by the TGetResultSetMetadataResp response.
In some embodiments, if one of the download requests fails from the current batch of cloud outputs <b0, b1, b2, b3, . . . >, for example b1, the client system requests refreshes using the startRowOffset field; the server regenerates URLs for batches b1, b2, b3 . . . and returns them to the client system; 2) the client system uses the refreshed URLs to download the missing outputs; 3) the client system requests new URL refreshes for a number of times each time waiting an exponentially increasing amount of time before retrying the request; and 4) the client system also uses a straggler mitigation strategy to identify slow downloads, that is a download that makes slower progress than the median download; in that case, the client system proactively cancels the download request and retries it, in this case without regenerating the URL if it has not expired yet.
Although the foregoing embodiments have been described in some detail for purposes of clarity of understanding, the invention is not limited to the details provided. There are many alternative ways of implementing the invention. The disclosed embodiments are illustrative and not restrictive.
This application is a continuation of prior, co-pending U.S. patent application Ser. No. 17/841,946, filed Jun. 16, 2022, which is incorporated herein in its entirety for all purposes.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 17841946 | Jun 2022 | US |
| Child | 18614380 | US |
