Methods exist for forecasting atmospheric winds using machine learning techniques, including algorithms that leverage deep neural networks. In addition, analog-based methods for forecasting winds have been explored for decades (Lorenz, 1969) to develop prediction methods for a range of weather parameters. Analog-based methods compare a current forecast of a region of the atmosphere with a repository of historical forecasts of the region of the atmosphere to determine the most similar scenario in the past—an analog.
Analog ensemble (AnEn) techniques have been applied for the prediction of weather parameters, tropical cyclone intensity, air quality, and renewable energy. In conventional AnEn algorithms, the AnEn constructs a probability distribution of a forecast parameter such as wind speed or heading, given a forecast, previous forecasts made by the same model, and corresponding ground truth for (i.e., observations corresponding to) those previous forecasts. However, finding and verifying analogs in a large corpus of historical data takes a lot of compute resources, and often requires an amount of post-processing time that is not practical for use in an operational system (e.g., planning for a fleet of aerial vehicles in real time).
Thus, a solution for generating more accurate and granular atmospheric wind forecasts in an operational time frame is desirable.
The present disclosure provides for techniques relating to a distributed computing system and method for generating atmospheric wind forecasts. A distributed computing system for wind forecasting in a region of the atmosphere may include an analog ensemble distilling architecture comprising: a learner configured to train a deep neural network using analog ensemble data and output a distilled analog ensemble, a reservoir comprising a cache, a builder comprising a plurality of jobs configured to sample a plurality of slices of an analog ensemble function, a corpus; a processor configured to: apply an analog ensemble operator, generate overlapping forecast output files, and generate wind forecasts; and a memory configured to store one or more components of the analog ensemble distilling architecture, wherein the distilled analog ensemble is configured to output an improved wind forecast. In some examples, the system also includes a metalearner configured to vary a learning parameter. In some examples, the learning parameter comprises a learning rate. In some examples, the learning parameter comprises a batch size. In some examples, the cache is an in-memory cache. In some examples, the cache is distributed over a plurality of jobs. In some examples, the corpus comprises a plurality of key-value pairs pairing a forecast with a ground truth observation. In some examples, each of the plurality of key-value pairs indicates a latitude and a longitude. In some examples, each of the plurality of key-value pairs indicates an altitude. In some examples, each of the plurality of key-value pairs indicates a lead time. In some examples, the improved wind forecast is deterministic. In some examples, the improved wind forecast is probabilistic. In some examples, the probabilistic improved wind forecast comprises a quantification of the ensemble mean uncertainty.
A method for distilling an analog ensemble may include receiving weather forecast data and weather observation data for a region of the atmosphere; generating, by a distributed computing system, a corpus comprising a forecast-observation key-value pair for each point on a grid map, each point indication a location on the grid map; generating an in-memory cache comprising an evolving plurality of training examples, each training example comprising a slice of an analog ensemble; training a deep neural network using the evolving plurality of training examples; and outputting a distilled analog ensemble. In some examples, the method also includes saving the corpus as a data file. In some examples, generating the corpus comprises generating a set of forecast outputs comprising a plurality of forecast outputs that are overlapping in time, the set of forecast outputs based on the weather forecast data. In some examples, the corpus represents historical forecast and observation data spanning at least two years.
A method for generating an improved wind forecast in a region of the atmosphere may include receiving a forecast comprising a wind vector for a plurality of grid points on a grid map; applying a distilled analog ensemble at all grid points of the forecast; and outputting an improved forecast. In some examples, the distilled analog ensemble uses inputs of one, or a combination of two or more, of a latitude, a longitude, a pressure, and a lead time. In some examples, the improved forecast has a lower error rate than the forecast. In some examples, the improved forecast provides an uncertainty quantification.
Various non-limiting and non-exhaustive aspects and features of the present disclosure are described hereinbelow with references to the drawings, wherein:
Like reference numbers and designations in the various drawings indicate like elements. Skilled artisans will appreciate that elements in the Figures are illustrated for simplicity and clarity, and have not necessarily been drawn to scale, for example, with the dimensions of some of the elements in the figures exaggerated relative to other elements to help to improve understanding of various embodiments. Common, well-understood elements that are useful or necessary in a commercially feasible embodiment are often not depicted in order to facilitate a less obstructed view of these various embodiments.
The Figures and the following description describe certain embodiments by way of illustration only. One of ordinary skill in the art will readily recognize from the following description that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles described herein. Reference will now be made in detail to several embodiments, examples of which are illustrated in the accompanying figures.
The above and other needs are met by the disclosed methods, a non-transitory computer-readable storage medium storing executable code, and systems for dispatching fleets of aircraft by a fleet management and flight planning system. The terms “aerial vehicle” and “aircraft” are used interchangeably herein to refer to any type of vehicle capable of aerial movement, including, without limitation, High Altitude Platforms (HAPs), High Altitude Long Endurance (HALE) aircraft, unmanned aerial vehicles (UAVs), passive lighter than air vehicles (e.g., floating stratospheric balloons, other floating or wind-driving vehicles), powered lighter than air vehicles (e.g., balloons and airships with some propulsion capabilities), fixed-wing vehicles (e.g., drones, rigid kites, gliders), various types of satellites, and other high altitude aerial vehicles.
The invention is directed to a distributed computing system and efficient, scalable method for generating accurate atmospheric wind forecasts. Forecasts of weather, winds, and other forecasts (e.g., of stochastic phenomena) in one or more regions, layers, or sub-layers (collectively “regions”) of the atmosphere (e.g., surface, troposphere, lower stratosphere, stratosphere, mesosophere, etc.) may be improved using machine learning to post-process forecast outputs (e.g., numerical weather model outputs (i.e., weather forecast data), and other forecast model outputs). An analog ensemble (AnEn) algorithm may be applied to forecast data (e.g., weather forecast data), including partitioning the forecast data and executing an independent computation for each grid point and forecast lead time using analogs comprising ground truth data (e.g., weather observation data including wind data, or other historical data against which forecast data may be compared). The post-processing of the forecast data using the AnEn algorithm and analogs may produce a more granular and accurate wind forecast (e.g., a reduction of forecast error) and a quantification of uncertainty (e.g., turning a single forecast into an ensemble forecast with appropriate probability distribution characteristics).
To perform the AnEn algorithm on a global scale, the computations may be performed over a distributed computing system (e.g., across many datacenter computers, and even across tens, hundreds, and thousands, of datacenters), wherein a 3D forecast with a plurality of pressure levels (i.e., corresponding to a plurality of altitudes) over a plurality of lead times may be generated in a matter of minutes (e.g., 10-20 minutes, or more or less). In examples described herein, atmospheric pressure may be a proxy for altitude. Such a 3D forecast may be provided at a variety of resolutions (i.e., increments).
An AnEn may be distilled (i.e., memorized or learned) into a deep neural network (DNN) to approximate the application of the AnEn algorithm point-wise, enabling using a new, larger corpus of analogs (e.g., up to and beyond 100s of terabytes of data) to create the distilled AnEn, which effectively enables a larger corpus of forecasts and may improve the AnEn quality without using additional computation and or time in the critical path (i.e., post-processing a forecast) of an operational system, such as a fleet management and/or flight planning system for a fleet of aerial vehicles. This makes the AnEn technique feasible in real operational systems. A DNN may be trained to learn a function mapping wind speed and direction from a current forecast to analogs (e.g., analog ensemble mean, ensemble forecasts). A response curve of an AnEn may be plotted to visualize the effect of the AnEn. Once a DNN is trained, it can be applied to any point (e.g., a place and time) in the weather data.
Analog ensemble algorithms may be used to improve output of existing weather data models (e.g., National Oceanic and Atmospheric Administration's (NOAA's) Global Forecast System (GFS) and the European Center for Medium-Range Weather Forecast's (ECMWF's) high resolution forecasts (HRES)). Using techniques described herein, aggregated forecast error may be reduced significantly (e.g., a range of 2%-11% on stratospheric wind speed and 10%-20% on heading), with improvements growing as lead times increase. Using a distributed system to generate AnEn output for a global forecast based on a few years (e.g., ranging from approximately one to five years) of forecast and weather observation data can be achieved by the techniques described herein in a matter of minutes (e.g., ranging from 5-20 minutes, or more or less). Distilling the AnEn into a DNN enables the system to scale at least to (i.e., post-process) a corpus of tens of years of historical forecasts, which can significantly improve forecast quality, and output improved wind forecast within an operational time frame (e.g., from a few minutes to a few hours). Distilling the AnEn generates a DNN that can implement a pointwise function that approximates the conventional AnEn algorithm. Thus, when a new forecast is received (e.g., a vector of wind speeds or other fields), the DNN may quickly produce an approximate improved forecast for each grid point, enabling a quick look-up of the improved forecast at any given grid point or other given point (i.e., between grid points) with no added latency. Further, using a longer historical forecast to create the pointwise function does not affect the real time operations of the system.
Post-processing an operational forecast takes place in the critical path of providing new operational data to a consumer system, e.g., a high altitude aerial vehicle's navigation system. New improved forecast data may be generated in the operational model described herein periodically (e.g., every few hours, 10 hours, 12 hours, or more or less frequently) or ad hoc (e.g., as desired, globally or for a desired location or region.
DNNs have a high capacity for encoding complex functions. A distilled AnEn as described herein is preferable to directly training a DNN to improve forecasts due to the relatively low amount of training data available for direct training. The AnEn generalizes well, or at minimum, provides an improved wind forecast according to key metrics when deployed to long validation periods on unseen meteorological forecasts. The distilled AnEn bootstraps training the DNN off the AnEn, effectively combining the AnEn's strength of being able to generate forecasts with a relatively small corpus of training examples with the DNNs ability to memorize this complex function in a much smaller number of parameters.
A set of forecast outputs (e.g., output files) based on the weather forecast data may be generated at step 104, each set of forecast outputs comprising a plurality of forecast outputs that are overlapping in time (i.e., the time being forecast).
Returning to
Any number of map and reduce phases may be distributed across a large number of machines in a datacenter, as described herein, allowing the Map-Reduce process to run at scale at appropriate operational speeds. In some examples, one may choose to materialize the corpus periodically to not process the raw model files over and over again, but not necessarily generate a new corpus every time a new model output file group comes in. Effectively this is a hybrid approach where the corpus is periodically materialized and models outputs newer than the last corpus are mapped ephemerally into the pipeline until they are included in a corpus.
In response to receiving new weather forecast data, the corpus may be updated at step 108. In an example, when a new forecast model output file is available for processing, the existing data may be mapped over, and for each point a <lat, lng, pressure, lead time>→<forecasted speed, forecasted heading> emitted. This is joined by key with the corpus and also any side data (e.g., optimal weights for matching), which also may be emitted under the appropriate key.
In
The corpus may be saved as a data file (e.g., a shared dataset) at step 256. An in-memory cache (i.e., reservoir) comprising an evolving plurality of training examples may be generated at step 258, each training example comprising a slice of an AnEn comprising a sample wind speed and heading. A cohort of builder jobs may sample slices of an AnEn function indicating a sample wind speed and wind heading forecast for each latitude, longitude, pressure altitude and lead time. In some examples, a slowly changing reservoir of training examples may utilize a large distributed buffer that is slowly updated over time. A cadre of worker tasks may load and/or replace (i.e., update) the in-memory cache with additional training examples periodically, randomly, or according to other cadences. For example, new training examples may be added to a reservoir, randomly replacing older data in the reservoir (e.g., in a buffer). Using the rate at which we add new data and overall capacity of a buffer, we can control dwell time distribution and diversity of samples. A similar technique to a replay buffer in deep reinforcement learning may be utilized, wherein a slowly changing flow of examples is provided, and each batch (i.e., on average) tends to draw from disparate parts of the function mapping being learned.
The evolving plurality of training examples may be used to train (e.g., be memorized or learned by) a DNN at step 260, for example using a learner, as described herein. A DNN training procedure may be implemented, pulling random batches of training examples from a reservoir. The learner may output a distilled AnEn at step 262.
In some examples, the improved wind forecast output by a distilled AnEn may be visualized with a three-dimensional plot, as shown in
The shape of this cone may change over different lead times and places around the world and encodes the transformation that is used to generate an, on average, improved wind forecast. A top rim of the cone 602 may correspond to a higher wind speed (e.g., 20 m/s or greater) and a bottom tip of the cone 604 may correspond to a lower wind speed (e.g., 5 m/s or less).
In some examples, every AnEn may be simultaneously approximated by using the tuple that specifies the grid point k as an input parameter ydistilled=h(k, xf). In an example, speed mean, direction mean, and the standard deviations of the ensemble forecasts for both quantities may be distilled into a single DNN with multiple outputs.
In
The grid map may be three-dimensional (e.g., including a range of altitudes), and may comprise grid point designations by latitude, longitude, pressure, time, uniquely identified stations or infrastructure, or designations in various fields and dimensions. The weather forecast data may comprise wind forecast data for some or all of the grid map. The distilled AnEn may output an improved wind forecast (e.g., including wind speed and heading) at step 306. The distilled AnEn may take inputs of latitude, longitude, pressure (altitude), and time in the future (i.e., lead time), together with a corresponding forecast wind speed and heading. Given this set of parameters it may generate a new forecast of wind speed and heading that has demonstrably higher forecast accuracy (e.g., lower centered root-mean-square error (CRMSE) with continued high correlation) than the original forecast, along with an uncertainty quantification (e.g., an ensemble forecast or probability distribution over the forecast quantity). The distilled AnEn may be represented in a data file of ˜100 kb, providing far more efficiency and flexibility in the operational post-processing critical path. As described herein, once trained, a distilled AnEn uses a fixed computational complexity even as the training corpus grows, which enables implementing a global AnEn with decades of historical forecasts. Output parameters of the distilled AnEn, whether analog means or probability distribution across a variable, may be functions of latitude, longitude, pressure, and lead time, as well as forecast fields (e.g., wind speed and heading).
Learner 408 may be configured to train a DNN to develop and output a distilled AnEn using examples from reservoir 406. Learner 408 also may be configured to evaluate the quality of a distilled AnEn (e.g., how well it can improve accuracy of wind forecasts), and to save the resulting distilled AnEn (e.g., in a storage or repository, such as storage 720 in
Computing device 701 may further include a display 706, a network interface 708, an input device 710, and/or an output module 712. Display 706 may be any display device by means of which computing device 701 may output and/or display data, such as via GUI 718. Network interface 708 may be configured to connect to a network such as a local area network (LAN) including one or more of a wired network, a wireless network, a wide area network (WAN), a wireless mobile network, a Bluetooth network, a satellite network, and/or the internet. Input device 710 may be a mouse, keyboard, or other hand-held controller, touch screen, voice interface, and/or any other device or interface by means of which a user may interact with computing device 701. Output module 712 may be a bus, port, and/or other interface by means of which computing device 701 may connect to and/or output data to other devices and/or peripherals.
Schematic diagrams of an exemplary operational system in which wind forecasting methods may be implemented are shown in
Sensors 825a-b may include Global Positioning System (GPS) sensors, wind speed and direction sensors such as wind vanes and anemometers, temperature sensors such as thermometers and resistance temperature detectors (i.e., RTDs), speed of sound sensors, acoustic sensors, pressure sensors such as barometers and differential pressure sensors, accelerometers, gyroscopes, combination sensor devices such as inertial measurement units (IMUs), light detectors, light detection and ranging (LIDAR) units, radar units, cameras, other image sensors (e.g., a star tracker), and more. These examples of sensors are not intended to be limiting, and those skilled in the art will appreciate that other sensors or combinations of sensors in addition to these described may be included without departing from the scope of the present disclosure.
Payload 820a-b may comprise a controller, or other computing device or logic circuit configured to control components of the aerial vehicles. In an embodiment, payload 820a-b may be coupled to balloons 810a-b, respectively, using one or more down connects 815a-b. Payload 820a-b may include or be coupled to sensors 825a-b, respectively.
Computing device 850 may be configured to control operations of one or more components of payloads 820a-b and balloons 810a-b. Computing device 850 may be any computing device configurable for use in controlling and/or planning a flight path for aerial vehicles. For example, computing device 850 may comprise a desktop computer, laptop computer, tablet computer, smart phone, server and terminal configuration, datacenter, and/or any other computing device known to those skilled in the art. In one embodiment, computing device 850 comprises a datacenter or other control facility (e.g., configured to run a distributed computing system, as described herein), and may communicate with payloads 820a-b via a network. As described herein, system 800, and particularly computing device 850, may be used for planning a flight path or course for aerial vehicles based on wind forecasts, for example, to navigate aerial vehicles along a desired heading or to a target location.
In some examples, balloons 810a-b may carry and/or include other components that are not shown (e.g., altitude control system, propeller, fin, electrical and other wired connections, avionics chassis, onboard flight computer, ballonet, communications unit including transceivers, gimbals, and parabolic terminals). Those skilled in the art will recognize that the systems and methods disclosed herein may similarly apply and be usable by various other types of aerial vehicles.
While specific examples have been provided above, it is understood that the present invention can be applied with a wide variety of inputs, thresholds, ranges, and other factors, depending on the application. For example, the time frames and ranges provided above are illustrative, but one of ordinary skill in the art would understand that these time frames and ranges may be varied or even be dynamic and variable, depending on the implementation.
As those skilled in the art will understand, a number of variations may be made in the disclosed embodiments, all without departing from the scope of the invention, which is defined solely by the appended claims. It should be noted that although the features and elements are described in particular combinations, each feature or element can be used alone without other features and elements or in various combinations with or without other features and elements. The methods or flow charts provided may be implemented in a computer program, software, or firmware tangibly embodied in a computer-readable storage medium for execution by a general-purpose computer or processor.
Examples of computer-readable storage mediums include a read only memory (ROM), random-access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks.
Suitable processors include, by way of example, a general-purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) circuits, any other type of integrated circuit (IC), a state machine, or any combination of thereof.
This application claims the benefit of U.S. Provisional Patent Application No. 62/891,893 entitled “Distributed computing system and method for generating atmospheric wind forecasts,” filed Aug. 26, 2019, the contents of which are hereby incorporated by reference in their entirety.
Number | Date | Country | |
---|---|---|---|
62891893 | Aug 2019 | US |