NSF Award
2404989
Machine learning with deep neural networks occurs in every scientific field. In astronomy, machine learning (ML) applications are wide-ranging. Advanced ML methods involve large numbers of tunable variables, in the range of millions to trillions of parameters. Current methods for tuning these parameters can find a single best-fitting model but are unable to produce the range of ML models that match observations. Knowing the range of uncertainties for a given method is of fundamental importance in science. More broadly, there are many areas beyond science where knowing the range of uncertainties is important (e.g., driverless cars, medical, and military applications). This proposal introduces a new algorithm that will explore uncertainties for trillion-dimensional models and beyond (encompassing the dimensionality of current large neural networks). For such trillion-dimensional parameter spaces, the method would be five hundred times faster than the best previous approaches. This method could revolutionize machine learning across scientific and commercial fields. The ability to show that all reasonable neural networks give similar results would directly address present problems of robustness and reproducibility, as well as rigorously quantify model uncertainties. This proposal also involves direct broader impact work with veterans transitioning from the military to college. <br/><br/>This proposal would advance the frontiers of sampling algorithm performance, with typical O(log D) or better scaling with dimensionality, as well as provide new knowledge about the geometry of high-dimensional posterior distributions, including those for deep neural networks. Advancing sampling algorithm performance would allow a broad range of problems to be addressed in astronomy that would otherwise be computationally intractable, especially reconstruction problems (e.g., field-level inference for cosmology or blended source reconstruction) and high-dimensional modeling (e.g., modeling the joint distribution of physical properties of multiple galaxies simultaneously, given their luminosity, spatial, and redshift distributions). Advancing understanding of the geometry of high-dimensional posterior distributions would enable the development of more optimized algorithms for exploration and sampling, particularly for advanced neural networks, further reducing the barriers to robust posterior distributions. The results of this proposal will be released as an open-source implementation of the algorithm as well as an open-access accompanying paper.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
