NSF Award
2311471
Scientists face a rapidly growing scale and complexity of data. Deep learning has proven to be a tremendously effective way of understanding large data sets, using backpropagation techniques that rely on automatic differentiation (AD) to provide efficient and scalable gradient-based optimization methods. Efforts to generalize deep learning have resulted in the emerging Differentiable Programming (DP) paradigm by incorporating a wide set of numerical computations and allowing the use of general numerical codes in optimization-based pipelines. Tasks such as parameter estimation, inverse problems, and apparatus design are naturally formulated as optimization problems amenable to gradient descent. The DP paradigm enables researchers to leverage domain knowledge while using powerful new techniques to enhance their science. This project will enable the use of differentiable programming techniques in large-scale science. C and C++ are the languages of choice for performance-sensitive scientific computing. However, C++ is currently a major challenge for AD implementations, which do not yet provide good feature coverage and adequate performance. Robustness, performance, native language support, and support for modern hardware architectures are critical for adoption in complex scientific pipelines.<br/><br/>This project will extend the source transformation AD tool “Clad” to provide extensive C++ language support and interoperability capabilities. Deeply integrated into the LLVM compilation toolchain, Clad re-uses the Clang compiler frontend to differentiate C++ constructs, where it has access to the language details to support the necessary C++ features. By using the compiler to differentiate code, Clad provides advanced control over the differentiation process. Clad is already a very capable software for AD research. This project will: (a) extend support for C++ entities, including concurrency primitives to easily take advantage of GPUs while respecting high-level program structure; (b) facilitate AD interoperability and integration with large-scale scientific codes; and (c) ease differentiable programming adoption in science. This proposal will change Clad into sustainable cyberinfrastructure, and will engage scientists from numerous domains to use Clad as part of a DP pipeline to gain new insights from their data.<br/><br/>This award by the Office of Advanced Cyberinfrastructure is jointly supported by the Division of Physics within the Directorate for Mathematical and Physical Sciences.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
