NSF Award
2418156
Approximate message passing (AMP) is a family of iterative algorithms that was been widely used to solve linear-regression problems in recent years. AMP algorithms are known to offer optimal solutions in certain situations and are well-understood theoretically under certain asymptotic assumptions, notably when it is assumed that the number of data samples approaches infinity. However, practical settings often operate under a very limited number of known data samples such that asymptotic knowledge no longer applies. Indeed, contemporary data-science applications often call for the ability to extract actionable information from datasets of limited size. AMP algorithms commonly serve as fast-convergent and statistically efficient estimators and can be employed as powerful theoretical machinery to study other statistical procedures in high dimension. However, the asymptotic nature of most of the existing AMP theory limits the capability of fully characterizing the convergence behavior of these algorithms in real-world settings. Accordingly, this project aims to develop a new, non-asymptotic statistical foundation for AMP that allows one to predict its finite-sample dynamics in a fine-grained manner. Broader-impact aspects of the work lie in its potential to substantially enrich the statistical and algorithmic foundations for information processing and machine learning in data-hungry applications and to contribute significantly to various science and engineering applications such as optimal imaging, social-network analysis, single-cell data analysis, and generative modeling. The investigator also plans to create new courses, develop tutorials and short courses in high-profile conferences and workshops, and organize corresponding materials into a monograph.<br/> <br/>The technical activities aim to significantly advance the foundation of AMP by developing a versatile non-asymptotic framework capable of handling a polynomial number of AMP iterations in the most-challenging, sample-limited regime. To achieve this goal, the research program will leverage insights and techniques from various fields, including high-dimensional statistics, statistical signal processing, information theory, convex geometry, random-matrix theory, and statistical physics. To facilitate specific investigations and discussions, the project revolves around several stylized problems, subsuming as special cases structured linear models, spiked low-rank models, tensor learning, and diffusion generative models, among others. It is anticipated that this project will contribute novel insights into this foundational area from both theoretical and algorithmic perspectives.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
