NSF Award
2418872
This research project will advance the methodology for and practical implementation of adaptive experiments. Adaptive experiments dynamically update treatment assignment procedures based on observed responses. They offer significant advantages over conventional randomized control trials for addressing decision-oriented questions such as identifying the best treatment among alternatives or personalizing interventions for different populations. However, their uptake in research and decision making settings has been limited. Methods for adaptive experimental designs to date have not served researchers who wish to understand how to best design an adaptive experiment that adequately addresses concerns around estimation and hypothesis testing. This project will develop methods and a framework for the use of adaptive experiments by applied researchers. Undergraduate and graduate students will be involved in the conduct of the project and user-friendly software will be developed. By providing tools and a framework that improves experimental efficiency and ethical standards, the project will facilitate more effective and informed decision making in social science research.<br/><br/>This research project will address methodological gaps for adaptive experiments through three primary contributions: advancing statistical methodology for experiment design, establishing a framework for adaptive experiment design, and developing software for complex experimentation. In terms of statistical methodology, the project will develop new methods for sample size calculations in adaptive setting and develop a heuristic algorithm for optimal treatment assignment using inverse probability weighted estimators. The project also will document a comprehensive framework for adaptive experiment design. Design decisions such as frequency of algorithm updates and probability floors will have a large impact on the eventual assignment of the adaptive algorithm and on the performance of estimators on the data ex-post. The goal of this framework will be to help applied researchers navigate these decisions. Finally, the project will support the development of user-friendly software to implement complex randomization procedures online, with a focus on accessibility for non-technical users and integration with online field experiments.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
