NSF Award
2209823
People are constantly learning – whether formal education of homework problems & videos, or reading websites like Wikipedia. This project develops the Experiments As a Service Infrastructure (EASI), which lowers the barriers to conducting randomized experiments that compare alternative ways of designing digital learning experiences, as well as analyzing the data derived from the systems to rapidly change what future people receive. It does this by bringing together multidisciplinary researchers around the shared problem of testing ideas for improving and personalizing educational resources. The research also advances (1) the science of learning and instruction; (2) methods for analyzing complex educational data, and (3) machine learning algorithms that use data to improve educational experiences. Improving learning and teaching increases people's knowledge and gives them the ability to solve problems they care about, driving their personal and career success and increasing society's human capital.<br/><br/>Instructional decisions about digital educational resources impact all students, from practice problems in K12 systems to tutorial webpages in university and community college online courses. The current versions of resources are too infrequently compared against alternative resources, which may provide better learning. With this in mind, the project has the goal of using data to test hypotheses about what is most helpful to students, and then use that data to change the experience for future students. The Experiments-As-a-Service-Infrastructure supports three complementary types of multi-disciplinary, collaborative research. A–Design: the infrastructure helps researchers investigate theories of learning and discover how to improve instruction by designing randomized field experiments on components of real-world digital educational resources. This provides more ecologically valid research on learning and instruction, in subfields of education, psychology, policy and discipline-based education research. B–Analysis: the infrastructure facilitates sophisticated analysis of experiments in the context of large-scale data about student profiles, such as to discover which interventions are effective for different subgroups of students. This can advance the use of innovative data-intensive methods for gaining actionable knowledge in education, learning analytics, educational data mining, and applied statistics. C–Adaptation: the infrastructure enables research into adaptive experimentation by providing a testbed for algorithms that dynamically analyze data from experiments, to enhance learning by presenting future students with whichever version of a resource (condition) is more effective, or to personalize learning by presenting different subgroups of future students with the version of a resource that is most effective for their subgroup. The infrastructure provides a testbed for empirical evaluation of which algorithms enact effective adaptive experimentation in education to inspire the development of new algorithms. Finally, the work aligns many educational communities around the shared problem of enhancing and personalizing education through experimentation and spurs multidisciplinary research by providing extensive support for collaboration and sharing of designs, data, analysis scripts and algorithms while fostering an online community for training and collaborations, to promote high-quality, innovative, impactful experiments.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
