NSF Award
2016849
The Automated Collaboration Assessment Using Behavioral Analytics project will<br/>measure and support collaboration as students engage in STEM learning activities.<br/>Collaboration promotes clarifications of misconceptions and deeper understanding of concepts<br/>in STEM which prepares students for future employment in STEM and beyond. This project<br/>aligns with the goal of the Cyberlearning for Work at the Human-Technology Frontier program to<br/>fund exploratory research that supports learners in working productively in technology-rich<br/>STEM environments. Collaboration is an important learning skill in K-12 STEM education, yet teachers have few<br/>consistent ways to measure and support students’ development in this area. This project will<br/>result in both an improved understanding of productive collaboration and a prototype<br/>instructional tool that can help teachers identify nonverbal behaviors and assess overall<br/>collaboration and engagement quality. Using nonverbal behaviors to assess engagement will<br/>decrease dependence on discourse and content-based dialogue and increase the transferability<br/>of this work into different domains. This project is particularly timely as the ability to collaborate<br/>and engage in group work are growing requirements in professional and learning settings; at the<br/>same time the very act of collaboration is being disrupted by the Coronavirus pandemic and<br/>there is a high likelihood that much of this “new normal” (social distancing; combining in-person<br/>and remote collaboration) will be with us for some time. This project will meet the urgent need<br/>currently felt by educators and educational institutions to support the development of<br/>collaboration skills among students, even as the very act of collaboration is shifting and nontraditional<br/>forms of education are taking hold.<br/><br/>This project is a collaboration between the Center for Education Research and<br/>Innovation (CERI) and Center for Vision Technology (CVT) at SRI International (SRI) and will<br/>capture multiple students’ actions as they work collaboratively face-to-face, both in-person and<br/>through a virtual platform. This project will use a collaboration conceptual<br/>model, multistage predictive and explainable machine learning models, and video analytics to<br/>assess and report on collaborative behaviors and interactions. The behavior analytics system<br/>will use facial expressions, body movements, and meta-information about the collaboration task<br/>to identify interactions that show how students contribute to the collaboration, individually and<br/>collectively. This 2-year project will use reliability and model prediction testing and sequential,<br/>correlation, and thematic analyses of video recordings, surveys, interviews, and student artifacts<br/>to answer the following research questions: Can machine learning models reliably assess<br/>collaboration when compared to human assessments? How do individual behaviors during<br/>collaboration lead and relate to group level interactions and collaboration quality? and Can we<br/>validate and relate the assessed collaboration behaviors to student outcomes as represented by<br/>group-generated artifacts? The intellectual merits include contributions to the advancement of<br/>two fields: (1) machine learning— by developing and exploring new algorithms that generate<br/>explainable collaboration skill assessments and teacher/student dashboards at different grain<br/>sizes of the interactions, and (2) learning sciences—by contributing a collaboration conceptual<br/>model that shows how specific skills, interactions, and behaviors correspond to collaboration<br/>quality at group and individual levels. Broader impacts of this work include increasing the<br/>availability and types of feedback presented to instructors and learners from diverse<br/>backgrounds. This will expand the settings and number of individuals who can be evaluated and<br/>supported on collaboration by making collaborative learning easier to monitor through tools that<br/>can be used by a wide audience of educators and professionals.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
