NSF Award
2025216
Representational competence is the ability to use drawings, diagrams, and symbols to learn, communicate, and solve problems in a specific area, such as organic chemistry. Studies have shown that undergraduate students’ ability to reason with specific representations is important for their understanding of and success in organic chemistry. However, no validated assessment instruments currently exist that measure students’ representational competence in organic chemistry. With support from NSF’s EHR Core Research Building Capacity in STEM Education Research (ECR: BCSER) program, this project seeks to develop such an instrument. In the process of conducting this project, the principal investigator will develop competencies in educational testing and psychometrics. These new competencies will complement existing expertise in qualitative investigation of students’ understanding of chemistry concepts and characterization of instructors’ teaching beliefs and classroom practices. The high-quality assessment instrument developed by the project team is expected to be useful for further research on organic chemistry learning, as well as for helping college chemistry instructors make informed decisions about their teaching.<br/><br/>This study will use a combination of classical test theory and factor analysis methods to develop and test the Organic-chemistry Representational Competence Assessment (ORCA). ORCA will measure student representational competence skills in the context of organic chemistry representations such as dash-wedge diagrams, Newman projections, and chair conformations. The targeted skills to be measured by ORCA include: the ability to (a) interpret these representations, (b) select an appropriate representation for a particular purpose, (c) make connections across these representations, and (d) use these representations to make predictions about chemical phenomena. The measurement quality framework, which targets select psychometric evidence informed by the Standards for Educational and Psychological Testing, will be used as the conceptual and methodological framework for evaluating the quality of ORCA’s items and its capacity to assist in drawing valid and reliable inferences about student representational competence skills. The instrument will be rigorously evaluated and tested with organic chemistry students from the University of North Carolina at Greensboro and the University of South Florida. The principal investigator will be mentored by an expert in the area of assessment of content in undergraduate chemistry courses, and will complete coursework to develop proficiency in psychometrics and measurement. The success of the project and achievement of project objectives will be assessed through periodic meetings with both the mentor and the advisory board. Dissemination of project work will be facilitated via publications in academic journals, presentations at relevant conferences, and workshops on assessment measurement design and evaluation. Results of this study will inform instructional practices and evidence-based design of organic chemistry curricula that supports the use of effective strategies to develop student representational competence. This project is supported through the EHR Core Research: Building Capacity in STEM Education Research (ECR: BCSER) competition that is designed to build individuals’ capacity to carry out high-quality, fundamental STEM education research in STEM learning and learning environments, broadening participation in STEM fields, and STEM workforce development.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
