NSF Award
2315532
This research and development (R&D) project aims to serve the national interest by advancing the development of an innovative tool and approach to document and assess capacity building in science, technology, engineering, and mathematics (STEM) programs in higher education. The Capacity Assessment, Tracking, and Enhancement through Network Analysis (CATENA) tool is based on a systems approach to understand complex organizations. The Five Capabilities Model of Capacity provides a framework for understanding capacity building that informs assumptions, conceptualization of metrics/measures, research questions, data collection, data analysis, interpretation, implementation, and use of findings. The dimensions of the Five Capacities Model include the capabilities of system actors to (a) commit and engage; (b) carry out tasks and produce results; (c) relate and attract support; (d) adapt and self-renew; and (e) balance diversity with coherence. The R&D effort integrates Social Network Analysis (SNA) with additional qualitative information in an effort to measure the capacity of an educational system for transformation. The mixed methods study is designed to yield the following outcomes: (1) An understanding of facets of capacity that are manifest in interactions and network features within education; (2) an instrument to capture these facets of capacity; and (3) a well-documented SNA data collection and analysis process rooted in graph theory to capture emergent system features. The project is designed around a three-phase research and development plan: The purpose of Phase 1 is to define salient interactions among key actors involved in advancing STEM educational capacity. Phase Two is designed to incorporate findings from Phase 1 into survey development. During Phase 3, the project team will pilot the survey across a College of Engineering and develop supporting documentation for its use. Evaluation and dissemination activities are supported by an external evaluation team and an external advisory board of experts in institutional change, SNA, and qualitative methods.<br/><br/>The goal of the project is to address a national need for methodologies to evaluate and inform systemic, institutional change in a way that foregrounds continuous capacity building for long-term, sustainable change in higher education. In accordance with the overarching goal, the project will investigate three research questions across the three sequential phases of the study: (RQ1) What types of interactions among faculty, staff, and graduate students advance STEM education capacity among each of the capabilities defined by the Five Capabilities Model? (RQ2) How can each of the types of interaction observed through answering RQ1 be measured quantitatively through the design of the CATENA instrument? (RQ3) What network topologies emerge as significant for evaluating STEM education capacity in the networks created from piloting the CATENA instrument created in answering RQ2? The R&D effort integrates quantitative and qualitative data from a broad range of methodologies, such as social network analysis, surveys, and interviews to develop the CATENA tool and then to study capacity building. Examples of metrics include clustering coefficients, average path length, betweenness centrality, and degree distribution. The NSF IUSE: EDU Program supports research and development projects to improve the effectiveness of STEM education for all students. Through its Institutional and Community Transformation track, the project supports efforts to transform and improve STEM education across institutions of higher education and disciplinary communities.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
