NSF Award
2418638
Civil and construction engineering students may encounter various construction components, including structural elements, materials, equipment, and operations, in their everyday lives, such as when walking in an urban environment. These unstructured observations can offer great learning opportunities; however, without expert support, it is unlikely that students will effectively learn from such observations on their own. If educators were physically available during students' everyday activities, they could direct students’ attention to the main construction components and explain their observations in real-time. However, since this is not feasible in the real world, this project aims to design, develop, and test a transformative learning system that uses Artificial Intelligence (AI) as an on-demand educator. The envisioned AI-enhanced learning system relies on a digital platform in the form of a mobile application. When students face a construction project, they can look at the project through their smartphones using the mobile app, which will help students learn from their observations by 1) directing their attention to the main construction components they encounter in their everyday life or formal site visits, 2) explaining the observations, 3) linking the observations to students’ formal engineering education materials available on web-based learning management systems, and 4) generating automated reports about students’ observations and performance for instructors to help them adjust the course activities accordingly. To promote equity and accessibility in education, the mobile app will be designed to operate on the most basic and affordable smartphones and will use color palettes compatible with the needs of users with color vision deficiency (CVD), along with subtitles and audio narrations.<br/><br/>The envisioned AI-enhanced learning system will be designed based on the Activity Learning Theory, which asserts that the human mind is an integral part of environmental interactions and positions activity—whether sensory, mental, or physical—as a precursor to learning. The AI-enhanced platform will be designed based on human-centered principles and will operate using a novel hybrid image-audio processing system that can efficiently and effectively recognize and classify various construction components. In addition to integrating imagery and audio data through this novel hybrid approach, the project will introduce two major technological innovations in audio processing and sound recognition. First, the hybrid use of collected audio and imagery data will improve the overall performance of the system by capturing a more comprehensive range of construction components and operations. Second, by using innovative audio processing and signal source separation algorithms, the need for multiple microphones will be eliminated, enabling the entire system to be encapsulated in a single device (i.e., a student’s smartphone) with the ability to sense and analyze audio signals from distances of up to 100 feet. Throughout this project, the proposed AI-enhanced teaching and learning approach will be implemented in multiple undergraduate construction engineering courses to empirically evaluate its effectiveness on students’ learning processes and outcomes, as well as the perceptions of both students and educators regarding this innovation as a formal pedagogical method. Although the AI-enhanced learning platform will be developed in the context of construction engineering, the proposed learning method and the intellectual merit of this project can be transferred to other disciplines. This project will also assess the broader applicability of the proposed innovation.<br/><br/>This project is funded by the Research on Innovative Technologies for Enhanced Learning (RITEL) program that supports early-stage exploratory research in emerging technologies for teaching and learning.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
