NSF Award
2423725
An important problem in human-robot interaction with multiple robots is the ability to cooperatively manipulate objects while navigating complex environments. This scenario involves a collaboration between a human leader and a group of teleoperated and semi-autonomous agile-legged robots (i.e., synthetic actors) working together to manipulate and transport complex objects. This research project aims to enable these human-robot teams to achieve intelligent and robust cooperative manipulation and transportation in challenging settings like factories, homes, and offices. The project fosters a bidirectional sensorimotor interaction in which haptic gloves convey force feedback experienced by the robotic agents to the human, while electroencephalogram (EEG), electromyography (EMG), and hand position signals from the human are transferred to the robots' control algorithms for embodied reasoning. The project's overarching research goal is to establish a formal foundation to deploy a distributed planning and control approach for co-learning and co-adaptation of human-robot interaction with multiple robot agents in cooperative loco-manipulation. This work will have important societal impacts by deploying semi-autonomous legged robots that can effectively work together with humans to accomplish labor-intensive tasks, such as assembly and manufacturing. The developed co-learning and co-adaptation algorithms will allow teams of humans and synthetic actors to manipulate and transport heavy objects in challenging environments. Moreover, incorporating EEG and EMG into the researched data-driven models could lay the groundwork for developing multi-agent legged assistive devices to allow paralyzed individuals to perform daily activities. The integrated educational plan will have a profound impact by 1) creating hands-on educational activities on robot locomotion and programming for K-12 students and underrepresented minorities and 2) sponsoring senior design projects for undergraduate teams to be involved in research and experiments. <br/> <br/>The research project will advance knowledge in the largely unexplored field of formation control and embodied reasoning (planning and control) of complex models of loco-manipulation in multi-agent human-robot interaction systems through four objectives: 1) Creation of data-driven dynamic prediction models for human intention based on deep learning techniques; 2) Creation of data-driven dynamical models for the complex network of multi-agent legged robots and the human operator (Co-learning); 3) Creation of human dynamics-aware and distributed data-driven predictive control algorithms for optimal control of the network of synthetic actors with the human in the loop for loco-manipulation (Co-adaptation); and 4) Experimental validation on a team of advanced quadrupedal robots for cooperative loco-manipulation and transportation tasks in the Principal Investigator's laboratory.<br/>This award has been co-funded by the Mind, Machine and Motor Nexus program and the Dynamics, Controls and System Diagnostics programs.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
