NSF Award
2221570
Remote presence systems, consisting of operator interfaces coupled to avatar robots, promise new ways of connecting people across the barrier of distance. For example, the technology will enable physicians to incorporate physical examinations into telehealth, and families to care for elderly relatives as they age in place. Present systems, however, can be slow and frustrating to use due to limited haptic capabilities: operators cannot easily feel remote objects as if they were in their own hands. To address this, a new paradigm in telemanipulation, Shape-Based Remote Manipulation (SBRM), will be developed. SBRM integrates novel multi-finger haptic devices with mathematical models of grasped objects to enable dexterous in-hand telemanipulation. In addition to developing novel technology, this project will also make an educational impact both within and outside the partner universities. The investigators will recruit and train a majority BIPOC (Black, Indigenous, and People of Color) group of graduate students on this research. Additionally, the research team will design and host a tele-touch exhibit to be called “The Shape Beaming Experience” where students at The Challenger Learning Center in Tallahassee, Florida, can physically interact with students at the Museum of Science and Industry in Chicago, Illinois.<br/><br/>The goal of Shape-Based Remote Manipulation (SBRM) is to overcome a fundamental issue in telemanipulation, namely the trade-off between stability/robustness and performance in the face of communication latencies. It accomplishes this through a novel integration of RGB-D data, geometric models, compliant robotic hands and arms, and bilateral control, along with innovations in multi-finger haptic feedback and shape-based controls. The project will be executed in four parts. First, a state-of-the-art remote manipulation testbed will be developed spanning two university campuses nearly 1,000 miles apart. Second, techniques for obtaining semantic, geometric, and physical information about objects in the avatar's environment will be developed. Third, a novel Shape Interface will be created, allowing operators to grasp, feel and manipulate virtual objects. Fourth, a set of shape-based control techniques that coordinate operator and avatar dexterity will be developed. The project will lead to new knowledge about the factors impacting remote in-hand dexterity, especially the role of shape-based haptic information. One aspect of the work will be a novel multi-finger haptic interface that constrains the fingers to extremely rigid virtual surfaces, enabling heretofore infeasible studies of shape information in haptic perception and dexterous manipulation. Another aspect of the work will be a new approach to telemanipulation based on differential geometry that aims to combine the robustness benefits of unilateral control with the performance benefits of bilateral control.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
