NSF Award
2315396
Reinforcement learning is the name of a learning technique used to teach robots new skills. Reinforcement Learning is also used in training the robot behave in certain ways and this training enables important applications from game playing to package handling. Yet, these results seldom lead to real-world large-scale applications. The main challenges stem from the limited abilities of reinforcement learning techniques to efficiently create behaviors that can be used in realistic environments, across objects, obstacles and tasks, while still ensuring operational safety. Optimal control is another method that is used to control systems. Such methods can be very efficient for performing numerical computations but are generally limited to rather narrow behaviors. The project aims at casting a new light on reinforcement learning and optimal control, which share common foundations but until now have failed to produce a single method combining the advantages of both approaches. This research will include the development of new methods to improve learning efficacy and guarantee safety for real physical systems. To demonstrate the broad applicability of the approach, the project will evaluate the methods in four realistic application domains: towing kites for energy supply, robots with arms and legs, avatars and microscopic movement of proteins. This project contributes to the advance of national health, prosperity and welfare by improving the capabilities, reliability and safety of robots in a wide area of applications with high industrial potential. The project will be conducted by a French-US team of researchers which will help train the next generation of the workforce by providing a unique international research experience.<br/><br/>The project is articulated around two main research goals. The first goal is to produce a new reinforcement learning algorithm which better exploits prior model knowledge, in particular model derivatives, to accelerate convergence, guarantee a convergence rate and enforce hard constraints. The second goal aims to efficiently solve a particular class of hard problems, namely problems with hybrid (discrete/continuous) dynamics. The combination of both objectives will result in a common theoretical framework to merge optimal control and reinforcement learning approaches as well as numerically efficient algorithms capable of generically solving complex high-dimensional problems. Finally, the side outcomes of the project will be several demonstrations which have value beyond the science.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
