The broader impact/commercial potential of this Small Business Innovation Research (SBIR) Phase II project results from the fact that it will unlock the potential of autonomous vehicles in dense urban environments in such a way that these vehicles will be safe, effective, and able to operate in environments with a wide range of often-vulnerable road users. By providing a system for autonomous vehicles to understand the goals and behaviors of humans on the road, the technology will allow autonomous vehicles to react to humans safely and effectively. Without the innovations commercialized with the help of this award, autonomous vehicles will be at best uselessly timid and dangerous additions to urban roads, and at worst a deadly obstacle to the goal of safe, livable cities. Every autonomous vehicle that is capable enough to be on the market will need to solve the problem that this project is helping to solve.<br/> <br/>This Small Business Innovation Research (SBIR) Phase II project will help to address one of the thorniest problems in autonomous vehicles. The question of how a computer system can gain an understanding of human mental states has occupied researchers and laypeople with an interest in machine intelligence since the coining of the term. By building an approach based on the leveraging of careful human measurement and state-of-the-art learning algorithms, the innovations developed in this project will help pave a new path towards the computational modeling of cognitive facilities that are central to human intelligence but historically intractable to model using conventional machine learning or computer vision techniques. In addition to helping solve the central problem of human understanding for autonomous vehicles, the research published from this project will open new avenues for understanding a broad class of problems where the question of "ground truth" about the world is difficult or impossible to answer.