NSF Award
1941178
A key goal of artificial intelligence (AI) is to build systems that can read and understand language as humans do. This capability underlies a broad range of technologies, including question answering, machine translation, and dialogue systems. While progress has been made, AI systems currently lack the robustness and flexibility of human language understanding---typical systems leverage shallow pattern-matching strategies to perform tasks, and as a result are only effective at the specific tasks they are built for, and fail easily even within those settings. This project addresses these issues by improving the ability of systems to construct rich representations of the "world" described in text: Who are the entities involved, and what are their attributes and relationships? What events are taking place, who is participating in those events, and why are they occurring? The design of the systems' notion of a world uses concepts like these that have been identified by cognitive scientists and psychologists as fundamental in human language understanding. The expected benefit of this work is the development of AI systems that can use language flexibly and robustly because, like humans, these systems will perform tasks based on the core information conveyed in language, rather than superficial pattern-matching. In addition to improving systems, this project will have the benefit of building bridges between the AI community and cognitive scientists, psychologists, and linguists---the project's modeling framework provides a pathway through which insights from cognitive science can be translated to model implementation, which can be utilized both for improvement of AI systems and for testing of cognitive hypotheses. <br/><br/>This exploratory EAGER project improves the capacity of systems to automatically construct the world underlying the text being analyzed, and designs targeted probing tasks to enable fine-grained assessment of the extent to which systems have captured this information. The modeling framework uses memory-augmented neural networks, leveraging the external memory components to represent worlds. Rather than explicit annotation, the project implements cognitively-inspired design of both world components themselves and inductive bias for encouraging particular components to capture what is intended. Learning is carried out via self-supervised objectives and auxiliary supervision on large datasets of narratives. System evaluation consists of both standard reading comprehension question answering tasks and the development of novel probing tasks. The use of controlled probing tasks draws critically from methodological approaches used in cognitive neuroscience and psycholinguistics, applying these scientific methods for interpretation of artificial systems. These probing tasks allow for targeted analysis of individual world components and provide guidance for model improvement. The methodology of the project iterates between model design and targeted testing via probing tasks, using the results of the latter to guide the former.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
