NSF Award
2402875
Rapid progress in artificial intelligence in recent years has been attributed to dramatic increases in the scale of deep neural networks (DNNs): DNNs have more parameters than ever before and are being trained on large proportions of the internet to acquire impressive, almost human-like visual abilities. However, even the largest, highest performing DNNs can fail in strange, inscrutable, and surprisingly “unhuman-like” ways, and this misalignment between humans and DNNs is worsening as DNNs grow in scale. This project aims to rectify the growing misalignment problem by designing the next generation of human-aligned DNNs capable of mimicking human behavior by relying on the computational, algorithmic, and representational principles that shape natural intelligence. The project will result in algorithms that behave like the human visual system, which are broadly applicable across computer vision, cognitive science, and neuroscience.<br/><br/>This project combines large-scale visual psychophysics to characterize human visual strategies and identify the computational principles underlying object recognition in humans with machine-learning methods to translate these insights into algorithms for aligning DNNs with humans. Data from human studies will first be combined with the recently developed “neural harmonizer” training algorithm to generate a large “zoo” of human-vision-aligned versions of publicly available DNNs. A differential analysis of neural circuits and representations in aligned versus standard DNNs will explain why today’s approaches to deep learning are misaligned with human brains. Finally, this project will build on the insights gained regarding the misalignment of DNNs and humans towards reverse engineering the data diets and objective functions needed to align DNNs with human vision from the outset. The proposed work will lead to significant advances in understanding of the perceptual and computational principles underlying human vision and the development of mathematical theories, computational tools, and learning approaches needed to inculcate artificial systems with those same principles.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
