NSF Award
2412008
This project seeks to understand and improve the ability of neural networks to make accurate predictions on multiple tasks simultaneously. This problem has applications in many areas, such as computer vision, natural language processing, social networks, transportation, and bioinformatics. A key objective is to model the complex relationship between tasks when deploying large neural networks. The modeling of task relationships, together with active measurements of the performances of subsets of tasks, will enable the design of new algorithms for multitask learning. This project will also examine model generalization of fine-tuning and develop new measurement tools to better understand information transfer from multitask neural networks. These new developments will enable applications across multiple disciplines and facilitate the creation of public datasets.<br/><br/>There are three components to this project. The first component develops surrogate models to measure task relationships in multitask learning (MTL). A scalable modeling framework will be developed to select subsets of tasks, by utilizing both statistical and geometric properties of large (pre-trained) models. This framework applies to many settings, such as learning the compositionality of data augmentation, and fine-tuning language models from multiple data sources. The second component examines clustering methods for MTL system design objectives, such as egalitarian criteria. The methods will be integrated into a boosting framework to improve task performance. The last component aims to design efficient fine-tuning methods, by utilizing new measurements of the generalization of large neural networks. Along with developing new MTL and fine-tuning methods, the research team will explore new AI applications, particularly those with heterogeneous features. The team will create a dataset encompassing years of traffic accident records from eight states to facilitate road safety research using MTL. Through the project, the investigator will integrate graduate and undergraduate students into the research process, train them in the new methods, and advise them on related thesis topics. The project will involve synergistic activities such as conference workshops, sessions, and local symposiums.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
