NSF Award
2321504
Inspired by the success of foundation language models in applications such as ChatGPT, one can envision the far-reaching impacts that can be brought by a pre-trained Foundation Graph Learning Model (FGLM) with broader applications in the areas such as scientific research, social network analysis, anomaly detection, drug discovery, and e-commerce. Despite the significant progress of pre-trained graph neural networks, there has not yet a FGLM that can achieve desired performance on various graph-learning-related tasks. To bridge this gap, the goal of this project is to design and develop a new machine-learning paradigm (techniques, methods, and models) towards effective yet efficient FGLMs, which will help researchers and practitioners advance their work in a wide range of real-world applications driven by the prevalent graph-structured data, thus helping to enhance national safety, public health, and welfare. The project outcomes (such as open-source code, benchmark data, and models) will be made publicly accessible and be broadly distributed through demos, publications, media presentations, and the like. This project will integrate research with education, including novel curriculum development, student mentoring, professional training and workforce development, and K-12 outreach activities aimed at women and underrepresented groups.<br/><br/>By developing a new machine-learning paradigm to jointly solve the multi-task, cross-graph, and cross-domain challenges in graph learning at the first attempt, this project includes three interconnected research components towards effective yet efficient FGLMs. First, to realize the strong and consistent task-generalization ability for FGLMs in an effective yet affordable way, given a graph, the team will design and develop a new multi-task self-supervised graph learning framework with a novel multi-gradient descent optimization algorithm coupled with adaptive data augmentation to learn each task equitably well. Second, as real-world graphs are always incomplete, to learn comprehensive knowledge for a specific domain, the team will develop a new multi-graph co-training framework, specifically a variational expectation-maximization framework with relational knowledge distillation, to jointly train the generated graphs in an effective yet efficient manner while tackling the challenge of diversified node attributes of different graphs. Third, to further enable cross-domain knowledge transfer for pre-trained FGLMs, the team will develop mixture-of-expert based meta-learning techniques to characterize the latent properties of graphs from different domains and adaptively utilize the knowledge learned from existing domains to infer on a rarely seen or unseen domain. The new machine-learning paradigm developed in this project will accelerate the development in the rapidly evolving area of pre-trained graph neural networks, advance the field of information integration and informatics, and help researchers and practitioners in different domains to advance their work in a variety of real-world applications driven by the ubiquitous graph data.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
