NSF Award
2419830
Inauthentic accounts are commonly used by adversaries on online platforms to carry out fraudulent activities like false advertising, scams, and personal threats. These accounts appear to belong to real people, but actually portray fictitious personas and are controlled by miscreants through semi-automated means to deliver potentially harmful content. Promptly detecting inauthentic accounts and fraudulent content is important to keep online users safe and prevent harmful and possibly illegal activity to thrive. Existing approaches to flag potentially harmful content either rely on learning behavioral traits of inauthentic accounts or on identifying keywords that are commonly used in fraudulent content. Existing research has, however, shown that adversaries adapt both their behavior and the content they post over time, with the goal of avoiding being flagged. In this project, the research team aims to address this problem by combining the two approaches into an end-to-end automated analysis pipeline.<br/><br/>The project is improving the state of the art of automated identification of fraudulent online material. First, the team is developing robust artificial intelligence techniques to identify narratives used by previously identified inauthentic online accounts. These techniques will leverage advances in large language models and multi-modal embeddings to identify content that is posted on multiple platforms, consisting not only of text but also of images and videos. Second, the team is developing machine learning techniques to identify the characteristics of narratives used by adversarial actors, with the goal of identifying future harmful narratives irrespective of the content being shared. Third, the investigators aim to use the identified narratives to flag new inauthentic accounts, and learn their behavioral patterns for more effective detection. Used in conjunction, these three methods will allow researchers to identify the changes in content and behavior of harmful online campaigns, allowing for a more robust identification than what is currently possible.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
