NSF Award
2430958
Speech-based technologies have been heralded as promising solutions to overcome the limitations of existing clinical modalities related to limited healthcare access, non-naturalistic in-clinic interactions, and social stigma. Speech measures combined with artificial intelligence can serve as valuable biomarkers for mental health conditions, such as depression and post-traumatic stress disorder. Yet, in order for artificial intelligence to truly succeed in a future-of-work landscape in which clinicians will be expected to work side-by-side with artificial intelligence systems, both clinicians and patients need to calibrate their trust in the algorithms that power this decision-making process. The goal of this project is to design reliable machine learning, notably for speech-based diagnosis and monitoring of mental health, for addressing three pillars of trustworthiness: explainability, privacy preservation, and fair decision making. Trustworthiness is critical for both patients and clinicians: patients must be treated fairly and without the risk of reidentification, while clinical decision-making needs to rely on explainable and unbiased machine learning. This research program further provides a fertile ground for training high school and college students providing them with the knowledge about (and inclination toward) ethically applying computing research in sensitive populations. The tangible applications developed as part of this research serve as a vehicle to encourage students to pursue careers in Science, Technology, Engineering, and Mathematics, and prepare them to work in transdisciplinary settings for solving real-world problems.<br/><br/>This project seeks to design explainable, anonymized, and fair speech biomarkers for mental health, integrating aspects of speech acquisition, transparent modeling, and unbiased decision making. The work is divided into three technical objectives. The first objective designs novel speaker anonymization algorithms that retain mental health information and suppress information related to the identity of the speaker. The anonymization algorithms learn a mapping between the original speech and a latent space, which embeds information about speaker identity, mental health, and phonological sequence through deterministic and probabilistic operations. The second objective improves explainability of speech-based models for tracking mental health through novel convolutional architectures that learn explainable spectrotemporal transformations relevant to speech production fundamentals. The third objective examines how bias in data and model design may perpetuate social disparities in mental health, and designs new machine learning to mitigate unwanted bias in speech-based mental health diagnosis. Through a series of experiments this work further contributes to understanding ways in which human-machine partnerships are formed in mental healthcare settings along dimensions of trust formation, maintenance, and repair.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
