NSF Award
2313190
Machine-Learning-as-a-Service (MLaaS) is a new software paradigm that gives developers access to powerful machine learning models trained on massive data sets without requiring those developers know how to train the models or have access to the required training data. This approach makes machine learning accessible to a much wider range of software systems, but it also creates new challenges. Specifically, there is a tension between the desire to provide a very general MLaaS interface (to make it as widely applicable as possible) and the specific needs of individual applications that use MLaaS. For example, many MLaaS providers offer general object detection models as a service, which can recognize tens of thousands of different objects in a picture, but most applications require only a small subset of that capability; e.g., applications concerned with traffic only care about objects that could appear on a roadway. This project will explore this tension – to preserve the generality of MLaaS while improving the robustness, accuracy, and performance of individual applications that use these services. Specifically, the project will first create a benchmark suite of real-world applications to drive an empirical study of the software bugs that arise due to the tension between general MLaaS interfaces and specific application needs. Based on that study, the project will create a set of tools that automatically adapt software to fix inconsistencies and ambiguities that arise due to the use of general MLaaS interfaces in application-specific contexts. Finally, the project will create methods and tools for refactoring software to use additional information –including the MLaaS’s confidence in its results– that is available from MLaaS providers, but is typically ignored by software applications.<br/><br/>Machine learning is now a major part of software systems that affect our daily lives, including transportation, medical systems, and even news distribution. The rise of MLaaS makes it even easier for non-experts to incorporate machine learning into these software systems, but it also increases the opportunities for a new class of software bugs and software failures. This project will identify and categorize the novel class of bugs that can arise from the use of MLaaS in larger software systems and create tools and methodologies to identify and fix those bugs. All benchmarks, data, and software tools developed through this project will be released as open source so that the larger community can freely benefit from this work. By improving the correctness and performance of software systems that use machine learning services, this project will not only make it easier to develop such software, but also improve the quality of people’s daily lives as the software they use will be more reliable.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
