NSF Award
2305020
With support from the Chemical Measurement and Imaging (CMI) program in the Chemistry Division, and partial co-funding from the Established Program to Stimulate Competitive Research (EPSCoR) and the Human-Centered Computing Program, Professor John Kalivas and his undergraduate group at Idaho State University are developing new virtual reality (VR) methods of chemical data analysis. In collaboration with the Applied Visualization Laboratory at the Idaho National Laboratory, the Kalivas group is complementing data visualization VR processes with tactile and auditory elements to enhance data analysis. By including all three human senses in conjunction with the human’s unique reasoning capabilities, it should be possible to outperform the computer in complex data analysis situations where the computer is restricted to numerical conclusions. There are many data analysis situations where critical decisions must be made that are not always obvious, such as deciding if an early biopsy is cancerous or not. In addition to medical diagnostics, the insights gained will be useful for a broad range of other applications such as process analytical technologies and the agriculture industry. Educational versions of the VR programs are being developed to allow blind and deaf students to learn and explore data science processes, bringing these students to the forefront of machine learning technology. This part of the work is being developed in collaboration with the Idaho Educational Services for the Deaf and Blind. The project will provide research opportunities for undergraduates and high school students in underserved regions of Idaho.<br/><br/>In this project, the Kalivas team is exploiting their recently developed Physicochemical Response Integrated Spectral Measurement (PRISM) approach in VR. A strategic feature of PRISM is that it is composed of hundreds of sample-wise similarity measures based on amplifying hidden-but-essential chemical (and physiochemical if present) properties encoded within measured spectra such as infrared (IR), near IR, Raman, etc. Each of these similarity measures can be incorporated as an object feature in the VR setting to fully characterize the inherent chemical nature of a sample. A key goal of this exploratory project is to effectively convert multidimensional chemical data to VR thereby significantly enhancing the connection between data and discovery. Using VR to depict data points/samples visually, haptically (touching including texture to data shapes), and with sonification (sound for each data point), the human user will be able to explore data in-depth to uncover hidden patterns and corresponding chemical attributes. With such a detailed data analysis process, improved classification and target sample predictions with better explanations or interpretations of sample relationships should be possible. All developed algorithms will be posted to the Kalivas web site, allowing free access to potential users.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
