NSF Award
2209912
Since object-detection algorithms outperformed humans, one decade has passed. During this period, the unprecedented achievements of the Computer Vision domain made many believe that object detection is a solved problem. However, when it comes to scientific imagery such as microscopic, telescopic, aerial, satellite, and medical images, the general-purpose object-detection algorithms are far from perfect. A pixel-precise segmentation of objects and identification of their physical properties based on their texture features are still outstanding challenges in many interdisciplinary areas of research. Space Weather is one such area. Extreme space-weather events, similar to extreme terrestrial events, can have drastic economic and collateral impacts on mankind. Continuous and automatic monitoring of solar filaments plays an integral role in achieving reliable space-weather forecast/prediction systems, which consequently results in the technical preparedness much needed in many infrastructural aspects of the society, such as the power grid and the GPS systems. Our Machine Learning Ecosystem brings automatic, accurate, and reliable analyses of filaments’ dynamic behavior to the experts’ fingertips. The main contributions of this ecosystem are two data products of annotated filaments, and four software products which carry out the annotation (localization, identification, and segmentation) of these filaments. This modular ecosystem can be easily expanded in the future, beyond the lifetime of the award, as faster and more efficient modules are expected to be implemented by the community and replace the existing ones. Throughout the development of this project, we consult with the instrument/data experts from the National Solar Observatory (NSO) for proper utilization of the observation images and metadata we integrate from the six ground-based observatories of the Global Oscillation Network Group, that together provide a full-disk and continuous (24/7) coverage of the Sun.<br/><br/>The primary focus of this project is on the localization and segmentation of a specific solar event, called a filament, and the identification of its magnetic field chirality. That said, the novel concepts investigated in this project, such as the detection algorithm, the augmentation engine, and the segmentation loss function which is sensitive to granularities of objects, remain agnostic to the type of the event/object of interest. Moreover, the released datasets of annotated filaments can serve the Computer Vision community as a testbed for algorithms that aim at high-precision segmentation of objects. Our Machine Learning Ecosystem consists of two data products and four software products. The largest collection of manually annotated filaments data, and a continuously-growing collection of automatically annotated filaments are the two main data products. The main software products are (1) an augmentation engine that provides users with practically unlimited semi-real filament instances, (2) a deep neural network algorithm for localization, segmentation, and classification of filaments, (3) a high-precision segmentation loss function (sensitive to granularities of the observed filaments) that guides the segmentation task, and (4) a deployable detection module which carries out the localization, segmentation, and classification tasks in real time.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
