Patent Application
20230169763
The present application claims priority to Korean Patent Application No. 10-2021-0170169, filed on Dec. 1, 2021. The entire contents of the above-listed application are hereby incorporated by reference for all purposes.
The following disclosure relates to a system and a method for automatically analyzing a cloud cover in an optical satellite image based on machine learning, and more particularly, to a system and a method for automatically analyzing a cloud cover in an optical satellite image based on machine learning capable of automatically analyzing a cloud cover in an optical satellite image through a machine learning model that re-defines object classes in the satellite image, classifies indicator characteristics of white ground surfaces, white objects, and the like, which are similar to each other in reflection characteristics, such as snow and ice, and detects a cloud cover on a pixel basis to improve a deterioration in accuracy of a conventional automatic cloud cover analysis method.
A multi-purpose practical satellite generates a standard image by eliminating or minimizing a geometric distortion of an image captured by the satellite and an error and noise included in a signal value of the image. An optical satellite provides a service for a user to easily read an optical image through a catalog search system by generating a list of standard images in a pre-processing stage.
At this time, since the list of standard images registered in the search system includes clouds, the search service is provided to enable a request for generating a standard image by analyzing a cloud occupancy according to a proportion of a cloud in an image and identifying a valid image satisfying a search condition.
Here, the proportion of the cloud is classified as a cloud cover grade defined as shown in Table 1 below, and catalog information is provided through the search system. The user performs a search by assigning a valid search condition to the corresponding grade, and as a result, selects a specific image from the retrieved valid images and requests a standard image generated for the specific image.
As the cloud cover grades are not further subdivided and the catalog is classified in a 16×16 grid form, detection performance is good when a cloud is thick with a predetermined size or more, but detection performance is poor when a cloud is small and thin.
In addition, snow, which spreads over a large area with a small amount of texture information, may be erroneously detected as a cloud. Therefore, a grid-based automatic cloud cover analysis technique has limitations in performance and location accuracy because an object is determined only from a portion of the grid.
Specifically, in a case where a cloud is thin with a small size, the cloud is not well detected. In addition, in a case where snow spreads over a large area with a small amount of texture information, the snow is erroneously detected as a cloud. Also, a shadow area caused by a cloud is not detected.
Therefore, it is acutely required to develop a more accurate and reliable cloud cover analysis system capable of detecting a cloud cover in which an intensity of a cloud, various indicator characteristics, and a shadow caused by a cloud are considered.
An embodiment of the present invention is directed to providing a system and a method for automatically analyzing a cloud cover in an optical satellite image based on machine learning capable of not only improving accuracy and reliability in cloud cover analysis results by re-defining object classes for analyzing an occupancy of a cloud included in a list of standard images based on machine learning, classifying indicator characteristics having similar reflection characteristics, such as snow and ice, and detecting a cloud on a pixel basis, rather than detecting a cloud on a grid basis according to a conventional method, but also minimizing human resources required for analyzing a cloud cover, and increasing the usability of the satellite image such as removing haze or creating a mask layer for a cloud area using an analysis result value.
In one general aspect, a system for automatically analyzing a cloud cover in an optical satellite image based on machine learning includes: an image management unit receiving image information collected from a satellite, creating and storing a list of the collected image information, and determining whether or not the collected image information is new image information; a cloud cover calculation unit receiving the new image information from the image management unit and calculating a cloud cover based on machine learning; and a cloud cover information storage unit receiving and storing the cloud cover image information for which the cloud cover has been calculated from the cloud cover calculation unit, wherein the cloud cover calculation unit includes a machine learning-based cloud cover calculation model inputting the new image information received from the image management unit as an input pattern and outputting the image information for which the cloud cover has been calculated as an output pattern, and the cloud cover calculation model is generated by performing machine learning with an input data set being input thereto, the input data set including a plurality of pieces of image information and labels corresponding to object classes of the respective pieces of image information.
The cloud cover calculation model may be trained, with a weight value being assigned to each of the object classes, the object classes including a thick cloud, a thin cloud, a cloud shadow, and a background.
After being generated, the cloud cover calculation model may be evaluated with test data being input thereto, while proportions of indicators are reflected in the test data according to indicator characteristics.
The indicator characteristics may include snow/ice indicators, city indicators, river/sea indicators, forest indicators, desert indicators, and other indicators.
The cloud cover calculation model may use a semantic segmentation technique in which objects are detected by object class on a pixel basis within the image information.
In another general aspect, a method for automatically analyzing a cloud cover in an optical satellite image based on machine learning for calculating a cloud cover within image information collected from a satellite includes: a) generating, by a cloud cover calculation unit, a cloud cover calculation model by performing machine learning with an input data set being input thereto, the input data set including a plurality of pieces of image information collected from a satellite and labels corresponding to object classes of the respective pieces of image information; b) determining, by an image management unit, whether the image information collected from the satellite is new image information; c) calculating, by the cloud cover calculation unit, a cloud cover, with the image information determined by the image management unit as new image information being input to the cloud cover calculation model; and d) outputting, by the cloud cover calculation unit, the cloud cover image information for which the cloud cover has been calculated from the cloud cover calculation model, and transferring the cloud cover image information to the cloud cover information storage unit.
The cloud cover calculation model may be generated by performing machine learning with an input data set being input thereto, the input data set including a plurality of pieces of image information and labels corresponding to object classes of the respective pieces of image information.
The cloud cover calculation model may be trained, with a weight value being assigned to each of the object classes, the object classes including a thick cloud, a thin cloud, a cloud shadow, and a background.
The method may further include, between the step a) and the step b), al) evaluating, by the cloud cover calculation unit, the generated cloud cover calculation model with test data being input thereto, while proportions of indicators are reflected in the test data according to indicator characteristics, wherein the indicator characteristics include snow/ice indicators, city indicators, river/sea indicators, forest indicators, desert indicators, and other indicators.
The cloud cover calculation model may use a semantic segmentation technique in which objects are detected by object class on a pixel basis within the image information.
According to the present invention, it is possible to not only improve accuracy and reliability in cloud cover analysis results, but also minimize human resources required for analyzing a cloud cover, and increase the usability of the satellite image, for example, by removing haze or creating a mask layer for a cloud area using an analysis result value.
In addition, for a satellite image service, it is possible to evaluate a catalog as to validity depending on a cloud cover. By applying a cloud cover mask generated through the automatic cloud cover analysis system, it is possible to automatically determine whether to obtain an image at the time of evaluation of validity, thereby improving an image service speed.
The term “unit”, “device”, “system”, or the like described herein refers to a unit for processing an operation in which one or more functions are combined, and may be implemented by hardware, software, or a combination of hardware and software.
The term “unit”, “device”, “system”, or the like as used herein may be treated as being equivalent to a computer related entity, i.e., hardware, a combination of hardware and software, software, or software at the time of execution. In addition, in the present invention, an application program to be executed may be configured in a “unit”, and may be recorded in one physical memory in a readable, writable, and erasable form, or may be recorded in a distributed manner between two or more memories or recording media.
Hereinafter, the present invention will be described with reference to the accompanying drawings.
In addition,
Referring to
Here, the image management unit 100 serves to receive raw captured image information from a satellite, create and manage a list of the received image information, determine whether the received image information is previously received image information or new image information, and transmit the new image information to the cloud cover calculation unit 200 when the received image information is determined as new image information.
If necessary, as illustrated in
Meanwhile, the cloud cover calculation unit 200 is provided to receive the new image information from the image management unit 100 and calculate a cloud cover based on machine learning. The cloud cover calculation unit 200 includes a machine learning-based cloud cover calculation model inputting the new image information received from the image management unit 100 as an input pattern and outputting image information for which a cloud cover has been calculated as an output pattern.
Here, the cloud cover calculation model is an artificial intelligence learning model generated through machine learning by receiving an input data set in advance, the input data set including a plurality of pieces of phase image information and labels corresponding to object classes of the respective pieces of image information. The cloud cover calculation model according to the present invention is configured to more accurately detect the objects by classifying objects into further subdivided object classes and assigning not only result value labels but also weight value labels to the object classes.
According to an embodiment of the present invention, as illustrated in
Furthermore, among the object classes, the thin cloud may further include fog or haze according to settings. Here, the haze refers to a phenomenon in which solid particles, rather than water vapor, reduce visibility.
In the present invention, the machine learning may be performed with a weight value set for each of the object classes. For example, as illustrated in
Here, the weight value for the thin cloud may be differently set, for example, by increasing or decreasing the weight value as a result of calculating a proportion of the thin cloud in the image information and comparing the calculated proportion with a preset proportion (e.g., 80%).
Meanwhile, after being generated, the cloud cover calculation model according to the present invention may be evaluated with test data according to indicator characteristics. Here, the indicator characteristics include snow/ice indicators, city indicators, river/sea indicators, forest indicators, desert indicators, and other indicators, as illustrated in
By performing machine learning for each indicator characteristic as described above, a cloud cover may be detected in consideration of reflectance or the like according to the indicator characteristic even when the objects are identically white in color.
In the test data illustrated in
As illustrated in
For the evaluation through the test data, two matrices (Jaccard index and overall accuracy) were applied. In the evaluation based thereon, the target values of the Jaccard index and the overall accuracy for the thick cloud were set to 90% and 95%, respectively, and the target values of the Jaccard index and the overall accuracy for the thin cloud were set to 70% and 80%, respectively.
By performing the evaluation using the above-described test data, accuracy can be improved.
Meanwhile, as another feature of the present invention, the cloud cover calculation model uses a semantic segmentation technique in which objects are detected by object class on a pixel basis within the image information.
Here, the semantic segmentation technique is for classifying the objects in the image information into semantic units, and predicting which object class each pixel of the image information belongs to.
In this manner, prediction is performed for all the pixels in the image information. Therefore, the semantic segmentation technique is also called dense prediction. As a result, various types of objects such as clouds, rivers, seas, mountains, and buildings may be included in the image information received from the satellite.
By detecting the different types of objects on a pixel basis, the objects can be more clearly classified.
Accordingly, through the cloud cover calculation model of the cloud cover calculation unit 200 according to an embodiment of the present invention, it is possible to clearly detect whether or not there is a shadow caused by a cloud as well as an intensity of the cloud (thick cloud or thin cloud) in the image information, as illustrated in
Meanwhile, the cloud cover information storage unit 300 is provided to receive and store the cloud cover image information for which the cloud cover has been calculated from the cloud cover calculation unit 200. As illustrated in
Referring to
As described for the automatic cloud cover analysis system 1000 above, in the automatic cloud cover analysis method, first of all, the cloud cover calculation unit 200 generates a cloud cover calculation model by performing machine learning with an input data set being input to the cloud cover calculation unit 200, the input data set including a plurality of pieces of image information collected from a satellite and labels corresponding to object classes of the respective pieces of image information (S100).
Here, the cloud cover calculation model is generated by performing machine learning with an input data set being input, the input data set including a plurality of pieces of image information and labels corresponding to object classes of the respective pieces of image information. The cloud cover calculation model is trained, with a weight value being assigned to each of the object classes, the object classes including a thick cloud, a thin cloud, a cloud shadow, and a background.
Furthermore, the thin cloud may further include fog or haze as an object class according to settings, and the weight value for the thin cloud may be differently assigned by comparing a proportion of the thin cloud in the image information with a preset proportion, so that the cloud cover calculation model is trained according to the weight value for the thin cloud.
In addition, in another embodiment of the present invention, the cloud cover calculation unit 200 may perform evaluation based on test data according to indicator characteristics input after the cloud cover calculation model is generated through the step S100. Here, the indicator characteristics include snow/ice indicators, city indicators, river/sea indicators, forest indicators, desert indicators, and other indicators, as illustrated in
By additionally performing machine learning through the test data for each indicator characteristic as described above, a cloud cover may be detected in consideration of reflectance or the like according to the indicator characteristic even when the objects are identically white in color.
In addition, as described above, the cloud cover calculation model uses a semantic segmentation technique in which objects are detected by object class on a pixel basis within the image information.
Meanwhile, when the cloud cover calculation unit 200 generates a cloud cover calculation model by performing the step S100, the image management unit 100 determines whether the image information collected from the satellite is new image information (S200).
When it is determined that the image information collected from the satellite is new image information, a cloud cover is calculated with the new image information being input to the cloud cover calculation model (S300).
In addition, when the cloud cover is calculated, cloud cover image information is output from the cloud cover calculation model (S400), and transferred to and stored in the cloud cover information storage unit 300.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2021-0170169 | Dec 2021 | KR | national |
