Patent Application
20210239606
This invention generally relates to the processing of hyperspectral imaging thermal infrared data between 7 and 14 microns at any spectral resolution to derive physical properties of targets of interest in a computationally compact and efficient manner.
Thermal InfraRed (TIR) hyperspectral imaging deals with acquiring images of a scene and obtaining a spectrum, which is the characteristic distribution of the electromagnetic radiation emitted or absorbed by the target between 7 and 14 microns, for each pixel in the image. TIR hyperspectral imaging combines the power of digital imaging and TIR spectroscopy. In fact, TIR hyperspectral imaging remote sensors can collect image data simultaneously in dozens or hundreds of narrow, adjacent spectral bands. These measurements make it possible to derive a continuous spectrum for each pixel in the image, as shown in Drawing 1. Spectral features then provide information regarding the physical properties of the target.
Converting acquired spectral data into final products is complex and often requires extensive computational capabilities (Fauvel et al. 2017; Qi et al. 2018). Generally, hyperspectral imaging data, acquired using in-situ and space-borne sensors, are firstly downlinked to a ground station and then processed and analyzed. Earth science data often exceeds 10 GB so, the action of downlinking the data implies long delays (Qi et al. 2018).
It would be ideal to have inversion algorithms capable of producing final science data products on-board (Du et al. 2009; Fauvel et al. 2017; Qi et al. 2018). In this way, only the final products, and the not the large raw spectral dataset, would be downlinked. However, current algorithms and available methods are computationally inefficient as they involve millions of floating points operation per pixel and can't be employed for on-board processing (Realmuto et al. 2018). Additionally, existing methods can't effectively deal with atmospheric properties, especially clouds (Hadji-Lazaro and Clerbaux 1999; Garcia-Cuesta et al. 2007; Prata and Bernardo 2014; Whitburn et al. 2016; Fauvel et al. 2017; Ren et al. 2019).
The present method is unique as it uses a supervised Partial Least Squares Regression (PLSR) model to derive characteristics of remote targets of interest (for example SO2 gas, CO2 gas, land surface temperature, remote wind measurements, etc.) under clear and cloudy background conditions. The model was trained from a large lookup table of radiative transfer spectra for realistic conditions, including clouds and aerosols. This method is computationally compact and efficient and can be employed for on-board processing dramatically reducing retrieval solutions. Various tests have shown the efficiency and reliability of the present method. For example, the present method for SO2 gas retrievals requires 149 floating point operations per pixels while Realmuto's SO2 inversion approach requires millions (Realmuto et al. 2017). Additionally, the technique works for both day and night and for both clear and cloudy sky background conditions.
The present method creates a way to significantly improve the efficiency of analyzing hyperspectral imaging data to retrieve characteristics of remote targets of interest, including in the presence of background clouds. The method uses a supervised PLSR model, which was trained from an extensive library of simulated radiative transfer spectra, to derive characteristics of remote targets of interest (for example SO2 gas, CO2 gas, land surface temperature, remote wind measurements, etc.). The radiative transfer library included a large number of complex conditions, including clouds and aerosols. These diverse conditions are cumbersome to implement in a traditional lookup table method but become amenable in the present method. This method is computationally compact and efficient and can be employed for on-board processing, dramatically reducing retrieval times. Various tests (detailed below) have shown the efficiency and reliability of the present method.
Hyperspectral imaging, from ground or space, is important for both military and civilian remote sensing. However, converting large hyperspectral imaging datasets into useable data products is complex and often employs computationally inefficient algorithms (Realmuto et al. 2017). Employing inefficient inversion algorithms is not suitable for in-situ and/or on-board platforms (Qi et al. 2018). For example, processing TIR hyperspectral imaging data for trace gas retrievals might require cumbersome lookup tables. The TIR spectral radiance retrieved from a target is a unique function of the composition and state of the target and the foreground/background atmosphere. Lookup tables can be generated from radiative transfer calculations for a variety of realistic target/atmosphere conditions. Once lookup tables are created, they are difficult to use due to their computational inefficiencies and large size.
Rather than employing a lookup table for these retrievals, the lookup tables can be used to train a supervised PLSR model. This PLSR model is computationally efficient, compact, and amenable for many input/output cases. PLSR is suitable for these type of applications as it can predict outputs based on many input variables, where the inputs and outputs can be redundant, collinear and/or not independent (Hoskuldsson 1988; Martens and Naes 1989; Mattu et al. 2000; Rosipal and Kramer 2006; Lopez et al. 2013). Thus, it allows for a fast and efficient way to implement the inversion algorithm. For example, the PLSR method for SO2 gas retrievals requires 149 floating point operations per pixels while Realmuto's SO2 inversion approach requires millions (Realmuto et al. 2017).
1. Sequence of Events (also summarized in Drawing 2):
1. A user selects and defines the desired retrieved properties (for example SO2 gas, CO2 gas, land surface temperature, remote wind measurements, etc.) in relationship to various environmental conditions including viewing orientation, target cloud and aerosol geometry and other environmental conditions.
2. The user also defines the type of hyperspectral imaging sensor, wavelength window, spectral resolution and viewing geometries.
3. The user then selects the number of PLSR components to be used in the calculations.
4. The user finally defines a large number of atmospheric soundings data.
5. The method employs a radiative transfer model to calculate the at-sensor radiance for the combination of conditions described in steps 1-4 above.
6. The dataset is split into two separate datasets. The first dataset is used as a model training dataset, which is made of 80% of the lookup table spectra. The second dataset is used as a testing dataset, which is made of the remaining 20% spectra.
7. The method creates a PLSR model using the training dataset. The training process is carried out by mapping and associating each spectrum in the training dataset (80% of the lookup table spectra) with the corresponding target condition, which would be retrieved from such a spectrum at each of the environmental and viewing conditions contained in training library. This is implemented using the following equation:
where n is the number of wavelength bands of the sensor, αn is the PLSR model coefficient at band n, Ln is the radiance at band n and k is the user-defined number of PLSR components.
8. The method explains between 70% and 90% of the variance in the training dataset so that the model is not over-trained with possible noise.
9. The performance of the PLSR model is evaluated by using the independent test dataset.
10. The αn model coefficients are saved as output allowing the PLSR model to be applicable to subsequent operations. This results in a compact method for deriving properties from TIR hyperspectral data.
1. The PLSR model is used to invert measured radiance spectra to derive the target property of interest. This is carried out by simply vector multiplying the measured spectral radiance by the αn model coefficients. This process limits the operations that need to be computed to less than 149 floating point operations per pixel.
As a detection target, for this particular test, we considered sulfur dioxide (SO2) volcanic gas emissions at Kīlauea volcano in Hawaii. A lookup table of simulated spectra was created by varying SO2 plume gas concentrations, plume locations, plume sizes, viewing geometries and background materials in an attempt to simulate atmospheric conditions that occur at the summit of the volcano and in analog tropical volcanic scenarios. Such conditions are reported in Table 1.
An individual measured TIR spectrum could be taken and matched against the best fit spectrum in such cumbersome library by brute force methods to invert spectral radiance to SO2 path-concentration. However, this is time consuming and computationally intensive. Rather than using the library of simulated spectra to convert radiance to path-concentration, the table was used to train a PLSR model, following the method described in this invention. The training process was carried out by mapping and associating each spectrum in the training dataset with the corresponding SO2 path-concentration, which would be retrieved from such a spectrum at each of the environmental and viewing conditions. The PLSR coefficients (and not the whole simulated spectral library) are then used to invert the radiance spectra of the remaining 20% spectra to path-concentration to evaluate the performance of the model. Once the performance of the PLSR model is acceptable (% variance explained in the output is higher than 70%), the PLSR coefficients (and not the whole spectral library) are used to invert the measured radiance spectra to path-concentration.
Results obtained using the present invention to create PLSR models and process TIR hyperspectral imaging data, acquired using the Thermal Hyperspectral Imager (THI) sensor, when clouds are present are very encouraging. In fact, the current method can convert radiance measurements into SO2 gas concentrations even when clouds are present. Drawing 2 shows two images of the plume at Kīlauea volcano, which contains SO2 gas, acquired with the THI instrument. Image A shows a raw image of the plume. No plume is clearly present. The clouds, which appear in yellow, are disturbing the data acquisition due to their water vapor absorption and complex thermochemical properties. An image like A would be useless to Earth scientists, as it doesn't help to identify or quantify the SO2 volcanic gas present in it. Image B is the processed version of image A using the present method. Radiance measurements were converted into SO2 gas concentrations measurements. It can be seen that, despite the presence of the clouds, the method was able to identify the volcanic gas and quantify it in parts per million meters (ppm-m) units. Ppm-m is a standard unit for these kind of calculations. This is a very significant result.
Due to the lack of robust inversion algorithms for processing space-borne TIR hyperspectral imaging sensor data in the presence of clouds, the present invention was also tested on retrieving physical information from targets in such conditions. As part of this work, the current method was tested on retrieving airplane-based measurements of SO2 emissions from Kilauea volcano, Hawaii and of Land Surface Temperature (LST), which is a key indicator for plant and crop health monitoring. MODIS/ASTER air-borne simulator (MASTER) datasets, acquired during the January/February 2018 HyspIRI (Hyperspectral InfraRed Imager) NASA campaign, were used to retrieve SO2 path-concentrations from the air. Hyperspectral Thermal Emission Spectrometer (HyTES) datasets were used to retrieve LST. Results are shown in Drawing 4.
The patent or application file contains various drawings executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.
Drawing 1. TIR Hyperspectral imaging combines the power of digital imaging and spectroscopy. TIR hyperspectral imaging remote sensors can collect image data simultaneously in dozens or hundreds of narrow, adjacent spectral bands. These measurements make it possible to derive a continuous spectrum for each pixel in the image. Spectra can be then used to identify processes or materials occurring at that specific pixel based on the unique spectral features that each material/process has.
Drawing 2. An individual measured TIR spectrum could be taken and matched against the best fit spectrum in a lookup table by brute force methods to invert spectral radiance to the target property of interest. However, this is time consuming and computationally intensive. Rather than using the library of spectra to carry out the inversion, the table was used to train a Partial Least Squared Regression (PLSR) model. a) The PLSR training of consists in determining the αn PLSR coefficients. This is carried out by mapping and associating each spectrum in the training dataset with the known target of interest, which would be retrieved from such a spectrum at each of the environmental and viewing conditions contained in the spectral library. b) The αn PLSR coefficients (and not the whole spectral library) are then used to invert measured radiance spectra to the target property of interest.
Drawing 3. Two images of the volcanic plume at Kilauea volcano, which contains SO2 gas, acquired with the THI instrument. Image A shows the raw image of the plume that was acquired in radiance measurements. No plume is clearly visible. Clouds are displayed in yellow. Image B is the processed version of image A using the present method. It can be seen that his software was able to isolate volcanic plume features from the clouds and quantify the amount of SO2 gas present in the plume in parts per million meters units, a standard unit for these kind of calculations.
Drawing 4. A) (above) HyTES image of a field processed using the current invention. The Land Surface Temperature (LST) measurements, which were obtained using the present method, are shown in false colors. B) (below) MASTER image of a volcanic plume emitted by Kilauea volcano, Hawaii. The SO2 path-concentrations of the volcanic plume, which were obtained using the current invention, can be seen in false colors against a gray scale image of the Kilauea summit.
Table 1. Conditions used to develop the lookup table used to train the ALTA-generated PLSR model.
The subject matter herein was developed in part under research contracts provided by the U.S. Government, National Aeronautics and Space Administration (NASA), Earth Science Technology Office (ESTO), Instrument Incubator Program (IIP), Contract NNX14AE61G as well as under subcontract to the Jet Propulsion Laboratory (JPL), Subcontract number No. 1602222.
